assignment not allowed to message map or repeated

Send Downlink command as JSON via gRPC and Python with use of payload formatter

Hey, I’m trying to send a downlink command via gRPC and Python to an existing device. It works to a point, but I tried to send a JSON, which then should get processed via the payload formatter. The problem is that if I send the JSON as an utf-8 encoded string, it does not get passed through the payload format but gets enqueued directly to the device. I tried to assign the req.queue_item.object, but it gives me an error (“AttributeError: Assignment not allowed to message, map, or repeated field “object” in the protocol message object.”). Is there a way to send a downlink to chirpstack with a JSON so that the payload formatter can encode it for the device?

Are you testing this against ChirpStack v3 or v4?

Hey, We use ChirpStack v4 and the corresponding PyPI library.

@brocaar , is this a bug, or is this functionality not yet implemented?

Where in v3 you had to pass the JSON object as string, you must pass it as JSON object in v4 ( https://github.com/chirpstack/chirpstack/blob/master/api/proto/api/device.proto#L483 ).

@brocaar I know, but if I set it to either a python JSON object or a google.protobuf.Struct it returns:

And what do I do with the data field, I can’t let it be empty.

I’m not familiar with the Python API for Protobuf struct types, but maybe this could help you? dictionary - How do you create a Protobuf Struct from a Python Dict? - Stack Overflow

@brocaar I tried that before but same message:

I think there is something else wrong. Maybe the word object is conflicting with something in python? Or something went wrong with the code generation for python. But I honestly don’t know how to debug this.

Found the solution. If you use:

Instead of:

Then everything works.

Sign up on Python Hint

Join our community of friendly folks discovering and sharing the latest topics in tech.

We'll never post to any of your accounts without your permission.

AttributeError: Assignment not allowed to composite field "task" in protocol message object

on 11 months ago

Understanding Protobuf and Message Objects

The error: attributeerror: assignment not allowed to composite field, fixing the attributeerror: working with composite fields in protobuf, conclusion:, port matlab bounding ellipsoid code to python.

31455 views

10 months ago

How do I use OpenCV MatchTemplate?

64853 views

How to debug Django unit tests?

47359 views

9 months ago

Passing on named variable arguments in python

94802 views

svg diagrams using python

55295 views

Related Posts

SQLAlchemy: query custom property based on table field

Why doesn't pytorch allow inplace operations on leaf variables, geodjango: postgresql not running migrations, object has no attribute 'geo_db_type, tensorflow object detection fine-tuning a model from an existing checkpoint, can i change __name__ atribute of object in python, airflow : externaltasksensor doesn't trigger the task, imported modules becomes none when replacing current module in sys.modules using a class object, how to catch - 'nonetype' object has no attribute 'something', finding the largest/smallest object in a python generator/list, assigning list to one value in that list, how could i detect subtypes in pandas object columns, the axis argument to unique is not supported for dtype object, how to get the most recent message of a channel in discord.py, python: how to deal with expected a readable buffer object in python, python-opencv attributeerror: 'module' object has no attribute 'createbackgroundsubtractorgmg', installation.

Copyright 2023 - Python Hint

Term of Service

Privacy Policy

Cookie Policy

Python Generated Code Guide

Any differences between proto2 and proto3 generated code are highlighted - note that these differences are in the generated code as described in this document, not the base message classes/interfaces, which are the same in both versions. You should read the proto2 language guide and/or proto3 language guide before reading this document.

The Python Protocol Buffers implementation is a little different from C++ and Java. In Python, the compiler only outputs code to build descriptors for the generated classes, and a Python metaclass does the real work. This document describes what you get after the metaclass has been applied.

Compiler Invocation

The protocol buffer compiler produces Python output when invoked with the --python_out= command-line flag. The parameter to the --python_out= option is the directory where you want the compiler to write your Python output. The compiler creates a .py file for each .proto file input. The names of the output files are computed by taking the name of the .proto file and making two changes:

• The extension ( .proto ) is replaced with _pb2.py .
• The proto path (specified with the --proto_path= or -I command-line flag) is replaced with the output path (specified with the --python_out= flag).

So, for example, let’s say you invoke the compiler as follows:

The compiler will read the files src/foo.proto and src/bar/baz.proto and produce two output files: build/gen/foo_pb2.py and build/gen/bar/baz_pb2.py . The compiler will automatically create the directory build/gen/bar if necessary, but it will not create build or build/gen ; they must already exist.

Protoc can generate Python stubs ( .pyi ) using the --pyi_out parameter.

Note that if the .proto file or its path contains any characters which cannot be used in Python module names (for example, hyphens), they will be replaced with underscores. So, the file foo-bar.proto becomes the Python file foo_bar_pb2.py .

The Python code generated by the protocol buffer compiler is completely unaffected by the package name defined in the .proto file. Instead, Python packages are identified by directory structure.

Given a simple message declaration:

The protocol buffer compiler generates a class called Foo , which subclasses google.protobuf.Message . The class is a concrete class; no abstract methods are left unimplemented. Unlike C++ and Java, Python generated code is unaffected by the optimize_for option in the .proto file; in effect, all Python code is optimized for code size.

If the message’s name is a Python keyword, then its class will only be accessible via getattr() , as described in the Names which conflict with Python keywords section.

You should not create your own Foo subclasses. Generated classes are not designed for subclassing and may lead to "fragile base class" problems. Besides, implementation inheritance is bad design.

Python message classes have no particular public members other than those defined by the Message interface and those generated for nested fields, messages, and enum types (described below). Message provides methods you can use to check, manipulate, read, or write the entire message, including parsing from and serializing to binary strings. In addition to these methods, the Foo class defines the following static methods:

• FromString(s) : Returns a new message instance deserialized from the given string.

Note that you can also use the text_format module to work with protocol messages in text format: for example, the Merge() method lets you merge an ASCII representation of a message into an existing message.

Nested Types

A message can be declared inside another message. For example:

In this case, the Bar class is declared as a static member of Foo , so you can refer to it as Foo.Bar .

Well Known Types

Protocol buffers provides a number of well-known types that you can use in your .proto files along with your own message types. Some WKT messages have special methods in addition to the usual protocol buffer message methods, as they subclass both google.protobuf.Message and a WKT class.

For Any messages, you can call Pack() to pack a specified message into the current Any message, or Unpack() to unpack the current Any message into a specified message. For example:

Unpack() also checks the descriptor of the passed-in message object against the stored one and returns False if they don’t match and does not attempt any unpacking; True otherwise.

You can also call the Is() method to check if the Any message represents the given protocol buffer type. For example:

Use the TypeName() method to retrieve the protobuf type name of an inner message.

Timestamp messages can be converted to/from RFC 3339 date string format (JSON string) using the ToJsonString() / FromJsonString() methods. For example:

You can also call GetCurrentTime() to fill the Timestamp message with current time:

To convert between other time units since epoch, you can call ToNanoseconds(), FromNanoseconds(), ToMicroseconds(), FromMicroseconds(), ToMilliseconds(), FromMilliseconds(), ToSeconds() , or FromSeconds() . The generated code also has ToDatetime() and FromDatetime() methods to convert between Python datetime objects and Timestamps. For example:

Duration messages have the same methods as Timestamp to convert between JSON string and other time units. To convert between timedelta and Duration, you can call ToTimedelta() or FromTimedelta . For example:

FieldMask messages can be converted to/from JSON string using the ToJsonString() / FromJsonString() methods. In addition, a FieldMask message has the following methods:

• IsValidForDescriptor(message_descriptor) : Checks whether the FieldMask is valid for Message Descriptor.
• AllFieldsFromDescriptor(message_descriptor) : Gets all direct fields of Message Descriptor to FieldMask.
• CanonicalFormFromMask(mask) : Converts a FieldMask to the canonical form.
• Union(mask1, mask2) : Merges two FieldMasks into this FieldMask.
• Intersect(mask1, mask2) : Intersects two FieldMasks into this FieldMask.
• MergeMessage(source, destination, replace_message_field=False, replace_repeated_field=False) : Merges fields specified in FieldMask from source to destination.

Struct messages let you get and set the items directly. For example:

To get or create a list/struct, you can call get_or_create_list() / get_or_create_struct() . For example:

A ListValue message acts like a Python sequence that lets you do the following:

To add a ListValue/Struct, call add_list() / add_struct() . For example:

For each field in a message type, the corresponding class has a property with the same name as the field. How you can manipulate the property depends on its type.

As well as a property, the compiler generates an integer constant for each field containing its field number. The constant name is the field name converted to upper-case followed by _FIELD_NUMBER . For example, given the field optional int32 foo_bar = 5; , the compiler will generate the constant FOO_BAR_FIELD_NUMBER = 5 .

If the field’s name is a Python keyword, then its property will only be accessible via getattr() and setattr() , as described in the Names which conflict with Python keywords section.

Singular Fields (proto2)

If you have a singular (optional or required) field foo of any non-message type, you can manipulate the field foo as if it were a regular field. For example, if foo ’s type is int32 , you can say:

Note that setting foo to a value of the wrong type will raise a TypeError .

If foo is read when it is not set, its value is the default value for that field. To check if foo is set, or to clear the value of foo , you must call the HasField() or ClearField() methods of the Message interface. For example:

Singular Fields (proto3)

If you have a singular field foo of any non-message type, you can manipulate the field foo as if it were a regular field. For example, if foo ’s type is int32 , you can say:

If foo is read when it is not set, its value is the default value for that field. To clear the value of foo and reset it to the default value for its type, you call the ClearField() method of the Message interface. For example:

Singular Message Fields

Message types work slightly differently. You cannot assign a value to an embedded message field. Instead, assigning a value to any field within the child message implies setting the message field in the parent. You can also use the parent message’s HasField() method to check if a message type field value has been set.

So, for example, let’s say you have the following .proto definition:

You cannot do the following:

Instead, to set bar , you simply assign a value directly to a field within bar , and - presto! - foo has a bar field:

Similarly, you can set bar using the Message interface’s CopyFrom() method. This copies all the values from another message of the same type as bar .

Note that simply reading a field inside bar does not set the field:

If you need the "has" bit on a message that does not have any fields you can or want to set, you may use the SetInParent() method.

Repeated Fields

Repeated fields are represented as an object that acts like a Python sequence. As with embedded messages, you cannot assign the field directly, but you can manipulate it. For example, given this message definition:

You can do the following:

The ClearField() method of the Message interface works in addition to using Python del .

When using the index to retrieve a value, you can use negative numbers, such as using -1 to retrieve the last element in the list. If your index goes out of bounds, you’ll get an IndexError: list index out of range .

Repeated Message Fields

Repeated message fields work similar to repeated scalar fields. However, the corresponding Python object also has an add() method that creates a new message object, appends it to the list, and returns it for the caller to fill in. Also, the object’s append() method makes a copy of the given message and appends that copy to the list. This is done so that messages are always owned by the parent message to avoid circular references and other confusion that can happen when a mutable data structure has multiple owners. Similarly, the object’s extend() method appends an entire list of messages, but makes a copy of every message in the list.

For example, given this message definition:

Unlike repeated scalar fields, repeated message fields don’t support item assignment (i.e. __setitem__ ). For example:

Groups (proto2)

Note that groups are deprecated and should not be used when creating new message types – use nested message types instead.

A group combines a nested message type and a field into a single declaration, and uses a different wire format for the message. The generated message has the same name as the group. The generated field’s name is the lowercased name of the group.

For example, except for wire format, the following two message definitions are equivalent:

A group is either required , optional , or repeated . A required or optional group is manipulated using the same API as a regular singular message field. A repeated group is manipulated using the same API as a regular repeated message field.

For example, given the above SearchResponse definition, you can do the following:

Given this message definition:

The generated Python API for the map field is just like a Python dict :

As with embedded message fields , messages cannot be directly assigned into a map value. Instead, to add a message as a map value you reference an undefined key , which constructs and returns a new submessage:

You can find out more about undefined keys in the next section.

Referencing undefined keys

The semantics of Protocol Buffer maps behave slightly differently to Python dict s when it comes to undefined keys. In a regular Python dict , referencing an undefined key raises a KeyError exception:

However, in Protocol Buffers maps, referencing an undefined key creates the key in the map with a zero/false/empty value. This behavior is more like the Python standard library defaultdict .

This behavior is especially convenient for maps with message type values, because you can directly update the fields of the returned message.

Note that even if you don’t assign any values to message fields, the submessage is still created in the map:

This is different from regular embedded message fields , where the message itself is only created once you assign a value to one of its fields.

As it may not be immediately obvious to anyone reading your code that m.message_map[10] alone, for example, may create a submessage, we also provide a get_or_create() method that does the same thing but whose name makes the possible message creation more explicit:

Enumerations

In Python, enums are just integers. A set of integral constants are defined corresponding to the enum’s defined values. For example, given:

The constants VALUE_A , VALUE_B , and VALUE_C are defined with values 0, 5, and 1234, respectively. You can access SomeEnum if desired. If an enum is defined in the outer scope, the values are module constants; if it is defined within a message (like above), they become static members of that message class.

For example, you can access the values in the three following ways for the following enum in a proto:

An enum field works just like a scalar field.

If the enum’s name (or an enum value) is a Python keyword, then its object (or the enum value’s property) will only be accessible via getattr() , as described in the Names which conflict with Python keywords section.

The values you can set in an enum depend on your protocol buffers version:

• In proto2 , an enum cannot contain a numeric value other than those defined for the enum type. If you assign a value that is not in the enum, the generated code will throw an exception.
• proto3 uses open enum semantics: enum fields can contain any int32 value.

Enums have a number of utility methods for getting field names from values and vice versa, lists of fields, and so on - these are defined in enum_type_wrapper.EnumTypeWrapper (the base class for generated enum classes). So, for example, if you have the following standalone enum in myproto.proto :

…you can do this:

For an enum declared within a protocol message, such as Foo above, the syntax is similar:

If multiple enum constants have the same value (aliases), the first constant defined is returned.

In the above example, myproto_pb2.SomeEnum.Name(5) returns "VALUE_B" .

Given a message with a oneof:

The Python class corresponding to Foo will have properties called name and serial_number just like regular fields . However, unlike regular fields, at most one of the fields in a oneof can be set at a time, which is ensured by the runtime. For example:

The message class also has a WhichOneof method that lets you find out which field (if any) in the oneof has been set. This method returns the name of the field that is set, or None if nothing has been set:

HasField and ClearField also accept oneof names in addition to field names:

Note that calling ClearField on a oneof just clears the currently set field.

Names which conflict with Python keywords

If the name of a message, field, enum, or enum value is a Python keyword , then the name of its corresponding class or property will be the same, but you’ll only be able to access it using Python’s getattr() and setattr() built-in functions, and not via Python’s normal attribute reference syntax (i.e. the dot operator).

For example, if you have the following .proto definition:

You would access those fields like this:

By contrast, trying to use obj.attr syntax to access these fields results in Python raising syntax errors when parsing your code:

Extensions (proto2 only)

Given a message with an extension range:

The Python class corresponding to Foo will have a member called Extensions , which is a dictionary mapping extension identifiers to their current values.

Given an extension definition:

The protocol buffer compiler generates an "extension identifier" called bar . The identifier acts as a key to the Extensions dictionary. The result of looking up a value in this dictionary is exactly the same as if you accessed a normal field of the same type. So, given the above example, you could do:

Note that you need to specify the extension identifier constant, not just a string name: this is because it’s possible for multiple extensions with the same name to be specified in different scopes.

Analogous to normal fields, Extensions[...] returns a message object for singular messages and a sequence for repeated fields.

The Message interface’s HasField() and ClearField() methods do not work with extensions; you must use HasExtension() and ClearExtension() instead. To use the HasExtension() and ClearExtension() methods, pass in the field_descriptor for the extension you are checking for the existence of.

If the .proto file contains the following line:

Then the protocol buffer compiler will generate code based on the service definitions found in the file as described in this section. However, the generated code may be undesirable as it is not tied to any particular RPC system, and thus requires more levels of indirection that code tailored to one system. If you do NOT want this code to be generated, add this line to the file:

If neither of the above lines are given, the option defaults to false , as generic services are deprecated. (Note that prior to 2.4.0, the option defaults to true )

RPC systems based on .proto -language service definitions should provide plugins to generate code appropriate for the system. These plugins are likely to require that abstract services are disabled, so that they can generate their own classes of the same names. Plugins are new in version 2.3.0 (January 2010).

The remainder of this section describes what the protocol buffer compiler generates when abstract services are enabled.

Given a service definition:

The protocol buffer compiler will generate a class Foo to represent this service. Foo will have a method for each method defined in the service definition. In this case, the method Bar is defined as:

The parameters are equivalent to the parameters of Service.CallMethod() , except that the method_descriptor argument is implied.

These generated methods are intended to be overridden by subclasses. The default implementations simply call controller.SetFailed() with an error message indicating that the method is unimplemented, then invoke the done callback. When implementing your own service, you must subclass this generated service and implement its methods as appropriate.

Foo subclasses the Service interface. The protocol buffer compiler automatically generates implementations of the methods of Service as follows:

• GetDescriptor : Returns the service’s ServiceDescriptor .
• CallMethod : Determines which method is being called based on the provided method descriptor and calls it directly.
• GetRequestClass and GetResponseClass : Returns the class of the request or response of the correct type for the given method.

The protocol buffer compiler also generates a "stub" implementation of every service interface, which is used by clients wishing to send requests to servers implementing the service. For the Foo service (above), the stub implementation Foo_Stub will be defined.

Foo_Stub is a subclass of Foo . Its constructor takes an RpcChannel as a parameter. The stub then implements each of the service’s methods by calling the channel’s CallMethod() method.

The Protocol Buffer library does not include an RPC implementation. However, it includes all of the tools you need to hook up a generated service class to any arbitrary RPC implementation of your choice. You need only provide implementations of RpcChannel and RpcController .

Plugin Insertion Points

Code generator plugins which want to extend the output of the Python code generator may insert code of the following types using the given insertion point names.

• imports : Import statements.
• module_scope : Top-level declarations.

Sharing Messages Between Python and C++

Prior to the 4.21.0 version of the Protobuf Python API, Python apps could share messages with C++ using a native extension. Starting in the 4.21.0 API version, sharing messages between Python and C++ is not supported by the default install. To enable this capability when working with the 4.x and later versions of the Protobuf Python API, define the environment variable, PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp , and ensure that the Python/C++ extension is installed.

google.protobuf.internal.containers ¶

Contains container classes to represent different protocol buffer types.

This file defines container classes which represent categories of protocol buffer field types which need extra maintenance. Currently these categories are:

Repeated scalar fields - These are all repeated fields which aren’t composite (e.g. they are of simple types like int32, string, etc).

Repeated composite fields - Repeated fields which are composite. This includes groups and nested messages.

Base container class.

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

Simple, type-checked, dict-like container for with submessage values.

get_or_create() is an alias for getitem (ie. map[key]).

key – The key to get or create in the map.

This is useful in cases where you want to be explicit that the call is mutating the map. This can avoid lint errors for statements like this that otherwise would appear to be pointless statements:

msg.my_map[key]

If key is not found, d is returned if given, otherwise KeyError is raised.

as a 2-tuple; but raise KeyError if D is empty.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

Simple, list-like container for holding repeated composite fields.

Appends the contents of another repeated field of the same type to this one, copying each individual message.

Adds a new element at the end of the list and returns it. Keyword arguments may be used to initialize the element.

Appends one element by copying the message.

Extends by appending the given sequence of elements of the same type

as this one, copying each individual message.

Inserts the item at the specified position by copying.

Removes and returns an item at a given index. Similar to list.pop().

Removes an item from the list. Similar to list.remove().

Simple, type-checked, list-like container for holding repeated scalars.

Appends the contents of another repeated field of the same type to this one. We do not check the types of the individual fields.

Appends an item to the list. Similar to list.append().

Extends by appending the given iterable. Similar to list.extend().

Inserts the item at the specified position. Similar to list.insert().

Simple, type-checked, dict-like container for holding repeated scalars.

UnknownField container

Table of Contents

• google.protobuf
• google.protobuf.any_pb2
• google.protobuf.descriptor
• google.protobuf.descriptor_database
• google.protobuf.descriptor_pb2
• google.protobuf.descriptor_pool
• google.protobuf.duration_pb2
• google.protobuf.empty_pb2
• google.protobuf.field_mask_pb2
• google.protobuf.internal.containers
• google.protobuf.json_format
• google.protobuf.message
• google.protobuf.message_factory
• google.protobuf.proto_builder
• google.protobuf.reflection
• google.protobuf.service
• google.protobuf.service_reflection
• google.protobuf.struct_pb2
• google.protobuf.symbol_database
• google.protobuf.text_encoding
• google.protobuf.text_format
• google.protobuf.timestamp_pb2
• google.protobuf.type_pb2
• google.protobuf.unknown_fields
• google.protobuf.wrappers_pb2

Related Topics

• Previous: google.protobuf.field_mask_pb2
• Next: google.protobuf.json_format
• Show Source

Can I add a new node in the middle of a Tensorflow model graph using the graphsurgeon tool?

Hello, I’m using TensorRT version 4.0.1.6.

I successfully replaced an unsupported Tensorflow operation with a TensorRT PlugIn layer using the graphsurgeon APIs.

Now, I want to add a totally new node in the middle of the model.

I saw that the graphsurgeon support these APIs: create_node, create_plugin_node, extend, append

But I couldn’t understand if I can use them for my purpose.

I know how to find the specific model location where I want to add my new node but I cannot find the right way to use the graphsurgeon APIs to do that.

Please advise,

You can convert graph_def to GraphSurgeons DynamicGraph, modify it and finally write it as uff:

Thanks juko.lam

My purpose is to add a debug node multiple times in the middle of the grpah in order to implement a simple functionallity: Save to disk the tensor input data.

By this way I will have the ability to investigate the graph nodes tensor outputs data correctness.

So, as I wrote previously, I have the following abilities:

• uff file generation using uff.from_tensorflow_frozen_model API
• Replace an existing nodes in the graph using graphsurgeon API
• Remove an existing nodes in the graph using graphsurgeon API

What I don’t have and till now I couldn’t succeeded to do is to add totally new node somewhere in the graph

Can you provide me please an example of how to add new node in the middle of the graph using graphsurgeon APIs?

Based on my constraints (I have only the frozen graph pb file), Do you have any other idea (not related to the graphsurgeon) to to achieve my purpose?

You can load the frozen pb file which gives you the graph, then convert the graph to DynamicGraph and modify it. So something like this (probably not runnable directly):

I suggest that you inspect graphs created by the TRT examples just before they are written to UFF file.

I tried this to replace the ResizeBilinear with Conv2DTranspose but I got error :

“convert_lanenet_uff.py”, line 32, in add_plugin node.input = next_.input AttributeError: Assignment not allowed to repeated field “input” in protocol message object.

Code snippet : import tensorflow as tf from tensorflow.core.framework import types_pb2 import graphsurgeon as gs

graph = gs.DynamicGraph(‘frozen_graph.pb’)

Add new node

node = tf.NodeDef() node.op = ‘Conv2DTranspose’ node.name = ‘Conv2DTranspose’ graph.append(node)

want to replace ResizeBilinear with Conv2DTranspose layer

next = graph.find_nodes_by_op(“ResizeBilinear”)[0] # assume only one node is found node.input = next.input next.input = [node.name]

Assignment not allowed to repeated field "conversions" in protocol message object

[email protected]

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protocol message object_detection.protos.CenterNet.ObjectDetection has no non-repeated field "scale_head_params" #10200

mihaidimoiu commented Aug 10, 2021 • edited

No branches or pull requests