Working with Runtime Schemas
Use exaloop.data.protobuf when a message's schema becomes available at runtime.
The module loads Protobuf descriptor sets and exposes native reflection through
field names and numbers. It suits schema inspection tools and services that
receive descriptors alongside their payloads.
When the schema is known before compilation, generated messages provide typed field access. Reflection instead requires an accessor matching each field's type; nested messages are independent copies that must be written back after modification.
Load Your Schema
The application loads a descriptor set generated by protoc. For this example,
save the following schema as common.proto:
syntax = "proto3";
package example;
message Child {
string name = 1;
}
The --include_imports option includes any dependent schemas in the descriptor
set, allowing the application to load it independently:
mkdir -p build
exaloop protoc -I. --include_imports \
--descriptor_set_out=build/schema.pb common.proto
from exaloop.data.protobuf import Schema
with open("build/schema.pb", "rb") as stream:
descriptors = stream.read()
with Schema(descriptors) as schema:
with schema.message("example.Child") as message:
message.set_string("name", "Ada")
wire = message.serialize()
assert message.get_string(1) == "Ada"
Save the program as program.codon and run it with the Data SDK:
exaloop --with data run -release program.codon
Schema reads the binary descriptor set, and message() creates an empty
message using its fully qualified name. In this example, the same field is
written by name and read by number. Both forms refer to the original .proto
definition, not a generated Codon attribute name.
The context managers release native resources when the block finishes. Messages
keep their schema alive, so an existing message remains usable after its
Schema handle is closed. Loading an incomplete or invalid descriptor set
raises ValueError; that includes missing or cyclic dependencies, duplicate
files, and empty sets.
Read and Change Fields
Choose an accessor for the field's Protobuf type: get_string() for text,
get_int() for signed integers, and so on. Each getter has a corresponding
setter. A singular field needs only its name or number, as in
message.set_string("name", "Ada").
For repeated fields, count(field) tells you how many elements are present.
Pass a zero-based index to read or replace an element. To add an element, pass
index=-2 to a setter. The default index, -1, identifies a singular field;
these negative values are API markers, not Python-style indexing from the end.
Map fields use the same repeated-field interface, with one entry message per
key/value pair. Your application is responsible for replacing an existing key.
get_message() returns a copy of a nested message. Changes to that copy become
part of the parent only when you assign it back with set_message(). Likewise,
returned strings and bytes are owned copies: they remain valid if the parent is
changed or closed.
Presence and Parsing
has(field) distinguishes an explicitly present field from an absent one, where
the schema supports presence. It raises an error for fields without presence
rather than treating their default value as absence. clear(field) resets a
singular field or empties a repeated field.
To load wire data, call parse(data). Parsing replaces the message only after
the input passes Protobuf parsing, required-field, and UTF-8 validation; a failed
parse leaves the previous contents intact. serialize() produces deterministic
wire bytes and checks that required fields have been initialized. String fields
must contain valid UTF-8, while bytes fields can contain arbitrary binary data.
API Reference
Schema
| API | Description |
|---|---|
runtime_version() -> int | Compiled protobuf version encoded by GOOGLE_PROTOBUF_VERSION |
Schema(descriptors: bytes) | Load a binary FileDescriptorSet |
schema.message(name: str) -> Message | Construct by fully qualified protobuf message name |
schema.close() | Idempotent; existing messages retain their schema |
with schema as entered: | Close on exit |
Message
In the signatures below, field accepts a schema name (str) or field number
(int). The index conventions are explained in Read and Change Fields.
| API | Description |
|---|---|
count(field) -> int | Size of repeated field |
has(field) -> bool | Explicit presence; rejects fields without presence |
clear(field) | Reset singular or repeated field |
get_int(field, index=-1) -> int | Signed integer, enum, or bool reflection family |
get_uint(field, index=-1) -> u64 | Unsigned integer family |
get_float(field, index=-1) -> float | Float/double family |
get_bool(field, index=-1) -> bool | Boolean conversion of integer-family getter |
get_bytes(field, index=-1) -> bytes | Independent string/bytes payload copy |
get_string(field, index=-1) -> str | UTF-8 decode of the string/bytes payload |
get_message(field, index=-1) -> Message | Independent child copy, not a live view |
set_int(field, value: int, index=-1) | Signed/enum range checks; bool must be 0 or 1 |
set_uint(field, value: u64, index=-1) | Unsigned range checks |
set_float(field, value: float, index=-1) | Float/double assignment |
set_bool(field, value: bool, index=-1) | Integer-family boolean setter |
set_bytes(field, value: bytes, index=-1) | String or binary assignment; string fields validate UTF-8 |
set_string(field, value: str, index=-1) | UTF-8 encode then assign string/bytes family |
set_message(field, value: Message, index=-1) | Copy child; checks message name and schema compatibility |
parse(data: bytes) | Replace only after successful parse, required-field and UTF-8 validation |
serialize() -> bytes | Deterministic serialization; requires initialized required fields |
close() / context manager | Idempotent native-handle cleanup |
Sharing Messages and Schemas
You can assign a message from another schema when the fully qualified message names match. If the two messages do not share the same descriptor, the library checks wire compatibility by serializing and parsing the value. This does not establish that the schemas have identical meanings: schema versioning remains an application concern.
Native API failures surface as ValueError, including a closed handle, an
incorrect field type or index, and missing required fields. Handles are shared
between aliases, so closing a message through one variable closes it for every
alias. If multiple threads use the same schema factory or message, protect that
access with application-level synchronization.
For compile-time schemas and live child views, see the generated message API.