-   [](https://platform.exaloop.io/)
-   Data APIs
-   Working with Runtime Schemas

# Working with Runtime Schemas

Use `exaloop.data.protobuf` when a message's schema becomes available at runtime. The module loads Protobuf descriptor sets and exposes native reflection through field names and numbers. It suits schema inspection tools and services that receive descriptors alongside their payloads.

When the schema is known before compilation, [generated messages](https://platform.exaloop.io/data/protobuf/) provide typed field access. Reflection instead requires an accessor matching each field's type; nested messages are independent copies that must be written back after modification.

## Load Your Schema

The application loads a descriptor set generated by `protoc`. For this example, save the following schema as `common.proto`:

```protobuf title="common.proto"
syntax = "proto3";
package example;
message Child {
  string name = 1;
}
```

The `--include_imports` option includes any dependent schemas in the descriptor set, allowing the application to load it independently:

```sh title="Terminal"
mkdir -p build
exaloop protoc -I. --include_imports \
  --descriptor_set_out=build/schema.pb common.proto
```

```python title="program.codon"
from exaloop.data.protobuf import Schema

with open("build/schema.pb", "rb") as stream:
    descriptors = stream.read()

with Schema(descriptors) as schema:
    with schema.message("example.Child") as message:
        message.set_string("name", "Ada")
        wire = message.serialize()
        assert message.get_string(1) == "Ada"
```

Save the program as `program.codon` and run it with the Data SDK:

```sh title="Terminal"
exaloop --with data run -release program.codon
```

`Schema` reads the binary descriptor set, and `message()` creates an empty message using its fully qualified name. In this example, the same field is written by name and read by number. Both forms refer to the original `.proto` definition, not a generated Codon attribute name.

The context managers release native resources when the block finishes. Messages keep their schema alive, so an existing message remains usable after its `Schema` handle is closed. Loading an incomplete or invalid descriptor set raises `ValueError`; that includes missing or cyclic dependencies, duplicate files, and empty sets.

## Read and Change Fields

Choose an accessor for the field's Protobuf type: `get_string()` for text, `get_int()` for signed integers, and so on. Each getter has a corresponding setter. A singular field needs only its name or number, as in `message.set_string("name", "Ada")`.

For repeated fields, `count(field)` tells you how many elements are present. Pass a zero-based `index` to read or replace an element. To add an element, pass `index=-2` to a setter. The default index, `-1`, identifies a singular field; these negative values are API markers, not Python-style indexing from the end. Map fields use the same repeated-field interface, with one entry message per key/value pair. Your application is responsible for replacing an existing key.

`get_message()` returns a copy of a nested message. Changes to that copy become part of the parent only when you assign it back with `set_message()`. Likewise, returned strings and bytes are owned copies: they remain valid if the parent is changed or closed.

### Presence and Parsing

`has(field)` distinguishes an explicitly present field from an absent one, where the schema supports presence. It raises an error for fields without presence rather than treating their default value as absence. `clear(field)` resets a singular field or empties a repeated field.

To load wire data, call `parse(data)`. Parsing replaces the message only after the input passes Protobuf parsing, required-field, and UTF-8 validation; a failed parse leaves the previous contents intact. `serialize()` produces deterministic wire bytes and checks that required fields have been initialized. String fields must contain valid UTF-8, while bytes fields can contain arbitrary binary data.

## API Reference

### Schema

| API | Description |
| --- | --- |
| `runtime_version() -> int` | Compiled protobuf version encoded by `GOOGLE_PROTOBUF_VERSION` |
| `Schema(descriptors: bytes)` | Load a binary `FileDescriptorSet` |
| `schema.message(name: str) -> Message` | Construct by fully qualified protobuf message name |
| `schema.close()` | Idempotent; existing messages retain their schema |
| `with schema as entered:` | Close on exit |

### Message

In the signatures below, `field` accepts a schema name (`str`) or field number (`int`). The index conventions are explained in [Read and Change Fields](https://platform.exaloop.io/data/protobuf-dynamic/#read-and-change-fields).

| API | Description |
| --- | --- |
| `count(field) -> int` | Size of repeated field |
| `has(field) -> bool` | Explicit presence; rejects fields without presence |
| `clear(field)` | Reset singular or repeated field |
| `get_int(field, index=-1) -> int` | Signed integer, enum, or bool reflection family |
| `get_uint(field, index=-1) -> u64` | Unsigned integer family |
| `get_float(field, index=-1) -> float` | Float/double family |
| `get_bool(field, index=-1) -> bool` | Boolean conversion of integer-family getter |
| `get_bytes(field, index=-1) -> bytes` | Independent string/bytes payload copy |
| `get_string(field, index=-1) -> str` | UTF-8 decode of the string/bytes payload |
| `get_message(field, index=-1) -> Message` | Independent child copy, not a live view |
| `set_int(field, value: int, index=-1)` | Signed/enum range checks; bool must be 0 or 1 |
| `set_uint(field, value: u64, index=-1)` | Unsigned range checks |
| `set_float(field, value: float, index=-1)` | Float/double assignment |
| `set_bool(field, value: bool, index=-1)` | Integer-family boolean setter |
| `set_bytes(field, value: bytes, index=-1)` | String or binary assignment; string fields validate UTF-8 |
| `set_string(field, value: str, index=-1)` | UTF-8 encode then assign string/bytes family |
| `set_message(field, value: Message, index=-1)` | Copy child; checks message name and schema compatibility |
| `parse(data: bytes)` | Replace only after successful parse, required-field and UTF-8 validation |
| `serialize() -> bytes` | Deterministic serialization; requires initialized required fields |
| `close()` / context manager | Idempotent native-handle cleanup |

## Sharing Messages and Schemas

You can assign a message from another schema when the fully qualified message names match. If the two messages do not share the same descriptor, the library checks wire compatibility by serializing and parsing the value. This does not establish that the schemas have identical meanings: schema versioning remains an application concern.

Native API failures surface as `ValueError`, including a closed handle, an incorrect field type or index, and missing required fields. Handles are shared between aliases, so closing a message through one variable closes it for every alias. If multiple threads use the same schema factory or message, protect that access with application-level synchronization.

For compile-time schemas and live child views, see the [generated message API](https://platform.exaloop.io/data/protobuf-api/).

Source: https://platform.exaloop.io/data/protobuf-dynamic/