Use Individual SDKs
The all-SDK distribution includes a matching compiler, CLI, and SDK registrations. This page covers separately supplied SDKs for an existing Codon installation, and lists the module names used by both installation types.
Choose a prebuilt SDK for your platform and Codon version, then use it to compile
your application. The examples below use /path/to/sdk/data, /path/to/sdk/duckdb,
and similar paths as placeholders for the installed SDK directories.
The compiler, standard library, runtime, and SDK must be compatible. Exaloop can provide the appropriate SDK for your toolchain and target platform.
The optional Exaloop CLI registers SDK locations and orchestrates native application and server-function builds while retaining Codon's core commands.
Requirements
| Component | Requirements for use |
|---|---|
| Data plugin 0.1.0 | Codon >=0.20.3 <0.21.0 |
| Optimizer plugin 0.1.0 | Codon >=0.20.2 <0.21.0; plugin matching the compiler's LLVM build |
| DuckDB integration | Codon 0.20.3; Python 3.9+ to compile extensions for a compatible DuckDB host; embedded mode includes DuckDB |
| Redis SDK | Codon 0.20.3, Redis 7.2+, Python 3.9+ to compile modules |
| PostgreSQL SDK | Codon 0.20.3, PostgreSQL 16+; Python 3.9+, a C compiler, and matching server headers and pg_config to compile functions |
| MySQL SDK | Codon 0.20.3, Oracle MySQL 8.0+ and libmysqlclient; Python 3.9+, C++17 and matching headers to compile UDFs |
| Kafka SDK | Codon 0.20.3, librdkafka 2.4+; validated with Kafka 4.3.1 |
| Lambda SDK | Codon >=0.20.3 <0.21.0, libcurl; matching Ubuntu 24.04 Linux environment for deployment |
| Exaloop AI tools | Python 3.9+; a working Codon installation for native validation |
The release and compatibility guide explains platform selection and release matching. Individual integration guides cover host-server requirements.
Import Paths
All SDK-provided Codon modules live under exaloop:
| API | Import path |
|---|---|
| Arrays, tables, and file formats | exaloop.data |
| Dynamic Protobuf schemas | exaloop.data.protobuf |
| Protobuf wire helpers | exaloop.data.protobuf_wire |
| Avro schemas, batch readers, and generated record I/O | exaloop.data.avro |
| DuckDB | exaloop.databases.duckdb |
| Redis | exaloop.databases.redis |
| PostgreSQL | exaloop.databases.postgres |
| MySQL | exaloop.databases.mysql |
| Kafka | exaloop.streaming.kafka |
| AWS Lambda | exaloop.aws_lambda |
Use a full import such as import exaloop.databases.duckdb, or an alias such as
import exaloop.databases.duckdb as duckdb. The database modules expose both
client and server-function APIs; separate client modules are also available as
exaloop.databases.redis_client, exaloop.databases.postgres_client, and
exaloop.databases.mysql_client.
These paths replace the old top-level imports. In particular, import arrow
becomes import exaloop.data as data. Rebuild generated Protobuf bindings with
the updated SDK so their wire-helper imports use the new namespace. Your own
generated message modules retain the names and packages selected by your schema.
SDK directory names and -plugin arguments are unchanged. Load each required
SDK by repeating -plugin; their modules share the exaloop namespace.
Use the full module path in from imports too, for example
from exaloop.databases.mysql import connect. Codon does not automatically
discover submodules with from exaloop.databases import mysql.
The optimizer has no importable Codon module and is enabled only with -plugin.
Data Plugin
Try It: Write a Parquet File
This program creates a table from a native NumPy array, writes it to Parquet, then reads the column back:
import exaloop.data as data
import numpy as np
with data.table({"temperature": data.array(np.array([20.5, 21.0, 19.8]))}) as dataset:
dataset.write("temperatures.parquet")
with data.read_parquet("temperatures.parquet") as dataset:
values = dataset["temperature"].to_numpy(float)
assert list(values) == [20.5, 21.0, 19.8]
For the example in program.codon, run:
codon run -release -plugin /path/to/sdk/data program.codon
To produce a standalone executable, use
codon build -release -plugin /path/to/sdk/data -o program program.codon, then run
./program. If the SDK is installed at $HOME/.codon/lib/codon/plugins/data,
you can use the short name -plugin data instead.
The SDK includes the runtime library, Codon modules, Protobuf tools, and the native
avrogen-codon generator. See the Avro guide for typed readers
and projected batches. Deploy compiled applications with libcodon_data and the
matching Codon runtime libraries, using library search paths appropriate for the
destination machine. Python and the code generators are development tools, not
application dependencies.
Continue with the Data guide, or generate your first Protobuf bindings.
Database Integrations
Each integration has its own SDK directory and deployment model:
- DuckDB compiles scalar and vectorized UDFs into extensions
loadable by the CLI or Python client. Use the SDK's
build.py, or pass-plugin /path/to/sdk/duckdbto Codon to embed DuckDB in your application. - Redis supports a standalone native client with
-plugin /path/to/sdk/redis(Redis in Codon) and server command modules compiled with the SDK'sbuild.py(Codon in Redis). - PostgreSQL supports a native libpq client with
-plugin /path/to/sdk/postgres(PostgreSQL in Codon) and typed server functions compiled with the SDK'sbuild.py(Codon in PostgreSQL). Client applications require libpq; function SDKs must match the server's major version. - MySQL supports a native client with
-plugin /path/to/sdk/mysql(MySQL in Codon) and loadable scalar functions compiled withbuild.py(Codon in MySQL). TCP connections verify certificates and hostnames by default; UDFs are trusted native server code.
Streaming Integrations
Kafka supports native producers, batched consumers,
partition replay, and Kafka transactions through exaloop.streaming.kafka.
Compile with -plugin /path/to/sdk/kafka and deploy with librdkafka; no Java or
Python runtime is needed by applications.
Profiling and Optimization
Use profiling to locate runtime costs, then profile-guided optimization to train and evaluate a new build. Both need SDKs matching the Codon compiler and its LLVM build. Register individual SDK directories with the CLI:
exaloop plugin add profiling --path /path/to/sdk/profiling
exaloop plugin add ext --path /path/to/sdk/ext
exaloop doctor --profiling
exaloop doctor --optimization
The probes compile and execute native checks; do not rely on version strings
alone. PGO additionally needs an application target and representative replay
inputs, as shown in its walkthrough. Compiler-pass switches and direct -plugin
use are covered in the advanced reference.
The optimizer adds no runtime library to the selected application.
Exaloop AI
Exaloop AI supplies native API recipes, Codon engineering guidance, examples and local validation tools. Its portable bundle is separate from native SDK selection. Follow the Exaloop AI guide to activate it in your coding assistant and choose the references for your application.