-   [](https://platform.exaloop.io/)
-   [Cookbook](https://platform.exaloop.io/cookbook/)
-   Profile, Tune, and Package a Batch Worker

# Profile, Tune, and Package a Batch Worker

Take a batch worker through performance investigation, PGO evaluation, and runtime packaging. A repeatable synthetic workload makes the workflow easy to reproduce; separate training, tuning, and holdout inputs establish how candidate selection works before you apply it to production data.

**Uses:** native profiler, optimizer SDK, CLI projects and runtime packaging. **Services:** none. **Difficulty:** advanced.

## Establish a Repeatable Workload

Download the [complete event-processing project](https://platform.exaloop.io/assets/files/optimization-acd45ef00669f7467df3664f4bcce45a.zip/), extract it, and run from the `optimization` directory. It contains `app.codon` and `exaloop.toml`; it needs neither a dataset download nor a broker. Its arguments are event count, seed, and optional distribution (`skew` or `uniform`).

```sh title="Terminal"
unzip optimization.zip
cd optimization
exaloop doctor --profiling
exaloop doctor --optimization
exaloop --target events run 1000 1 skew
```

```text title="Output"
[479935, 19488, 26600, 47320, 46728]
```

The program prints five integer category totals. The same arguments must produce the same totals. Different seeds provide different input sequences, and `uniform` exercises a different category mix. The small run checks installation; use larger runs for measurement so startup does not dominate.

This synthetic workload demonstrates the workflow, not the speedup available to your application. For a real worker, retain its parsing, allocations, numerical work, and output checks in the measured path. Do not repeatedly call a pure function on the same constant input and mistake compiler hoisting for throughput.

## Find the Cost Before Tuning

```sh title="Terminal"
exaloop --no-open --profile --target events run 20000000 7 skew
exaloop --no-open --profile=memory --target events run 20000000 7 uniform
```

Open the resulting `profiles/capture-*/index.html`. Inspect expensive functions, call counts, capture warnings, and RSS. A wide function may include I/O waiting: the profiler measures elapsed time, not CPU samples. Native library allocations can increase RSS without appearing in the GC heap. Confirm any improvement in an unprofiled run because instrumentation changes execution cost.

If your real application spends most of its time waiting for the database or broker, reduce unnecessary queries or transfers before expecting compiler tuning to change end-to-end latency.

## Separate Training, Selection, and Evaluation

Replace the sample manifest with this configuration to include both traffic mixes. Keep the holdout cases out of training and candidate selection.

```toml title="exaloop.toml"
[targets.events]
kind = "native"
source = "app.codon"
codon_args = ["-release"]

[targets.events.optimize]
train = [["8000000", "1", "skew"], ["8000000", "19", "uniform"]]
tune = [["16000000", "7", "skew"], ["16000000", "23", "uniform"]]
holdout = [["24000000", "29", "skew"], ["24000000", "41", "uniform"]]
samples = 9
presets = ["pgo"]
```

```sh title="Terminal"
exaloop --no-open --dry-run --target events optimize --budget 10m
exaloop --no-open --target events optimize --budget 10m
```

Inspect `build/optimize/events/session-*/index.html` and `report.json`. The result may select the baseline: that is the correct outcome when no candidate passes the configured correctness, performance, and resource gates. Review individual holdout cases, not just an aggregate improvement. Keep the report with the input definitions and toolchain version used to produce it.

The default verifier compares stdout and stderr with the baseline. That covers this program's complete observable result, but does not establish that the baseline implements the right business rule. Maintain independent correctness tests. A worker writing Parquet or updating a database needs a [custom verifier](https://platform.exaloop.io/opt/reference/#configuration) for those effects. Use immutable inputs and disposable outputs: optimization runs the program many times. Never train against a production payment or notification endpoint.

## Package the Intended Build

For the ordinary non-PGO project build:

```sh title="Terminal"
exaloop --target events package --bundle-runtime --output packages/events-v1
```

Move the complete new package directory to a compatible staging machine, keeping its libraries, hashes, and license inventory together. Run the packaged executable with the same sanity and holdout arguments and compare the category totals. The destination does not need the compiler, CLI, or Python interpreter, but it still needs a compatible OS and native runtime environment.

Selected PGO artifacts

The optimization session's selected executable is `program` inside its session directory. The `package` command above performs a normal project build; it does not package that selected executable. Do not label the resulting package as PGO-optimized. Deploying the selected artifact requires a separately validated native dependency bundle and staging checks for that exact executable. Keep its session report and artifact identity together; do not swap it into a package and leave stale integrity hashes.

For a service-backed worker, supply credentials, certificates, application data, and service configuration separately. Runtime packaging is not cross-compilation, does not start services, and does not make a binary portable to every Linux or macOS host. Keep the previous validated package available for rollback.

## Apply This to Other Recipes

-   Profile [portfolio screening](https://platform.exaloop.io/cookbook/risk-sql/) with representative SQL input sizes and null distributions, then compare unprofiled end-to-end time.
-   Replay captured [telemetry](https://platform.exaloop.io/cookbook/telemetry/) from immutable files when testing parsing and validation; include malformed and missing-value cases.
-   Profile a [Kafka worker](https://platform.exaloop.io/cookbook/transactional-events/) with an isolated topic and group. Its external effects make it unsuitable for the simple stdout-only tuning configuration used here.

Source: https://platform.exaloop.io/cookbook/performance-delivery/