Batch runs
A batch run scores a processed dataset offline against the model this server has deployed. Where the online endpoint answers one request at a time, a batch run answers for an entire population in one pass.
Batch runs matter beyond scoring: a server's drift baseline is built from its first completed batch run. Until one completes, Drift Radar has nothing to compare against.
Triggering a run
Pick a processed dataset and a time range. Everything else has a working default.
Advanced flags
| Flag | What it does |
|---|---|
| Infer every token | Run the heads on every position instead of only the last. |
| Include all probabilities | Ship per-class probabilities for every categorical target. |
| Save logits | Persist the raw logits alongside the decoded output. |
| Logits only | Skip the decoded predictions parquet entirely. |
| Infer target date | Override every parquet timestamp with one date. Leave empty to keep the event time. |
Save logits and Logits only are easy to confuse: the first adds raw output next to the
decoded predictions, the second replaces them.
Output destination
Without an output destination, predictions stay on internal storage and you have to fetch them by hand. Configuring one exports each run automatically as it finishes.
Reading a run
The list shows every run against this server and can be filtered by status. Opening one gives its Inference Details, its shards — batch inference is distributed, and shard detail is where a partial failure shows up — and its drift analysis once that completes.
If the list fails to load, that is usually transient. Retry; if it persists, the
inference-api logs are the place to look.
Next steps
- Drift Radar — what the baseline built from these runs is used for.
- History — every inference this server has served.