Skip to main content

Pre-training

Pre-training builds a foundation model from your data. It learns the structure and relationships across the datasets in a design, without being aimed at one specific prediction task yet. What it produces are checkpoints — the starting points for Fine-tuning and for deployment.

Pre-training vs Fine-tuning

These are two separate sections of Studio, each with its own runs list, its own leaderboard and its own metadata. They are not two settings of one screen.

Pre-trainingFine-tuning
Starts fromA published design and your datasetsAn existing base checkpoint
LearnsGeneral structure across your dataA specific task
ProducesFoundation checkpointsTask-adapted checkpoints
InternallyPRETRAINPOST_TRAINING

If you remember finding both under a single "Training" section, that is the change: look for them as siblings under Studio.

Creating a run

A run always begins from a Published Model Design — that design supplies the datasets, the split, the targets and the metrics. What you set at run time is everything about how the training executes: the cluster it runs on, its resources and its hyperparameters.

The run's configuration is presented in three tabs, which stay readable after the run starts so you can always see what a run was built from:

  • Model Input — the datasets and split inherited from the design.
  • Model Output — the targets, tasks and metrics inherited from the design.
  • Training Settings — the execution configuration. See Hyperparameters.

Scheduling a later start

You can set a start time in the future instead of running immediately. The run is created in Scheduled status and the reconciler starts it at the chosen moment. Until it starts you can edit the start time, run it now, or cancel it — a canceled scheduled run keeps its record and is marked Canceled.

Run statuses

Training run states: creating, then pending for capacity or scheduled for a later start, then running, ending as completed, failed or canceled; a running run can be paused and resumed

StatusMeaning
CreatingThe run record is being provisioned.
PendingWaiting for cluster capacity.
ScheduledDeferred start; will begin at its scheduled time.
RunningTraining in progress.
PausedSuspended; resumable.
CompletedFinished; its checkpoints are available.
FailedStopped on an error. Open the run to inspect logs.
CanceledStopped before or during execution by a user.
For DeletionMarked for removal.

A queued run waits for capacity on the cluster. If runs sit in Pending longer than you expect, the cluster is saturated — see Clusters.

Monitoring a run

Open a run to follow its progress. Metrics are reported on two axes:

  • Training metrics, emitted as the run progresses, plotted by step and epoch.
  • Validation metrics, evaluated against the design's test split.

Checkpoints are marked on the metric charts, so you can see which checkpoint corresponds to which point on the loss curve, and you can export the metric tables for a selected set of checkpoints. Full detail in Monitoring.

Checkpoints

A run emits checkpoints as it trains. Each one is a usable model artifact:

  • Fine-tune from it, as a base checkpoint.
  • Compare it against other checkpoints and runs on the Leaderboard.
  • Deploy it to an Inference Server.

The best checkpoint is rarely the last one. Use the leaderboard rather than assuming the final checkpoint won.

Next steps