Training¶

The Training panel configures and runs sleap-nn training without leaving the app. Runs go to a local GPU (desktop, see Environment setup) or to a remote worker.

You need at least one user-labeled frame. Label a handful first — the Tutorial walks the whole loop.

Model types¶

Top-down trains two models — a centroid model and a centered instance model — so the panel shows two config slots. The other types use one.

Configuration¶

The panel holds the settings you change most. Everything else lives in the Full Configuration... dialog.

In the panel¶

• Model Type & Configs — the Model Type, plus one config slot per model. A ★ marks the type recommended for your data; types that need skeleton edges are disabled until you add them. Each slot is filled in for you (from your most recent trained run when there is one, otherwise a baseline profile), and you can browse to a different config
• Data
  • Training Labels — which project to train on (defaults to the open one)
  • Validation Labels (optional) — leave empty for Same as training (auto-split)
  • Post-Training Inference Target — see Post-training inference
• Hyperparameters — Max Epochs, Batch Size, Rotation Augmentation (Off, ±15°, ±180°, Custom), and Scale Augmentation. For top-down, Anchor Part sits above them, and two-model pipelines get one tab per model. Start at 5 epochs for a first end-to-end run, then raise it
• Remote (desktop) — send the run to a remote worker instead of the local GPU
• Estimated Memory Usage — params, weights, batch images, activations, confidence maps, gradients, and image cache, so you find out that a configuration won't fit before you start the run rather than at epoch 1

Key ideas¶

A few terms that come up throughout the configuration:

• Configs. A model's settings live in a config file (YAML), shown in its slot under Model Type & Configs. These settings are hyperparameters: you choose them before training. The model's parameters, its weights, are what training learns. The app fills each slot from your last trained run, or from a built-in baseline profile.
• Backbone. The network the model is built on. UNet is the default and works well for most projects. ConvNeXt and Swin Transformer can start from weights pretrained on ImageNet, but they need color (RGB) input. See sleap-nn's model reference for details.
• Input scaling. Shrinks images before the model sees them: faster and lighter on memory, but less precise. The centroid model in a top-down pipeline only has to find each animal, so it usually takes scaling well.
• Crop size (top-down). The size of the box the second model looks at around each animal. On Auto, the app takes your largest labeled animal, adds room for the rotation and scale augmentation you've set, and rounds up to fit the model. One stray point that makes an instance huge also makes the crop huge, so check Instance Size Distribution before you trust it.
• Augmentation. Random rotations, scaling, noise and contrast changes applied to your labeled frames during training, so the model doesn't memorize the exact images. Match rotation to your camera: ±180° for overhead views, small angles for side views.

Full Configuration...¶

Full Configuration... opens the Training Configuration dialog: a Pipeline tab, plus one tab per model. Use Search parameters... to jump to a field. Edits save as you type; click Done to close, or Reset to profile defaults… to start over.

The Pipeline tab:

Each model's tab:

Scratch, fine-tune, or resume¶

The top of each model's tab picks how the run starts:

Fine-tune and Resume need a checkpoint path; Start Training stays disabled until you set one.

Checking before you train (desktop)¶

Dry run, next to Start Training, checks your setup in seconds before you commit to a long run. For each model (both, for top-down) it runs sleap-nn train --dry-run with the same config a real run would use, data pipeline and workers included: all setup, plus one training batch and one validation batch, on one device. It also checks it can save to the model folder next to your labels file: it writes the configs and a checkpoint there, then removes them, so nothing is kept.

• Passed — each model shows what the real run would do: labeled frames and samples, steps per epoch, max epochs, model size, device, run folder, data pipeline and workers, whether the image cache fits in memory, and the write check
• Failed — the model shows the error that stopped it, with Copy for the error output. Show log has the full output, and Copy log copies it. If one model fails, the other is still checked
• Cancel stops the check and removes what it had written so far
• Change the config afterwards and the result says so; run it again to check the new one

Dry runs need a sleap-nn release after 0.3.5. With an older one the result says so, and training itself still works. Dry runs run on this computer only, not on a remote worker.

Running¶

Click Start Training (or Start Remote Training with a worker selected). While it runs:

• Each model gets a progress row with its epoch count and loss. Click the row to open the Training Monitor — live training and validation loss curves, plus each epoch's sample predictions when Visualize Predictions is on (under Output ▸ Visualization)
• The log shows raw sleap-nn output — click it, or Expand, for the full log
• If a run fails, the error output shows why without making you dig through the log

To end a run early, use the buttons under the panel (also in the Training Monitor):

• Stop Early — stops the current model and keeps its checkpoint. The run carries on: the next model of a top-down pair, then post-training inference
• Cancel — terminates immediately. Nothing after it runs

When training finishes:

• View Metrics — accuracy metrics for the models you just trained
• Export Model… (desktop) — convert the trained model to ONNX or TensorRT. See Exporting a model
• Train Again — clears the finished run so you can start the next round

Post-training inference¶

Post-Training Inference Target tells the app what to predict on as soon as training finishes. The predictions are merged into the project automatically (a remote run shows Fetch & Load Predictions instead):

• Nothing (skip inference)
• Suggested frames
• User labeled frames
• Frames with predictions
• Entire current video
• All videos
• Random sample (current video)
• Random sample (all videos)

After training¶

• Predict ▸ Evaluation Metrics for Trained Models… — accuracy metrics for models you've trained, with detailed per-node breakdowns
• Predict ▸ Set Overlay Models…, then check Predict ▸ Visualize Model Outputs (desktop) — draws a model's confidence map over the current frame, which is how you diagnose why a model is wrong rather than just that it is. The checkbox stays disabled until you've set overlay models. Single-animal and top-down models are supported
• Predict ▸ Export Labels Package… — bundle labels and frames for training elsewhere

Then go correct the predictions and train again. See Inference.