Skip to content
Hayonix

API reference · Schema

FineTuneHyperparameters

  • epochsinteger

    Default 1

    1–20

  • batch_sizeinteger

    Default 8

    1–256

  • gradient_accumulation_stepsinteger

    Batches accumulated per optimizer step (default 1)

    1–64

  • learning_ratenumber

    Default 0.0001

    1e-7–0.01

  • lr_schedulerstring

    How the learning rate changes over the run (default linear)

    One oflinearcosineconstant

  • warmup_rationumber

    Share of the steps over which the learning rate ramps up from 0 (default 0)

    0–0.5

  • weight_decaynumber

    Weight decay (default 0)

    0–1

  • max_grad_normnumber

    Gradients are clipped to this norm (default 1)

    0–100

  • max_seq_lengthinteger

    Longer examples are truncated (default: the model limit, up to 2048)

    32–131072

  • max_stepsinteger | null

    Stop after this many optimizer steps, whatever `epochs` says; null or omitted: no step limit

    1–1000000

  • seedinteger

    Random seed, for repeatable runs (default 42)

    0–2147483647

Example
{
  "epochs": 1,
  "batch_size": 1,
  "gradient_accumulation_steps": 1,
  "learning_rate": 1e-7,
  "lr_scheduler": "linear",
  "warmup_ratio": 0,
  "weight_decay": 0,
  "max_grad_norm": 0,
  "max_seq_length": 32,
  "max_steps": 1,
  "seed": 0
}