Skip to content
Docs

Validation

class Validation

Configuration to perform validation during model training.

Arguments:

  • dataset - The validation data, as a TabularDataset or TextDataset object.
  • lr_schedule - A LrSchedule object. Leave to None for a constant learning rate.
  • early_stop - An EarlyStop object, an EarlyStopMode object, or a string representing an EarlyStopMode. If None, early stop will not be performed.
  • save_best - A path to save the best model checkpoint. If None, the best model will not be saved.
  • dp - A DpValid object to perform validation in a Differentially Private (DP) manner with respect to the validation set. Importantly, only the global loss has DP guarantees.
  • tensorboard - A path to a TensorBoard log directory. If None, TensorBoard data will not be recorded.
  • each - The validation frequency.
  • trigger - The event triggering validation. Can be a HookEvent object or a string representing a HookEvent.

class EarlyStopMode

Enumeration class representing different early stop modes. Supported modes are: PRECISE, NORMAL and QUICK.

class EarlyStop

Early stop configuration.

Arguments:

  • delta - The smallest variation which is considered an improvement: if the relative improvement of the monitored quantity is less than this value, the early stopping will be triggered.
  • patience - The number of consecutive times that the early stopping criterion must be triggered before actually failing.

class LrSchedule

Learning rate scheduler that reduces the learning rate of a given factor when the monitored quantity reaches a plateau.

Arguments:

  • factor - The factor by which the learning rate will be reduced, must be between 0 and 1.
  • patience - The number of steps for which the condition must be triggered before actually reducing the learning rate.
  • threshold - The threshold for measuring the new optimum, to only focus on significant changes.
  • cooldown - The number of steps to wait before resuming normal operation after the learning rate has been reduced.
  • lr_min_ratio - The ratio to compute the minimal learning rate, i.e. lr * self.lr_min_ratio. The learning rate will never go below such value.

class DpValid

Configuration to perform validation during model training with Differential Privacy (DP) guarantees on the validation set. Importantly, only the global loss has DP guarantees.

Arguments:

  • eps - The epsilon value of the DP guarantee on the validation data.
  • loss_clipping - The clipping value for the loss. If a tuple of two float, it is interpreted as (min, max). If a single float, it is interpreted as (0, max). While max must be positive, min must be non-positive.