Untuned Warmup

class pytorch_warmup.untuned.UntunedExponentialWarmup(optimizer, last_step=-1)

Untuned exponetial warmup schedule for Adam.

This warmup scheme is described in On the adequacy of untuned warmup for adaptive optimization.

Parameters:

• optimizer (Optimizer) – an Adam optimizer

• last_step (int) – The index of last step. (Default: -1)

class pytorch_warmup.untuned.UntunedLinearWarmup(optimizer, last_step=-1)

Untuned linear warmup schedule for Adam.

This warmup scheme is described in On the adequacy of untuned warmup for adaptive optimization.

Parameters:

• optimizer (Optimizer) – an Adam optimizer

• last_step (int) – The index of last step. (Default: -1)