lr_schedulers

This file contains learning rate schedulers for tensorflow optimization.

class tensiometer.synthetic_probability.lr_schedulers.ExponentialDecayAnnealer(start, end, roll_off_step, steps)

Exponential learning-rate annealer with smooth roll-off.

Initialize the annealer.

Parameters:

• start – starting learning rate.

• end – final learning rate.

• roll_off_step – step where decay begins.

• steps – total number of steps to reach end.

step()

Advance one step and compute the decayed learning rate.

Returns:

updated learning rate.

class tensiometer.synthetic_probability.lr_schedulers.ExponentialDecayScheduler(lr_max, lr_min, roll_off_step, steps)

Keras callback applying exponential decay to the optimizer learning rate.

Exponentially decaying learning rate.

Parameters:

• lr_max – maximum learning rate

• lr_min – minimum earning rate

• roll_off_step – step at which the scheduler starts rolling off

• steps – total number of steps

get_lr()

Retrieve the current optimizer learning rate.

on_train_batch_begin(batch, logs=None)

Record the current learning rate at batch start.

on_train_batch_end(batch, logs=None)

Update the learning rate at batch end.

on_train_begin(logs=None)

Reset state and set the initial learning rate.

set_lr(lr)

Set the optimizer learning rate if available.

Parameters:

lr – learning rate value to assign.

class tensiometer.synthetic_probability.lr_schedulers.LRAdaptLossSlopeEarlyStop(monitor='val_loss', factor=np.float64(0.31622776601683794), patience=25, cooldown=10, verbose=0, min_lr=1e-05, threshold=0.0, **kwargs)

Adaptive learning-rate scheduler with optional early stopping based on loss slope.

Adaptive reduction of learning rate when likelihood improvement stalls for a given number of epochs.

Parameters:

• monitor – metric name to monitor.

• factor – multiplicative decay factor (<1).

• patience – epochs to wait before reducing the rate.

• cooldown – epochs to wait after a reduction.

• verbose – verbosity level.

• min_lr – lower bound for the learning rate.

• threshold – minimum loss change considered improvement.

on_epoch_end(epoch, logs=None)

Update learning rate based on monitored loss slope at epoch end.

Parameters:

• epoch – current epoch index.

• logs – training logs containing monitored metrics.

on_train_begin(logs=None)

Reset scheduler state at the start of training.

class tensiometer.synthetic_probability.lr_schedulers.LRSeesawAdaptLossSlopeEarlyStop(monitor='val_loss', reduction_factor=np.float64(0.31622776601683794), increase_factor=0.003, patience=25, cooldown=10, verbose=0, min_lr=1e-05, threshold=0.0, **kwargs)

Adaptive scheduler that decays on plateau but allows gentle increases when improving.

Adaptive reduction of learning rate when likelihood improvement stalls for a given number of epochs.

Parameters:

• monitor – metric name to monitor.

• reduction_factor – multiplicative decay factor (<1).

• increase_factor – small increment applied when loss improves.

• patience – epochs to wait before reducing the rate.

• cooldown – epochs to wait after a reduction.

• verbose – verbosity level.

• min_lr – lower bound for the learning rate.

• threshold – minimum loss change considered improvement.

class tensiometer.synthetic_probability.lr_schedulers.PowerLawDecayAnnealer(start, end, power, steps)

Power-law learning-rate annealer.

Initialize the annealer.

Parameters:

• start – starting learning rate.

• end – final learning rate.

• power – power-law exponent.

• steps – total number of steps to reach end.

step()

Advance one step and compute the decayed learning rate.

Returns:

updated learning rate.

class tensiometer.synthetic_probability.lr_schedulers.PowerLawDecayScheduler(lr_max, lr_min, power, steps)

Keras callback applying power-law decay to the learning rate.

Power law decaying learning rate.

Parameters:

• lr_max – maximum learning rate

• lr_min – minimum earning rate

• power – power law index

• steps – total number of steps

get_lr()

Retrieve the current optimizer learning rate.

on_train_batch_begin(batch, logs=None)

Record the current learning rate at batch start.

on_train_batch_end(batch, logs=None)

Update the learning rate at batch end.

on_train_begin(logs=None)

Reset state and set the initial learning rate.

set_lr(lr)

Set the optimizer learning rate if available.

Parameters:

lr – learning rate value to assign.

class tensiometer.synthetic_probability.lr_schedulers.StepDecayAnnealer(start=None, change_every=None, steps=None, steps_per_epoch=None, boundaries=None, values=None)

Piecewise constant learning-rate annealer.

Initialize the step decay schedule.

Parameters:

• start – initial learning rate.

• change_every – number of steps between decay events.

• steps – total number of steps.

• steps_per_epoch – steps contained in one epoch.

• boundaries – optional explicit boundary steps.

• values – optional explicit values at boundaries.

step()

Advance one step and return the appropriate learning rate.

Returns:

learning rate selected by the current step.

class tensiometer.synthetic_probability.lr_schedulers.StepDecayScheduler(lr_max=None, change_every=None, steps=None, steps_per_epoch=None, boundaries=None, values=None)

Keras callback applying step-wise changes to the learning rate.

Learning rate step function changes.

Parameters:

• lr_max

• change_every

• steps

• steps_per_epoch

• boundaries

• values

on_train_batch_begin(batch, logs=None)

Record the current learning rate at batch start.

on_train_batch_end(batch, logs=None)

Update the learning rate at batch end.

on_train_begin(logs=None)

Reset state and set the initial learning rate.

set_lr(lr)

Set the optimizer learning rate if available.

Parameters:

lr – learning rate value to assign.