Class AdaDeltaOptimizer

java.lang.Object

deepnetts.net.train.opt.AdaDeltaOptimizer

All Implemented Interfaces:

Optimizer, TrainingListener, EventListener

public final class AdaDeltaOptimizer
extends Object
implements Optimizer, TrainingListener

Implementation of ADADELTA which is a modification of AdaGrad that uses only a limited window of previous gradients. Warning: Implementation of this optimizer is experimental and still unstable.

See Also:

• Optimizer
• AdaGradOptimizer
• ADADELTA Paper

Field Summary

Fields inherited from interface Optimizer

COL_IDX, ROW_IDX

Constructor Summary

Constructors

Constructor	Description
AdaDeltaOptimizer(AbstractLayer layer)

Method Summary

Modifier and Type	Method	Description
float	calculateDeltaBias(float grad, int idx)
Tensor1D	calculateDeltaBias(Tensor1D grad)
float	calculateDeltaWeight(float grad, int... idxs)	Smoothing term to prevent division by zero if sqr grad sum becomes zero 1e-8 should be also tried https://d2l.ai/chapter_optimization/adagrad.html 1e-6 The value to use is 1e-6, 1e-8, Keras uses 1e-7 for adam
TensorBase	calculateDeltaWeight(TensorBase grad)
void	handleEvent(TrainingEvent event)	Invoked when a training event occurs.
void	setLearningRate(float learningRate)

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AdaDeltaOptimizer

public AdaDeltaOptimizer(AbstractLayer layer)

Method Details

calculateDeltaWeight

public float calculateDeltaWeight(float grad,
 int... idxs)

Description copied from interface: Optimizer

Smoothing term to prevent division by zero if sqr grad sum becomes zero 1e-8 should be also tried https://d2l.ai/chapter_optimization/adagrad.html 1e-6 The value to use is 1e-6, 1e-8, Keras uses 1e-7 for adam

Specified by:

calculateDeltaWeight in interface Optimizer

calculateDeltaBias

public float calculateDeltaBias(float grad,
 int idx)

Specified by:

calculateDeltaBias in interface Optimizer

handleEvent

public void handleEvent(TrainingEvent event)

Description copied from interface: TrainingListener

Invoked when a training event occurs.

Specified by:

handleEvent in interface TrainingListener

Parameters:

event - the training event

setLearningRate

public void setLearningRate(float learningRate)

Specified by:

setLearningRate in interface Optimizer

calculateDeltaWeight

public TensorBase calculateDeltaWeight(TensorBase grad)

Specified by:

calculateDeltaWeight in interface Optimizer

calculateDeltaBias

public Tensor1D calculateDeltaBias(Tensor1D grad)

Specified by:

calculateDeltaBias in interface Optimizer