Weight Initialization

Weight initialization is a technique that deals with presetting of the weight parameters in the network prior to training in a manner that favors the ultimate convergence of the neural network.

Function Descriptions

class ztlearn.initializers.GlorotNormal[source]

Bases: ztlearn.initializers.WeightInitializer

Glorot Normal (GlorotNormal)

GlorotNormal, more famously known as the Xavier initialization is based on the effort to try mantain the same variance of the gradients of the weights for all the layers. Glorot normal is an implementation based on Gaussian distribution

References

[1] Understanding the difficulty of training deep feedforward neural networks
[2] Initialization Of Deep Feedfoward Networks
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.GlorotUniform[source]

Bases: ztlearn.initializers.WeightInitializer

Glorot Uniform (GlorotUniform)

GlorotUniform, more famously known as the Xavier initialization is based on the effort to try mantain the same variance of the gradients of the weights for all the layers. Glorot uniform is an implementation based on Uniform distribution

References

[1] Understanding the difficulty of training deep feedforward neural networks
[2] Initialization Of Deep Feedfoward Networks
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.HeNormal[source]

Bases: ztlearn.initializers.WeightInitializer

He Normal (HeNormal)

HeNormal is a robust initialization method that particularly considers the rectifier nonlinearities. He normal is an implementation based on Gaussian distribution

References

[1] Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
[2] Initialization Of Deep Networks Case of Rectifiers
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.HeUniform[source]

Bases: ztlearn.initializers.WeightInitializer

He Normal (HeNormal)

HeNormal is a robust initialization method that particularly considers the rectifier nonlinearities. He uniform is an implementation based on Uniform distribution

References

[1] Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
[2] Initialization Of Deep Networks Case of Rectifiers
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.Identity[source]

Bases: ztlearn.initializers.WeightInitializer

Identity (Identity)

Identity is an implementation of weight initialization that returns an identity matrix of size shape

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.InitializeWeights(name)[source]

Bases: object

initialize_weights(shape, random_seed=None)[source]
name
class ztlearn.initializers.LeCunNormal[source]

Bases: ztlearn.initializers.WeightInitializer

LeCun Normal (LeCunNormal)

Weights should be randomly chosen but in such a way that the sigmoid is primarily activated in its linear region. LeCun uniform is an implementation based on Gaussian distribution

References

[1] Efficient Backprop
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.LeCunUniform[source]

Bases: ztlearn.initializers.WeightInitializer

LeCun Uniform (LeCunUniform)

Weights should be randomly chosen but in such a way that the sigmoid is primarily activated in its linear region. LeCun uniform is an implementation based on Uniform distribution

References

[1] Efficient Backprop
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.One[source]

Bases: ztlearn.initializers.WeightInitializer

One (One)

One is an implementation of weight initialization that returns all ones

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.RandomNormal[source]

Bases: ztlearn.initializers.WeightInitializer

Random Normal (RandomNormal)

Random uniform, an implementation of weight initialization based on Gaussian distribution

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.RandomUniform[source]

Bases: ztlearn.initializers.WeightInitializer

Random Uniform (RandomUniform)

Random uniform, an implementation of weight initialization based on Uniform distribution

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.WeightInitializer[source]

Bases: object

compute_fans(shape)[source]
class ztlearn.initializers.Zero[source]

Bases: ztlearn.initializers.WeightInitializer

Zero (Zero)

Zero is an implementation of weight initialization that returns all zeros

init_name
weights(shape, random_seed)[source]