# Weight Initialization¶

Weight initialization is a technique that deals with presetting of the weight parameters in the network prior to training in a manner that favors the ultimate convergence of the neural network.

## Function Descriptions¶

class ztlearn.initializers.GlorotNormal[source]

Glorot Normal (GlorotNormal)

GlorotNormal, more famously known as the Xavier initialization is based on the effort to try mantain the same variance of the gradients of the weights for all the layers. Glorot normal is an implementation based on Gaussian distribution

References

[1] Understanding the difficulty of training deep feedforward neural networks
[2] Initialization Of Deep Feedfoward Networks
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.GlorotUniform[source]

Glorot Uniform (GlorotUniform)

GlorotUniform, more famously known as the Xavier initialization is based on the effort to try mantain the same variance of the gradients of the weights for all the layers. Glorot uniform is an implementation based on Uniform distribution

References

[1] Understanding the difficulty of training deep feedforward neural networks
[2] Initialization Of Deep Feedfoward Networks
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.HeNormal[source]

He Normal (HeNormal)

HeNormal is a robust initialization method that particularly considers the rectifier nonlinearities. He normal is an implementation based on Gaussian distribution

References

[1] Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
[2] Initialization Of Deep Networks Case of Rectifiers
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.HeUniform[source]

He Normal (HeNormal)

HeNormal is a robust initialization method that particularly considers the rectifier nonlinearities. He uniform is an implementation based on Uniform distribution

References

[1] Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
[2] Initialization Of Deep Networks Case of Rectifiers
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.Identity[source]

Identity (Identity)

Identity is an implementation of weight initialization that returns an identity matrix of size shape

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.InitializeWeights(name)[source]

Bases: object

initialize_weights(shape, random_seed=None)[source]
name
class ztlearn.initializers.LeCunNormal[source]

LeCun Normal (LeCunNormal)

Weights should be randomly chosen but in such a way that the sigmoid is primarily activated in its linear region. LeCun uniform is an implementation based on Gaussian distribution

References

[1] Efficient Backprop
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.LeCunUniform[source]

LeCun Uniform (LeCunUniform)

Weights should be randomly chosen but in such a way that the sigmoid is primarily activated in its linear region. LeCun uniform is an implementation based on Uniform distribution

References

[1] Efficient Backprop
init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.One[source]

One (One)

One is an implementation of weight initialization that returns all ones

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.RandomNormal[source]

Random Normal (RandomNormal)

Random uniform, an implementation of weight initialization based on Gaussian distribution

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.RandomUniform[source]

Random Uniform (RandomUniform)

Random uniform, an implementation of weight initialization based on Uniform distribution

init_name
weights(shape, random_seed)[source]
class ztlearn.initializers.WeightInitializer[source]

Bases: object

compute_fans(shape)[source]
class ztlearn.initializers.Zero[source]

Zero (Zero)

Zero is an implementation of weight initialization that returns all zeros

init_name
weights(shape, random_seed)[source]