Deep Learning

The initial layers of convolutional neural networks just learn the general features like edges and very general image features, it’s the deeper part of the networks that learn the specific shapes and parts of objects which are trained in this method.

• The augmentation on the training images using the ImageDataGenerator class:

Activation Functions

• Desirable properties of an activation function

1. Nonlinear
2. Range
3. Continuously differentiable
4. Monotonic
5. Smooth
6. Approximates identity near the origin

• Common activation functions

1. Rectified Linear Unit (ReLU): f(x) = max(0, x)

The consistent problem for ReLu is that its derivative is 0 for half of the values of the input x. So using the gradient descent update algorithm, the parameter with a value less than 0 will never be updated, leading close to 40% dead neurons.

2. Swish: f(x) = x·sigmoid(x)