Goal:

Learn to implement/understand the Multi-Layer Perceptron (MLP) model by using TensorFlow Keras

Reading Task:

Go through the Jupyter notebooks from my GitHub repository "MLP Classifier - TensorFlow Keras No Brainer"

Practice Task
You will perform multi-class classification using the MLP model.

You will create the following MLP models based on the given specification. The MLPs will be trained using the Backpropagation algorithm that is implemented using the mini-batch Stochastic Gradient Descent (SGD) optimization algorithm.

Mini-batch Size & Training:
The size of mini-batches should be set to 64. A network should be trained for 50 epochs with early stopping on.

Experiments:
You will perform multi-class classification on the MNIST Fashion dataset using the following 6 models. After loading the dataset, create a separate validation subset with 5000 samples. Then, scale the data.

Create the models based on the architecture and layer configurations given below.

Architecture:

2 hidden layers
Hidden layer 1: neurons=300
Hidden layer 2: neurons=100
SGD Learning rate: 0.1

Experiment 1 :
kernel_initializer (all layers) = zeros
activation (hidden layers) = sigmoid
Dropout = No

Experiment 2:
kernel_initializer (all layers) = ones
activation (hidden layers) = sigmoid
Dropout = No

Experiment 3:
kernel_initializer (all layers) = random_normal
activation (hidden layers) = sigmoid
Dropout = No

Experiment 4:
kernel_initializer (all layers) = random_normal
activation (hidden layers) = tanh
Dropout = No

Experiment 5:
kernel_initializer (all layers) = random_normal
activation (hidden layers) = relu
Dropout = No

Experiment 6:
kernel_initializer (all layers) = random_normal
activation (hidden layers) = relu
Dropout = yes (Hidden layer 1: rate=0.1, Hidden layer 2: rate=0.1)

Report:

Train accuracy, test accuracy & no. of epochs at which the training stopped.
Train and test accuracies should be reported up to 3 decimal places.
Report the learning curves (accuracy & loss) for each model.

Dataset:

The MNIST Fashion dataset:

https://keras.io/api/datasets/fashion_mnist/