Lecture 04 - Stochastic gradient descent using Flux.jl

Lecture04

Stochastic gradient descent using Flux.jl.

Available Functions

• demo(): Stochastic gradient descent demo for MNIST handwritten digit recognition

Usage

using MachineLearningCourse
Lecture04.demo()

using Plots
using MachineLearningCourse
_, losses = Lecture04.demo(validation_size=1000, epochs=100, patience=100)
plot = Lecture04.plot_losses(losses)
plot!(plot, size=(1200, 800))
savefig(plot, "training_progress.png")

DNN(layers)

Flux Deep Neural Network structure with fully connected layers.

Arguments

• layers::Vector{Int}: Number of neurons per layer [input, hidden..., output]
• ϕ: Activation function for hidden layers

Fields

• layers::Vector{Int}: Layer architecture specification
• model::Flux.Chain: The underlying Flux model

Uses ReLU activation for hidden layers, softmax activation for output layer.

Example

# Create network: 784 inputs → 128 hidden → 64 hidden → 10 outputs
network = DNN([784, 128, 64, 10])

accuracy(network::DNN, X_test, Y_test)

Calculate accuracy of DNN model on test data.

Arguments

• network::DNN: Trained DNN network
• X_test::Matrix{Float32}: Test input data (features × samples)
• Y_test::Matrix{Float32}: Test target labels (one-hot encoded, classes × samples)

Returns

• Float32: Test accuracy (0.0 to 1.0)

demo(seed=42, hidden_layers=[128, 64], train_size=5000, test_size=1000, η=0.001, epochs=50, verbose=true, validation_size=0, patience=10)

MNIST handwritten digit recognition demonstration using Flux.jl.

Parameters

• seed: Random seed (default: 42)
• hidden_layers: Dimensions of hidden layers (default: [128, 64])
• train_size: Number of training samples to use (default: 5000)
• test_size: Number of test samples to use (default: 1000)
• η: Learning rate for Adam optimizer (default: 0.001)
• epochs: Number of training epochs (default: 50)
• batch_size: Mini-batch size for SGD training (default: 128)
• verbose: Print training progress (default: true)
• validation_size: Number of validation samples for early stopping (default: 0 - no validation)
• patience: Number of epochs to wait before early stopping (default: 10)

evaluate(network::DNN, X_test, Y_test, classes)

Comprehensive evaluation of DNN model with confusion matrix and per-class metrics.

Arguments

• network::DNN: Trained DNN network
• X_test::Matrix{Float32}: Test input data (features × samples)
• Y_test::Matrix{Float32}: Test target labels (one-hot encoded, classes × samples)
• classes: Vector of class labels or number of classes

Returns

• NamedTuple: (accuracy=Float32, predictions=Vector, truelabels=Vector, confusionmatrix=Matrix{Int})

plot_losses(losses; title="Training Progress")

Create a plot of training losses. Handles both single training losses and training+validation losses.

Arguments

• losses: Either Vector{Float32} (training only) or NamedTuple with trainlosses and vallosses
• title: Plot title (default: "Training Progress")

Returns

• Plot object showing training and validation loss curves

predict(network::DNN, X)

Make predictions using trained DNN network.

Arguments

• network::DNN: Trained DNN network
• X::Matrix{Float32}: Input data (features × samples)

Returns

• Predictions from the network (output × samples)

train!(network::DNN, X_train, Y_train, η, epochs; batch_size=128, verbose=true)

Train a DNN model using supervised learning.

Arguments

• network::DNN: DNN network to train
• X_train::Matrix{Float32}: Training input data (features × samples)
• Y_train::Matrix{Float32}: Training target labels (one-hot encoded)
• η::Float32: Adam optimizer learning rate
• epochs::Int: Number of training epochs
• batch_size::Int: Mini-batch size (default: 128)
• verbose::Bool: Print training progress (default: true)

Returns

• Vector{Float32}: Training losses per epoch

train!(network::DNN, X_train, Y_train, X_validate, Y_validate, η, epochs; batch_size=128, verbose=true, early_stopping_patience=10)

Train a DNN model using supervised learning with validation data and early stopping.

Arguments

• network::DNN: DNN network to train
• X_train::Matrix{Float32}: Training input data (features × samples)
• Y_train::Matrix{Float32}: Training target labels (one-hot encoded)
• X_validate::Matrix{Float32}: Validation input data (features × samples)
• Y_validate::Matrix{Float32}: Validation target labels (one-hot encoded)
• η::Float32: Adam optimizer learning rate
• epochs::Int: Number of training epochs
• batch_size::Int: Mini-batch size (default: 128)
• verbose::Bool: Print training progress (default: true)
• early_stopping_patience::Int: Number of epochs to wait before early stopping (default: 10)

Returns

• NamedTuple: (trainlosses=Vector{Float32}, vallosses=Vector{Float32})