15 Neural networks and deep learning
Chapter status: Structural placeholder.
This chapter currently contains the approved heading structure but no reviewed technical content.
15.1 Feed-forward networks
15.1.1 Neurons, layers and activation functions
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.1.2 ReLU, sigmoid, tanh, GELU
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.1.3 Softmax outputs and cross-entropy
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.1.4 Universal approximation and the role of depth
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.2 Backpropagation and automatic differentiation
15.2.1 The chain rule on the computational graph
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.2.2 Forward and reverse-mode differentiation
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.2.3 Practical implementation in modern frameworks
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3 Optimisation of deep models
15.3.1 Stochastic gradient descent and mini-batching
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3.2 Momentum, Nesterov and Adam
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3.3 Learning-rate schedules and warm-up
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3.4 Weight decay, dropout, batch and layer normalisation
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3.5 Data augmentation for images
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.3.6 Early stopping and validation monitoring
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.4 Convolutional neural networks
15.4.1 The convolutional layer and parameter sharing
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.4.2 Pooling, stride and receptive field
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.4.3 Padding conventions and output-size arithmetic
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.4.4 Feature hierarchies and interpretability
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.5 Canonical architectures
15.5.1 LeNet-5, AlexNet, VGG
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.5.2 GoogLeNet / Inception
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.5.3 ResNet and residual connections
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.5.4 DenseNet, MobileNet, EfficientNet
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.5.5 Vision transformers and hybrid models
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.6 Training strategy for industrial data
15.6.1 Transfer learning and fine-tuning
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.6.2 Small-data regimes and few-shot learning
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.6.3 Class-imbalance countermeasures
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.
15.6.4 Reproducibility: seeds, versioning, provenance
Content status: Placeholder.
This subsection has not yet been drafted or technically reviewed.