Practical guide

Train and run small neural networks

The neural examples implement complete forward passes and parameter gradients with ordinary shdr buffers and compute shaders. Native C++/GLSL pipelines run training and inference on the CPU or GPU. These fixed architectures demonstrate how compute stages and resources form a model. A deeper LeNet-style classifier deploys prepared weights through native inference, with optional offline training in PyTorch.

# Run this example on the CPU.
./build/example_neural
# Run this example on the CPU.
./build/example_cnn
# Run this example on the CPU.
./build/example_transformer
# Run this example on the requested headless GPU profile.
./build/example_neural --gpu es31
# Run this example on the requested headless GPU profile.
./build/example_cnn --gpu es31
# Run this example on the requested headless GPU profile.
./build/example_transformer --gpu es31

Each executable initializes repeatable inputs and parameters, checks gradients, trains the model and reports loss and classification accuracy.

Choose a model

Every parameter participates in gradient computation, including hidden layers, convolution coefficients, attention matrices, positional features and classifier heads. The reported loss and classification counts describe the seeded training tasks. The public-data evaluator also fits real MNIST zero/one mini-batches with these models. The deeper classifier has a separate held-out ten-class digit evaluation.

Train an ANN or CNN

neural_support.hpp provides shdr_examples::binary_network. Its first constructor argument chooses the CNN. Its second is the batch size. The constructor supplies seeded data and initial parameters, so this is a runnable CPU training loop:

#include "examples/neural_support.hpp" // Use the small fully trainable binary-network wrapper.
#include <iostream> // Print the evaluated mean loss.
int main() { // Train and evaluate the same model through compute pipelines.
 shdr_examples::binary_network model(false, 32); // Choose the two-feature ANN with a batch of 32 samples.
 for (unsigned step = 0; step < 160; ++step) // Perform the demonstrated number of training steps.
 model.training_pipeline().run(); // Compute analytic gradients and update every parameter on the CPU.
 model.evaluation_pipeline().run(); // Run inference and calculate mean binary cross entropy.
 const shdr::buffer<float>& loss = model.loss; // Read loss through a const host view.
 std::cout << "Mean loss: " << loss.data()[0] << '\n'; // Print the completed mean loss.
} // Release the model resources after evaluation.

Attach its shader sources to the executable with the build helper:

shdr_target_shaders(app PRIVATE SOURCES # Compile and embed each ANN training and evaluation stage.
    examples/shaders/ann_infer.cpp # Compute the network's forward prediction.
    examples/shaders/ann_gradient.cpp # Compute all analytic per-sample parameter gradients.
    examples/shaders/nn_sgd.cpp # Average batch gradients and apply SGD updates.
    examples/shaders/nn_binary_loss.cpp) # Reduce prediction errors into a mean loss.

For a CNN, construct binary_network(true, 32), substitute cnn_infer.cpp and cnn_gradient.cpp, and keep the shared SGD and loss modules. The bundled CNN example trains for 180 updates.

For headless GPU execution, link shdr::egl and use a device:

shdr::headless_device gpu; // Create a default ES 3.1 headless device.
model.training_pipeline().prepare(gpu); // Prepare programs and resources before the training loop.
for (unsigned step = 0; step < 160; ++step) // Perform the same number of GPU training steps.
    model.training_pipeline().run(gpu); // Compute gradients and update parameters on the GPU.
model.evaluation_pipeline().run(gpu); // Evaluate predictions and loss on the GPU.

The training pipeline computes per-sample gradients, then updates parameters using their batch mean. Prediction is a separate inference stage, and evaluation adds mean binary cross-entropy. gradient_pipeline() writes derivatives and preserves parameter values. reset_parameters() restores the deterministic initial weights.

Supply custom data by writing the allocated input and target buffers. Use parameter_count() and batch_count() for dimensions. The default SGD learning rates are 0.4 for the ANN and 1.5 for the CNN. These settings belong to the small seeded fixtures. Tune training settings when building another model or dataset.

Train the transformer

transformer_support.hpp provides a four-token classifier with a configurable batch size and learning rate:

#include "examples/transformer_support.hpp" // Use the four-token transformer demonstration wrapper.

shdr_examples::transformer_model model(8, 0.15f); // Allocate eight samples and use a learning rate of 0.15.
for (unsigned step = 0; step < 32; ++step) // Perform the demonstrated transformer training steps.
    model.training_pipeline().run(); // Compute analytic gradients and update all model parameters.
model.prediction_pipeline().run(); // Run a separate forward prediction dispatch.
const float mean_loss = model.loss(); // Calculate the mean loss from completed predictions.
const unsigned correct = model.correct(); // Count samples whose predicted class matches their label.

Attach transformer_infer.cpp, transformer_gradient.cpp and nn_sgd.cpp to the executable. Both model pipelines accept run(gpu) for GPU execution. reset_inputs(seed) recreates the balanced sequence fixture, while reset_parameters(seed) restores initial weights. Replace inputs with sample-major, token-major floating features and labels with class IDs zero or one to supply a compatible batch.

The forward pass adds learned positional features, computes one scaled Q/K/V attention head, projects its output and adds a residual. RMS normalization has learned scales. A width-four ReLU feed-forward block adds another residual. Mean token pooling feeds the two-class classifier. Softmax and cross-entropy use shifted logits for numerical stability.

inputs holds sixteen floats per sequence. parameters holds 134 floats, and gradients holds batch_count * 134 floats. Each predictions element is vec4(probability_class_0, probability_class_1, cross_entropy, predicted_class). The predicted class is zero or one represented as a float. loss() and correct() read the completed prediction buffer and can download GPU results.

The transformer gradient uses analytic forward-mode differentiation: vec2 holds a value and one parameter derivative. Each (sample, parameter) invocation evaluates a forward pass, and the shared SGD stage reduces the batch. This independent-invocation design trains every parameter and repeats forward work for each derivative. Large models benefit from the efficiency of reverse-mode differentiation.

Understand the update pipeline

Training kernels write gradients[sample * parameter_count + parameter], so each invocation owns a distinct output. The nn_sgd_kernel gives each parameter its own invocation and applies:

parameter -= learning_rate * (mean_sample_gradient + weight_decay * parameter)

Its named ports are gradients and parameters. Its uniforms are batch_count, parameter_count, learning_rate and weight_decay. The examples use zero weight decay. The gradient stage finishes before the update dispatch reads its output. Parameter writes then become visible to subsequent inference or training stages. At most four storage-buffer ports are used per model stage, within the guaranteed ES 3.1 compute baseline.

Inference reads trained parameters and writes predictions. Attach the inference module, create its kernel with make_kernel, bind input, parameter and output buffers, and set batch_count. The transformer also binds labels for its loss diagnostic. Predicted probabilities depend on inputs and parameters. Applications use the embedded shader programs.

Run a LeNet-style digit classifier

shdr_examples::lenet_classifier in lenet_support.hpp runs a modern LeNet-5-style network with 61,706 parameters. It uses full second-layer connectivity, ReLU, fixed average pooling and linear final logits. These choices differ from the original 1998 LeNet-5 architecture.

The seven stages reuse three native modules: lenet_conv.cpp, lenet_pool.cpp and lenet_dense.cpp. Attach them to the executable with shdr_target_shaders. Input and intermediate buffers stay on the selected backend until their results are needed on the host.

#include "examples/lenet_support.hpp" // Use the seven-stage native classifier.
#include "examples/real_data_support.hpp" // Read the prepared plain binary fixtures.
auto weights = shdr_examples::real_data::floats("lenet_weights.bin"); // Load the prepared flat float32 parameters.
auto images = shdr_examples::real_data::bytes("mnist_test.bin"); // Read held-out digit image bytes.
shdr_examples::lenet_classifier model(1); // Allocate activations for one digit image.
model.load_parameters(weights); // Check the parameter count and finite values.
auto* input = model.inputs.data(); // Access the host input before dispatching inference.
for (unsigned i = 0; i < 784; ++i) { // Copy the first 28-by-28 held-out image.
 input[i] = images[i] / 255.f; // Match the zero-to-one training input scale.
} // The convolution stage supplies the zero border internally.
model.inference_pipeline().run(); // Or run(gpu) for the same complete GPU network.
const shdr::buffer<float>& scores = model.logits; // Read scores through a const host view.
const float* logits = scores.data(); // Obtain ten completed unnormalized class scores.

Choose the class with the largest logit. The helper accepts batches from one to 512 images. inputs contains batch_count * 784 floats and logits contains batch_count * 10. Parameters use convolution order [out,in,ky,kx] and dense order [out,in], with each layer's biases immediately after its weights.

The included model was prepared with three complete epochs of the 60,000-image MNIST training split, Adam at learning rate 0.001, batch size 128 and seed 12345. Independent PyTorch evaluation of all 10,000 held-out images recorded 97.78% accuracy. These are the model-preparation results in training provenance. The real-data executable separately checks native CPU/GPU logits against the framework scores and reports accuracy against the original test labels. Full native runs reproduced 97.78% on every recorded CPU/GPU path. The data results contain the measured accuracy, numerical errors and evaluation times.

Preparation is optional. To retrain and export plain weights, install PyTorch and NumPy in a preparation environment, then use the checked source cache:

# Train on the official training split and export weights and independent test logits.
python3 scripts/train_lenet.py --cache build/datasets/upstream

The deployed C++ program reads lenet_weights.bin and runs the seven-stage classifier through the single-header runtime. The three smaller models demonstrate native analytic training. See the data guide for preprocessing, source terms, preparation packages and the full held-out evaluation.

Validation and limits

The ANN and CNN compare every analytic parameter gradient with central differences from an independent double-precision prediction oracle. GPU runs compare gradients, trained weights and probabilities with CPU results. The transformer checks predictions and all 134 analytic derivatives against its independent double-precision oracle. GPU runs compare parameters, gradients and predictions with a separately trained CPU model, and check parameter-family changes, loss reduction and classification. Finite differences validate the implementations. Training uses the analytic gradients.

The ANN uses eight fixed ReLU neurons. The CNN uses stride-one valid cross-correlation and two 6×6 activation maps. The transformer uses one attention head over all four fixed tokens. These examples train fixed float32 architectures with SGD. Model import, serialization, mixed precision, large tensor operations, distributed training and adaptive optimizers require additional application-specific implementations. Compare CPU and GPU float results with tolerances suitable for the algorithm, particularly near ReLU boundaries.

Keep inputs and parameters stable during a dispatch. Serialize training jobs that share parameter storage. Concurrent inference can share immutable parameters and inputs while using distinct writable outputs. Read GPU losses and predictions at deliberate synchronization points. See asynchronous execution for background work and finite resource rings, and portability for source and layout rules.