• Rumelhart, Hinton & Williams (1986): Learning representations by back-propagating errors
• PyTorch: Autograd mechanics
• Kingma & Ba (ICLR 2015): Adam
• Rajbhandari et al. (2020): ZeRO, mixed-precision model-state memory

The tiny network runs real arithmetic with fixed initial parameters and four fixed XOR examples. Predictions are linear scores, not probabilities. This is a teaching simulation, not a measured training benchmark. The large-model memory budget is an approximation for one unsharded mixed-precision Adam setup; activations, temporary buffers and communication are excluded. The tiny network uses SGD, not Adam.