Beyond the Transistor
Section 3 · Lab

The lab

Three findings from 106 training runs on a Fourier KAN, all rerun from scratch in October 2026 with the code this site exports. Every number on this page is computed from the run log when the site is built.

What ran, and where

Every run fits the same target, the chirp g(x) = sin(2πx) + cos(πx²) + cos(πx³)·sin(πx³) on [-2, 2]. A Fourier KAN is trained on a CPU in float32 with AdamW, fresh batches of 128 samples per step, and Gaussian training noise. Each trained model is then scored on 500 evenly spaced points, three ways:

PathWhat it is
CPU float32The trained weights in full precision.
bfloat16, emulatedInputs, weights and outputs rounded to bfloat16 in PyTorch. Rounding only.
Toolkit CPU emulator, bfloat16The same weights through the Q.ANT toolkit, version 2.3.1.

No result on this page comes from photonic hardware. The toolkit reports what it runs on, and every run recorded its answer: driver cpu-backend;commit:none;, which is the toolkit’s CPU emulator. The emulator computes an exact cosine in bfloat16. A real card computes the optical tcos shape and adds analog effects, so hardware numbers will differ. When someone runs the bundle on a card, the result will say so.

106 runs, 33 minutes of training, on a Linux x86_64 machine with Python 3.13.16 and PyTorch 2.14.1+cpu. A model that predicts zero everywhere scores 1.1556.

1. The precision floor

The best long run: [1, 16, 1], grid 16, 1,041 parameters, learning rate 0.002, 60,000 steps with a cosine schedule. The same weights, three ways:

PathMSE
CPU float320.0000200
bfloat16, emulated0.000962
Toolkit CPU emulator, bfloat160.00128
Toolkit vs float3264x

Same weights, three number paths

measured · log scale
0.00001 0.0001 0.001 [1,16,1] g16 1041 params · 60k steps · seed 0 64x [1,12,1] g12 589 params · 30k steps · seed 1 16x [1,8,8,1] g8 1297 params · 60k steps · seed 0 25x [1,12,1] g12 589 params · 60k steps · seed 0 54x
CPU float32bfloat16 rounding, emulated in PyTorchToolkit CPU emulator, bfloat16

Across the 4 long runs, float32 scores range from 0.0000200 to 0.0000891. On the toolkit path they all land between 0.00128 and 0.00195. Better float32 training barely moves the bfloat16 result, because the limit is the number format rather than the model.

Rounding alone, emulated in PyTorch, accounts for about 75% of the gap on the best run. The rest comes from how the toolkit computes in bfloat16 internally.

If you deploy to bfloat16 hardware: stop training once float32 loss is well under the format’s floor. Past that point you pay for accuracy the device cannot represent.

2. Learning rate beats architecture

The largest network in the sweep, [1, 16, 16, 16, 1] with grid 16 on every layer, 17,457 parameters, 8,000 steps, no schedule. Identical apart from one setting.

Learning rateMSE, float32MSE, toolkit emulator
0.0051.16491.1777
0.0010.003320.00536
Swing220x

[1, 16, 16, 16, 1], grid 16, 8,000 steps

measured · toolkit emulator
-2-1012 -202
target lr 0.005, MSE 1.1777 lr 0.001, MSE 0.00536 predicting zero everywhere: 1.1556

At 0.005 the network scored 1.1777, worse than predicting zero everywhere (1.1556). It did not train at all. Dropping the rate to 0.001 fixed it. The big first grid network also failed at 0.005.

3. Past a low threshold, size stops mattering

64 runs, all at 8,000 steps, sweeping width, depth, grid and two learning rates.

Parameters against error, 64 runs

measured · toolkit emulator · log scales
2005001,0002,0005,000 0.0020.0030.0050.01 411 params, [1,10,1] grid 10, lr 0.003: MSE 0.00369 411 params, [1,10,1] grid 10, lr 0.005: MSE 0.00415 491 params, [1,10,1] grid 12, lr 0.003: MSE 0.00300 491 params, [1,10,1] grid 12, lr 0.005: MSE 0.00301 651 params, [1,10,1] grid 16, lr 0.003: MSE 0.00354 651 params, [1,10,1] grid 16, lr 0.005: MSE 0.00386 331 params, [1,10,1] grid 8, lr 0.003: MSE 0.00353 331 params, [1,10,1] grid 8, lr 0.005: MSE 0.00358 2421 params, [1,10,10,1] grid 10, lr 0.003: MSE 0.00316 2421 params, [1,10,10,1] grid 10, lr 0.005: MSE 0.00383 2901 params, [1,10,10,1] grid 12, lr 0.003: MSE 0.00342 2901 params, [1,10,10,1] grid 12, lr 0.005: MSE 0.00392 3861 params, [1,10,10,1] grid 16, lr 0.003: MSE 0.00561 3861 params, [1,10,10,1] grid 16, lr 0.005: MSE 0.0117 1941 params, [1,10,10,1] grid 8, lr 0.003: MSE 0.00365 1941 params, [1,10,10,1] grid 8, lr 0.005: MSE 0.00318 493 params, [1,12,1] grid 10, lr 0.003: MSE 0.00300 493 params, [1,12,1] grid 10, lr 0.005: MSE 0.00390 589 params, [1,12,1] grid 12, lr 0.003: MSE 0.00263 589 params, [1,12,1] grid 12, lr 0.005: MSE 0.00382 781 params, [1,12,1] grid 16, lr 0.003: MSE 0.00279 781 params, [1,12,1] grid 16, lr 0.005: MSE 0.00366 397 params, [1,12,1] grid 8, lr 0.003: MSE 0.00292 397 params, [1,12,1] grid 8, lr 0.005: MSE 0.00293 3385 params, [1,12,12,1] grid 10, lr 0.003: MSE 0.00440 3385 params, [1,12,12,1] grid 10, lr 0.005: MSE 0.00458 4057 params, [1,12,12,1] grid 12, lr 0.003: MSE 0.00301 4057 params, [1,12,12,1] grid 12, lr 0.005: MSE 0.00551 5401 params, [1,12,12,1] grid 16, lr 0.003: MSE 0.00523 5401 params, [1,12,12,1] grid 16, lr 0.005: MSE 0.00794 2713 params, [1,12,12,1] grid 8, lr 0.003: MSE 0.00292 2713 params, [1,12,12,1] grid 8, lr 0.005: MSE 0.00289 247 params, [1,6,1] grid 10, lr 0.003: MSE 0.00576 247 params, [1,6,1] grid 10, lr 0.005: MSE 0.00674 295 params, [1,6,1] grid 12, lr 0.003: MSE 0.00366 295 params, [1,6,1] grid 12, lr 0.005: MSE 0.00446 391 params, [1,6,1] grid 16, lr 0.003: MSE 0.00415 391 params, [1,6,1] grid 16, lr 0.005: MSE 0.00462 199 params, [1,6,1] grid 8, lr 0.003: MSE 0.00466 199 params, [1,6,1] grid 8, lr 0.005: MSE 0.00462 973 params, [1,6,6,1] grid 10, lr 0.003: MSE 0.00369 973 params, [1,6,6,1] grid 10, lr 0.005: MSE 0.00377 1165 params, [1,6,6,1] grid 12, lr 0.003: MSE 0.00456 1165 params, [1,6,6,1] grid 12, lr 0.005: MSE 0.00593 1549 params, [1,6,6,1] grid 16, lr 0.003: MSE 0.00406 1549 params, [1,6,6,1] grid 16, lr 0.005: MSE 0.00460 781 params, [1,6,6,1] grid 8, lr 0.003: MSE 0.00346 781 params, [1,6,6,1] grid 8, lr 0.005: MSE 0.00344 329 params, [1,8,1] grid 10, lr 0.003: MSE 0.00330 329 params, [1,8,1] grid 10, lr 0.005: MSE 0.00367 393 params, [1,8,1] grid 12, lr 0.003: MSE 0.00273 393 params, [1,8,1] grid 12, lr 0.005: MSE 0.00367 521 params, [1,8,1] grid 16, lr 0.003: MSE 0.00396 521 params, [1,8,1] grid 16, lr 0.005: MSE 0.00280 265 params, [1,8,1] grid 8, lr 0.003: MSE 0.00421 265 params, [1,8,1] grid 8, lr 0.005: MSE 0.00364 1617 params, [1,8,8,1] grid 10, lr 0.003: MSE 0.00609 1617 params, [1,8,8,1] grid 10, lr 0.005: MSE 0.00465 1937 params, [1,8,8,1] grid 12, lr 0.003: MSE 0.00341 1937 params, [1,8,8,1] grid 12, lr 0.005: MSE 0.00351 2577 params, [1,8,8,1] grid 16, lr 0.003: MSE 0.00376 2577 params, [1,8,8,1] grid 16, lr 0.005: MSE 0.00731 1297 params, [1,8,8,1] grid 8, lr 0.003: MSE 0.00281 1297 params, [1,8,8,1] grid 8, lr 0.005: MSE 0.00312 393 → 0.00273 4,057 → 0.00301
lr 0.003 lr 0.005 within 25% of the best run (16 of 64)

Hover a point for its configuration. Every run trained for 8,000 steps.

A 393 parameter model scored 0.00273. A 4,057 parameter model, 10 times larger, scored 0.00301.

The runs do not all cluster. 16 of the 64 runs landed within 25% of the best, and the full range ran from 0.00263 to 0.0117. The claim is about the paired configurations above and the early ceiling, not about every run.

Repeat it yourself

Everything needed to reproduce this page is public.

FileWhat it is
runs-2026-10.jsonlEvery run in full: config, loss curve, predictions, each path’s MSE and the toolkit’s own device report
rerun.pyThe script that ran all 106 configurations
The playground bundleThe exact model, training and backend code the runs used

The toolkit is open source under Apache 2.0, and its CPU emulator builds from the public repository. The bundle README has the commands.

Ran it and got different numbers, or ran it on real hardware? Post them in the forum, with the backend each number came from.