Recognise handwritten digits

This example classifies handwritten digits from the MNIST test set. The network is already trained, and its weights ship with the example. You only run inference, so it’s a short path from loading weights to a prediction.

Objectives

• Load pretrained weights and convert them to bfloat16.
• Chain linear, bias, ReLU and softmax operations into a complete network.
• Measure the accuracy of the network over many images.

Before you begin

1. Install the toolkit. The CPU emulator works for this example, so you don’t need hardware.

2. Check that Python can import the toolkit:

   python -c "import qant_native_computing_toolkit as q; print(q.generic.get_available_npus())"

   On the emulator this prints {0: 'cpu_backend'}.

3. Get the examples repository:

   git clone --recurse-submodules https://github.com/Q-ANT-GmbH/qant_native_computing_toolkit_examples.git
   cd qant_native_computing_toolkit_examples/python/digit_recognition

4. Install the example’s dependencies:

   pip install -r requirements.txt

   The file pins torch==2.11.0 and torchvision==0.26.0 from the PyTorch CPU index. We ran it with PyTorch 2.14.1.

5. Make sure the machine can reach the internet. On first use, torchvision downloads the MNIST test set to /tmp/mnist_files/.

Run the example

1. Start Jupyter and open digit_recognition_on_npu.ipynb:

   jupyter notebook digit_recognition_on_npu.ipynb

2. Run all cells. The notebook shows a random digit and prints the prediction with its certainty.

To run it without opening a browser, execute the notebook from the command line:

jupyter nbconvert --to notebook --execute digit_recognition_on_npu.ipynb --output executed.ipynb

How it works

The network

The network has two fully connected layers. The weights are stored in network_weights.npz.

Layer	Weight shape	Operations
Input	784 values, one per pixel of a 28 by 28 image	none
Hidden	(133, 784)	linear, bias, ReLU
Output	(10, 133)	linear, bias, softmax

The weight shapes follow the (n_out, n_in) layout that linear_fprop expects. See Key concepts.

Load the data

The helper file utils.py does the preparation. load_NN_weights reads the four arrays and converts each one to bfloat16:

weight_dict = np.load(Path(__file__).resolve().parent / "network_weights.npz")
weights_layer_1 = weight_dict["l1_w"].astype(bfloat16)
bias_layer_1 = weight_dict["l1_b"].astype(bfloat16)

load_random_image picks a random MNIST test image, scales its pixel values to the range 0 to 1, and converts it to bfloat16.

Run inference

The whole network is six toolkit calls:

fm = q_ai.linear_fprop(img.flatten(), weights_layer_1)
fm = q_ai.add_bias_fprop(fm, bias_layer_1)
fm = q_ai.relu_fprop(fm)

fm = q_ai.linear_fprop(fm, weights_layer_2)
fm = q_ai.add_bias_fprop(fm, bias_layer_2)
res = q_ai.softmax_fprop(fm).squeeze()

The image is flattened to a 1D array of 784 values. Because the input is a single vector, softmax_fprop receives one row, which is the only shape it accepts.

The prediction is the index of the largest output, and the certainty is that output as a percentage:

prediction = np.argmax(res)
certainty = np.max(res) * 100

Expected output

These are the values we got on the CPU emulator. The notebook picks a random image, so your digit can differ.

Measure	Value
Predicted digit	9
Certainty	77.0%

We also measured accuracy over 1,000 random test images with a separate script that makes the same six toolkit calls. It compares the toolkit with a float32 NumPy version of the same network on the same images.

Implementation	Accuracy over 1,000 images
Toolkit, CPU emulator, bfloat16	0.953
NumPy, float32	0.952

The bfloat16 network on the emulator classifies as well as the float32 reference here. The softmax output sums to 0.9999 because of bfloat16 rounding.

Run the tests

The example includes two tests:

python -m pytest -v test_digit_recog.py

• test_digit_recog_accuracy classifies 1,000 random images and asserts that accuracy is above 0.85.
• test_notebook runs the notebook and checks that its prediction matches the label.

On the emulator both tests passed in 18.9 seconds.

Troubleshooting

• If the MNIST download fails with an HTTP 404 error, retry. See Troubleshooting.
• If importing torchvision fails with operator torchvision::nms does not exist, your torch and torchvision versions don’t match. See Troubleshooting.

What’s next

• Train your own network on the host and run it through the toolkit in the Fourier network example.
• Build a much larger network from the same operations in Image recognition.