Test every operation against PyTorch

The introduction notebook is the best first example. It tours the API one operation at a time. For each operation it creates random bfloat16 data, computes the result with the toolkit and with PyTorch, and prints how far apart the two are.

Objectives

• Call the operations in the native and ai modules with the correct shapes and arguments.
• Compare toolkit results with PyTorch reference results.
• Learn which differences come from bfloat16 rounding.

Before you begin

1. Install the toolkit. The CPU emulator works for this example, so you don’t need hardware.

2. Check that Python can import the toolkit:

   python -c "import qant_native_computing_toolkit as q; print(q.generic.get_available_npus())"

   On the emulator this prints {0: 'cpu_backend'}.

3. Get the examples repository:

   git clone --recurse-submodules https://github.com/Q-ANT-GmbH/qant_native_computing_toolkit_examples.git
   cd qant_native_computing_toolkit_examples/python/introduction

4. Install the example’s dependencies:

   pip install -r requirements.txt

   The file pins torch==2.11.0 and torchvision==0.26.0 from the PyTorch CPU index. We ran it with PyTorch 2.14.1.

Run the example

1. Start Jupyter and open intro.ipynb:

   jupyter notebook intro.ipynb

2. Run all cells.

To run it without opening a browser, execute the notebook from the command line:

jupyter nbconvert --to notebook --execute intro.ipynb --output executed.ipynb

How it works

The comparison helper

A helper called print_errors prints three numbers for each operation:

• Relative error. The mean of the absolute error divided by the reference value.
• Absolute error. The mean absolute difference.
• Max absolute error. The largest single difference.

The notebook stores every error list in error_dict_for_test, which the test reads later.

Native operations

The first section calls the two native math operations. This is the element-wise multiplication:

x = np.random.rand(100, 10).astype(bfloat16)
y = np.random.rand(100, 10).astype(bfloat16)

# calculate the reference elementwise product
e = x * y

# compute the elementwise product using the qant SDK
r = q_native.mul_elementwise(x, y)

The second operation, calc_scaled_periodic_nl_fprop, computes the cosine-like function multiplied by a weight. The notebook then plots it next to a true cosine in a chart titled “Approximated Cosine vs Original”.

Layer operations

The rest of the notebook calls the ai module. The linear layer follows the PyTorch weight layout, so the reference is torch.nn.functional.linear:

# batch size 128, input features 32, output features 64
weight = np.random.rand(64, 32).astype(bfloat16)
input = np.random.rand(128, 32).astype(bfloat16)

r = q_ai.linear_fprop(input, weight)

Convolutions take stride, padding and dilation as arguments:

r = q_ai.conv_fprop(input, weight, stride=1, padding=1, dilation=1)

For the transposed convolution, the weight shape is (in_channels, out_channels, kernel_size, kernel_size), as a comment in the notebook notes.

The KAN layer section compares the toolkit against a NumPy sum of exact cosines:

r = q_ai.calc_kan_layer_fprop(input, phis, amplitudes, ks)
e = np.sum(
    amplitudes * np.cos(input[None, :, None] * ks[None, None, :] + phis), axis=(1, 2)
)

Expected output

These are the values the notebook printed on the CPU emulator. The notebook fixes its random seeds, so your numbers should match closely on the emulator.

Operation	Relative error	Absolute error	Max absolute error
mul_elementwise	0	0	0
calc_scaled_periodic_nl_fprop	0	0	0
linear_fprop	0.000244141	0.00195312	0.0625
conv_fprop	0.00012207	0.000488281	0.03125
conv_transpose_fprop	0.00012207	0.000976562	0.0625
add_bias_fprop	0	0	0
relu_fprop	0	0	0
sigmoid_fprop	0.000488281	0.000244141	0.00390625
softmax_fprop	0.125	0.0019989	0.0332031
calc_kan_layer_fprop	0	0	0
maxpool2d_fprop	0	0	0
avgpool2d_fprop	0	0	0
adaptive_maxpool2d_fprop	0	0	0
adaptive_avgpool2d_fprop	0	0	0
batchnorm2d_fprop	4.07696e-05	0.000163078	0.03125

The last cell printed Total operations tested: 15. On the emulator, the “Approximated Cosine vs Original” plot shows two identical curves.

The non-zero errors in linear, convolution and batch normalisation come from bfloat16 rounding of sums. They are not hardware effects.

Run the tests

The example includes a test that executes the notebook and checks each operation’s mean error against a threshold:

python -m pytest -q test_nb.py

The default threshold is 0.005. Transposed convolution, the scaled periodic function, softmax and batch normalisation are allowed 0.02, and the KAN layer 0.2. On the emulator the test passed in 6.6 seconds.

What’s next

• Put the linear, bias, ReLU and softmax operations together in Digit recognition.
• Read about bfloat16 and the cosine-like function in Key concepts.
• Try the KAN layer arithmetic in your browser in the live example.