Rockchip’s RKNN Toolkit2 converts ONNX models into .rknn files for the RK3588 NPU. The board then runs those files through RKNN Toolkit Lite2 for Python, or the RKNN Runtime C/C++ API.
The Rockchip RK3588 platform can use that detector in a camera appliance, with the CPU handling tracking and networking. That makes local occupancy detection a useful application, especially if the device only needs to send counts rather than a continuous video stream.
But an exported file isn’t proof that the detector works. The conversion, image preparation, output decoding, and board software all need checking – and changing all four at once makes a bad result much harder to explain.
Table of Contents
Check the export
Install RKNN Toolkit2 in a separate Python environment, using the package and Python version supported by your chosen release. Keep the version with the model’s build notes. The older RKNN Toolkit is a different toolchain, so a tutorial written for it isn’t a dependable installation guide for RK3588.
And you don’t necessarily need an x86 workstation. Rockchip added ARM64 support to Toolkit2 in version 2.3.0, though you still need a supported environment and enough resources for conversion.
For a first conversion, use a fixed input shape that matches the model’s export, such as batch 1 with a 640×640 image. Dynamic inputs add another variable (and their support depends on the graph and toolkit). Get one input size working before adding more.
Run the ONNX model on a few saved images first, and keep its outputs. Use those same images later, with matching resize, padding, channel order, and normalization, so you’re comparing the conversion rather than two different input pipelines.
Check the output shapes too. Rockchip’s YOLOv5 example makes this particularly clear:
Its optimized ONNX export moves part of the original graph into post-processing. The documented example changes one output from [1,19200,85] to [1,255,80,80]; the detector’s outputs no longer have the same structure as the original export.
Because of that, a decoder copied from another YOLO tutorial might be wrong for your file. Start with the matching model-zoo example, including its export instructions and post-processing, before replacing the supplied model with your own.
Build before quantizing
Build without quantization first. That gives you a useful comparison before calibration adds another possible source of error.
This conversion skeleton assumes a single RGB image input, raw pixel values from 0 to 255, and a model expecting division by 255. It follows the configuration used in Rockchip’s YOLOv5 converter; another model may need different settings. It assumes Toolkit2 is already installed and a compatible model.onnx exists.
from rknn.api import RKNN
def check(stage, result):
if result != 0:
raise RuntimeError(f"{stage} failed: {result}")
rknn = RKNN(verbose=True)
try:
rknn.config(
target_platform="rk3588",
mean_values=[[0, 0, 0]],
std_values=[[255, 255, 255]],
)
check("load", rknn.load_onnx(model="model.onnx"))
check("build", rknn.build(do_quantization=False))
check("export", rknn.export_rknn("model-fp.rknn"))
finally:
rknn.release()The separate return-code checks tell you which stage stopped. A failure while loading the ONNX graph needs a different investigation from a model that exports successfully and then gives wrong detections.
If the log names an unsupported operation, inspect that node and the export that produced it. Check the operator documentation for your toolkit release and target. Re-exporting with the settings used by a supported example may help; changing the opset number at random isn’t a diagnosis. Some graphs will need changes to the model itself.
Normalization can also break an otherwise successful conversion. With the configuration above, don’t divide your runtime image by 255 again unless the particular input API requires it. Applying the same scaling twice changes what reaches the network.
Check RGB versus BGR, tensor layout, and letterbox padding before blaming the NPU. Rockchip’s Python example prepares ONNX and RKNN inputs differently, so copying the ONNX input array directly into the RKNN path isn’t a safe assumption. Keep preprocessing and the model together.
Calibrate and compare
Once the non-quantized build gives acceptable results, try INT8 with a representative calibration set. For a doorway camera, that means images with the lighting, viewing angle, and people sizes you expect at that doorway – not just convenient pictures downloaded for the demo.
Replace the build call with a quantized build and export it under a different filename:
check("build", rknn.build(
do_quantization=True,
dataset="calibration.txt",
))
check("export", rknn.export_rknn("model-int8.rknn"))For this image-input example, calibration.txt lists image paths, one per line. Confirm that the paths resolve from your conversion environment, and keep a separate validation set. Calibration images help build the model; they shouldn’t be your only evidence that it works.
Compare missed detections and false positives, then check the downstream counts. People counting accuracy depends on tracking and installation too, so similar-looking boxes on one image aren’t enough. If INT8 loses useful detections, revisit calibration and input preparation before lowering the confidence threshold to hide the problem.
Run on the board
On the board, load the exported model with Lite2 or the C/C++ Runtime and initialize the NPU. Record the runtime and driver versions alongside the converter version, and check their compatibility against the SDK and board vendor’s guidance. Installing Toolkit2 on the workstation doesn’t update the board’s runtime or kernel driver.
If initialization fails, run the vendor’s known-good NPU example before changing your camera code. If that example also fails, investigate the board software stack first. If it works, compare your model target, build version, and application with the working example.
After inference works, measure capture, preprocessing, inference, and post-processing separately. A faster network won’t fix a frame queue that keeps growing. Keep the ONNX reference, both RKNN builds, validation images, and version notes together – you’ll need them when the next model export changes something.
