RKNN Toolkit Lite2 on RK3588

RKNN Toolkit Lite2 runs converted .rknn models on the RK3588 NPU through Python. Rockchip supplies a ResNet18 example that loads a model, prepares an RGB image, and returns classification scores on the board.

The Rockchip RK3588 platform still uses its CPU for the Python application, image handling, and whatever happens after inference. Lite2 gives that application access to the NPU. If you haven’t built a compatible model yet, convert ONNX to RKNN first; installing Lite2 won’t make an ONNX file load directly.

Table of Contents

• Prepare the board environment
• Run a known model first
• Check the result before timing it
• Diagnose failures before adding a camera
• FAQ

Prepare the board environment

Use a board image with working NPU support, then check the Python version and architecture before downloading a Lite2 wheel. The wheel must match both. A package built for a different Python ABI isn’t interchangeable just because the board has the right processor.

Keep the installation in a virtual environment. Put only the matching wheel from your chosen Rockchip release in the working directory, then install it there. This keeps an experiment from changing packages used by another application (and makes the setup easier to reproduce).

python3 --version
uname -m
python3 -m venv .venv
. .venv/bin/activate
python -m pip install ./rknn_toolkit_lite2-*.whl
python -c "from rknnlite.api import RKNNLite; print('Lite2 import OK')"

Install the NumPy and OpenCV versions supported by your selected SDK release in the same environment before running the image example. That last command checks the Python import. It doesn’t test the NPU. Lite2, the board’s runtime library, and its kernel driver are separate parts of the software stack; use the SDK and board vendor’s compatibility guidance before changing them. A virtual environment won’t replace a missing driver.

Run a known model first

Start with Rockchip’s Lite2 ResNet18 example from the same SDK release. Copy its resnet18_for_rk3588.rknn and space_shuttle_224.jpg into your working directory. Run the supplied example before modifying it, so you have a reference result from the board.

The shorter script below uses those same files and prints output shapes and the highest-scoring class index. It’s a classification example for that particular model and its 224×224 RGB input. It isn’t a camera application, and it hasn’t been tested on hardware for this article.

from time import perf_counter

import cv2
import numpy as np
from rknnlite.api import RKNNLite


def require_ok(stage, status):
    if status != 0:
        raise RuntimeError(f"{stage}: return code {status}")


frame = cv2.imread("space_shuttle_224.jpg")
if frame is None or frame.shape[:2] != (224, 224):
    raise ValueError("Expected the supplied 224x224 example image")

rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
image_batch = np.expand_dims(rgb, axis=0)

engine = RKNNLite()
try:
    require_ok("load", engine.load_rknn("resnet18_for_rk3588.rknn"))
    require_ok("initialize", engine.init_runtime(
        core_mask=RKNNLite.NPU_CORE_0
    ))
    started = perf_counter()
    outputs = engine.inference(inputs=[image_batch])
    elapsed_ms = (perf_counter() - started) * 1000
    if outputs is None or len(outputs) == 0:
        raise RuntimeError("Inference returned no outputs")
    print("Shapes:", [array.shape for array in outputs])
    print("Top class index:", int(np.argmax(outputs[0])))
    print(f"Inference call: {elapsed_ms:.2f} ms")
finally:
    engine.release()

The return codes separate loading from initialization, while the finally block releases the context if inference fails. Otherwise, a script that stops at the first error can leave you guessing which part actually worked.

And the core mask is deliberate: it follows Rockchip’s example and uses one NPU core. Don’t read this script’s timing as the maximum throughput of the RK3588. Changing core selection is a separate experiment, once the output is right.

Check the result before timing it

The class index needs the matching label list to become a readable category. Compare it with the supplied example’s result on the same image. A returned tensor proves that something ran; it doesn’t establish that your input preparation is correct.

For your own model, keep the input dimensions, channel order, tensor layout, and scaling with the conversion notes. Normalization may already be configured in the RKNN build. Dividing pixels by 255 again can change the input the network receives, even though the array still has an acceptable shape.

A detector needs different output handling too. ResNet18 returns classification scores; YOLO variants can return several tensors that require model-specific decoding and filtering. Carry over the matching post-processing from the model’s deployment example, rather than adding a classification label list to detection outputs.

If detections disappear, compare a saved input with the conversion reference before adjusting confidence thresholds. Lowering a threshold can produce more boxes without fixing a wrong channel order or an incorrectly decoded tensor.

Measure the application too

The printed duration covers one inference call, including its Python/runtime overhead. It excludes reading the image, preparing the input, and interpreting the scores. The distinction between CPU and NPU workloads matters here: moving the network to the accelerator doesn’t move all the surrounding work with it.

For repeatable measurements, run warm-up calls, time multiple iterations, and record the model, core mask, and software versions. Then measure the complete application under sustained load. A single call after initialization is a poor basis for choosing a camera frame rate.

Diagnose failures before adding a camera

An import failure usually sends you back to the environment: active interpreter, installed wheel, dependencies, or native libraries. Keep the full traceback. Installing the same wheel repeatedly won’t fix a different Python interpreter running the script.

If import succeeds and initialization fails, check the runtime log, model target, and board software. Run the unmodified vendor example again. If that fails too, investigate the platform before spending time on your detector’s preprocessing.

But if the supplied model works and yours doesn’t, you have a narrower problem. Compare the conversion release, target chip, and input contract with the working case. Preserve the failure log before changing anything; changing the model and runtime together removes a useful comparison.

Once saved images work, add camera capture and local occupancy detection. Keep capture, inference, and counting separately measurable, so a stalled camera or growing frame queue doesn’t look like a slow NPU.

Save the working model, image, script, dependency versions, and board-image details together. The next export can change an output tensor without changing your Python application, and that small working case is much easier to debug than a live camera service.

FAQ

Can Lite2 convert an ONNX model?

No. Use RKNN Toolkit2 for conversion. Lite2 loads the resulting RKNN model on the board and provides the Python inference interface.

Does Lite2 require the full Toolkit2 package?

You don’t need the conversion package just to run an existing model. The board still needs a compatible Lite2 installation, runtime library, and NPU driver.

Can this ResNet18 script run a YOLO detector?

The load-and-initialize sequence is reusable, but the image preparation and output handling must match the detector. Start with its own deployment example before adapting this script.

Sources

• Rockchip RKNN software stack