Rockchip has announced the official release of the RKNN3 SDK V1.0.0 for its RK1820 / RK1828 AI coprocessors. This release is fully optimized for the RK3588 / RK3576 + RK1820 / RK1828 hardware combination, providing a complete software stack for on-device AI model deployment.
The SDK delivers comprehensive upgrades in performance, model compatibility, functionality, and precision – combining high performance, broad adaptability, and excellent power efficiency.
The SDK package includes PC-side development tools, board-side runtime APIs, and model conversion/deployment examples. It supports Android and Linux systems, with PCIe / USB high-speed interfaces for low-latency data exchange.
Key new capabilities:
- Inference efficiency boost – Concurrent data transfer and inference, optimized core operators, and multi-core, multi-model parallel inference for high concurrency.
- Enhanced LLM support – Native support for mRoPE and Function Call, compatible with mainstream LLM features.
- Easier development & deployment – On-board accuracy analysis, lightweight Python API toolkit, user-defined model post-processing on the coprocessor, and embedding model support were added to the rkllm3 server.
Table of Contents
- Core Performance Leap: +15% LLM Decode, 3B >100 TPS, Up to 8B Support
- VLM & Omni‑Modal Performance
- CNN Model Performance
- Near‑Lossless Quantization
- Full‑Stack AI Ecosystem for AIoT 2.0
Core Performance Leap: +15% LLM Decode, 3B >100 TPS, Up to 8B Support
A major breakthrough of this SDK release is an overall LLM Decode performance improvement exceeding 15%. Models ranging from 0.5B to 8B parameters have been deeply optimized, with Rockchip RK1820 and RK1828 tailored to their respective compute characteristics.
- 3B-scale model breakthrough – Qwen2.5-3B achieves 102.01 Decode TPS, enabling real-time on-device LLM interaction.
- Ultra-lightweight model (0.5B) – Qwen2.5-0.5B delivers 21.89ms TTFT, 4.63ms TPOT, and 215.86 Decode TPS.
- Mid-to-large model (8B) – Qwen3-8B runs stably on RK1828 with 61.11 Decode TPS, meeting on-device deployment needs for larger LLMs.
VLM & Omni‑Modal Performance
Under standard test conditions, Rockchip RK182X has been deeply optimized for mainstream VLM and omni‑modal models. RK1820 and RK1828 are differentiated by compute capability, maintaining stable visual inference latency across resolutions.
- Qwen3-VL-4B achieves nearly 90 TPS LLM Decode.
- RK1828 enables full coprocessor-side inference for mid-to-large VLM models.
- Qwen2.5-Omni-3B runs on RK1828 with 102.63 Decode TPS (over 100 TPS), plus full audio+visual+language pipeline on the coprocessor: stable inference at 392×392 visual resolution, audio inference only 98.91ms.
CNN Model Performance
RK182X delivers solid single-core compute and dramatically higher multi‑batch, multi‑core throughput. The ViT performance of DINOv3 is particularly outstanding. In multi‑batch, multi‑core mode, frame rates increase several times over, making it ideal for high‑concurrency vision applications like smart surveillance and industrial inspection.
Near‑Lossless Quantization
The SDK applies differentiated quantization strategies:
- LLM / VLM → W4A16 G32
- CNN → W8A8
Inference performance is significantly improved while accuracy remains nearly identical to the original float32 version – some models even show slight accuracy gains.
Full‑Stack AI Ecosystem for AIoT 2.0
Rockchip’s RKNN3 SDK doesn’t exist in a vacuum – it’s built for a specific ecosystem of NPUs and CPUs. Here’s how it all fits into the perception‑decision‑execution framework of AIoT 2.0:
- Perception – Full support for Mobilenet, YOLO, depth estimation; audio models from iFLYTEK, AISpeech, and Crescendo (ASR, TTS).
- Decision – Qwen3-VL, GLM Edge, 0.5B–8B LLMs, Qwen2.5-Omni-3B, MiniCPM, Step-GUI-Edge.
- Execution – Customizable model post‑processing on the coprocessor; Android/Linux support for smart hardware and industrial inspection.
The SDK is fully compatible with Hugging Face, ModelScope, and GitHub. Ready‑to‑use RKNN models are available at: