Kneron KL630 Ultra-Flexible Configuration for AI Camera Applications

Using TSMC's 28nm advanced technology node, the Kneron KL630 family of chips is a next-generation SoC designed for a variety of professional and consumer IP cameras, including security cameras, home cameras, video doorbell cameras, anti-shake cameras, 180° and 360° panoramic cameras, and as home appliance and robotic cameras.

The KL630 family of chips supports rapid development of high quality and multi-camera applications, providing up to 4 channels of video input, independent advanced multi-ISP pipelines including multi-exposure high dynamic range fusion, and specialized video quality enhancement for multiple wide-angle lenses and sensors, plus on-chip warping, stitching and blending, all for high quality real-time panoramic camera video processing. Kneron's third-generation neural network architecture kernel is built in, so it reacts instantly and distinguishes specific objects, reducing false positives. Overall solution features.

Reconfigurable NPU

Kneron is the leader in reconfigurable NPUs, innovating in the industry by proposing dynamic storage DMA to enhance memory access efficiency and dynamically supporting different data accuracy requirements of the same neural network, making NPU chips with ASIC high performance without sacrificing the programmability of data-intensive algorithms. With its unique and innovative architecture and excellent performance, the Nenergy team received the IEEE CAS 2021 Darlington Best Paper Award.

Nenergy's 4th generation reconfigurable NPU can support running CNN and Transformer networks simultaneously, doing both machine vision and semantic analysis with excellent computing power efficiency, providing end-users with higher performance, lower power consumption and lower cost solutions for various end-device AI applications.

Unlike common AI models that are only application-specific, Nenergy's Reconfigurable Artificial Neural Network (RANN) technology is more flexible and can meet different application requirements and adapt to various computing architectures.

In line with the characteristics of image recognition software and hardware products, Nenergy's AI models and algorithms balance performance, model size and complexity, in addition to low latency, high computational power and strong performance.

Nenergy's lightweight image recognition algorithms mainly include face recognition, body and gesture recognition, object and scene recognition, etc. These models are included in the release of the developed standard SDK.

The fifth generation ISP can record images and videos with perceptible details, realistic tonal levels and lower noise. The features are as follows.

• · "Color Consistent" technology for dynamic compensation, reducing color discontinuities and intensity
• · "HDR" technology provides sharper detail in high-contrast scenes, presenting richer and more accurate color rendering for a more realistic visual experience in the real world.
• · "Smart Edge Enhancement" technology improves localized image blurring (out of focus) caused by slight lens and sensor misalignment
• · "Multi-Lens and Sensor ISP" technology compensates for color and luminance deviations in video stitched and/or fused from multiple lens and sensor sources due to production differences

Intelligent HEVC encoder significantly reduces useless or unimportant data wherever possible and maintains high video quality without interrupting network bandwidth.

With the latest developments in dynamic ROI, GOP, FPS and noise reduction, · intelligently detects regions of interest, especially the amount of motion in dynamic scenes; applies noise reduction at different intensities; and adjusts picture groups and frame rates accordingly to achieve outstanding bitrate savings without losing every important detail.

• · Intelligent HEVC encoders enable extremely low bit rates compared to H.264 encoders in video surveillance applications, resulting in savings of up to 60% in network bandwidth and local storage costs
• · Fisheye engine de-warping, stitching, blending

The 180°/360° multi-channel fisheye de-warping support engine introduces an innovative way to combine a high-performance hardware-accelerated fisheye de-warper, allowing a real-time de-warping process for significant CPU load savings and the flexibility to support a variety of fisheye geometry correction/wide angle lenses to deliver industry-leading panoramic video formation quality.

De-warping

• · supports up to 4 video inputs
• · algorithmic optimization based on pixel/block conversion

Stitching and blending

• · Feature-based matching
• · geometric model stitching
• · compensates for color deviations of different sensors

The schematic diagram is as follows

Intelligent Video Analysis

AI On-Device" is a neural network based design that analyzes visual features of live video in real time, classifies people (especially faces), pets, cars, and any object of interest based on training for target detection as well as recognition, in order to allow end customers to have better privacy protection, data localization, and lower data access latency, in response to tasks running on the device site. to respond to tasks running on the device site.

• · seamlessly customizes integration with internal ISPs and de-warping engines to enable highly accurate detection and recognition even with wide-angle/fisheye lenses

Audio enhancement and intelligence

Creative application of machine learning methods to enhance speech and audio signal processing. VATICS audio technology inspires novel and elegant solutions to the field's most challenging and long-standing technical challenges, such as noise suppression, acoustic echo cancellation, automatic gain control, dynamic range control, and more. Benefits include.

• · offers competitive performance with a smaller footprint. Up to 30% less computational resources compared to market solutions
• · lower hardware design constraints
• ·'s internal machine learning-based engine reduces significant manual tuning efforts

High-performance application tools PQ Tool and Fisheye Correction Tool

PQ Tool, providing fast and complete image adjustment tools

EZ Tuning

There are 10 settings aligned with 17 blocks to accommodate iso changes in the current architecture. We can change the values by typing or pulling the scroll directly. You can view the attached image below.

Fisheye Correction Tool

Can be adjusted in time for timely review

 KL630 series chips provide design flexibility and support external memory based on PCB design considerations

Kneron provides square schematics and main AP functions for customers to get started quickly.

KL630 series of AI cameras (with anti-shake camera)

In surveillance systems, cameras are mounted on high poles or vehicles or moving equipment and may shake due to wind, motion or hand action.

Digital/Electronic Image Stabilization (DIS/EIS) is an efficient video stream correction method that reduces video shake and rolling shutter effects. It relies on a gyro sensor to generate the angular velocity signal for EIS, while EIS correction causes the image to be cropped and rescaled in the current frame. the MotionProcess library computes a warp matrix for ImgSigProcess (ISPE) to perform video stabilization and rolling shutter correction to improve image quality.

The main principle is to use the three-axis angular velocity and acceleration. Over a period of time, the gyroscope sensor adds up the total angles and calculates how much angular displacement it should correct in a frame. This is shown in the figure below.

Note that the level/vertical shift does not cause the gyro sensor to be manipulated. The gyroscope should be kept as far away from the lens as possible.

Kneron's solution also allows for different configuration adjustments depending on the position of the image. As shown in the figure below, consider the image with the adjusted position, including the adjusted RATIO, fisheye correction can be applied together.

►Scene application diagram

►Display board photo

►Scene cube diagram

►Core technical advantages

1. reconfigurable NPU

2. image recognition software

3. 5th generation ISP to record images and videos with perceptible details, true tonal levels, and lower noise

4. intelligent HEVC encoder to significantly reduce useless or unimportant data as much as possible and maintain high quality video without interrupting network bandwidth

5. fisheye engine to de-warp, stitch, and blend

6. intelligent video analysis

7. audio Enhancement and intelligence

8. High-performance application tools PQ Tool and Fisheye Correction Tool

►Specifications

KL630 Series of AI Cameras KLR5S3 Series (Car DVR) KLI5S3 Series (Industrial Cam)