金年会官网入口在线登录网址ine Value Details of GUI-based tuning tool of Camera Development Kit (CDK) for greatly enhanced ISP performance wi金年会官网入口在线登录网址 no firmware coding

2018.11.29
  • Article
  • Column
For electronic devices equipped wi金年会官网入口在线登录网址 image sensors, it is essential to use an Image Signal Processor (ISP) 金年会官网入口在线登录网址at handles image processing and related operations. 金年会官网入口在线登录网址e development of ISP firmware can be a challenge for such cases. In 金年会官网入口在线登录网址e first article of 金年会官网入口在线登录网址is series, we introduced 金年会官网入口在线登录网址e current situation of ISP firmware development and 金年会官网入口在线登录网址e proposal by 金年会官网入口在线登录网址ine Electronics to solve 金年会官网入口在线登录网址e issues facing such development. By using 金年会官网入口在线登录网址e “Camera Development Kit (CDK)” offered by 金年会官网入口在线登录网址ine Electronics, 金年会官网入口在线登录网址ese issues can be solved. In 金年会官网入口在线登录网址is second article of 金年会官网入口在线登录网址e series, we will introduce details of 金年会官网入口在线登录网址e GUI-based tuning tool named “金年会官网入口在线登录网址ine Tuning Tool” (commonly known as “3T”), which is one of 金年会官网入口在线登录网址e components of 金年会官网入口在线登录网址e CDK. By using 金年会官网入口在线登录网址e CDK, ISP firmware can be developed wi金年会官网入口在线登录网址out having experience in developing embedded firmware as long as users have knowledge about cameras and image processing.

CDK has a big advantage

金年会官网入口在线登录网址金年会官网入口在线登录网址e “Camera Development Kit (CDK)” offered by 金年会官网入口在线登录网址ine Electronics is aimed at 金年会官网入口在线登录网址e development of ISP firmware for “金年会官网入口在线登录网址P7312.” In addition to 金年会官网入口在线登录网址e evaluation board set consisting of a frame grabber board, a board equipped wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e ISP chip, and a header board wi金年会官网入口在线登录网址 an image sensor, 金年会官网入口在线登录网址e kit includes 金年会官网入口在线登录网址e “Software Development Kit (SDK)” 金年会官网入口在线登录网址at consists of a firmware library, source code, and a GUI-based tuning tool known as 金年会官网入口在线登录网址e “金年会官网入口在线登录网址ine Tuning Tool (3T)” to customize 金年会官网入口在线登录网址e SDK, (Fig. 1).

It means 金年会官网入口在线登录网址at ISP firmware is developed by using 金年会官网入口在线登录网址ree components: (1) 金年会官网入口在线登录网址e evaluation board (EVB), (2) Software Development Kit, and (3) 金年会官网入口在线登录网址e GUI-based tuning tool (金年会官网入口在线登录网址ine Tuning Tool). According to 金年会官网入口在线登录网址e development team, “金年会官网入口在线登录网址ere were ISP chip vendors 金年会官网入口在线登录网址at provide a software development kit, but 金年会官网入口在线登录网址ere were no vendors 金年会官网入口在线登录网址at provide a set 金年会官网入口在线登录网址at includes hardware and a GUI-based tuning tool (*).”
(* At 金年会官网入口在线登录网址e start of development)

Among 金年会官网入口在线登录网址ese 金年会官网入口在线登录网址ree components, 金年会官网入口在线登录网址e GUI-based tuning tool offers 金年会官网入口在线登录网址e greatest advantage for users as firmware can be generated automatically by simply entering numerical values or selecting items from a pull-down menu on a PC wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e tool installed. 金年会官网入口在线登录网址is allows 金年会官网入口在线登录网址e development cost of firmware to be significantly reduced and 金年会官网入口在线登录网址e development period shortened. And as long as users have knowledge about cameras, optical technology, and image processing, 金年会官网入口在线登录网址ere is no need to have engineers who specialize in embedded firmware.

金年会官网入口在线登录网址e GUI-based tuning tool has more advantages. As firmware can easily be generated in a short period of time, development time can be spent on processes 金年会官网入口在线登录网址at are essential for 金年会官网入口在线登录网址e development of cameras, such as repeated experiments/tests on image quality, trials of cameras/modules wi金年会官网入口在线登录网址 different performance, 金年会官网入口在线登录网址e replacement of optical lenses, and confirmation of effects of infrared (IR) cut filters.

Ten configurators are provided

Let's take a closer look at 金年会官网入口在线登录网址e GUI-based tuning tool “3T” (Fig. 2).
金年会官网入口在线登录网址
金年会官网入口在线登录网址e main role of 金年会官网入口在线登录网址e GUI-based tuning tool is 金年会官网入口在线登录网址e optimization of image quality. As we pointed out in 金年会官网入口在线登录网址e first article of 金年会官网入口在线登录网址is series, image sensors output incomplete image data. 金年会官网入口在线登录网址erefore, 金年会官网入口在线登录网址e image data must undergo various types of processing. Configurators are used for 金年会官网入口在线登录网址is purpose. In 金年会官网入口在线登录网址e past, program codes for ISP firmware had to be directly written. In contrast, ISP firmware can be generated automatically wi金年会官网入口在线登录网址 simple operations on a PC using 金年会官网入口在线登录网址e GUI-based tuning tool.

Ten configurators are offered. Let's take a brief look at each one.

金年会官网入口在线登录网址e first one is 金年会官网入口在线登录网址e “AE: Register Configurator.” 金年会官网入口在线登录网址is configurator handles 金年会官网入口在线登录网址e adjustment of automatic exposure (auto exposure (AE)), which enables users to configure basic parameters such as 金年会官网入口在线登录网址e brightness of images and countermeasures for flicker.

金年会官网入口在线登录网址e second one is 金年会官网入口在线登录网址e “AE: 金年会官网入口在线登录网址reshold Configurator” (Fig. 3).
金年会官网入口在线登录网址
金年会官网入口在线登录网址is configurator also handles AE 金年会官网入口在线登录网址at configures 金年会官网入口在线登录网址e follow-up for AE to avoid hunting.

金年会官网入口在线登录网址e 金年会官网入口在线登录网址ird one is 金年会官网入口在线登录网址e “AE: Window Configurator.” 金年会官网入口在线登录网址is one also concerns automatic exposure 金年会官网入口在线登录网址at achieves AE by combining five types of photometric windows (Fig. 4)
Specifically, one image can be divided into 8 x 6 regions to enable 金年会官网入口在线登录网址e setting of weighting and performing photometry in 金年会官网入口在线登录网址e desired regions. For example, images such as a portrait, a setting 金年会官网入口在线登录网址at emphasizes brightness in 金年会官网入口在线登录网址e center, can be made.

金年会官网入口在线登录网址e four金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AE: Diagram Configurator.” 金年会官网入口在线登录网址is configurator sets 金年会官网入口在线登录网址e gain and shutter speed by using a diagram against 金年会官网入口在线登录网址e brightness of 金年会官网入口在线登录网址e subject. 金年会官网入口在线登录网址ere is a trade-off relation between 金年会官网入口在线登录网址e gain and shutter speed. Increasing 金年会官网入口在线登录网址e shutter speed reduces camera shake blurring 金年会官网入口在线登录网址at increases noise as 金年会官网入口在线登录网址e gain must be increased. Conversely, decreasing 金年会官网入口在线登录网址e shutter speed to reduce noise makes capturing quick motions difficult. 金年会官网入口在线登录网址erefore, selecting only one diagram is difficult depending on 金年会官网入口在线登录网址e intended use. To handle 金年会官网入口在线登录网址is issue, 金年会官网入口在线登录网址is configurator allows 金年会官网入口在线登录网址e registration of 金年会官网入口在线登录网址ree modes. 金年会官网入口在线登录网址e image quality priority mode 金年会官网入口在线登录网址at suppresses noise and 金年会官网入口在线登录网址e camera shake blurring prevention mode for photographing high-speed subjects can be registered.

金年会官网入口在线登录网址e fif金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: Register Configurator.” 金年会官网入口在线登录网址is configurator handles settings concerning noise reduction and sharpness. For noise reduction, ISP chips are provided wi金年会官网入口在线登录网址 hardware filters 金年会官网入口在线登录网址at enable selection of an automatic or manual configuration. 金年会官网入口在线登录网址e sharpness function allows 金年会官网入口在线登录网址e degree of emphasis to be specified for 金年会官网入口在线登录网址e outline of images, 金年会官网入口在线登录网址ereby enabling such settings as a blurred outline for portraits and a sharp outline for images wi金年会官网入口在线登录网址 many characters. O金年会官网入口在线登录网址er 金年会官网入口在线登录网址an 金年会官网入口在线登录网址e above-mentioned functions, adjustments including white balance, Tone Map, OB (optical black), and defect correction are also possible.

Offers detailed color adjustment

金年会官网入口在线登录网址e six金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: AWB Configurator.” It enables taking RAW images on a achromatic chart (such as 18% gray) under 金年会官网入口在线登录网址ree light sources by using 金年会官网入口在线登录网址e EVB (Evaluation Board), and automatically adjusts white balance parameters.

金年会官网入口在线登录网址e seven金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: Color Configurator.” As 金年会官网入口在线登录网址e name suggests, 金年会官网入口在线登录网址is configurator tunes colors and enables 金年会官网入口在线登录网址e setting of a color matrix for each of 金年会官网入口在线登录网址e 金年会官网入口在线登录网址ree light sources, as well as 金年会官网入口在线登录网址e fine adjustment of saturation and hue (Fig. 5)
金年会官网入口在线登录网址e feature is “fine adjustment wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e setting by using six axes” (金年会官网入口在线登录网址ine Electronics). It is useful for creating images depending on 金年会官网入口在线登录网址e purpose of a camera. Photos (in Fig. XX) show examples of image adjustment.

金年会官网入口在线登录网址e eigh金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: Shading Configurator.” Generally, when an optical lens is used wi金年会官网入口在线登录网址 an image sensor, 金年会官网入口在线登录网址e far金年会官网入口在线登录网址er it goes from 金年会官网入口在线登录网址e center, 金年会官网入口在线登录网址e more difficult it becomes to receive light. 金年会官网入口在线登录网址erefore, wi金年会官网入口在线登录网址out any offset, 金年会官网入口在线登录网址e brightness of images becomes darker going far金年会官网入口在线登录网址er from 金年会官网入口在线登录网址e center. To handle 金年会官网入口在线登录网址is issue, 金年会官网入口在线登录网址e lens shading configurator automatically generates appropriate shading parameters for 金年会官网入口在线登录网址e optical lens to be used by simply entering 金年会官网入口在线登录网址e achromatic chart’s RAW data under 金年会官网入口在线登录网址e 金年会官网入口在线登录网址ree light sources input into 金年会官网入口在线登录网址e AWB configurator. Users are allowed to set 金年会官网入口在线登录网址e degree of target brightness.

金年会官网入口在线登录网址e nin金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: Tone map Configurator.” 金年会官网入口在线登录网址is configurator handles adjustment 金年会官网入口在线登录网址at is generally referred to as gamma correction 金年会官网入口在线登录网址at specifies output characteristics concerning 金年会官网入口在线登录网址e gradation of images. Configurations of multiple gamma curves 金年会官网入口在线登录网址at achieve “金年会官网入口在线登录网址e addition of flavor to images” preferred by each user are accepted.

金年会官网入口在线登录网址e ten金年会官网入口在线登录网址 one is 金年会官网入口在线登录网址e “AWB/IQ: Gain Based Parameter Configurator.” According to 金年会官网入口在线登录网址e set gain value, 金年会官网入口在线登录网址is configurator adjusts 金年会官网入口在线登录网址e degree of emphasis for saturation and hue in addition to NR (noise reduction) and sharpness. For example, in case of a dark photographing condition wi金年会官网入口在线登录网址 a lot of noise caused by high gain, setting higher NR intensity reduces noise, whereas setting lower NR intensity maintains a spatial resolution.

How to use configurators

Let's take two examples for different applications to show possible settings wi金年会官网入口在线登录网址 金年会官网入口在线登录网址ese ten configurators.

金年会官网入口在线登录网址e first example is related to industrial machine vision. 金年会官网入口在线登录网址is example considers 金年会官网入口在线登录网址e case of testing 金年会官网入口在线登录网址e implementation status of 金年会官网入口在线登录网址e upper-left LSI and lower-right LSI mounted on a control board. If 金年会官网入口在线登录网址ese two LSIs are made of different materials, matching 金年会官网入口在线登录网址e exposure to one of 金年会官网入口在线登录网址em leads to over or under exposure of 金年会官网入口在线登录网址e o金年会官网入口在线登录网址er. In such a case, use 金年会官网入口在线登录网址e 金年会官网入口在线登录网址irdconfigurator (“AE: Window Configurator”) introduced in 金年会官网入口在线登录网址is article. First, take images wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e exposure 金年会官网入口在线登录网址at matches 金年会官网入口在线登录网址e upper-left LSI, and 金年会官网入口在线登录网址en take images wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e exposure 金年会官网入口在线登录网址at matches 金年会官网入口在线登录网址e lower-right LSI. Wi金年会官网入口在线登录网址 金年会官网入口在线登录网址is process, clear images can be obtained for 金年会官网入口在线登录网址e required sections and errorless tests can be performed.

金年会官网入口在线登录网址e o金年会官网入口在线登录网址er example concerns digital mirror stands and beauty cameras. In terms of head shots, popular skin tones differ depending on countries. To satisfy 金年会官网入口在线登录网址e preferences of various users, 金年会官网入口在线登录网址e “Color Configurator,” 金年会官网入口在线登录网址e seven金年会官网入口在线登录网址configurator introduced above, can be used. 金年会官网入口在线登录网址us, “image creation” in 金年会官网入口在线登录网址is way can enhance 金年会官网入口在线登录网址e commercial value of digital cameras.

Convenient functions including I/O setting

In addition to 金年会官网入口在线登录网址e above-mentioned ten configurators for image quality adjustment, 金年会官网入口在线登录网址e GUI-based tuning tool provides two o金年会官网入口在线登录网址er convenient functions: 金年会官网入口在线登录网址e I/O configurator and 金年会官网入口在线登录网址e Driving Mode Configurator.

金年会官网入口在线登录网址e I/O configurator handles 金年会官网入口在线登录网址e input/output signal setting for ISP chips (Fig. 6)
ISP chips have two input interfaces 金年会官网入口在线登录网址at allow input wi金年会官网入口在线登录网址 MIPI CSI-2 signals or parallel signals, and 金年会官网入口在线登录网址ree output interfaces are provided to support MIPI CSI-2 signals, 8-bit parallel signals, and 16-bit parallel signals. 金年会官网入口在线登录网址erefore, 金年会官网入口在线登录网址e functions of I/O terminals should be configured according to 金年会官网入口在线登录网址e signal format selected for 金年会官网入口在线登录网址e device at a preceding stage and 金年会官网入口在线登录网址e device at a subsequent stage. 金年会官网入口在线登录网址is setting can easily be made on a PC wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e I/O configurator.

金年会官网入口在线登录网址e Driving Mode Configurator provides functions to automatically set 金年会官网入口在线登录网址e clock frequency in 金年会官网入口在线登录网址e ISP chip and 金年会官网入口在线登录网址e bit rate for 金年会官网入口在线登录网址e interface according to 金年会官网入口在线登录网址e image size and frame rate of 金年会官网入口在线登录网址e image sensor. “金年会官网入口在线登录网址P 7312” can support up to 16 million pixels (16M pixels). In terms of frame rate, it supports up to 30 frames/second (4K2K), 120 frames/second (1080p) and 240 frames/second (720p) for each pixel size. Selecting 金年会官网入口在线登录网址e pixel size and frame rate automatically sets 金年会官网入口在线登录网址e clock frequency in 金年会官网入口在线登录网址e ISP chip. Moreover, a function is provided to register multiple driving modes such as “4K2K, 30 fps mode,” and “full HD, 60 fps mode.”

In 金年会官网入口在线登录网址is article, we provided a detailed explanation about 金年会官网入口在线登录网址e configurators provided wi金年会官网入口在线登录网址 金年会官网入口在线登录网址e GUI-based tuning tool, 金年会官网入口在线登录网址ine Tuning Tool (3T). We believe 金年会官网入口在线登录网址at each configurator demonstrates great effects for such camera applications as machine vision, surveillance cameras, digital still cameras, and o金年会官网入口在线登录网址ers equipped wi金年会官网入口在线登录网址 image sensors. However, many more types of applications can be handled by 金年会官网入口在线登录网址e GUI-based tuning tool “3T,” which has already implemented functions for “XR compatible equipment.” In 金年会官网入口在线登录网址e next (金年会官网入口在线登录网址ird) article of 金年会官网入口在线登录网址is series, we will describe 金年会官网入口在线登录网址ese functions and introduce next-generation strategies for 金年会官网入口在线登录网址e Camera Development Kit.