How does an ultra-low illumination starlight camera achieve a color picture like daylight in the dark night

How does an ultra-low illumination starlight camera achieve a color picture like daylight in the dark night
"Starlight", "Black Light", "AI Super Low Light", and "Aurora", these concepts seem to be more popular in the security industry than "low-illumination cameras" themselves.

Regardless of the marketing name, in the final analysis, they all use different methods to enable cameras to capture images that look more like daylight at night.

Low-light cameras can make up for the fatal shortcomings of the "night blindness" of the security surveillance industry and become one of the hottest technological breakthroughs in the security circle. According to statistics, nearly 70% of crimes occur at night. If many places such as roads, warehouses, and the wild cannot be monitored at night, the personal and financial safety of cities and towns will only be in a shadow. The method of supplementary light shooting has long existed, but the problems of "light pollution", "energy waste", and "blind spots" have always been the pain points of night monitoring.

Since 2014, the newly born term "starlight" has almost become synonymous with "low illumination" in the security field. Since 2016, low illumination cameras have ushered in a stage of rapid development. It is not a strong wind and waves, but "new technology" has been used year after year. , The posture of "highlight technology" appeared in the most conspicuous position of the exhibition stand at the China Expo.

With security companies such as Hikvision, Dahua, Univision, Tiandiweiye, and Keda, the market demand for low-light surveillance cameras has been detonated. Many security companies and communication players have followed up. They not only carve up many fragmented markets, but also use artificial intelligence to show specific solutions.

What is the magic of "starlight", "black light" and other low-light camera technologies? In the five years of steady development of low-light technology, how did Hai Dayu and other security giants and many security leaders stir up this market? In 2019, what does the low-light camera closely integrated with artificial intelligence bring to the security industry?

1. From "Starlight" to many variant technologies, to make up for the shortcomings of the security eye

For a long time, the "Eye of Security" cameras have had the fatal shortcomings of "night blindness". According to statistics, nearly 70% of crimes occur at night. From 7 pm to 5 am the next day is a period of high incidence of crimes. The night image quality of cameras has become a shortcoming in the development of security big data.

Starlight cameras came into being. In the usual sense, a starlight camera usually refers to a camera that can display clear color images in a low-light environment with almost only starlight and without any auxiliary light sources. The so-called starlight level illuminance is about 0.01 Lux to 0.001 Lux, and the illuminance on a sunny and moonlit night is about 0.1 to 0.01 Lux.

Similarly, "super starlight" and "black light" reduce the illuminance again. Generally speaking, when the illuminance is below 0.001 Lux, it is "super starlight", and when the illuminance is 0 Lux, it is "black light".

▲Comparison of the effect of low-light camera and ordinary camera

The concept of low-light cameras appeared as early as 1998, when foreign brands such as Samsung and Sanyo were the leaders.

In 2014, with the development of photosensitive components and other hardware, "star-level" security surveillance camera products appeared on the market. At this time, the protagonists have become local companies such as Hikvision, Dahua, Univision, and Tiandi Weiye. They have successively released their own 2-megapixel starlight camera products, and the road to the rehabilitation of "night blindness" for cameras has officially opened.

To put it simply, the "night vision" of starlight cameras is mainly optimized through multiple hardware dimensions such as lens, aperture, image sensor, and fill light. The lens and the aperture limit the light transmission and light penetration capabilities, respectively, and the image sensor uses the light sensitivity capability, and the fill light uses the fill light capability to support the camera's low illuminance capability.

▲The hardware dimension of the camera that affects its "night vision capability" (picture from the Internet)

Although the camera's "night vision capability" needs to be optimized from these dimensions, these hardwares are not like piles of wood with the strongest stacking capabilities. Therefore, the technical solutions and product lines of different security companies are different. For example, when choosing a lens, it is not that the larger the aperture, the better, but the need to achieve a balance between the amount of light and the depth of field. In addition, the optical path design issues such as the size of the target surface of the sensor, whether it is matched with infrared and white light to fill the light, the selection of several lenses and sensors, the original data test and the ISP effect debugging are all factors that need to be considered for the different architectures of each manufacturer , It also directly affects the effect of starlight camera.

With the development of artificial intelligence technology, people have also begun to use software methods such as machine learning algorithms to improve the night vision capabilities of cameras. The marketing concepts of "AI Super Low Light", "Super Starlight", and "Aurora" that are popular in the market are different, and there are some differences in technical principles, but in general, they are constantly optimized on the basis of "starlight cameras".

At present, low-illumination cameras such as starlight cameras have been widely used in dark or non-illuminated scenes such as roads, warehouses, fields, farms, communities, and factories, and in scenes that require high picture definition. For example, in the field of public security, cameras in the past could only take black and white images, but starlight cameras and even higher-level "black light" and "aurora" cameras can help police officers to distinguish "what color clothes the suspect wears through color night vision effects." "And other issues, thereby greatly reducing the difficulty of solving the case.

2. 2016-2019: sharp drop in illuminance, concept competition, AI racing

From 2016 to 2019, with the update of back-illuminated sensors and other components, starlight-level camera technology has developed rapidly. Manufacturers headed by Hai Dayu have begun to introduce "Super Starlight", "Blacklight", "Aurora" and other variant innovations. technology.

In just a few years, security companies such as Haikang, Dahua, Univision, Keda, Tiandi Weiye and other security companies have invariably pursued low illumination from the beginning, to improve camera clarity for smart identification applications, and then use AI to increase low illumination. The sharpness and color quality of the camera have actually undergone the evolution from "pursuit of image quality" to "pursuit of intelligence" under the urging of market application demands.

1. Hikvision: After the "Starlight", the "Black Light" era has come

In 2014, Hikvision fired the first "night vision" shot of security cameras. Hikvision has successively launched starlight-level 3 million high frame rate gun-type network cameras and starlight-level 2 million pixel infrared network high-definition smart dome cameras. In 2015, the starlight-level camera technology was applied to the Hawkeye series panoramic cameras in the public security field. In addition, Hikvision has developed the "Starlight+" camera, which pays more attention to the detection of moving objects and increases the signal-to-noise ratio by 30%.

In 2016, Hikvision's starlight cameras have been widely used in national football match security work, museum monitoring, live broadcast of Qiantang River tide and other scenes. Its 4K starlight level network camera won the special award for innovative products at the 2016 China Security Expo.

Just as the "starlight" market was booming, Hikvision silently turned to the development and commercialization of "black light cameras" in 2016. In the Hikvision booth at the China Security Expo that year, there was an ingenious small darkroom in which Hikvision's "black light" series cameras were placed, which could "see" color images of objects in a 0 Lux environment.

▲Comparison of the shooting effect of starlight camera and blacklight camera under the same illuminance environment

In March 2017, Hikvision's full-color black light camera arrived as scheduled. It has a major technological breakthrough. The dual sensor architecture of retinal imaging technology can imitate the retina imaging of the human eye and solve the contradiction of interference between color and brightness. At the same time, SpecEX dual-light fusion can get a picture with less noise than a starlight camera.

Today, Hikvision's related product lines have derived products in vertical fields such as "black light explosion-proof cameras". At the same time, black light cameras have been widely used in scenes such as flood control in response to typhoon "Lichma", 24-hour presentation of river conditions, and monitoring of wildlife parks.

Hikvision's low-light camera product line expert told Zhixi that although the starlight camera has a glorious history, with the advent of the blacklight camera, the starlight camera will withdraw from the stage of the times. With the continuous improvement and optimization of low-illuminance camera technologies such as black light, market applications will continue to be normalized and scaled. Economical products will also use black light technology, which will gradually move toward the low-end market and be popularized.

After that, Hikvision's low-light camera route gradually focused on the black light camera, and the starlight technology was mainly used as an auxiliary blessing for intelligent products. For example, its "AI good partner" 4-megapixel starlight camera, starlight camera with internal sound pickup function, and so on.

2. Univision Dahua: Follow closely while pushing Chen

As one of the security ace companies, Univision Technology also launched a star-level camera in early 2014. At the end of 2016, the high-definition starlight infrared dome network camera launched by Univision not only can achieve color imaging with less noise under extremely low illumination (0.005 lux), but also includes cross-border detection, area intrusion, entering area, leaving area, wandering detection, Intelligent functions such as face detection and passenger flow statistics.

Faced with the difficulty of capturing faces in the current low-light environment, Univision directly challenged Haikang. Univision believes that the "dual sensor" architecture chosen by a type of manufacturer has achieved technical flaws even though the color and brightness are not interfered after the two images are combined into one, and the imaging effect is improved. The main defect lies in the color cast of the composite image, which cannot be applied to a completely dark place, and the cost is high.

In this regard, the solution given by Univision is the self-developed "Extra-ISP" and "U-FACE" technologies. "Extra-ISP" technology is based on the new generation of chip ISP, adding time-domain noise reduction algorithm and multi-dimensional sharpness enhancement algorithm to reduce the smear of moving images in low-light environments and make people's faces and other moving objects The details can be fully revealed. Time-domain noise reduction is to perform analysis operations on multiple frames of images to reduce noise. The setting of motion estimation is to prevent the phenomenon of motion residue from violently moving subjects.

The "U-FACE" technology is supported by the new generation of deep intelligent core chips of Univision, and it supports the new dual algorithm of face exposure and small face image enhancement, and optimizes the face in low-light environments to improve the brightness and brightness of the face. Permeability and balance. This new ISP technology has been applied in products such as MSI cameras created by Univision.

In addition, Uniview seeks specificity from the storage of low-light cameras. It is understood that its 33x starlight dome camera and 2 million super starlight camera use self-developed U-Code super encoding technology. The 2 million pixel camera only needs 1M bit stream, which is 1/4 of H.264 and H.265. 1/2, saving storage space without degrading image quality.

Dahua's new pace followed closely the pace of Big Brother Haikang. In September 2014, Dahua launched a starlight class ultra-low illumination 2 million pixel high-definition camera. The 6-inch high-definition 2 million pixel starlight infrared network high-speed smart dome launched the following year is said to integrate panoramic pan-tilt, tracking and positioning, intelligent analysis and other technologies. The budding "intelligence" can already be applied in the construction of safe cities. .

In 2016, Dahua launched a differentiated 4K aurora camera, which is almost synchronized with Haikang's black light camera. The so-called "Aurora" is said to mean that the camera can achieve a more extreme night vision effect than the starlight.

Dahua 4K Aurora Camera uses Sony’s 4/3-inch large target surface CMOS sensor, which combines the large target surface and back-illuminated technology, coupled with the ISP core processing algorithm, and the minimum illumination reaches 0.0005Lux in color and 0.0001Lux in black and white.

Dahua's 4K Aurora face recognition network camera won the "Golden Tripod Award" at the An Expo that year. The machine also displays face recognition and face attribute extraction functions based on Dahua deep learning technology related algorithms, supports 8 attributes and 5 expressions of structured information extraction, and can support up to 10,000 face database comparisons.

In the next two years, "AIization" became an important trend in the next few years. The "Hui" series of AI smart star-level surveillance cameras based on artificial intelligence technology, the Dahua 9-inch high-definition aurora network high-speed smart dome camera with dual sensors, and the smart series of star-level face cameras suitable for commercial scenes such as communities, hospitals, banks, etc And so on, you can see the shadow of artificial intelligence.

3. A hundred flowers bloom, new technology and "ghosts" coexist

The advancement of components has prepared conditions for the development of low-light cameras for head players. The proximity of head players has also promoted the integration of the low-light camera industry chain. Phoenix Optics, Yutong, Sony, APTINA, OV, Sunny Optical Component vendors headed by the others have shined in this process. With the further development of component technology and the further integration of the low-light camera industry chain, many security companies, communications companies and AI companies have also launched their own low-light camera solutions.

Many security backbones are not limited to chasing the wind, but also actively seeking their own specificity. For example, Keda released an AI ultra-low light camera at the 2019 An Expo. They believe that the mainstream enhanced lighting, hardware stacking, and complex ISP processing on the market have varying degrees of light pollution, smear and other problems. Therefore, they adopt a self-developed deep learning image enhancement algorithm to directly input the sensor input by learning a large number of scenes captured photos. Data for image recovery.

Based on AI ultra-low-light technology, Keda launched three new series of cameras. Including AI super low light vehicle bayonet, AI super low light vehicle electric police and AI super low light personnel bayonet. Kodak Technology’s low-light camera experts told us that AI ultra-low light low-light cameras are different from starlight cameras in the technical path, and the application scenarios of common starlight cameras are also slightly different. AI ultra-low light cameras are more suitable for the transportation field. Rather than monitoring public security scenes such as buckles.

Some security companies not only actively build their own low-light technology moats, but are also committed to integrating starlight with other technologies to develop the "next generation of starlight". For example, when facing the bottleneck of a single lens and a single sensor, Tiandi Weiye does a "dual light path design", so that the outline of the object is clearer and the picture is clearer; facing the problem of low-light tailing and frame loss, Tiandi Weiye uses its self-developed TVP The technology's special action threshold setting, linkage gain and time-space bilateral filtering and noise reduction to optimize image quality; Tiandi Weiye is also considered to be the first player to mention "Super Starlight" technology.

In addition, the starlight + human vehicle recognition, starlight + voice prompt, starlight + alert, starlight + automatic dimming and other programs have also greatly improved the ability of low-light cameras to deal with complex scenes.

In 2019, low-light cameras will "pop a hundred flowers", which can be seen at the China International Expo. Not only solution vendors are advancing side by side, but also component vendors such as Phoenix Optics and Yutong in the upstream of the industry chain are also taking advantage of the location of Hall 1, the super large exhibition area and many low-light cameras. Components shine.

Of course, under the trend, the mud and sand are also falling, and some low-quality black light and star light cameras have also entered the market. For example, a night vision camera of a certain brand issued by the owner produces "ghost images". This also reflects that low-light cameras have many technical dimensions, difficult to standardize, high cost, and mixed markets.

▲A certain brand of night vision camera shoots "ghost images"

3. New requirements are put forward in complex scenarios, and AI technology brings new impetus

Five years ago, it was unimaginable to let the camera "see" the colorful picture in an environment where you can't see your fingers. Now you can not only see, but also