This is an example of applying our technology to a 360° camera. See also this blog for more details.

Optical Learning^®

"Optical Learning^® (Achromatic restoration technology)" removes all aberrations and blurring of the lens by AI. See also this blog for more details.(Japanese Registered trademark No. 5896153, Janapese Patent No. 6164564, US Patent US10,291,844, Korean patent 2,011,938)

Optical Learning^® Principles

An optical simulation is performed from the lens design data, based on the results, a dictionary corresponding to the image position is generated by AI (machine learning). Use the dictionary to restore aberration-free.

Conventional aberration correction technology

As an aberration correction technology that appeals to high image quality, it is incorporated as a function in offline editing machines. In addition, for surveillance cameras, etc. are on sale as those that can realize real time. However, there is no technology to correct all aberrations with high accuracy from lens design data.

As a problem of conventional aberration correction technology,
・Since the filter is used to shift the pixels, the central part with high resolution is also blurred.
・There are many methods using approximate expressions, and it cannot be corrected. Distortion remains. The line is jagged. There are problems such as.
・The stretched part of the surrounding area is blurred.
・Lens aberrations other than distortion cannot be corrected.
・High image quality cannot be corrected in real time.
Etc.

Effect of Optical Learning^®

• Realization of reality of no aberration...It restores all aberrations such as wide-angle lenses, fisheye lenses, and endoscopes, as well as distortion and blur caused by dimming and MTF. In other words, any camera can be made aberration-free.
• Achieves image quality (SN standard about 40 dB) comparable to professional Codec.
• Improves inexpensive lenses to a higher level of performance. (Example: HD lens → 4K lens)

* This is an example of our Optical Learning^® GoPro Hero4. The lens was analyzed by Fit.

Application of Optical Learning^®

• It can be applied to all types of cameras and photography equipment such as single-lens cameras, action cameras, drones, surveillance cameras, endoscopes, automobiles, and smartphones.
• It is effective for optical systems such as endoscopes where there are many restrictions on lens design.
• By reducing the number of lenses and improving the yield, it will lead to cost reduction and compactness.
• It is also effective for stitching, CG, VR, image recognition, measurement, robots, etc.

In fact, most cameras are distorted.
There may come a time when only distortion-free cameras can be sold, just as ordinary CRT TVs can no longer be sold when flat CRT TVs are released.

AIRD™(Artificial Intelligence Retina Development)

"AIRD (Artificial Intelligence Retina Development)" is a RAW development (demosaic) technology that reproduces the function of the human retina using AI super resolution. Eliminates zipper noise and artifacts, and eliminates sensor noise, optical LPF and diffraction blur. And the biggest feature is that it is not easily affected by chromatic aberration of magnification and axial chromatic aberration and is not easily deteriorated.(Japanese Patent No. 6435560, Former Phase Shift Development)

AIRD concept

　Instead of using R and B pixels, only G pixels and AI super resolution are used to shift twice the G pixels by half a pixel to generate them. By not using R and B pixels to generate G pixels, zipper noise and artifacts are not generated in principle.

　In addition, since the image is generated only from the highly sensitive G image, the actual sensitivity is high. It goes well with RCCB and RYYB sequences.

Conventional demosaic technology

　Artifacts such as zipper noise and false colors are almost always present. One of the causes is that it may be caused by the influence of lens aberration.

　As a problem of conventional demosaic technology,
・Artifacts such as zipper noise occur
・False color
・Reduced resolution
・Degraded resolution of red and blue images
・Zipper noise and false color due to chromatic aberration
Etc.

AIRD effect

• It completely suppresses artifacts such as zipper noise, and eliminates false colors caused by zipper noise.

• It also suppresses false colors.

• It is not easily affected by chromatic aberration of magnification and axial chromatic aberration, and suppresses zipper noise and false color.

• By AI(machine learning) the optical LPF and diffraction phenomenon, blurring due to the optical LPF and diffraction is removed.

• Since it is generated only from the highly sensitive green image, the actual sensitivity is high, and sensor noise can also be suppressed by AI(machine learning) the sensor noise.

• When used in combination with Optical Learning^® or super-resolution, it is possible to suppress the emphasis on image quality deterioration due to demosaic.

Defective pixel correction by AIRD

• Ultra-high quality defective pixel correction using AI is also possible with demosaic processing.AIRD no kekkan gaso hosei no shōsai ni tsuite wa buroguni mo nosete orimasu. Details of AIRD's defective pixel correction are also posted on our blog.

The detailed of AIRD are also posted on the blog. Please also refer to here for the detailed evaluation of AIRD.


・Difference in image quality due to demosaic (outdoor version ①)


・Difference in image quality due to demosaic (outdoor version ②)


・Difference in image quality due to demosaic (night view)


・Difference in image quality due to demosaic (indoor version ①)


・Difference in image quality due to demosaic (indoor version ②)


・Difference in image quality due to demosaic (indoor version ③)

KRM™(Keep Resolution Mapping)

　"KRM(AI super-resolution geometric transformation)" is a unique technology that uses AI super-resolution to perform geometric transformation without reducing resolution.

SRSZ™(Super Resolution Stepless Zoom)

　"SRSZ(AI super-resolution stepless zoom)" is a stepless electronic zoom that uses AI super-resolution. See also this blog for more details.

Principle of KRM / SRSZ

　By combining phase shift using AI super-resolution and aperture correction using AI super-resolution, geometric transformation (keystone correction) and stepless zoom can be restored without resolution deterioration.

　KRM and SRSZ are actually exactly the same algorithms.

Effect of KRM / SRSZ

　Geometric transformation (keystone correction) and stepless zoom can be freely restored without deterioration of resolution. (Conventional learning-type super-resolution and deep learning could only restore to the specified teacher resolution.)

Visual Sharpness™

　"Visual Sharpness" is an AI super-resolution sharpness that changes only the tilt without adding edges based on the visual model. See also this blog for more details.

Traditional Sharpness

　This is an adjustment function to supplement the viewing environment such as screen size and distance, and the viewer's eyesight.

　On YouTube etc., there are many people who edit with smartphones, and I see many images with borders that are over-sharpened.

Visual Sharpness Principle

　AI super-resolution Visual Sharpness that changes only the tilt without adding edges based on the visual model.

Effect of Visual Sharpness

Visual AI^®

　"Visual AI^®" projects far and near like the human eye. Depth of field control technology using AI(machine learning) realizes pan focus by AI. (Japanese Registered trademark No. 6365294, Japanese Patent No. 6694626)

Visual AI^® Principles

　By combining optical learning^® with a distance measuring means such as a stereo camera, you can create an image that can be projected as far or near as the human eye.

　Can a person see far or near by measuring the distance with both eyes and changing the resolution with the brain? I made a hypothesis. "Visual AI^®" has realized this mechanism by AI(machine learning).

Effects of Visual AI^®

・In order to increase the depth of field, the image sensor cannot be enlarged due to optical issues.

・The aperture ratio of the image sensor is back-illuminated and has reached its limit, making it difficult to increase the sensitivity.

・Visual AI^® is the only way to design an easy-to-use camera with deep depth of field and high sensitivity.

・Visual AI^® achieves ideal deep focus with a lot of information.

360° Images

　An example of a 360-degree image taken using a fisheye lens. It is displayed using Pannellum.

360-degree images from Insta360

　This is a 6528x3264 360-degree JPEG image taken with the Insta360 one RS 1inch and converted into a 360-degree image.

　This is a 6528x3264 mp4 image taken with the Insta360 one RS 1inch and converted into a 360-degree image.

220 degree partial panorama image created using HAL220LF+GFX100+Realop super resolution processing

　This is a 9196x9196 220-degree partial panorama PNG image taken with HAL220LF+GFX100, developed in Realop super resolution, and converted to a 220-degree image. (Provided by: Entaniya Co., Ltd.)

220 degree partial panorama image taken with HAL220LF+GFX100+Realop super-resolution development+super-resolution upconversion

　This is an 18392x18392 multi-resolution 220-degree partial panorama JPEG image taken with HAL220LF + GFX100 and converted into a 220-degree image with Realop super-resolution development and super-resolution up-conversion. The magnification increases considerably.(Provided by: Entaniya Co., Ltd.)