Super-resolved Chromatic Mapping of Snapshot Mosaic Image Sensors via a Texture Sensitive Residual Network

M Shoeiby, L Petersson, A Armin… - Proceedings of the …, 2020 - openaccess.thecvf.com
Proceedings of the IEEE/CVF Winter Conference on Applications …, 2020openaccess.thecvf.com
This paper introduces a novel method to simultaneously super-resolve and colour-predict
images acquired by snapshot mosaic sensors. These sensors allow for spectral images to
be acquired using low-power, small form factor, solid-state CMOS sensors that can operate
at video frame rates without the need for complex optical setups. Despite their desirable
traits, their main drawback stems from the fact that the spatial resolution of the imagery
acquired by these sensors is low. Moreover, chromatic mapping in snapshot mosaic sensors …
Abstract
This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fact that the spatial resolution of the imagery acquired by these sensors is low. Moreover, chromatic mapping in snapshot mosaic sensors is not straightforward since the bands delivered by the sensor tend to be narrow and unevenly distributed across the range in which they operate. We tackle this drawback as applied to chromatic mapping by using a residual channel attention network equipped with a texture sensitive block. Our method significantly outperforms the traditional approach of interpolating the image and, afterwards, applying a colour matching function. This work establishes state-of-the-art in this domain while also making available to the research community a dataset containing 296 registered stereo multi-spectral/RGB images pairs.
openaccess.thecvf.com