TY - GEN
T1 - MantissaCam
T2 - 14th IEEE International Conference on Computational Photography, ICCP 2022
AU - So, Haley M.
AU - Martel, Julien N.P.
AU - Wetzstein, Gordon
AU - Dudek, Piotr
N1 - Funding Information:
ceived the graduation (with Hons.) degree from the Bauhaus-Universität Weimar, Weimar, Ger-many and the Ph.D. degree in computer sci-ence from the University of British Columbia, BC, Canada, in 2011. He is currently an As-sociate Professor of Electrical Engineering and, by courtesy, of Computer Science, with Stanford University, Stanford, CA, USA. He is the Leader of Stanford Computational Imaging Lab and a Faculty Co-Director of the Stanford Center for Image Systems Engineering. At the intersection of computer graphics and vision, computational optics, and applied vision science, his research has a wide range of applications in next-generation imaging, display, wearable computing, and microscopy systems. He is the recipient of an NSF CAREER Award, an Alfred P. Sloan Fellowship, an ACM SIGGRAPH Significant New Researcher Award, a Presidential Early Career Award for Scientists and Engineers (PECASE), an SPIE Early Career Achievement Award, a Terman Fellowship, an Okawa Research Grant, the Electronic Imaging Scientist of the Year 2017 Award, an Alain Fournier Ph.D. Dissertation Award, Laval Virtual Award, and the Best Paper and Demo Awards at ICCP 2011, 2014, and 2016 and at ICIP 2016.
Funding Information:
This project was in part supported by NSF Award 1839974, the NSF Graduate Research Fellowship, and a PECASE by the ARL.
Publisher Copyright:
© 2022 IEEE.
PY - 2022/8/1
Y1 - 2022/8/1
N2 - The ability to image high-dynamic-range (HDR) scenes is crucial in many computer vision applications. The dynamic range of conventional sensors, however, is fundamentally limited by their well capacity, resulting in saturation of bright scene parts. To overcome this limitation, emerging sensors offer in-pixel processing capabilities to encode the incident irradiance. Among the most promising encoding schemes is modulo wrapping, which results in a computational photography problem where the HDR scene is computed by an irradiance unwrapping algorithm from the wrapped low-dynamic-range (LDR) sensor image. Here, we design a neural network-based algorithm that outperforms previous irradiance unwrapping methods and we design a perceptually inspired 'mantissa,' or log-modulo, encoding scheme that more efficiently wraps an HDR scene into an LDR sensor. Combined with our reconstruction framework, MantissaCam achieves state-of-the-art results among modulo-type snapshot HDR imaging approaches. We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.
AB - The ability to image high-dynamic-range (HDR) scenes is crucial in many computer vision applications. The dynamic range of conventional sensors, however, is fundamentally limited by their well capacity, resulting in saturation of bright scene parts. To overcome this limitation, emerging sensors offer in-pixel processing capabilities to encode the incident irradiance. Among the most promising encoding schemes is modulo wrapping, which results in a computational photography problem where the HDR scene is computed by an irradiance unwrapping algorithm from the wrapped low-dynamic-range (LDR) sensor image. Here, we design a neural network-based algorithm that outperforms previous irradiance unwrapping methods and we design a perceptually inspired 'mantissa,' or log-modulo, encoding scheme that more efficiently wraps an HDR scene into an LDR sensor. Combined with our reconstruction framework, MantissaCam achieves state-of-the-art results among modulo-type snapshot HDR imaging approaches. We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.
KW - computational photography
KW - end-to-end optimization
KW - in-pixel intelligence
KW - programmable sensors
UR - http://www.scopus.com/inward/record.url?scp=85141099918&partnerID=8YFLogxK
U2 - 10.1109/ICCP54855.2022.9887659
DO - 10.1109/ICCP54855.2022.9887659
M3 - Conference contribution
AN - SCOPUS:85141099918
T3 - IEEE International Conference on Computational Photography, ICCP 2022
BT - IEEE International Conference on Computational Photography, ICCP 2022
PB - IEEE
Y2 - 1 August 2022 through 5 August 2022
ER -