Spatially Varying Radiometric Calibration for

Abstract—Modern society has ubiquitous electronic displays including billboards, signage and kiosks. The concurrent preva-lence of handheld cameras creates a novel opportunity to use cameras and displays as communication channels. The electronic display in this channel serves a twofold purpose: to display an image to humans while simultaneously transmitting hidden bits for decoding by a camera. Unlike standard digital watermark-ing, the message recovery in camera-display systems requires physics-based modeling of image formation in order to optically communicate hidden messages in real world scenes. By modeling the photometry of the system using a camera-display transfer function (CDTF), we show that CDTF depends on camera pose and varies spatially over the display. We devise a radiometric calibration to handle the nonlinearities of both the display and the camera, and we use the method for recovering video messages hidden within display images. Results are for 9 different display-camera systems for messages with 4500 bits. Message accuracy improves significantly with calibration and we achieve accuracy near 99 % in our experiments, independent of the type of camera or display used. Index Terms—photometric modeling, radiometric calibration, spatially variations, convex optimization I