THE SMART BOOKSHELF

The smart bookshelf serves as a test-bed to study environments that are intelligently augmented by projector-camera devices. The system utilizes a camera pair and a projector coupled with an RFID reader to monitor and maintain the state of a real world library shelf. Using a simple calibration scheme, the homography induced by the world plane in which book spines approximately lie is estimated. As books are added to the shelf, a foreground detection algorithm which takes into account the projected information yields new pixels in each view that are then verified using a planar parallax constraint across both cameras to yield the precise location of the book spine. The system allows users to query for the presence of a books through a user interface, highlighting the spines of present book using the known locations obtained through foreground detection and transforming image pixels to their corresponding points in the projectors frame via a derived homography. The system also can display the state of the bookshelf at any time in the past. Utilizing RFID tags increases robustness and usefulness of the application. Tags encode information about a book such as the title, author, etc, that can be used to query the system. It is used in conjunction with the visual system to infer the state of the shelf. This work provides a novel foreground detection algorithm that works across views, using loose geometric constraints instead pixel color similarity to robustly isolate foreground pixels. The system also takes into account projected information which if not handled would be detrimental to the system. The intent of this work was to study the feasibility of an augmented reality system and use this application as a testbed to study the issues of building such a system