Volume xx (200y), Number z, pp. 1–13 Interactive Sketch-Driven Image Synthesis

Figure 1: Our interactive system guides a user to specify pose and appearance using sketching, in order to synthesize novel images from a labeled collection of training images. The user first sketches elliptical “masses ” (left), then contours (center), mimicking a traditional sketching workflow. Once the pose is specified, the artist can constrain the appearance and render a novel image (right). Top row: user sketch input and feedback guidelines; Bottom row: rendered previews. We present an interactive system for composing realistic images of an object under arbitrary pose and appearance specified by sketching. Our system draws inspiration from a traditional illustration workflow: The user first sketches rough “masses ” of the object, as ellipses, to define an initial abstract pose that can then be refined with more detailed contours as desired. The system is made robust to partial or inaccurate sketches using a reduced-dimensionality model of pose space learnt from a labelled collection of photos. Throughout the composition process, interactive visual feedback is provided to guide the user. Finally, the user’s partial or complete sketch, complemented with appearance requirements, is used to constrain the automatic synthesis of a novel, high-quality, realistic image