Understanding hand drawn chemical structures : segmentation and recognition of handwriting and sketching in freehand diagrams

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 51-52).Chemists often use hand-drawn structural diagrams to capture and communicate ideas about organic compounds. However, the software available today for specifying these structures to a computer relies on a traditional mouse and keyboard interface, and as a result lacks the ease of use, naturalness, and speed of drawing on paper. In response, we have developed a novel sketch-based system capable of interpreting hand-drawn organic chemistry diagrams, allowing users to draw molecules with a penbased input device in much the same way that they would on paper. The system's ability to interpret a sketch is based on knowledge about both chemistry and chemical drawing conventions. The system employs a trainable symbol recognizer incorporating both feature-based and image-based methods to locate and identify symbols in the sketch. Analysis of the spatial context around each symbol allows the system to choose among competing interpretations and determine an initial structure for the molecule. Finally, knowledge of chemistry (in particular atomic valence) enables the system to check the validity of its interpretation and, when necessary, refine it to recover from inconsistencies. We demonstrate that the system is capable of recognizing diagrams of common organic molecules and show that using domain knowledge produces a noticeable improvement in recognition accuracy.by Tom Yu Ouyang.S.M