Anatomo-functional bimodality imaging for plant phenotyping: An insight through depth imaging coupled to thermal imaging

The shoots of the plants constitute challenging scenes for computer vision. Leaves, distributed with multiple orientations and various sizes, are spatially arranged along the three-dimensional (3D) networked archi-tecture of the plants. When observed with imaging systems, the shoots of the plants form complex 3D textures with various illuminations, depths, These have been shown, for instance, to display fractal properties with scale invariance in 2D projections (Ruderman and Bialek, 1994) or in 3D (Boudon et al., 2013). As a result, the detection, or the seg-mentation, of items in such complex scenes raises specific questions that may require selection, adaptation, or even the developments of new imag-ing systems or of new image processing algorithms. These rather funda-mental informational concerns also meet the need for plant scientists to perform noninvasive, rapid, automatic observations on large populations of plants to confront phenotypic data and genomic data now available at high throughput. Among applications requiring high-throughput pheno-typing of plants with computer vision is the segmentation of individual leaves on the entire plant (Biskup et al., 2007; Omasa et al., 2007; Klose et al., 2009; Kraft et al., 2010; Fiorani et al., 2012; Chéné et al., 2012). Different techniques of depth cameras have been used in the researches with a vari-ety of observation scales, including small plants (Klose et al., 2009; Kraft et al., 2010; Fiorani et al., 2012), more structured shoots of entire plants (Chéné et al., 2012), and canopy (Biskup et al., 2007; Omasa et al., 2007). It would therefore be interesting to push forward the proofs of feasibility given in the works (Biskup et al., 2007; Omasa et al., 2007; Klose et al., 2009; Kraft et al., 2010; Fiorani et al., 2012; Chéné et al., 2012) by comparing how the depth cameras compete for leaf segmentation on the same plants. We propose to contribute in this direction in this chapter. Leaf segmentation from depth information gives access to shape measurement as illustrated in several reports (Biskup et al., 2007; Omasa et al., 2007; Klose et al., 2009; Kraft et al., 2010; Fiorani et al., 2012; Chéné et al., 2012), but as underlined in Fiorani et al. (2012) and Chéné et al. (2012), it is also possible to couple the anatomic information extracted from depth imaging with functional information like water content, chlorophyll efficiency, nutrient content, or pathogen presence at the scale of the leaf. Anatomofunctional imaging providing registered bimodal information has been developed for bio-medical imaging with the anatomic information of, for instance, computed tomography coupled with the functional imaging of magnetic resonance imaging or positron emitted tomography. Such anatomofunctional imag-ing modalities have been applied to plant imaging, but their large-scale application is rather limited by their cost and low throughput (Jahnke et al., 2009; Fiorani et al., 2012). In this report we demonstrate how the associ-ation of depth imaging with an existing functional imaging system could constitute a new direction of instrumentation development for a variety of low-cost anatomofunctional bimodal imaging in plant phenotyping. The chapter is organized as follows. Section 9.2 gives a classification and a comparison of depth cameras. We then review in Section 9.3 the existing functional imaging available in plant science and illustrate with a study case the monitoring of the development of a pathogen, apple scab, on an entire plant at the scale of the leaf with functional information extracted from thermal imaging and anatomic information extracted from depth imaging. Section 9.4 details a registration approach between the anatomic and functional images with examples on apple scab. Section 9.5 concludes and gives a set of perspective opened by this work