First working version of the avatar-garment simulation software

The objective of eTryOn’s WP2 is to investigate and develop an automatic, cost-effective and systematic method to fit digitally-designed garments to the 3D avatars generated in WP1. As such the objective consists of both: ● Generating a collection of digital garments ● Automatic fitting of the digital garments in both still images (DressMeUp app) and in real-time (MagicMirror app) At a high level, the goal can be described as creating realistic visualisations of 3D garments on a person's body. These visualisations need to be rendered in software accessible to end users. The current deliverable builds on the research delivered in D2.1 and uses the garments created in that deliverable to create a first working version of the avatar-garment simulation software. Section 3 describes in detail the core innovation in the avatar-garment simulation work. We have achieved many targets in this innovation objective and describe that in detail in this document. We explain how we use VStitcher animations as a ground truth to model cloth simulation in Unity-Obi. We identify vertex movement of the garment for a given animation and then use Nevergrad optimisation to work out the optimal Obi parameters for that garment on a given avatar. Combining this with the body tracking described in sections 4 (for Snapchat) and 5 (for Unity) allowed us to create initial working versions of the MagicMirror app. The detail of the difficulties encountered and solutions to these issues is provided in this deliverable. As well as the MagicMirror (in sections 4 and 5), we also describe the progress of the DressMeUp (section 6) and VR Designer (section 7) applications. In both of these applications we have successfully created an end-to-end prototype. In the DressMeUp app, we have taken the scans from WP1 and integrated them into our pipeline. As part of this process we have identified some of the current limitations around avatar-garment alignment. We have built a framework for evaluating the results and will use this to target improvements over the next few months. In the VR designer app, we explain the decision to directly use VStitcher animations for best results and the next steps to integrate more realistic looking avatars into that application