DeepEIR: A Holistic Medical Multimedia System for Gastrointestinal Tract Disease Detection and Localization

We developed a complete holistic medical multimedia system for gastrointestinal (GI) tract disease detection and localization based on Computer Vision (CV) and Artificial Intelligence (AI). Our DeepEIR system is designed as flexible, generalizable, adaptable, efficient and accurate solution that supports various endoscopic devices including wireless capsular endoscopes, and it can be easily expanded with new diseases and objects. The system can both process a vast amount of data off-line and perform real-time support during live medical procedures. We also contributed to the problem of medical data availability for research community. We collected, annotated, and published several datasets and data annotation tools as open source. Our datasets (Kvasir, Nerthus and Medico) immediately got a lot of attention and are used by many research teams. This work connects Multimedia and Medicine and uses Image Analysis, Machine Learning (ML), Convolutional Neural Networks (CNN), Deep Learning (DL) and Generative Adversarial Networks (GAN) to support doctors in their daily routine, reduce lesion overlooking and, therefore, have a societal impact by helping people to survive lethal diseases