Online Respiratory Signal Detection based on Color Video

在這個研究中，我們提出了一個使用彩色視訊觀察使用者臉部的連續圖片在心跳時對綠色通道產生的微弱變化的即時且低花費的呼吸訊號偵測方法。因為心跳會隨著使用者的吸氣和呼氣而增加或減慢，所以呼吸訊號便可從經過一些濾波器處理的心跳間距序列獲得。此外，本研究也提出可靠度的估測並用來觀察訊號的穩定性。隨著頭部移動或光線的調整，可靠度將會立即地下降且不論在即時或事後分析都很容易觀察。 在實驗部分，我們在不同條件下測量得到的結果和呼吸感測器的訊號相似度，像是光源、背景、和攝影機的距離以及使用的通道組合。我們也在接下來的小節測量我們能達到的最高相似度而且發現可靠度越高，我們的結果和呼吸感測器訊號的相似度越高。在應用的部分，這個方法可以像一個商業化的儀器-StressEraser一樣用來引導使用者的呼吸，不僅可以消除壓力還能透過每天練習以增加心跳變異率，以及偵測使用者在睡覺時的呼吸訊號以幫助醫師觀察睡眠呼吸中止症。 這個研究的貢獻是提出一個低消費且非接觸式的即時呼吸訊號偵測方法，並且可以引導使用者達到自己的共振呼吸頻率，進而達到放鬆和增加心跳變異率的功能。In this study, we propose an online low cost, non-contact respiratory signal extraction method using widely used RGB camera by measuring the subtle variations of green channel intensity from users’ face image sequence during heart pulse. Since heart rate may increase or decrease with users’ inhalation or exhalation, the respiratory signal can then be obtained from sequence of heart beats intervals after applying some filters. Besides, the evaluation of tidiness also proposed and used to observe the stability of signal. With head movement or light adjustment, tidiness will immediately decrease and easily to observe no matter in real-time or offline analysis. In the experiment, we measure the similarity of our result and respiration sensor signal in different conditions such as light source, background, distance from webcam and combination of channels. We also measure the highest similarity we can achieve in following sections and found that our result is more similar to respiration sensor signal for subjects with higher tidiness. In the application, this method can be used to guide users’ breaths like a commercial device-StressEraser to not only relieve stress but also increase heart rate variability through daily practices and detect respiratory signal when users are sleeping, which can help doctors observe the symptom of sleep apnea. The contribution of this study is a low cost, noncontact online respiratory signal detection method which can be used to guide users to its own resonant respiratory frequency, leading them to relax and increasing heart rate variability.口試委員會審定書 i 誌謝 ii 中文摘要 iii ABSTRACT iv CONTENTS v LIST OF FIGURES vii LIST OF TABLES xiii Chapter 1 INTRODUCTION 1 Chapter 2 RELATED WORK 3 2.1 RGB Camera-Based Method 3 2.2 Depth Camera-Based Method 7 2.3 BVP Sensor-Based Method 8 Chapter 3 METHODOLOGY 11 3.1 Heart Beat Detection based on Color Video 11 3.1.1 Raw Data 12 3.1.2 Band Pass Filter 13 3.1.3 Difference Signal 15 3.1.4 Set Threshold 15 3.1.5 Divide & Match 16 3.1.6 Peak Alignment 17 3.1.7 Search Back 17 3.2 Respiratory Signal Extraction 18 3.3 Signal Tidiness Estimation 23 3.4 Inter-beat Interval Confidence Level Estimation 26 Chapter 4 EXPERIMENTS 28 4.1 Experimental Setup 28 4.2 Reliability Evaluation 31 4.2.1 Setup for Reference Case 31 4.2.2 Effects of Changing Background 33 4.2.3 Effects of Changing Ambient Light 35 4.2.4 Effects of Changing Measuring Distance 39 4.2.5 Effects of Changing Color of Light 41 4.3 Similarity Evaluation for Different Subjects 44 4.4 Validation of Tidiness and Confidence Level Estimation 46 4.5 Visibility of Respiratory Sinus Arrhythmia 49 Chapter 5 APPLICATIONS 53 5.1 Stress-Release 53 5.2 Fast-Asleep 56 Chapter 6 CONCLUSIONS AND FUTURE WORK 58 REFERENCE 5