Google Research creates passive heart rate monitoring system via smartphone camera

Google Research published a description in Nature of a passive heart rate monitoring (PHRM) system that uses a smartphone's front-facing camera to record short videos of the face within seconds of unlocking. Using temporal convolutional neural networks and photoplethysmography (PPG) methods, the system estimates heart rate (HR) and daily resting heart rate (RHR). The training used over 350,000 video clips from nearly 700 participants, ensuring at least 25% of participants had light and medium skin (by Monk scale) and 33% had dark skin. In laboratory conditions, PHRM achieved a mean absolute percentage error (MAPE) of less than 10% for all skin tone groups, outperforming 15 known rPPG models. In a real-world "free-living" study with 231 participants wearing ECG chest straps and Fitbit Charge 6, PHRM showed a MAPE of 6.09% after reliability filtering, and daily RHR had a mean absolute error of 4.39 beats per minute relative to Fitbit, better than the target of 5 beats. The system correctly reflected the association of elevated RHR with high BMI and low aerobic fitness. Errors were slightly higher for participants with dark skin but met accuracy standards. The researchers also release the largest labeled video dataset and a pretrained PHRM-mini model for non-commercial research.