Heart Rate

Estimate your heart rate using a video taken with your webcam or your smartphone (or if you don’t have access to one, you can analyse my video instead, linked below).

Here’s what you will do to generate the data. First, make sure you’re in a brightly lit space (i.e. turn on the lights if you’re in a dark room). Better yet, find a light source like a desk lamp or a window, and stand in front of it with your webcam or smartphone camera. Take a minute or two to sit down and relax, to get your heart rate near its resting rate (usually around 60 beats per minute). Now gently rest the tip of your index finger against the lens of your webcam or smartphone camera. Now record a video. Record for somewhere between 15 and 30 seconds. Try not to move your finger, keep it as still as you can for the duration of the video.

If you used your smartphone, transfer the video file to your computer. Open it up and look at it on the screen. Here’s a sample frame from mine, which was approximately 15 seconds in duration:

Here is a link to the complete video:

The idea here is that as your heart pumps blood through your arteries and veins, the amount of light that the camera detects, as it comes through your fingertip, will be modulated by the differences in blood flow. The variation in brightness will be quite subtle, perhaps even not visible with the naked eye, and so your task is to use signal processing techniques to try to detect it.

Here are some suggestions as to how to proceed. First load in the video file into MATLAB. Each frame of the video will have three channels: red, green and blue. Each frame will contain the brightness of each pixel for each of the three channels red, green and blue. You might as well average over all the pixels in the frame to get an average brightness measure. The signal that you ultimately want is probably the brightness over time (i.e. for each frame of the video).

Here are some other things to consider:

• make sure to use the correct sampling rate (frame rate) of your video file when doing signal analysis or filtering
• in what frequency range do you expect your heart rate to reside?

Your tasks

1. Compute the average pixel brightness, averaged over all pixels, for the red channel, for each frame. This gives you a time series, one value for each frame of the video. Subtract the mean of the time series from each value. This will center the time series around zero. Plot the resulting time series on the y-axis, and time in seconds on the x-axis.
2. Band-pass filter the zero-centered time series from task 1 using cutoff frequencies to bracket what you think is likely to be your own heart rate during the recording. Remember, cutoff frequencies are defined in Hz (cycles per second) whereas heart rate is typically characterized as beats per minute. So a 60 beats per minute resting heart rate corresponds to 1.0 Hz. Plot the filtered time series.
3. Compute and plot a spectrum of the filtered time series. Identify the frequency corresponding to the peak of the spectrum and convert the value to beats per minute.

Hints

• in MATLAB you can load a video using a built-in function called VideoReader() (and you can see the MATLAB help to Read Video Files for more information):

  video = VideoReader('IMG_1383.MOV');
  nframes = video.NumberOfFrames;
  frameHeight = video.Height;
  frameWidth = video.Width;
  frameRate = video.FrameRate; % frames per second
  duration = video.Duration;   % seconds
  time = linspace(0,duration,nframes);

  You can then read each video frame as follows:

  for i = 1:nframes
    disp(sprintf('reading frame %4d/%4d ...', i,nframes));
    frame = read(video, i);
    redChannel = frame(:, :, 1);
    % <insert your code here to process the frame>
  end

Sample plot

When I do this for my video (linked above), band-pass filtering between 40 and 80 beats per minute, I get the following:

Note that in some cases the VideoReader() function in MATLAB may produce different numerical values for pixel brightness across different operating systems (e.g. MacOS, Windows, Linux) and/or may produce incorrect values for the NumberOfFrames field of the video object. For the purposes of this assignment don’t worry about those things, just proceed to estimate heart rate using the values you get.