Spatial-Temporal I

I was able to generate a rather stabilized image sequence using the KLT method mentioned previously. I disabled recreating features for each one lost and this provided a rather stable transformation for this particular video.

Before I delved into more research I decided to investigate the periodicity of the video sequence myself. I first grabbed the first 60 frames from the video. For each frame, I created a vector form of it using matlab. Each video frame is a 720x480 RGB frame. I converted each frame to Gray and then reshaped it into a 1x(720*480) vector. This 1x345600 vector was stored in a 60x345600 matrix: M.

The rows in M represent each frame of the video. Each column of M represented each individual pixel of the image.

I conjectured that if the video is perfectly stabilized, I should see a periodic signal in individual pixels caused by the spinning radar.

I used intuition to save time and memory by viewing the pixels that are located at the center of the video. This is because the center of the video is where the radar is located.

Below is an image of that matrix:

One can see a periodic signal within this matrix representation at pixel ranges 70, 528, 1004, and 1484. I was perplexed by the pixel locations of this periodic signal. However notice that these periodic pixels are ~480 apart. This is the width of the image.

I plotted the signals at the latter three pixel ranges and I get a desired periodic (cosine) signal. This periodic signal is verified as the same period as the radar which is 2.4 seconds. Each frame is 30ms.

What really helped in this investigation is the radar was a Furuno which has a dark blue strip on one side and is white on the other. This gives the nice pixel variation seen above.

There is a tool called Recurrence matrices which are a "qualitative and quantitative tool used to perform time series analysis of non-linear dynamical systems" (Cutler, Davis 2000) This was first introduced by Eckmann et al. in "Recurrence plots of dynamical systems". I will look into this tool to see what benefits I can gain from it. However, I may get by with just running a bandpass filter or an fft on the pixels of interest like the above.