利用上一帧和光流计算下一帧图像

参考：

1.摄像头畸变修正

https://blog.csdn.net/jacke121/article/details/78162392

2.利用光流补充帧间信息提高fps

https://stackoverflow.com/questions/47732102/unknown-output-of-opencvs-calcopticalflowfarneback

3.opencv Motion Analysis and Object Tracking API

https://docs.opencv.org/3.0-beta/modules/video/doc/motion_analysis_and_object_tracking.html#calcopticalflowfarneback

代码如下：

使用200帧与205帧计算光流，然后使用200帧和光流生成205帧图像

#!/usr/bin/env python2.7
import cv2
import numpy as np

# load image
prev = cv2.imread('stuttgart_00_000000_000200_leftImg8bit.png')
next = cv2.imread('stuttgart_00_000000_000205_leftImg8bit.png')

# change RGB to gray
prev_gray = cv2.cvtColor(prev,cv2.COLOR_BGR2GRAY)
next_gray = cv2.cvtColor(next,cv2.COLOR_BGR2GRAY)

# calculate optical flow
flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)

# calculate mat
w = int(prev.shape[1])
h = int(prev.shape[0])
y_coords, x_coords = np.mgrid[0:h, 0:w]
coords = np.float32(np.dstack([x_coords, y_coords]))
pixel_map = coords + flow
new_frame = cv2.remap(prev, pixel_map, None, cv2.INTER_LINEAR)

cv2.imwrite('new_frame.png', new_frame)
           

第200帧原图：

第205帧原图：

结果如下：

使用opencv calcOpticalFlowFarneback（注：只能使用灰度图，失真较严重）函数生成光流，后生成图像：

使用caffe flownet2的光流，生成图像：