Lucas-Kanade method

The Lucas-Kanade method is an optic flow estimator which estimates motion based on the local gradient and local difference of two consecutive frames.  To solve the aperture problem, the optic flow is assumed to be locally constant.  Homogeneous regions where there is no solution are masked out.

See also

#!/usr/bin/env ruby
require 'hornetseye'
require 'matrix'
include Hornetseye
sigma, cov_sigma = 3.0, 5.0
input = XineInput.new '../../data/videos/polygon.avi'
img2 = input.read_ubyte.gauss_blur sigma
display = X11Display.new
output = [ XImageOutput.new, XImageOutput.new, XImageOutput.new ]
window = output.collect { |o_| X11Window.new( display, o_, *img2.shape ).show }
window[0].title = "image"
window[1].title = "flow x"
window[2].title = "flow y"
rm = 1.0
while display.status?
  img1, img2 = img2, input.read_ubyte.gauss_blur( sigma )
  dx = ( img1 + img2 ).sobel( 0 ) / 16
  dy = ( img1 + img2 ).sobel( 1 ) / 16
  dt = ( img2 - img1 )
  a = ( dx ** 2 ).gauss_blur cov_sigma
  b = ( dy ** 2 ).gauss_blur cov_sigma
  c = ( dx * dy ).gauss_blur cov_sigma
  bx = ( dx * dt ).gauss_blur cov_sigma
  by = ( dy * dt ).gauss_blur cov_sigma
  det = a * b - c * c
  mask = det >= det.max * 1.0e-4
  a, b, c, det = [ a, b, c, det ].collect { |x| x.mask( mask ) }
  minv = Matrix[ [ b, -c ], [ -c, a ] ].collect { |x| x / det }
  b = Vector[ bx, by ].collect { |x| x.mask( mask ) }
  v = ( minv * b )
  rm = [ -v[0].min, v[0].max, -v[1].min, v[1].max, rm ].max
  v = v.collect { |x| x.unmask( mask ) }
  output[0].write( ( img1 + img2 ) / 2 )
  output[1].write( v[0] * 127 / rm + 127 )
  output[2].write( v[1] * 127 / rm + 127 )
  display.processEvents
end
Close