Bookmark.icon

Motion estimation

The process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence.

Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. It is an ill-posed problem as the motion is in three dimensions but the images are a projection of the 3D scene onto a 2D plane. The motion vectors may relate to the whole image (global motion estimation) or specific parts, such as rectangular blocks, arbitrarily shaped patches or even per pixel. The motion vectors may be represented by a translational model or many other models that can approximate the motion of a real video camera, such as rotation and translation in all three dimensions and zoom.

More often than not, the term motion estimation and the term optical flow are used interchangeably. It is also related in concept to image registration and stereo correspondence. In fact, all of these terms refer to the process of finding corresponding points between two images or video frames. The points that correspond to each other in two views (images or frames) of a real scene or object are "usually" the same point in that scene or on that object. Before we do motion estimation, we must define our measurement of correspondence, i.e., the matching metric, which is a measurement of how similar two image points are. There is no right or wrong here; the choice of matching metric is usually related to what the final estimated motion is used for as well as the optimization strategy in the estimation process.
Algorithms

The methods for finding motion vectors can be categorized into pixel-based methods ("direct") and feature-based methods ("indirect"). A famous debate resulted in two papers from the opposing factions being produced to try to establish a conclusion.

  • Direct methods
  • Block-matching algorithm
  • Phase correlation and frequency domain methods
  • Pixel recursive algorithms

Optical flow

Indirect methods

Indirect methods use features, such as corner detection, and match corresponding features between frames, usually with a statistical function applied over a local or global area. The purpose of the statistical function is to remove matches that do not correspond to the actual motion.

Statistical functions that have been successfully used include RANSAC.
Additional note on the categorization

It can be argued that almost all methods require some kind of definition of the matching criteria. The difference is only whether you summarise over a local image region first and then compare the summarisation (such as feature-based methods), or you compare each pixel first (such as squaring the difference) and then summarise over a local image region (block base motion and filter based motion). An emerging type of matching criteria summarises a local image region first for every pixel location (through some feature transform such as Laplacian transform), compares each summarised pixel and summarises over a local image region again. Some matching criteria have the ability to exclude points that do not actually correspond to each other albeit producing a good matching score, others do not have this ability, but they are still matching criteria.

Applications

Video coding

Applying the motion vectors to an image to synthesize the transformation to the next image is called motion compensation. It is most easily applied to discrete cosine transform (DCT) based video coding standards because the coding is performed in blocks.

As a way of exploiting temporal redundancy, motion estimation and compensation are key parts of video compression. Almost all video coding standards use block-based motion estimation and compensation such as the MPEG series including the most recent HEVC.

3D reconstruction

In simultaneous localization and mapping, a 3D model of a scene is reconstructed using images from a moving camera.

Key Terms

criteria
images
local image region
motion
motion estimation
motion vectors
pixel
points
process
scene

Additional Resources

No ressources found.

Acronymn

(none found)

Synonymns

Motion estimation
(none found)

Comments

No comment found.

Sources & Credits

Last modified on March 4 2020
Content adapted from Wikipedia
No credits found.
Videocide.com is service provided by Codecide, a company located in Chicago, IL USA.
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram
Malcare WordPress Security