Event-based vision
Visual Odometry with an Event Camera Using Continuous Ray Warping and Volumetric Contrast Maximization
Sensors · 2022
RESEARCH SUMMARY
This event-camera visual odometry method jointly estimates motion and scene structure by warping viewing rays continuously and maximizing their concentration in a 3D volume.
Read the supporting paper ↗
THE RESEARCH QUESTION
How can event-based odometry handle both translation and rotation?
Simple image-plane warps cannot describe every event generated by a moving camera in a scene with varying depth. Translation makes the apparent motion depend on the unknown 3D structure. [Paper]
How the method works
- Events become viewing rays whose positions depend on a continuous camera-motion model.
- Volumetric contrast maximization favors motion and structure estimates that make those rays agree in 3D, allowing joint optimization.
RESEARCH INTERPRETATION
What this contribution means
Moving the alignment objective from a 2D image to a volume expands the scene geometry that the method can represent. The paper evaluates localization and reconstruction with a single event camera, including challenging visual conditions.
This section explains the contribution in context. For experimental protocols, numerical results and comparisons, consult the original paper.
Scope and limitations
The optimization still needs informative event measurements and an appropriate motion representation. A 3D objective should not be confused with a guarantee of globally correct reconstruction for every scene. [Paper]
Questions about this research
Why optimize contrast in 3D rather than only in an image?
A volume can represent depth-dependent event motion caused by translation through a nonplanar scene.
Read the original abstract
We present a new solution to tracking and mapping with an event camera. The motion of the camera contains both rotation and translation, and the displacements happen in an arbitrarily structured environment. As a result, the image matching may no longer be represented by a low-dimensional homographic warping, thus complicating an application of the commonly used Image of Warped Events (IWE). We introduce a new solution to this problem by performing contrast maximization in 3D. The 3D location of the rays cast for each event is smoothly varied as a function of a continuous-time motion parametrization, and the optimal parameters are found by maximizing the contrast in a volumetric ray density field. Our method thus performs joint optimization over motion and structure. The practical validity of our approach is supported by an application to AGV motion estimation and 3D reconstruction with a single vehicle-mounted event camera. The method approaches the performance obtained with regular cameras, and eventually outperforms in challenging visual conditions.
Original publication record ↗Sources, paper & code
Primary research sources support the summary above. Project resources provide demonstrations or implementations; Google Scholar is a bibliographic index.
- Primary sourcePublisher — original publication record ↗www.mdpi.com
- Primary sourceFull paper — author-hosted PDF ↗1fwang.github.io
- Metadata indexGoogle Scholar — bibliographic listing ↗scholar.google.com