Continuous Stereo Matching using
Local Expansion Moves

Tatsunori Taniai1  Yasuyuki Matsushita2  Yoichi Sato1  Takeshi Naemura1
1The University of Tokyo, Japan
2Osaka University, Japan

Technical Report (arXiv 2016)

Overview

 

 

Abstract -- We present an accurate and efficient stereo matching method using local expansion moves, a new move making scheme using graph cuts. The local expansion moves are presented as many alpha-expansions defined for small grid regions. The local expansion moves extend the traditional expansion moves by two ways: localization and spatial propagation. By localization, we use different candidate alpha-labels according to the locations of local alpha-expansions. By spatial propagation, we design our local alpha-expansions to propagate currently assigned labels for nearby regions. With this localization and spatial propagation, our method can efficiently infer Markov random field models with a huge or continuous label space using a randomized search scheme. Our local expansion move method has several advantages over previous approaches that are based on fusion moves or belief propagation; it produces submodular moves deriving a subproblem optimality; it helps find good, smooth, piecewise linear disparity maps; it is suitable for parallelization; it can use cost-volume filtering techniques for accelerating the matching cost computations. Our method is evaluated using the Middlebury stereo benchmark and shown to have the best performance in sub-pixel accuracy.

 
An extended version of our CVPR 2014 paper below.
Now our new CPU-based algorithm is 2.1x faster than our previous CPU+GPU implementation with comparable or greater accuracy.
We have a plan to release our code written by C++ and OpenCV 3.0 in future.

Summary of Changes:
  • Theoretical verification on the preferability for piecewise planar scenes.
  • Proof of the subproblem optimality (submodularity).
  • Speed-up by efficient parallelization on CPU/GPU.
  • Speed-up by incorporating fast cost filtering.
  • Much clearer algorithm explanations.
[Link to arXiv]

 


Graph Cut based Continuous Stereo Matching
using
Locally Shared Labels

Tatsunori Taniai1   Yasuyuki Matsushita 2  Takeshi Naemura1
1The University of Tokyo, Japan
2Microsoft Research Asia, China


CVPR 2014

Overview

 

 

Abstract -- We present an accurate and efficient stereo matching method using locally shared labels, a new labeling scheme that enables spatial propagation in MRF inference using graph cuts. They give each pixel and region a set of candidate disparity labels, which are randomly initialized, spatially propagated, and refined for ontinuous disparity estimation. We cast the selection and propagation of locally defined disparity labels as fusion-based energy minimization. The joint use of graph cuts and locally shared labels has advantages over previous approaches based on fusion moves or belief propagation; it produces submodular moves deriving a subproblem optimality; enables powerful randomized search; helps to find good smooth, locally planar disparity maps, which are reasonable for natural scenes; allows parallel computation of both unary and pairwise costs. Our method is evaluated using the Middlebury stereo benchmark and achieves first place in sub-pixel accuracy.

 
Code will be made availabe as our journal work with further improvements!
Thank you very much for so many requests for our code.
Unfortunately, the original code is not distributable due to lisence and heavy GPU dependency issues.
Our new C++ implementation for journal work solves these issues (OpenCV 3.0 is the only dependency).
The descriptions of our algorithm would be also much easier to understand.
We have a plan to release the code once our journal is published. Sorry for the inconvenience.
Tatsunori Taniai, November 5th, 2015.
Paper PDF
Poster PDF

Spotlight Video

 

1st rank at the Middlebury benchmark
(0.5 pixel error theshold)

Benchmark

Additional results without post-processing

Results