Refining 3D Model and Scene Alignment Using Metropolis Hastings

Fatma Faruq, Maheen Rashid

Abstract
Given a matching between a 3D model and an image scene, how can one align the two optimally? We have implemented a possible method to achieve this optimization. It combines the use of Metropolis Hastings algorithm with statistics related to typical indoor scene components.

Results
Our algorithms gives more realistic matchings as shown below.

Incorporating Real-World Relations

From a database of Google Warehouse 3D models, we learned statistics relating model components to walls, and to each other. These were then modeled as probability distributions and incorporated in to the running of our algorithm. Following are some probability distributions, and the corresponding data histograms they were based on.

Original Alignment

Inter Object Intersection Constraint

Greedy Algorithm

Previous Work
A greedy optimization may converge to a non optimal result, and it may violate constraints that are never violated in the real world. For example intersecting model components:

Wall Intersection Constraint

Both Constraints

Before

After Before

After

Metropolis Hastings Algorithm

The algorithm works as follows. If it encounters a better state it will update the current state to it. Otherwise, it may still update the state depending on the ratio of probabilities of the two states.
To model probability of the image score we use multiple sigmoid functions. This ensures that the relative decrease in probability per point difference increases

over time so that the algorithm becomes more and more greedy. As shown in the following graphs:

Metropolis Hastings

Conclusion
By adding constraints to Metropolis Hastings, the algorithm provides realistic results that are more likely to be optimal. In the future changing how new states are generated, and swapping components can probably create even better matchings.
References
"Data-Driven Scene Understanding from 3D Models", Satkin et al, BMVC, 2012.

Greedy Optimization