Convex Optimization

Project Proposal: 3D Manhattan Wireframe Model From a Single Image

Lokender Tiwari
PhD14005
IIIT-Delhi

Abstract
This project is focuses on creating a 3D arrangement
of line from a single 2D image by using vanishing points
and an optimization procedure which incorporate the Manhattan assumption(all planes and lines intersects at 90 degree). In an image we see many lines intersecting each
other, but only few of then intersects in reality in 3D. These
connectivity (intersecting lines or broken incidence lines)
constraints must be satisfied while lifting lines from 2D to
3D. This problem is formulated as an linear programming
(LP) which gives a minimal set of least-violated connectivity constraints that are sufficient to unambiguously reconstruct the 3D lines.

1. Introduction
Figure 1. (a) Input image, (b)(c)(d) Desired output of project

Going from 2D to 3D is always been a challenging task

because while capturing an image of 3D scene we loose the
third dimension (depth). By extracting lines from an image
we cannot say anything about their order or arrangement
in 3D. With the Manhattan assumption we put some constraints on their 3D arrangement, we can find an arrangement of lines in 3D corresponding to 2D image lines. This
project is purely based on paper [1].
Intersection of two lines li and lj shown in figure(2) .
The projection rays for two image pixels pi and pj , lying on
lines li and lj respectively, are also shown. The 3D points
corresponding to these image pixels can slide anywhere on
its projection rays and the two unknown parameters (i , j )
denote the distances of these points from the camera center
(O). The directions of these two points are given by di and
dj . A 3D line can be parameterize using one of the end
points. As this point moves along the projection ray, the
corresponding 3D line slides while maintaining its Manhattan orientation Di . The distance between the two 3D lines
is given by sij .
Problem formulation is shown in equation(1). Where Di
and Dj are the Manhattan direction of lines i and j. Goal is

Figure 2.

to find the unknown depth parameters, such that the maximum number of such constraints are satisfied. L0 norm
formulation is np-hard therefore an equivalent relaxed L1
formulation is solved as shown in equation(2), where wij

are weight parameters based on evidence from image.

References
[1] S. Ramalingam and M. Brand. Lifting 3d manhattan lines
from a single image. In Computer Vision (ICCV), 2013 IEEE
International Conference on, pages 497504, Dec 2013.