Segmentation with depth: A level set approach

Segmentation with depth is the challenging problem of obtaining three dimensional information from a single two dimensional image. Unlike the standard segmentation problem, the goal of segmentation with depth is to determine not only the boundaries of objects that appear in the image, but also their relative distances to the observer by making use of occlusion relations. Nitzberg, Mumford, and Shiota [The 2.1D sketch, in Proceedings of the Third International Conference on Computer Vision, Osaka, Japan, 1990; Filtering, Segmentation, and Depth, Lecture Notes in Comput. Sci. 662, Springer-Verlag, Berlin, 1993] proposed a variational formulation of this problem; according to their model, called the 2.1D sketch model, the regions that the objects occupy and their relative depth are to be extracted from the two dimensional image by minimizing a curvature based functional. Numerically, this is a highly nontrivial problem as the functional involves curvatures of the unknown contours. In this paper, we develop a level set based procedure for minimizing the Nitzberg-Mumford-Shiota energy. Unlike the minimization technique of Nitzberg-Mumford-Shiota [Filtering, Segmentation, and Depth, Lecture Notes in Comput. Sci. 662, Springer-Verlag, Berlin, 1993], our technique represents contours implicitly and thus allows for connections between T-junctions to take place automatically.