Decomposing a Scene Into Geometric and Semantically Consistent Regions

High-level, or holistic, scene understanding involves reasoning about objects, regions, and the 3D relationships between them. This requires a representation above the level of pixels that can be endowed with high-level attributes such as class of object/region, its orientation, and (rough 3D) location within the scene. Towards this goal, the authors propose a region-based model which combines appearance and scene geometry to automatically decompose a scene into semantically meaningful regions. The model is defined in terms of a unified energy function over scene appearance and structure.