Knowledge Transfer Project
CycART - Cyclorama-based Automatic 3D Reconstruction Tools
Panorama-based 3D reconstruction of urban environments
Panoramic images of an environment can give the viewer a great sense of immersion. However, one thing is missing: depth. A 3D reconstruction of the world based on panoramic images opens up many new possibilities and applications. The goal of this project is to reconstruct urban environments in 3D using panoramic images. A panorama with depth can be viewed on a 3D television, or with simple red-green glasses. In a 3D reconstruction users can easily find the dimensions of a building. It even becomes possible to add virtual objects to the world that interact with the reconstructed environment. However, the real world is a complicated place, and existing image-based reconstruction techniques often fail when used on a real outdoor environment. This project aims to improve current reconstruction techniques, and to demonstrate some of the possible applications.
Lines and planes
Straight lines are common in an urban environment. Building facades usually contain windows, doors and drainpipes, and of course the edges of the building are often straight as well. By finding and matching line segments in multiple images, it is possible to reconstruct these line segments in 3D. Since building façades are often flat, line segments belonging to the same facade will (approximately) lie on the same plane. We use this fact to reconstruct a rough model of the buildings in an environment.
A reconstruction based purely on lines will be incomplete. Therefore, we also attempt to find a depth for pixels that do not lie on any line. Again we can use the observation that many objects in man-made environments can be approximated by plane segments. Specifically, groups of pixels that are close together in the image and have similar color are likely to belong to the same plane. This assumption helps us solve the reconstruction more easily.
We have also demonstrated some of the possible applications of 3D reconstruction in a web-based panorama viewer. The implemented features include occlusion handling of map data, simple point measurement and improved navigation between different panoramas.
The entire world is not made out of planes, and in some cases the assumption that it is will lead to errors in the reconstruction. We intend to detect situations where this happens and so improve the quality of the 3D model. The current models also contain many holes because parts of the environment could not be seen from the images used in the reconstruction. These holes could be filled by combining multiple reconstructions from different images. We will also investigate the use of a building contour map. Even if the contours on the map are inaccurate, the knowledge that a building exists at a certain location will be enough to help improve the reconstruction.
CycloMedia Technology BV
Remco Veltkamp, Utrecht University