DIPM: Data-driven inverse procedural modeling

This project develops new methods for the 3D reconstruction of complex objects. These methods use learned semantic prior knowledge, represented by procedural models, to deliver robust solutions despite partial or noisy data. To this end, AI methods for geometry estimation are developed by learning data-driven structural and topological rules and incorporating them as prior knowledge in the reconstruction process. Advances in natural language processing and reinforcement learning are being used to make procedural modeling representation accessible for reconstruction. For example, in the case of trees, the system learns that there must be branches behind the leaves that connect the leaves, and that small branches are connected to the trunk via larger branches. This project will develop the following contributions:
- The optimization of a generalized representation of procedural models, which improves the training of the methods to be developed and combines different procedural models.
- Multiview and monocular 3D reconstruction through data-driven inverse procedural modeling.
- Methods for deriving procedural models based on sample data.