Indoor Scene Synthesis: Generation of Interior Designs with Transformer Models

Kurzfassung

The task of indoor scene generation consists of creating a sequence of objects categories, their locations, and orientations conditioned on the shape and size of a room. In order to automize the easy and fast generation of authentic 3D scenes without the need of human supervision. For that large scale indoor scene data-sets are used, which allow us to extract patterns from expert-designed indoor scenes. Based on those it is possible to generate new scenes based on the learned patterns hidden in these datasets. Existing methods rely on representing the scene by its 2D or 3D appearance. There there are other methods representing it as a graph while making assumptions about the possible relations between objects. In contrast to those, the aim of this thesis is to learn relations between objects using the self-attention mechanism of transformers. This approach leads to a similar or even better levels of realism, while being more salable. Our approach named EnvironmentFormer is simple and an effective generative model conditioned on the room shape, and room type. The room type here could be for example be a kitchen or a living room. Our method is built on the transformer model from Vaswani et al. The result of our method is a complete 3D scene, containing information where which objects should be placed. Those results are then further improved by using appearance-based methods for refinement and style consistency.