Tell2Design: A Dataset for Language-Guided Floor Plan Generation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14680–14697 July 9-14, 2023 ©2023 Association for Computational Linguistics Tell2Design: A Dataset for Language-Guided Floor Plan Generation Sicong Leng1,*, Yang Zhou2,*,†, Mohammed Haroon Dupty3, Wee Sun Lee3, Sam Conrad Joyce4, Wei Lu1 1StatNLP Research Group, Singapore University of Technology and Design 2Institute of High Performance Computing (IHPC), A*STAR Singapore 3School of Computing, National University of Singapore 4Meta Design Lab, Singapore University of Technology and Design {sicong_leng,sam_joyce,luwei}@sutd.edu.sg zhou_yang@ihpc.a-star.edu.sg {dmharoon,leews}@comp.nus.edu.sg Abstract We consider the task of generating designs di- rectly from natural language descriptions, and consider floor plan generation as the initial research area. Language conditional genera- tive models have recently been very success- ful in generating high-quality artistic images. However, designs must satisfy different con- straints that are not present in generating artis- tic images, particularly spatial and relational constraints. We make multiple contributions to initiate research on this task. First, we in- troduce a novel dataset, Tell2Design (T2D), which contains more than 80k floor plan de- signs associated with natural language instruc- tions. Second, we propose a Sequence-to- Sequence model that can serve as a strong base- line for future research. Third, we benchmark this task with several text-conditional image generation models. We conclude by conduct- ing human evaluations on the generated sam- ples and providing an analysis of human perfor- mance. We hope our contributions will propel the research on language-guided design gener- ation forward1. 1 Introduction Recently, text-conditional generative AI models (Nichol et al., 2022; Saharia et al., 2022b; Ramesh et al., 2022; Dhariwal and Nichol, 2021; Ho et al., 2022) have demonstrated impressive results in gen- erating high-fidelity images. Such models gen- erally focus on understanding high-level visual concepts from sentence-level descriptions, and the generated images are valued for looking realistic and being creative, thereby being more suitable for generating artwork. However, besides less constrained generation like artworks, generating designs that meet various requirements specified ∗ Equal contribution † Most work done at NUS 1Code and dataset are available at https://github.com/ LengSicong/Tell2Design. Living Room Common Room Kitchen Bath Room Master Room Balcony The balcony, located on the Eastern side of the unit, North of the bathroom, juts out from the rest of the unit. It is 4' wide from West to East, and the entire balcony is located Eastward of the Easternmost wall of the bedroom, common area, and bathroom. The balcony spans 11' from North to South, and its Northern wall is shared with the Southern wall of the kitchen. The kitchen and balcony both jut out approximately 4' to the East of the unit. The bathroom is located on the central-eastern side of the unit between the common area and the master bedroom. While the Northern wall of the master bedroom and the Southern wall of the common area are both roughly 12' wide ... Language Instructions: Floor Plan: Figure 1: A data sample from our Tell2Design dataset. in natural languages is also much needed in prac- tice …

Content truncated to 500 words