Automating Dataset Generation for Object Detection in the Construction Industry with AI and Robotic Process Automation (RPA)

The construction industry is increasingly adopting artificial intelligence (AI) to enhance productivity and safety, with object detection in visual data serving as a vital tool. However, developing robust object detection models demands extensive, high-quality datasets, which are often difficult to generate and maintain in construction due to the dynamic and complex nature of job sites. This paper presents an innovative approach to automating dataset generation using robotic process automation (RPA) and generative AI techniques, specifically, DALL-E 2. This approach not only accelerates dataset creation but also improves model performance by delivering balanced, high-quality inputs. To validate the proposed methodology, a case study of a building construction site is conducted. In this study, three commonly used convolutional neural network architectures—RetinaNet, Faster R-CNN, and YOLOv5—are trained with the artificially generated dataset to automate the identification of formworks and rebars during construction.

This creative work has been published under the Creative Commons Attribution 4.0 International (CC-BY 4.0) license which allows copying, and redistribution as well as adaptation of the original work provided appropriate credit is given to the original author and the conditions of the license are met.