Research on Image Classification and Retrieval Using Deep Learning with Attention Mechanism on Diaspora Chinese Architectural Heritage in Jiangmen, China

The study of the architectural heritage of the Chinese diaspora has an important role and significance in China’s historical and cultural background in the preservation of cultural data, the restoration of images, and in the analysis of human social and ideological conditions. The images from the architectural heritage of the Chinese diaspora usually include frescos, decorative patterns, chandelier base patterns, various architectural styles and other major types of architecture. Images of the architectural heritage of the Chinese diaspora in Jiangmen City, Guangdong Province, China are the research object of this study. A total of 5073 images of diaspora Chinese buildings in 64 villages and 16 towns were collected. In view of the fact that different types of image vary greatly in features while there are only small differences among the features of the same type of image, this study uses the depth learning method to design the Convolutional Neural Network Attention Retrieval Framework (CNNAR Framework). This approach can be divided into two stages. In the first stage, the transfer learning method is used to classify the image in question by transferring the trained parameters of the Paris500K datasets image source network to the target network for training, and thus the classified image is obtained. The advantage of this method is that it narrows the retrieval range of the target image. In the second stage, the fusion attention mechanism is used to extract the features of the images that have been classified, and the distance between similar images of the same type is reduced by loss of contrast. When we retrieve images, we can use the features extracted in the second stage to measure the similarities among them and return the retrieval results. The results show that the classification accuracy of the proposed method reaches 98.3% in the heritage image datasets of the JMI Chinese diaspora architectures. The mean Average Precision (mAP) of the proposed algorithm can reach 76.6%, which is better than several mainstream model algorithms. At the same time, the image results retrieved by the algorithm in this paper are very similar to those of the query image. In addition, the CNNAR retrieval framework proposed in this paper achieves accuracies of 71.8% and 72.5% on the public data sets Paris500K and Corel5K, respectively, which can be greatly generalized and can, therefore, also be effectively applied to other topics datasets. The JMI architectural heritage image database constructed in this study, which is rich in cultural connotations of diaspora Chinese homeland life, can provide strong and reliable data support for the follow-up study of the zeitgeist of the culture reflected in architecture and the integration of Chinese and Western aesthetics. At the same time, through the rapid identification, classification, and retrieval of precious architectural images stored in the database, similar target images can be retrieved reasonably and accurately; then, accurate techniques can be provided to restore old and damaged products of an architectural heritage.

This creative work has been published under the Creative Commons Attribution 4.0 International (CC-BY 4.0) license which allows copying, and redistribution as well as adaptation of the original work provided appropriate credit is given to the original author and the conditions of the license are met.