Bridge Damage Cause Estimation Using Multiple Images Based on Visual Question Answering
This addresses the problem of automating infrastructure inspection for practitioners to prevent overlooked damage and improve maintenance techniques, though it is incremental as it adapts existing VQA and SfM methods to a specific domain.
The paper tackles bridge damage cause estimation by developing a Visual Question Answering (VQA) model that uses multiple images and Structure from Motion (SfM) to identify damage and members, achieving correct answer rates of 67.4% for member names, 68.9% for damage names, and 99.1% for yes/no questions, and applies it to an actual bridge to estimate damage causes.
In this paper, a bridge member damage cause estimation framework is proposed by calculating the image position using Structure from Motion (SfM) and acquiring its information via Visual Question Answering (VQA). For this, a VQA model was developed that uses bridge images for dataset creation and outputs the damage or member name and its existence based on the images and questions. In the developed model, the correct answer rate for questions requiring the member's name and the damage's name were 67.4% and 68.9%, respectively. The correct answer rate for questions requiring a yes/no answer was 99.1%. Based on the developed model, a damage cause estimation method was proposed. In the proposed method, the damage causes are narrowed down by inputting new questions to the VQA model, which are determined based on the surrounding images obtained via SfM and the results of the VQA model. Subsequently, the proposed method was then applied to an actual bridge and shown to be capable of determining damage and estimating its cause. The proposed method could be used to prevent damage causes from being overlooked, and practitioners could determine inspection focus areas, which could contribute to the improvement of maintenance techniques. In the future, it is expected to contribute to infrastructure diagnosis automation.