Multimodal and Crossmodal AI for Smart Data Analysis
It addresses the need for scalable AI frameworks in data analysis, but appears incremental as it builds on existing multimodal and crossmodal techniques.
This paper tackles the challenge of balancing multimodal and crossmodal AI approaches for smart data analysis by introducing the MMCRAI framework integrated into the xDataPF platform, and discusses its applications across domains.
Recently, the multimodal and crossmodal AI techniques have attracted the attention of communities. The former aims to collect disjointed and heterogeneous data to compensate for complementary information to enhance robust prediction. The latter targets to utilize one modality to predict another modality by discovering the common attention sharing between them. Although both approaches share the same target: generate smart data from collected raw data, the former demands more modalities while the latter aims to decrease the variety of modalities. This paper first discusses the role of multimodal and crossmodal AI in smart data analysis in general. Then, we introduce the multimodal and crossmodal AI framework (MMCRAI) to balance the abovementioned approaches and make it easy to scale into different domains. This framework is integrated into xDataPF (the cross-data platform https://www.xdata.nict.jp/). We also introduce and discuss various applications built on this framework and xDataPF.