Ruizhuo Song

Ruizhuo Song, Beiming Yuan

This paper introduces innovative frameworks for visual abstract reasoning, aiming to boost deep learning model performance. It emphasizes the importance of separating abstract concept and reasoning feature extraction processes. The effectiveness of the Cross-Feature Network (CFN) and its enhanced version, Triple-CFN, validates this approach. Challenges in visual abstract reasoning arise from complex pattern induction and conflicts in low-dimensional representations. To address these, a dual Expectation-Maximization (EM) process is introduced during CFN training, optimizing module parameters to synthesize non-conflicting concepts. However, the dual EM process may overfit, so mutual and decorrelation supervisions are designed to assist feature extraction, with decorrelation supervision proving effective. Leveraging metadata in Raven's Progressive Matrices (RPM), the paper proposes Meta Triple-CFN, improving reasoning accuracy and interpretability. Additionally, a Re-space layer is designed for feature space construction, further enhancing Triple-CFN's reasoning accuracy. These innovative designs provide effective solutions for abstract reasoning problem solvers, benefiting multiple deep learning domains. Codes are available at: https://github.com/Yuanbeiming/Triple-CFN-Separating-Concepts-and-Features-Enhances-Machine-Abstract-Reasoning-Ability.

Ruizhuo Song, Beiming Yuan

Visual abstract reasoning problems pose significant challenges to the perception and cognition abilities of artificial intelligence algorithms, demanding deeper pattern recognition and inductive reasoning beyond mere identification of explicit image features. Research advancements in this field often provide insights and technical support for other similar domains. In this study, we introduce PMoC, a deep-learning-based probabilistic model, achieving high reasoning accuracy in the Bongard-Logo, which stands as one of the most challenging clustering reasoning tasks. PMoC is a novel approach for constructing probabilistic models based on deep learning, which is distinctly different from previous techniques. PMoC revitalizes the probabilistic approach, which has been relatively weak in visual abstract reasoning.

Ruizhuo Song, Beiming Yuan

This paper thoroughly investigates the challenges of enhancing AI's abstract reasoning capabilities, with a particular focus on Raven's Progressive Matrices (RPM) tasks involving complex human-like concepts. Firstly, it dissects the empirical reality that traditional end-to-end RPM-solving models heavily rely on option pool configurations, highlighting that this dependency constrains the model's reasoning capabilities. To address this limitation, the paper proposes the Johnny architecture - a novel representation space-based framework for RPM-solving. Through the synergistic operation of its Representation Extraction Module and Reasoning Module, Johnny significantly enhances reasoning performance by supplementing primitive negative option configurations with a learned representation space. Furthermore, to strengthen the model's capacity for capturing positional relationships among local features, the paper introduces the Spin-Transformer network architecture, accompanied by a lightweight Straw Spin-Transformer variant that reduces computational overhead through parameter sharing and attention mechanism optimization. Experimental evaluations demonstrate that both Johnny and Spin-Transformer achieve superior performance on RPM tasks, offering innovative methodologies for advancing AI's abstract reasoning capabilities.

Ruizhuo Song, Beiming Yuan

This paper is dedicated to addressing the challenge of enhancing the abstract reasoning capabilities of AI, particularly for tasks involving complex human concepts. We introduce Lico-Net, a novel reasoning engine grounded in deep learning theory, which encodes the logical structure of Raven's Progressive Matrices (RPM) problems into probabilistic representations. Lico-Net excels in solving RPM tasks. Furthermore, we propose Lico-Net-Bongard, a tailored version of Lico-Net for the Bongard-Logo problem, which also achieves high reasoning accuracy through probabilistic representations. However, we observe a mismatch between the way deep learning algorithms and humans induce reasoning concepts, primarily attributed to the inadequate quality of negative samples. Improper configuration of negative samples can convey erroneous conceptual information to deep learning algorithms, thereby distorting their learning objectives. To address this issue, we propose two novel approaches: first, treating different sample points within reasoning problems as mutual negative samples to alter the existing negative sample structure in the data; second, designing a negative sample generator based on a step-wise linear attention mechanism to produce high-quality negative samples. Experimental results demonstrate that these methods significantly improve the performance of Lico-Net (-Bongard) and other baseline models on the RPM and Bongard-Logo datasets, as well as in the domain of foundational vision model processing, particularly when addressing the NICO dataset's distribution shift problem. Our findings emphasize the importance of improving negative sample quality for enhancing the abstract reasoning capabilities of deep learning algorithms and suggest that systems represent a promising direction for future research in this field.