Curse of Dimensionality for TSK Fuzzy Neural Networks: Explanation and Solutions
This work addresses a known limitation of TSK fuzzy systems, specifically their poor performance on high-dimensional data, which is a problem for practitioners using these systems in machine learning.
This paper investigates why Takagi-Sugeno-Kang (TSK) fuzzy systems with Gaussian membership functions struggle with high-dimensional datasets. The authors found that the issue stems from the saturation of the softmax function, which is an equivalent form of defuzzification. They propose a new defuzzification operation, HTSK, and show that both HTSK and an existing method, LogTSK, can prevent this saturation.
Takagi-Sugeno-Kang (TSK) fuzzy system with Gaussian membership functions (MFs) is one of the most widely used fuzzy systems in machine learning. However, it usually has difficulty handling high-dimensional datasets. This paper explores why TSK fuzzy systems with Gaussian MFs may fail on high-dimensional inputs. After transforming defuzzification to an equivalent form of softmax function, we find that the poor performance is due to the saturation of softmax. We show that two defuzzification operations, LogTSK and HTSK, the latter of which is first proposed in this paper, can avoid the saturation. Experimental results on datasets with various dimensionalities validated our analysis and demonstrated the effectiveness of LogTSK and HTSK.