Condensed Representation of Machine Learning Data
This work addresses the computational burden for machine learning practitioners dealing with big data, though it appears incremental as it builds on existing clustering techniques.
The paper tackles the problem of computational inefficiency in training machine learning models on large datasets by introducing a novel condensed representation method that reduces data redundancy while maintaining acceptable accuracy, achieving comparable model training results with significantly less computational resource usage.
Training of a Machine Learning model requires sufficient data. The sufficiency of the data is not always about the quantity, but about the relevancy and reduced redundancy. Data-generating processes create massive amounts of data. When used raw, such big data is causing much computational resource utilization. Instead of using the raw data, a proper Condensed Representation can be used instead. Combining K-means, a well-known clustering method, with some correction and refinement facilities a novel Condensed Representation method for Machine Learning applications is introduced. To present the novel method meaningfully and visually, synthetically generated data is employed. It has been shown that by using the condensed representation, instead of the raw data, acceptably accurate model training is possible.