Hybrid Losses for Hierarchical Embedding Learning
This work addresses the need for better hierarchical label handling in supervised learning, particularly for fine-grained classification tasks, but it is incremental as it builds on existing loss functions and multi-task frameworks.
The paper tackled the problem of traditional cross-entropy loss ignoring label hierarchy by proposing hybrid losses that leverage a tree structure for fine-grained labels, resulting in improved performance on classification, retrieval, embedding structure, and generalization on the OrchideaSOL dataset with nearly 200 categories.
In traditional supervised learning, the cross-entropy loss treats all incorrect predictions equally, ignoring the relevance or proximity of wrong labels to the correct answer. By leveraging a tree hierarchy for fine-grained labels, we investigate hybrid losses, such as generalised triplet and cross-entropy losses, to enforce similarity between labels within a multi-task learning framework. We propose metrics to evaluate the embedding space structure and assess the model's ability to generalise to unseen classes, that is, to infer similar classes for data belonging to unseen categories. Our experiments on OrchideaSOL, a four-level hierarchical instrument sound dataset with nearly 200 detailed categories, demonstrate that the proposed hybrid losses outperform previous works in classification, retrieval, embedding space structure, and generalisation.