Qinghong Yang

Zhongzhi Chen, Guang Liu, Bo-Wen Zhang et al. · meta-ai

In this work, we present a conceptually simple and effective method to train a strong bilingual/multilingual multimodal representation model. Starting from the pre-trained multimodal representation model CLIP released by OpenAI, we altered its text encoder with a pre-trained multilingual text encoder XLM-R, and aligned both languages and image representations by a two-stage training schema consisting of teacher learning and contrastive learning. We validate our method through evaluations of a wide range of tasks. We set new state-of-the-art performances on a bunch of tasks including ImageNet-CN, Flicker30k-CN, COCO-CN and XTD. Further, we obtain very close performances with CLIP on almost all tasks, suggesting that one can simply alter the text encoder in CLIP for extended capabilities such as multilingual understanding. Our models and code are available at https://github.com/FlagAI-Open/FlagAI.

Ting Jiang, Deqing Wang, Leilei Sun et al.

Hierarchical text classification aims to leverage label hierarchy in multi-label text classification. Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels. Since global hierarchy is static and irrelevant to text samples, it makes these methods hard to exploit hierarchical information. Contrary to global hierarchy, local hierarchy as a structured labels hierarchy corresponding to each text sample. It is dynamic and relevant to text samples, which is ignored in previous methods. To exploit global and local hierarchies,we propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL), which utilizes the large-scale parameters and prior language knowledge of BERT to model both global and local hierarchies.Moreover,HBGL avoids the intentional fusion of semantic and hierarchical modules by directly modeling semantic and hierarchical information with BERT.Compared with the state-of-the-art method HGCLR,our method achieves significant improvement on three benchmark datasets.

Jingwen Wang, Jingxin Liu, Juntao Pu et al.

In financial field, a robust software system is of vital importance to ensure the smooth operation of financial transactions. However, many financial corporations still depend on operators to identify and eliminate the system failures when financial software systems break down. This traditional operation method is time consuming and extremely inefficient. To improve the efficiency and accuracy of system failure detection and thereby reduce the impact of system failures on financial services, we propose a novel machine learning-based framework to predict the occurrence of system exceptions and failures in a financial software system. In particular, we first extract rich information from system logs and eliminate noises in the data. Then the cleaned data is leveraged as the input of our proposed anomaly prediction framework which consists of three modules: key performance indicator(KPI) data prediction module, anomaly identification module and severity classification module. Notably, we design a hierarchical architecture of alarm classifiers and try to alleviate the influence of class-imbalance problem on the overall performance. Empirically, the experimental results demonstrate the superior performance of our proposed method on a real-world financial software system log data set.