Mingyu Lee

Mingyu Lee, Jun-Hyung Park, Junho Kim et al.

Masked language modeling (MLM) has been widely used for pre-training effective bidirectional representations, but incurs substantial training costs. In this paper, we propose a novel concept-based curriculum masking (CCM) method to efficiently pre-train a language model. CCM has two key differences from existing curriculum learning approaches to effectively reflect the nature of MLM. First, we introduce a carefully-designed linguistic difficulty criterion that evaluates the MLM difficulty of each token. Second, we construct a curriculum that gradually masks words related to the previously masked words by retrieving a knowledge graph. Experimental results show that CCM significantly improves pre-training efficiency. Specifically, the model trained with CCM shows comparative performance with the original BERT on the General Language Understanding Evaluation benchmark at half of the training cost.

Eojin Jeon, Mingyu Lee, Juhyeong Park et al.

Biases in the dataset often enable the model to achieve high performance on in-distribution data, while poorly performing on out-of-distribution data. To mitigate the detrimental effect of the bias on the networks, previous works have proposed debiasing methods that down-weight the biased examples identified by an auxiliary model, which is trained with explicit bias labels. However, finding a type of bias in datasets is a costly process. Therefore, recent studies have attempted to make the auxiliary model biased without the guidance (or annotation) of bias labels, by constraining the model's training environment or the capability of the model itself. Despite the promising debiasing results of recent works, the multi-class learning objective, which has been naively used to train the auxiliary model, may harm the bias mitigation effect due to its regularization effect and competitive nature across classes. As an alternative, we propose a new debiasing framework that introduces binary classifiers between the auxiliary model and the main model, coined bias experts. Specifically, each bias expert is trained on a binary classification task derived from the multi-class classification task via the One-vs-Rest approach. Experimental results demonstrate that our proposed strategy improves the bias identification ability of the auxiliary model. Consequently, our debiased model consistently outperforms the state-of-the-art on various challenge datasets.

Mingyu Lee, Myeongjin Shin, Junseo Lee et al.

One of the most promising applications in the era of Noisy Intermediate-Scale Quantum (NISQ) computing is quantum generative adversarial networks (QGANs), which offer significant quantum advantages over classical machine learning in various domains. However, QGANs suffer from mode collapse and lack explicit control over the features of generated outputs. To overcome these limitations, we propose InfoQGAN, a novel quantum-classical hybrid generative adversarial network that integrates the principles of InfoGAN with a QGAN architecture. Our approach employs a variational quantum circuit for data generation, a classical discriminator, and a Mutual Information Neural Estimator (MINE) to explicitly optimize the mutual information between latent codes and generated samples. Numerical simulations on synthetic 2D distributions and Iris dataset augmentation demonstrate that InfoQGAN effectively mitigates mode collapse while achieving robust feature disentanglement in the quantum generator. By leveraging these advantages, InfoQGAN not only enhances training stability but also improves data augmentation performance through controlled feature generation. These results highlight the potential of InfoQGAN as a foundational approach for advancing quantum generative modeling in the NISQ era.