Jieli Zhou

Jieli Zhou, Baoyu Jing, Zeya Wang

Due to the shortage of COVID-19 viral testing kits and the long waiting time, radiology imaging is used to complement the screening process and triage patients into different risk levels. Deep learning based methods have taken an active role in automatically detecting COVID-19 disease in chest x-ray images, as witnessed in many recent works in early 2020. Most of these works first train a Convolutional Neural Network (CNN) on an existing large-scale chest x-ray image dataset and then fine-tune it with a COVID-19 dataset at a much smaller scale. However, direct transfer across datasets from different domains may lead to poor performance for CNN due to two issues, the large domain shift present in the biomedical imaging datasets and the extremely small scale of the COVID-19 chest x-ray dataset. In an attempt to address these two important issues, we formulate the problem of COVID-19 chest x-ray image classification in a semi-supervised open set domain adaptation setting and propose a novel domain adaptation method, Semi-supervised Open set Domain Adversarial network (SODA). SODA is able to align the data distributions across different domains in a general domain space and also in a common subspace of source and target data. In our experiments, SODA achieves a leading classification performance compared with recent state-of-the-art models in separating COVID-19 with common pneumonia. We also present initial results showing that SODA can produce better pathology localizations in the chest x-rays.

Jieli Zhou, Yuntao Zhou, Yi Xu

In recent years, with the rapid proliferation of research publications in the field of Artificial Intelligence, it is becoming increasingly difficult for researchers to effectively keep up with all the latest research in one's own domains. However, history has shown that scientific breakthroughs often come from collaborations of researchers from different domains. Traditional search algorithms like Lexical search, which look for literal matches or synonyms and variants of the query words, are not effective for discovering cross-domain research papers and meeting the needs of researchers in this age of information overflow. In this paper, we developed and tested an innovative semantic search engine, Analogy Search Engine (ASE), for 2000 AI research paper abstracts across domains like Language Technologies, Robotics, Machine Learning, Computational Biology, Human Computer Interactions, etc. ASE combines recent theories and methods from Computational Analogy and Natural Language Processing to go beyond keyword-based lexical search and discover the deeper analogical relationships among research paper abstracts. We experimentally show that ASE is capable of finding more interesting and useful research papers than baseline elasticsearch. Furthermore, we believe that the methods used in ASE go beyond academic paper and will benefit many other document search tasks.

Xin Qian, Ziyi Zhong, Jieli Zhou

Multimodal machine translation is one of the applications that integrates computer vision and language processing. It is a unique task given that in the field of machine translation, many state-of-the-arts algorithms still only employ textual information. In this work, we explore the effectiveness of reinforcement learning in multimodal machine translation. We present a novel algorithm based on the Advantage Actor-Critic (A2C) algorithm that specifically cater to the multimodal machine translation task of the EMNLP 2018 Third Conference on Machine Translation (WMT18). We experiment our proposed algorithm on the Multi30K multilingual English-German image description dataset and the Flickr30K image entity dataset. Our model takes two channels of inputs, image and text, uses translation evaluation metrics as training rewards, and achieves better results than supervised learning MLE baseline models. Furthermore, we discuss the prospects and limitations of using reinforcement learning for machine translation. Our experiment results suggest a promising reinforcement learning solution to the general task of multimodal sequence to sequence learning.