Han Qin

Christian J. Mahoney, Katie Jensen, Fusheng Wei et al.

In eDiscovery, it is critical to ensure that each page produced in legal proceedings conforms with the requirements of court or government agency production requests. Errors in productions could have severe consequences in a case, putting a party in an adverse position. The volume of pages produced continues to increase, and tremendous time and effort has been taken to ensure quality control of document productions. This has historically been a manual and laborious process. This paper demonstrates a novel automated production quality control application which leverages deep learning-based image recognition technology to extract Bates Number and Confidentiality Stamping from legal case production images and validate their correctness. Effectiveness of the method is verified with an experiment using a real-world production data.

Robert Keeling, Rishi Chhatwal, Nathaniel Huber-Fliflet et al.

Research has shown that Convolutional Neural Networks (CNN) can be effectively applied to text classification as part of a predictive coding protocol. That said, most research to date has been conducted on data sets with short documents that do not reflect the variety of documents in real world document reviews. Using data from four actual reviews with documents of varying lengths, we compared CNN with other popular machine learning algorithms for text classification, including Logistic Regression, Support Vector Machine, and Random Forest. For each data set, classification models were trained with different training sample sizes using different learning algorithms. These models were then evaluated using a large randomly sampled test set of documents, and the results were compared using precision and recall curves. Our study demonstrates that CNN performed well, but that there was no single algorithm that performed the best across the combination of data sets and training sample sizes. These results will help advance research into the legal profession's use of machine learning algorithms that maximize performance.

Han Qin, Kit Riehle, Haozhen Zhao

Web traffic is a valuable data source, typically used in the marketing space to track brand awareness and advertising effectiveness. However, web traffic is also a rich source of information for cybersecurity monitoring efforts. To better understand the threat of malicious cyber actors, this study develops a methodology to monitor and evaluate web activity using data archived from Google Analytics. Google Analytics collects and aggregates web traffic, including information about web visitors' location, date and time of visit, visited webpages, and searched keywords. This study seeks to streamline analysis of this data and uses rule-based anomaly detection and predictive modeling to identify web traffic that deviates from normal patterns. Rather than evaluating pieces of web traffic individually, the methodology seeks to emulate real user behavior by creating a new unit of analysis: the user session. User sessions group individual pieces of traffic from the same location and date, which transforms the available information from single point-in-time snapshots to dynamic sessions showing users' trajectory and intent. The result is faster and better insight into large volumes of noisy web traffic.