Sensitive Data Detection with High-Throughput Neural Network Models for Financial Institutions
This work provides a practical evaluation of sensitive data detection methods for financial institutions, addressing the lack of publicly available datasets for this specific domain.
This paper addresses the challenge of detecting Nonpublic Personally Identifiable (NPI) information in financial institutions, evaluating character-level neural network models like CNN, LSTM, BiLSTM-CRF, and CNN-CRF on both unstructured and structured data. The CNN model was found to be the most effective in terms of accuracy and throughput for production deployment.
Named Entity Recognition has been extensively investigated in many fields. However, the application of sensitive entity detection for production systems in financial institutions has not been well explored due to the lack of publicly available, labeled datasets. In this paper, we use internal and synthetic datasets to evaluate various methods of detecting NPI (Nonpublic Personally Identifiable) information commonly found within financial institutions, in both unstructured and structured data formats. Character-level neural network models including CNN, LSTM, BiLSTM-CRF, and CNN-CRF are investigated on two prediction tasks: (i) entity detection on multiple data formats, and (ii) column-wise entity prediction on tabular datasets. We compare these models with other standard approaches on both real and synthetic data, with respect to F1-score, precision, recall, and throughput. The real datasets include internal structured data and public email data with manually tagged labels. Our experimental results show that the CNN model is simple yet effective with respect to accuracy and throughput and thus, is the most suitable candidate model to be deployed in the production environment(s). Finally, we provide several lessons learned on data limitations, data labelling and the intrinsic overlap of data entities.