Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events
This provides a software utility for researchers and practitioners in domains like healthcare to adopt foundation models for event sequences, addressing a lack of existing tools, though it is incremental as it builds on established GPT frameworks.
The paper tackles the challenge of applying generative pre-trained transformers (GPTs) to continuous-time sequences of complex events, such as medical records, by introducing Event Stream GPT (ESGPT), an open-source library that streamlines dataset construction, modeling, and evaluation, resulting in a tool that enables few- and zero-shot performance assessment on user-specified tasks.
Generative, pre-trained transformers (GPTs, a.k.a. "Foundation Models") have reshaped natural language processing (NLP) through their versatility in diverse downstream tasks. However, their potential extends far beyond NLP. This paper provides a software utility to help realize this potential, extending the applicability of GPTs to continuous-time sequences of complex events with internal dependencies, such as medical record datasets. Despite their potential, the adoption of foundation models in these domains has been hampered by the lack of suitable tools for model construction and evaluation. To bridge this gap, we introduce Event Stream GPT (ESGPT), an open-source library designed to streamline the end-to-end process for building GPTs for continuous-time event sequences. ESGPT allows users to (1) build flexible, foundation-model scale input datasets by specifying only a minimal configuration file, (2) leverage a Hugging Face compatible modeling API for GPTs over this modality that incorporates intra-event causal dependency structures and autoregressive generation capabilities, and (3) evaluate models via standardized processes that can assess few and even zero-shot performance of pre-trained models on user-specified fine-tuning tasks.