A Virtual Member of a Community of Practice for the Society of Petroleum Engineers: From Prototype to Deployment
This work provides an incremental improvement in knowledge management and task support for petroleum engineers, specifically within the oil and gas sector.
This paper details the development of ATHENA, a virtual assistant for the Society of Petroleum Engineers, which significantly improved productivity and performance equality for well-planning tasks compared to a state-of-the-art RAG baseline system. Subsequent enhancements in multi-document retrieval, answer validation, and proactive dissemination further improved its support for knowledge-intensive tasks.
We describe the evolution of a virtual assistant, called ATHENA, designed to support the capture, retrieval, and dissemination of knowledge for members of a Community of Practice (CoP) related to the Oil and Gas sector. An evaluation of a first prototype involving 75 professionals from the Society of Petroleum Engineering (SPE) showed that ATHENA dramatically improved both their productivity and performance equality on a set of realistic well-planning tasks compare to their use of a state-of-the-art RAG baseline system. However, the evaluation also identified areas for improvement. This paper describes technical advances to our first prototype in the areas of multi-document retrieval, support for answer validation, and more focused proactive dissemination. Evaluation results show that this enhanced version of ATHENA provides better support for completing knowledge-intensive tasks related to well planning than does a state-of-the-art baseline. ATHENA has been integrated into the SPE Research Portal and is being deployed for use by the society's membership.