Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program
This addresses the labor-intensive process for admission committees by automating leadership assessment, though it is incremental as it applies existing NLP methods to a specific domain.
The study tackled the time-consuming review of letters of recommendation (LORs) in online master's program admissions by developing LORI, an AI-based tool for assessing leadership skills, achieving a weighted F1 score of 91.6%, precision of 92.4%, and recall of 91.6%.
Letters of recommendation (LORs) provide valuable insights into candidates' capabilities and experiences beyond standardized test scores. However, reviewing these text-heavy materials is time-consuming and labor-intensive. To address this challenge and support the admission committee in providing feedback for students' professional growth, our study introduces LORI: LOR Insights, a novel AI-based detection tool for assessing leadership skills in LORs submitted by online master's program applicants. By employing natural language processing and leveraging large language models using RoBERTa and LLAMA, we seek to identify leadership attributes such as teamwork, communication, and innovation. Our latest RoBERTa model achieves a weighted F1 score of 91.6%, a precision of 92.4%, and a recall of 91.6%, showing a strong level of consistency in our test data. With the growing importance of leadership skills in the STEM sector, integrating LORI into the graduate admissions process is crucial for accurately assessing applicants' leadership capabilities. This approach not only streamlines the admissions process but also automates and ensures a more comprehensive evaluation of candidates' capabilities.