SAINE: Scientific Annotation and Inference Engine of Scientific Research
This work aims to enhance transparency and understanding in scientific research for meta-science projects, but it appears incremental as it builds on existing open-source software and previous hierarchical classification work.
The authors tackled the problem of improving classification accuracy for scholarly publications by developing SAINE, an annotation and inference engine built on open-source tools like Label Studio and MLflow, and demonstrated its application in understanding scholarly publication spaces with user studies showing it aids in better understanding the classification process.
We present SAINE, an Scientific Annotation and Inference ENgine based on a set of standard open-source software, such as Label Studio and MLflow. We show that our annotation engine can benefit the further development of a more accurate classification. Based on our previous work on hierarchical discipline classifications, we demonstrate its application using SAINE in understanding the space for scholarly publications. The user study of our annotation results shows that user input collected with the help of our system can help us better understand the classification process. We believe that our work will help to foster greater transparency and better understand scientific research. Our annotation and inference engine can further support the downstream meta-science projects. We welcome collaboration and feedback from the scientific community on these projects. The demonstration video can be accessed from https://youtu.be/yToO-G9YQK4. A live demo website is available at https://app.heartex.com/user/signup/?token=e2435a2f97449fa1 upon free registration.