An Open Source Computer Vision and Machine Learning Framework for Affordable Life Science Robotic Automation
This work addresses the need for affordable robotic automation in life sciences, though it is incremental as it combines existing methods like U-net and Mixture Density Networks in a new framework.
The paper tackled the problem of enabling low-cost laboratory automation tasks like colony picking and liquid handling by developing an open-source robotic framework that integrates computer vision and machine learning for inverse kinematics, achieving mean positional error below 1 mm, joint angle prediction errors below 4 degrees, and colony detection with IoU score of 0.537 and Dice coefficient of 0.596.
We present an open-source robotic framework that integrates computer vision and machine learning based inverse kinematics to enable low-cost laboratory automation tasks such as colony picking and liquid handling. The system uses a custom trained U-net model for semantic segmentation of microbial cultures, combined with Mixture Density Network for predicating joint angles of a simple 5-DOF robot arm. We evaluated the framework using a modified robot arm, upgraded with a custom liquid handling end-effector. Experimental results demonstrate the framework's feasibility for precise, repeatable operations, with mean positional error below 1 mm and joint angle prediction errors below 4 degrees and colony detection capabilities with IoU score of 0.537 and Dice coefficient of 0.596.