Design of reliable technology valuation model with calibrated machine learning of patent indicators
This work addresses the need for trustworthy ML models in technology valuation for experts in academia and industry, though it is incremental as it builds on existing ML methods with calibration.
The authors tackled the problem of unreliable confidence estimates in machine learning models for patent valuation by proposing a calibrated ML framework that provides robust confidence levels, achieving improved reliability and accuracy as demonstrated through a case study with specific metrics like expected calibration error and F1-scores.
Machine learning (ML) has revolutionized the digital transformation of technology valuation by predicting the value of patents with high accuracy. However, the lack of validation regarding the reliability of these models hinders experts from fully trusting the confidence of model predictions. To address this issue, we propose an analytical framework for reliable technology valuation using calibrated ML models, which provide robust confidence levels in model predictions. We extract quantitative patent indicators that represent various technology characteristics as input data, using the patent maintenance period as a proxy for technology values. Multiple ML models are developed to capture the nonlinear relationship between patent indicators and technology value. The reliability and accuracy of these models are evaluated, presenting a Pareto-front map where the expected calibration error, Matthews correlation coefficient and F1-scores are compared. After identifying the best-performing model, we apply SHapley Additive exPlanation (SHAP) analysis to pinpoint the most significant input features by confidence bin. Through a case study, we confirmed that the proposed approach offers a practical guideline for developing reliable and accurate ML-based technology valuation models, with significant implications for both academia and industry.