Alexei Botchkarev

Alexei Botchkarev

Performance metrics (error measures) are vital components of the evaluation frameworks in various fields. The intention of this study was to overview of a variety of performance metrics and approaches to their classification. The main goal of the study was to develop a typology that will help to improve our knowledge and understanding of metrics and facilitate their selection in machine learning regression, forecasting and prognostics. Based on the analysis of the structure of numerous performance metrics, we propose a framework of metrics which includes four (4) categories: primary metrics, extended metrics, composite metrics, and hybrid sets of metrics. The paper identified three (3) key components (dimensions) that determine the structure and properties of primary metrics: method of determining point distance, method of normalization, method of aggregation of point distances over a data set.

Alexei Botchkarev

Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine Learning Studio tool for rapid assessment of multiple types of regression models. The tool offers environment for comparing 14 types of regression models in a unified experiment: linear regression, Bayesian linear regression, decision forest regression, boosted decision tree regression, neural network regression, Poisson regression, Gaussian processes for regression, gradient boosted machine, nonlinear least squares regression, projection pursuit regression, random forest regression, robust regression, robust regression with mm-type estimators, support vector regression. The tool presents assessment results arranged by model accuracy in a single table using five performance metrics. Evaluation of regression machine learning models for performing hospital case cost prediction demonstrated advantage of robust regression model, boosted decision tree regression and decision forest regression. The operational tool has been published to the web and openly available for experiments and extensions.

Alexei Botchkarev, Patrick Finnigan

Complexity is an inherent attribute of any project. The purpose of defining and documenting complexity is to have an early warning tool allowing a project team to focus on certain areas and aspects of the project in order to prevent and alleviate future risks and issues caused by this complexity. The main contribution of this paper is to present a systematic view of complexity in project management by identifying its key attributes and classifying complexity by these attributes. A "complexity taxonomy", based on a survey of the existing complexity literature, is developed and discussed including the product, project, and external environment dimensions. We show how complexity types are described through simple real life examples and business cases. Then we develop a framework (tool) for applying the notion of complexity as an early warning tool for a project manager in order to timely foresee future risks and problems. The paper is intended for researchers in complexity, project management, information systems, technology solutions and business management, and also for information specialists, project managers, program managers, financial staff and technology directors.