Maksim Belyaev

Andrei Velichko, Mehmet Tahir Huyut, Maksim Belyaev et al.

Healthcare digitalization requires effective applications of human sensors, when various parameters of the human body are instantly monitored in everyday life due to the Internet of Things (IoT). In particular, machine learning (ML) sensors for the prompt diagnosis of COVID-19 are an important option for IoT application in healthcare and ambient assisted living (AAL). Determining a COVID-19 infected status with various diagnostic tests and imaging results is costly and time-consuming. This study provides a fast, reliable and cost-effective alternative tool for the diagnosis of COVID-19 based on the routine blood values (RBVs) measured at admission. The dataset of the study consists of a total of 5296 patients with the same number of negative and positive COVID-19 test results and 51 routine blood values. In this study, 13 popular classifier machine learning models and the LogNNet neural network model were exanimated. The most successful classifier model in terms of time and accuracy in the detection of the disease was the histogram-based gradient boosting (HGB) (accuracy: 100%, time: 6.39 sec). The HGB classifier identified the 11 most important features (LDL, cholesterol, HDL-C, MCHC, triglyceride, amylase, UA, LDH, CK-MB, ALP and MCH) to detect the disease with 100% accuracy. In addition, the importance of single, double and triple combinations of these features in the diagnosis of the disease was discussed. We propose to use these 11 features and their binary combinations as important biomarkers for ML sensors in the diagnosis of the disease, supporting edge computing on Arduino and cloud IoT service.

Mehmet Tahir Huyut, Andrei Velichko, Maksim Belyaev

Early evaluation of patients who require special care and who have high death-expectancy in COVID-19, and the effective determination of relevant biomarkers on large sample-groups are important to reduce mortality. This study aimed to reveal the routine blood-value predictors of COVID-19 mortality and to determine the lethal-risk levels of these predictors during the disease process. The dataset of the study consists of 38 routine blood-values of 2597 patients who died (n = 233) and those who recovered (n = 2364) from COVID-19 in August-December, 2021. In this study, the histogram-based gradient-boosting (HGB) model was the most successful machine-learning classifier in detecting living and deceased COVID-19 patients (with squared F1 metrics F1^2 = 1). The most efficient binary combinations with procalcitonin were obtained with D-dimer, ESR, D-Bil and ferritin. The HGB model operated with these feature pairs correctly detected almost all of the patients who survived and those who died (precision > 0.98, recall > 0.98, F1^2 > 0.98). Furthermore, in the HGB model operated with a single feature, the most efficient features were procalcitonin (F1^2 = 0.96) and ferritin (F1^2 = 0.91). In addition, according to the two-threshold approach, ferritin values between 376.2 mkg/L and 396.0 mkg/L (F1^2 = 0.91) and pro-calcitonin values between 0.2 mkg/L and 5.2 mkg/L (F1^2 = 0.95) were found to be fatal risk levels for COVID-19. Considering all the results, we suggest that many features combined with these features, especially procalcitonin and ferritin, operated with the HGB model, can be used to achieve very successful results in the classification of those who live, and those who die from COVID-19. Moreover, we strongly recommend that clinicians consider the critical levels we have found for procalcitonin and ferritin properties, to reduce the lethality of the COVID-19 disease.

Andrei Velichko, Maksim Belyaev, Yuriy Izotov et al.

Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classification of special datasets in relation to the entropy of the time series recorded in the reservoir of the neural network. NNetEn estimates the chaotic dynamics of time series in an original way and does not take into account probability distribution functions. We propose two new classification metrics: R2 Efficiency and Pearson Efficiency. The efficiency of NNetEn is verified on separation of two chaotic time series of sine mapping using dispersion analysis. For two close dynamic time series (r = 1.1918 and r = 1.2243), the F-ratio has reached the value of 124 and reflects high efficiency of the introduced method in classification problems. The electroenceph-alography signal classification for healthy persons and patients with Alzheimer disease illustrates the practical application of the NNetEn features. Our computations demonstrate the synergistic effect of increasing classification accuracy when applying traditional entropy measures and the NNetEn concept conjointly. An implementation of the algorithms in Python is presented.

Andrei Velichko, Maksim Belyaev, Matthias P. Wagner et al.

Approximation of entropies of various types using machine learning (ML) regression methods are shown for the first time. The ML models presented in this study define the complexity of the short time series by approximating dissimilar entropy techniques such as Singular value decomposition entropy (SvdEn), Permutation entropy (PermEn), Sample entropy (SampEn) and Neural Network entropy (NNetEn) and their 2D analogies. A new method for calculating SvdEn2D, PermEn2D and SampEn2D for 2D images was tested using the technique of circular kernels. Training and testing datasets on the basis of Sentinel-2 images are presented (two training images and one hundred and ninety-eight testing images). The results of entropy approximation are demonstrated using the example of calculating the 2D entropy of Sentinel-2 images and R^2 metric evaluation. The applicability of the method for the short time series with a length from N = 5 to N = 113 elements is shown. A tendency for the R^2 metric to decrease with an increase in the length of the time series was found. For SvdEn entropy, the regression accuracy is R^2 > 0.99 for N = 5 and R^2 > 0.82 for N = 113. The best metrics were observed for the ML_SvdEn2D and ML_NNetEn2D models. The results of the study can be used for fundamental research of entropy approximations of various types using ML regression, as well as for accelerating entropy calculations in remote sensing. The versatility of the model is shown on a synthetic chaotic time series using Planck map and logistic map.

Andrei Velichko, Petr Boriskov, Maksim Belyaev et al.

The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2 ~ 0.9. The Hindmarsh-Rose spike model was used to generate time series of spike intervals, and datasets for training and testing the perceptron. The selection of the hyperparameters of the perceptron model and the estimation of the sensor accuracy were performed using the K-block cross-validation method. Even for a hidden layer with one neuron, the model approximates the fuzzy entropy with good results and the metric R2 ~ 0.5-0.8. In a simplified model with one neuron and equal weights in the first layer, the principle of approximation is based on the linear transformation of the average value of the time series into the entropy value. An example of using the chaos sensor on spike train of action potential recordings from the L5 dorsal rootlet of rat is provided. The bio-inspired chaos sensor model based on an ensemble of neurons is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing. The study will be useful for specialists in the field of computational neuroscience, and also to create humanoid and animal robots, and bio-robots with limited resources.

Mikhail Borisenkov, Maksim Belyaev, Nithya Rekha Sivakumar et al.

Wearable sensors and IoT/IoMT platforms enable continuous, real-time monitoring, but objective digital markers for eating disorders are limited. In this study, we examined whether actimetry and machine learning (ML) could provide objective criteria for food addiction (FA) and symptom counts (SC). In 78 participants (mean age 22.1 +/- 9.5 y; 73.1% women), one week of non-dominant wrist actimetry and psychometric data (YFAS, DEBQ, ZSDS) were collected. The time series were segmented into daytime activity and nighttime rest, and statistical and entropy descriptors (FuzzyEn, DistEn, SVDEn, PermEn, PhaseEn; 256 features) were calculated. The mean Matthews correlation coefficient (MCC) was used as the primary metric in a K-nearest neighbors (KNN) pipeline with five-fold stratified cross-validation (one hundred repetitions; 500 evaluations); SHAP was used to assist in interpretation. For binary FA, activity-segment features performed best (MCC = 0.78 +/- 0.02; Accuracy ~ 95.3% +/- 0.5; Sensitivity ~ 0.77 +/- 0.03; Specificity ~ 0.98 +/- 0.004), exceeding OaS (Objective and Subjective Features) (MCC = 0.69 +/- 0.03) and rest-only (MCC = 0.50 +/- 0.03). For SC (four classes), OaS slightly surpassed actimetry (MCC = 0.40 +/- 0.01 vs 0.38 +/- 0.01; Accuracy ~ 58.1% vs 56.9%). Emotional and restrained eating were correlated with actimetric features. These findings support wrist-worn actimetry as a digital biomarker of FA that complements questionnaires and may facilitate privacy-preserving clinical translation.

Maksim Belyaev, Murugappan Murugappan, Andrei Velichko et al.

The study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) in an Internet of Things (IoT) environment using rest-state EEG signals (rs-EEG). We computed different types of entropy from EEG signals and found that Fuzzy Entropy performed the best in diagnosing and monitoring PD using rs-EEG. We also investigated different combinations of signal frequency ranges and EEG channels to accurately diagnose PD. Finally, with a fewer number of features (11 features), we achieved a maximum classification accuracy (ARKF) of ~99.9%. The most prominent frequency range of EEG signals has been identified, and we have found that high classification accuracy depends on low-frequency signal components (0-4 Hz). Moreover, the most informative signals were mainly received from the right hemisphere of the head (F8, P8, T8, FC6). Furthermore, we assessed the accuracy of the diagnosis of PD using three different lengths of EEG data (150-1000 samples). Because the computational complexity is reduced by reducing the input data. As a result, we have achieved a maximum mean accuracy of 99.9% for a sample length (LEEG) of 1000 (~7.8 seconds), 98.2% with a LEEG of 800 (~6.2 seconds), and 79.3% for LEEG = 150 (~1.2 seconds). By reducing the number of features and segment lengths, the computational cost of classification can be reduced. Lower-performance smart ML sensors can be used in IoT environments for enhances human resilience to PD.

Irewola Aaron Oludehinwa, Andrei Velichko, Maksim Belyaev et al.

This study proposes an enhancement to the existing method for detecting Solar Active Regions (ARs). Our technique tracks ARs using images from the Atmospheric Imaging Assembly (AIA) of NASA's Solar Dynamics Observatory (SDO). It involves a 2D circular kernel time series transformation, combined with Statistical and Entropy measures, and a Machine Learning (ML) approach. The technique transforms the circular area around pixels in the SDO AIA images into one-dimensional time series (1-DTS). Statistical measures (Median Value, Xmed; 95th Percentile, X95) and Entropy measures (Distribution Entropy, DisEn; Fuzzy Entropy, FuzzyEn) are used as feature selection methods (FSM 1), alongside a method applying 1-DTS elements directly as features (FSM 2). The ML algorithm classifies these series into three categories: no Active Region (nARs type 1, class 1), non-flaring Regions outside active regions with brightness (nARs type 2, class 2), and flaring Active Regions (ARs, class 3). The ML model achieves a classification accuracy of 0.900 and 0.914 for Entropy and Statistical measures, respectively. Notably, Fuzzy Entropy shows the highest classification accuracy (AKF=0.895), surpassing DisEn (AKF=0.738), X95 (AKF=0.873), and Xmed (AKF=0.840). This indicates the high effectiveness of Entropy and Statistical measures for AR detection in SDO AIA images. FSM 2 captures a similar distribution of flaring AR activities as FSM 1. Additionally, we introduce a generalizing characteristic of AR activities (GSA), finding a direct agreement between increased AR activities and higher GSA values. The Python code implementation of the proposed method is available in supplementary material.

Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen et al.

In the present paper, we report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in oscillatory neural networks. Based on these results, we further select an adequate SPICE model to describe the modes of operation of coupled oscillator circuits. Physical mechanisms influencing the time of forward and reverse electrical switching, that determine the applicability limits of the proposed model, are identified. For the resistive coupling, it is shown that synchronization takes place at a certain value of the coupling resistance, though it is unstable and a synchronization failure occurs periodically. For the capacitive coupling, two synchronization modes, with weak and strong coupling, are found. The transition between these modes is accompanied by chaotic oscillations. A decrease in the width of the spectrum harmonics in the weak-coupling mode, and its increase in the strong-coupling one, is detected. The dependences of frequencies and phase differences of the coupled oscillatory circuits on the coupling capacitance are found. Examples of operation of coupled VO2 oscillators as a central pattern generator are demonstrated.

Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen et al.

We explore a prototype of an oscillatory neural network (ONN) based on vanadium dioxide switching devices. The model system under study represents two oscillators based on thermally coupled VO2 switches. Numerical simulation shows that the effective action radius RTC of coupling depends both on the total energy released during switching and on the average power. It is experimentally and numerically proved that the temperature change dT commences almost synchronously with the released power peak and T-coupling reveals itself up to a frequency of about 10 kHz. For the studied switching structure configuration, the RTC value varies over a wide range from 4 to 45 mkm, depending on the external circuit capacitance C and resistance Ri, but the variation of Ri is more promising from the practical viewpoint. In the case of a "weak" coupling, synchronization is accompanied by attraction effect and decrease of the main spectra harmonics width. In the case of a "strong" coupling, the number of effects increases, synchronization can occur on subharmonics resulting in multilevel stable synchronization of two oscillators. An advanced algorithm for synchronization efficiency and subharmonic ratio calculation is proposed. It is shown that of the two oscillators the leading one is that with a higher main frequency, and, in addition, the frequency stabilization effect is observed. Also, in the case of a strong thermal coupling, the limit of the supply current parameters, for which the oscillations exist, expands by ~ 10 %. The obtained results have a universal character and open up a new kind of coupling in ONNs, namely, T-coupling, which allows for easy transition from 2D to 3D integration. The effect of subharmonic synchronization hold promise for application in classification and pattern recognition.

Andrei Velichko, Vadim Putrolaynen, Maksim Belyaev

In the circuit of two thermally coupled VO2 oscillators, we studied a higher order synchronization effect, which can be used in object classification techniques to increase the number of possible synchronous states of the oscillator system. We developed the phase-locking estimation method to determine the values of subharmonic ratio and synchronization effectiveness. In our experiment, the number of possible synchronous states of the oscillator system was twelve, and subharmonic ratio distributions were shaped as Arnold's tongues. In the model, the number of states may reach the maximum value of 150 at certain levels of coupling strength and noise. The long-range synchronization effect in a one-dimensional chain of oscillators occurs even at low values of synchronization effectiveness for intermediate links. We demonstrate a technique for storing and recognizing vector images, which can used for reservoir computing. In addition, we present the implementation of analog operation of multiplication, the synchronization based logic for binary computations, and the possibility to develop the interface between spike neural network and a computer. Based on the universal physical effects, the high order synchronization can be applied to any spiking oscillators with any coupling type, enhancing the practical value of the presented results to expand spike neural network capabilities.