Andrei Velichko

Andrei Velichko, Mehmet Tahir Huyut, Maksim Belyaev et al.

Healthcare digitalization requires effective applications of human sensors, when various parameters of the human body are instantly monitored in everyday life due to the Internet of Things (IoT). In particular, machine learning (ML) sensors for the prompt diagnosis of COVID-19 are an important option for IoT application in healthcare and ambient assisted living (AAL). Determining a COVID-19 infected status with various diagnostic tests and imaging results is costly and time-consuming. This study provides a fast, reliable and cost-effective alternative tool for the diagnosis of COVID-19 based on the routine blood values (RBVs) measured at admission. The dataset of the study consists of a total of 5296 patients with the same number of negative and positive COVID-19 test results and 51 routine blood values. In this study, 13 popular classifier machine learning models and the LogNNet neural network model were exanimated. The most successful classifier model in terms of time and accuracy in the detection of the disease was the histogram-based gradient boosting (HGB) (accuracy: 100%, time: 6.39 sec). The HGB classifier identified the 11 most important features (LDL, cholesterol, HDL-C, MCHC, triglyceride, amylase, UA, LDH, CK-MB, ALP and MCH) to detect the disease with 100% accuracy. In addition, the importance of single, double and triple combinations of these features in the diagnosis of the disease was discussed. We propose to use these 11 features and their binary combinations as important biomarkers for ML sensors in the diagnosis of the disease, supporting edge computing on Arduino and cloud IoT service.

Mehmet Tahir Huyut, Andrei Velichko

Since February 2020, the world has been engaged in an intense struggle with the COVID-19 dis-ease, and health systems have come under tragic pressure as the disease turned into a pandemic. The aim of this study is to obtain the most effective routine blood values (RBV) in the diagnosis and prognosis of COVID-19 using a backward feature elimination algorithm for the LogNNet reservoir neural network. The first dataset in the study consists of a total of 5296 patients with the same number of negative and positive COVID-19 tests. The LogNNet-model achieved the accuracy rate of 99.5% in the diagnosis of the disease with 46 features and the accuracy of 99.17% with only mean corpuscular hemoglobin concentration, mean corpuscular hemoglobin, and activated partial prothrombin time. The second dataset consists of a total of 3899 patients with a diagnosis of COVID-19 who were treated in hospital, of which 203 were severe patients and 3696 were mild patients. The model reached the accuracy rate of 94.4% in determining the prognosis of the disease with 48 features and the accuracy of 82.7% with only erythrocyte sedimentation rate, neutrophil count, and C reactive protein features. Our method will reduce the negative pressures on the health sector and help doctors to understand the pathogenesis of COVID-19 using the key features. The method is promising to create mobile health monitoring systems in the Internet of Things.

Mehmet Tahir Huyut, Andrei Velichko, Maksim Belyaev

Early evaluation of patients who require special care and who have high death-expectancy in COVID-19, and the effective determination of relevant biomarkers on large sample-groups are important to reduce mortality. This study aimed to reveal the routine blood-value predictors of COVID-19 mortality and to determine the lethal-risk levels of these predictors during the disease process. The dataset of the study consists of 38 routine blood-values of 2597 patients who died (n = 233) and those who recovered (n = 2364) from COVID-19 in August-December, 2021. In this study, the histogram-based gradient-boosting (HGB) model was the most successful machine-learning classifier in detecting living and deceased COVID-19 patients (with squared F1 metrics F1^2 = 1). The most efficient binary combinations with procalcitonin were obtained with D-dimer, ESR, D-Bil and ferritin. The HGB model operated with these feature pairs correctly detected almost all of the patients who survived and those who died (precision > 0.98, recall > 0.98, F1^2 > 0.98). Furthermore, in the HGB model operated with a single feature, the most efficient features were procalcitonin (F1^2 = 0.96) and ferritin (F1^2 = 0.91). In addition, according to the two-threshold approach, ferritin values between 376.2 mkg/L and 396.0 mkg/L (F1^2 = 0.91) and pro-calcitonin values between 0.2 mkg/L and 5.2 mkg/L (F1^2 = 0.95) were found to be fatal risk levels for COVID-19. Considering all the results, we suggest that many features combined with these features, especially procalcitonin and ferritin, operated with the HGB model, can be used to achieve very successful results in the classification of those who live, and those who die from COVID-19. Moreover, we strongly recommend that clinicians consider the critical levels we have found for procalcitonin and ferritin properties, to reduce the lethality of the COVID-19 disease.

Andrei Velichko, Maksim Belyaev, Yuriy Izotov et al.

Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classification of special datasets in relation to the entropy of the time series recorded in the reservoir of the neural network. NNetEn estimates the chaotic dynamics of time series in an original way and does not take into account probability distribution functions. We propose two new classification metrics: R2 Efficiency and Pearson Efficiency. The efficiency of NNetEn is verified on separation of two chaotic time series of sine mapping using dispersion analysis. For two close dynamic time series (r = 1.1918 and r = 1.2243), the F-ratio has reached the value of 124 and reflects high efficiency of the introduced method in classification problems. The electroenceph-alography signal classification for healthy persons and patients with Alzheimer disease illustrates the practical application of the NNetEn features. Our computations demonstrate the synergistic effect of increasing classification accuracy when applying traditional entropy measures and the NNetEn concept conjointly. An implementation of the algorithms in Python is presented.

Andrei Velichko, Maksim Belyaev, Matthias P. Wagner et al.

Approximation of entropies of various types using machine learning (ML) regression methods are shown for the first time. The ML models presented in this study define the complexity of the short time series by approximating dissimilar entropy techniques such as Singular value decomposition entropy (SvdEn), Permutation entropy (PermEn), Sample entropy (SampEn) and Neural Network entropy (NNetEn) and their 2D analogies. A new method for calculating SvdEn2D, PermEn2D and SampEn2D for 2D images was tested using the technique of circular kernels. Training and testing datasets on the basis of Sentinel-2 images are presented (two training images and one hundred and ninety-eight testing images). The results of entropy approximation are demonstrated using the example of calculating the 2D entropy of Sentinel-2 images and R^2 metric evaluation. The applicability of the method for the short time series with a length from N = 5 to N = 113 elements is shown. A tendency for the R^2 metric to decrease with an increase in the length of the time series was found. For SvdEn entropy, the regression accuracy is R^2 > 0.99 for N = 5 and R^2 > 0.82 for N = 113. The best metrics were observed for the ML_SvdEn2D and ML_NNetEn2D models. The results of the study can be used for fundamental research of entropy approximations of various types using ML regression, as well as for accelerating entropy calculations in remote sensing. The versatility of the model is shown on a synthetic chaotic time series using Planck map and logistic map.

Andrei Velichko, Petr Boriskov, Maksim Belyaev et al.

The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2 ~ 0.9. The Hindmarsh-Rose spike model was used to generate time series of spike intervals, and datasets for training and testing the perceptron. The selection of the hyperparameters of the perceptron model and the estimation of the sensor accuracy were performed using the K-block cross-validation method. Even for a hidden layer with one neuron, the model approximates the fuzzy entropy with good results and the metric R2 ~ 0.5-0.8. In a simplified model with one neuron and equal weights in the first layer, the principle of approximation is based on the linear transformation of the average value of the time series into the entropy value. An example of using the chaos sensor on spike train of action potential recordings from the L5 dorsal rootlet of rat is provided. The bio-inspired chaos sensor model based on an ensemble of neurons is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing. The study will be useful for specialists in the field of computational neuroscience, and also to create humanoid and animal robots, and bio-robots with limited resources.

Andrei Velichko, N'Gbo N'Gbo, Bruno Carpentieri et al.

Estimating the largest Lyapunov exponent from a scalar time series is difficult when the governing equations, tangent dynamics, and full state vector are unavailable. We propose FEG-Pro, a forecast-error growth profiling framework for nonlinear scalar time series. The method constructs autocorrelation-guided sparse histories, performs distance-weighted k-nearest-neighbor multi-horizon forecasting, and analyzes the logarithmic growth of geometrically averaged forecast errors. Its primary output is the finite-horizon forecast-error growth slope, lambda_FEG. When the error-growth curve supports a quasi-linear regime, this slope can be compared with reference largest Lyapunov exponents as an estimate of the dominant instability rate. The same pipeline also extracts the formal fit-selection regime, curvature, residual roughness after quadratic detrending, monotonicity, and forecast-error distribution entropy (FEDE) from signed multi-horizon errors. These secondary descriptors are intended not only as diagnostic controls for the slope, but also as candidate machine-learning features for nonlinear signal analysis, because they encode profile geometry and distributional uncertainty not captured by lambda_FEG alone. We evaluate the method on chaotic maps, Mackey-Glass delay dynamics, and scalar Lorenz-63 observables with known or reference exponents. Full-record experiments show good agreement in quasi-linear cases and meaningful curve-shape information in curved or weak profiles. A dyadic length-halving experiment on representative logistic, Mackey-Glass, and Lorenz records shows that residual roughness and mean FEDE often change monotonically and remain interpretable as record length decreases, even when the slope becomes biased or highly variable. The results support treating forecast-error growth as a structured profile and feature-generation framework rather than a single-number estimator.

Bruno Carpentieri, Andrei Velichko, Mudassir Shams et al.

We propose an interpretable AI-assisted reliability diagnostic framework for parameterized root-finding schemes based on kNN-LLE proxy stability profiling and multi-horizon early prediction. The approach augments a numerical solver with a lightweight predictive layer that estimates solver reliability from short prefixes of iteration dynamics, enabling early identification of stable and unstable parameter regimes. For each configuration in the parameter space, raw and smoothed proxy profiles of a largest Lyapunov exponent (LLE) estimator are constructed, from which contractivity-based reliability scores summarizing finite-time convergence are derived. Machine learning models predict the reliability score from early segments of the proxy profile, allowing the framework to determine when solver dynamics become diagnostically informative. Experiments on a two-parameter parallel root-finding scheme show reliable prediction after only a few iterations: the best models achieve R^2=0.48 at horizon T=1, improve to R^2=0.67 by T=3, and exceed R^2=0.89 before the characteristic minimum-location scale of the stability profile. Prediction accuracy increases to R^2=0.96 at larger horizons, with mean absolute errors around 0.03, while inference costs remain negligible (microseconds per sample). The framework provides interpretable stability indicators and supports early decisions during solver execution, such as continuing, restarting, or adjusting parameters.

Mudassir Shams, Andrei Velichko, Bruno Carpentieri

Inverse parallel schemes remain indispensable tools for computing the roots of nonlinear systems, yet their dynamical behavior can be unexpectedly rich, ranging from strong contraction to oscillatory or chaotic transients depending on the choice of algorithmic parameters and initial states. A unified analytical-data-driven methodology for identifying, measuring, and reducing such instabilities in a family of uni-parametric inverse parallel solvers is presented in this study. On the theoretical side, we derive stability and bifurcation characterizations of the underlying iterative maps, identifying parameter regions associated with periodic or chaotic behavior. On the computational side, we introduce a micro-series pipeline based on kNN-driven estimation of the local largest Lyapunov exponent (LLE), applied to scalar time series derived from solver trajectories. The resulting sliding-window Lyapunov profiles provide fine-grained, real-time diagnostics of contractive or unstable phases and reveal transient behaviors not captured by coarse linearized analysis. Leveraging this correspondence, we introduce a Lyapunov-informed parameter selection strategy that identifies solver settings associated with stable behavior, particularly when the estimated LLE indicates persistent instability. Comprehensive experiments on ensembles of perturbed initial guesses demonstrate close agreement between the theoretical stability diagrams and empirical Lyapunov profiles, and show that the proposed adaptive mechanism significantly improves robustness. The study establishes micro-series Lyapunov analysis as a practical, interpretable tool for constructing self-stabilizing root-finding schemes and opens avenues for extending such diagnostics to higher-dimensional or noise-contaminated problems.

Mikhail Borisenkov, Maksim Belyaev, Nithya Rekha Sivakumar et al.

Wearable sensors and IoT/IoMT platforms enable continuous, real-time monitoring, but objective digital markers for eating disorders are limited. In this study, we examined whether actimetry and machine learning (ML) could provide objective criteria for food addiction (FA) and symptom counts (SC). In 78 participants (mean age 22.1 +/- 9.5 y; 73.1% women), one week of non-dominant wrist actimetry and psychometric data (YFAS, DEBQ, ZSDS) were collected. The time series were segmented into daytime activity and nighttime rest, and statistical and entropy descriptors (FuzzyEn, DistEn, SVDEn, PermEn, PhaseEn; 256 features) were calculated. The mean Matthews correlation coefficient (MCC) was used as the primary metric in a K-nearest neighbors (KNN) pipeline with five-fold stratified cross-validation (one hundred repetitions; 500 evaluations); SHAP was used to assist in interpretation. For binary FA, activity-segment features performed best (MCC = 0.78 +/- 0.02; Accuracy ~ 95.3% +/- 0.5; Sensitivity ~ 0.77 +/- 0.03; Specificity ~ 0.98 +/- 0.004), exceeding OaS (Objective and Subjective Features) (MCC = 0.69 +/- 0.03) and rest-only (MCC = 0.50 +/- 0.03). For SC (four classes), OaS slightly surpassed actimetry (MCC = 0.40 +/- 0.01 vs 0.38 +/- 0.01; Accuracy ~ 58.1% vs 56.9%). Emotional and restrained eating were correlated with actimetric features. These findings support wrist-worn actimetry as a digital biomarker of FA that complements questionnaires and may facilitate privacy-preserving clinical translation.

Maksim Belyaev, Murugappan Murugappan, Andrei Velichko et al.

The study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) in an Internet of Things (IoT) environment using rest-state EEG signals (rs-EEG). We computed different types of entropy from EEG signals and found that Fuzzy Entropy performed the best in diagnosing and monitoring PD using rs-EEG. We also investigated different combinations of signal frequency ranges and EEG channels to accurately diagnose PD. Finally, with a fewer number of features (11 features), we achieved a maximum classification accuracy (ARKF) of ~99.9%. The most prominent frequency range of EEG signals has been identified, and we have found that high classification accuracy depends on low-frequency signal components (0-4 Hz). Moreover, the most informative signals were mainly received from the right hemisphere of the head (F8, P8, T8, FC6). Furthermore, we assessed the accuracy of the diagnosis of PD using three different lengths of EEG data (150-1000 samples). Because the computational complexity is reduced by reducing the input data. As a result, we have achieved a maximum mean accuracy of 99.9% for a sample length (LEEG) of 1000 (~7.8 seconds), 98.2% with a LEEG of 800 (~6.2 seconds), and 79.3% for LEEG = 150 (~1.2 seconds). By reducing the number of features and segment lengths, the computational cost of classification can be reduced. Lower-performance smart ML sensors can be used in IoT environments for enhances human resilience to PD.

Irewola Aaron Oludehinwa, Andrei Velichko, Maksim Belyaev et al.

This study proposes an enhancement to the existing method for detecting Solar Active Regions (ARs). Our technique tracks ARs using images from the Atmospheric Imaging Assembly (AIA) of NASA's Solar Dynamics Observatory (SDO). It involves a 2D circular kernel time series transformation, combined with Statistical and Entropy measures, and a Machine Learning (ML) approach. The technique transforms the circular area around pixels in the SDO AIA images into one-dimensional time series (1-DTS). Statistical measures (Median Value, Xmed; 95th Percentile, X95) and Entropy measures (Distribution Entropy, DisEn; Fuzzy Entropy, FuzzyEn) are used as feature selection methods (FSM 1), alongside a method applying 1-DTS elements directly as features (FSM 2). The ML algorithm classifies these series into three categories: no Active Region (nARs type 1, class 1), non-flaring Regions outside active regions with brightness (nARs type 2, class 2), and flaring Active Regions (ARs, class 3). The ML model achieves a classification accuracy of 0.900 and 0.914 for Entropy and Statistical measures, respectively. Notably, Fuzzy Entropy shows the highest classification accuracy (AKF=0.895), surpassing DisEn (AKF=0.738), X95 (AKF=0.873), and Xmed (AKF=0.840). This indicates the high effectiveness of Entropy and Statistical measures for AR detection in SDO AIA images. FSM 2 captures a similar distribution of flaring AR activities as FSM 1. Additionally, we introduce a generalizing characteristic of AR activities (GSA), finding a direct agreement between increased AR activities and higher GSA values. The Python code implementation of the proposed method is available in supplementary material.

Petr Boriskov, Kirill Rudkovskii, Andrei Velichko

This paper studies the removal of salt-and-pepper noise from images using median filter (MF) and simple three-layer autoencoder (AE) within recursive threshold algorithm. The performance of denoising is assessed with two metrics: the standard Structural Similarity Index SSIMImg of restored and clean images and a newly applied metric SSIMMap - the SSIM of entropy maps of these images computed via 2D Sample Entropy in sliding windows. We shown that SSIMMap is more sensitive to blur and local intensity transitions and complements SSIMImg. Experiments on low- and high-resolution grayscales images demonstrate that recursive threshold MF robustly restores images even under strong noise (50-60 %), whereas simple AE is only capable of restoring images with low levels of noise (<30 %). We propose two scalable schemes: (i) 2MF, which uses two MFs with different window sizes and a final thresholding step, effective for highlighting sharp local details at low resolution; and (ii) MFs-AE, which aggregates features from multiple MFs via an AE and is beneficial for restoring the overall scene structure at higher resolution. Owing to its simplicity and computational efficiency, MF remains preferable for deployment on resource-constrained platforms (edge/IoT), whereas AE underperforms without prior denoising. The results also validate the practical value of SSIMMap for objective blur assessment and denoising parameter tuning.

Yuriy Izotov, Andrei Velichko

This paper presents a low-resource speech-command recognizer combining energy-based voice activity detection (VAD), an optimized Mel-Frequency Cepstral Coefficients (MFCC) pipeline, and the LogNNet reservoir-computing classifier. Using four commands from the Speech Commands da-taset downsampled to 8 kHz, we evaluate four MFCC aggregation schemes and find that adaptive binning (64-dimensional feature vector) offers the best accuracy-to-compactness trade-off. The LogNNet classifier with architecture 64:33:9:4 reaches 92.04% accuracy under speaker-independent evaluation, while requiring significantly fewer parameters than conventional deep learn-ing models. Hardware implementation on Arduino Nano 33 IoT (ARM Cor-tex-M0+, 48 MHz, 32 KB RAM) validates the practical feasibility, achieving ~90% real-time recognition accuracy while consuming only 18 KB RAM (55% utilization). The complete pipeline (VAD -> MFCC -> LogNNet) thus enables reliable on-device speech-command recognition under strict memory and compute limits, making it suitable for battery-powered IoT nodes, wire-less sensor networks, and hands-free control interfaces.

Imen Mahmoud, Andrei Velichko

This study proposes a novel methodological framework integrating a LightGBM regression model and genetic algorithm (GA) optimization to systematically evaluate the contribution of COVID-19-related indicators to Bitcoin return prediction. The primary objective was not merely to forecast Bitcoin returns but rather to determine whether including pandemic-related health data significantly enhances prediction accuracy. A comprehensive dataset comprising daily Bitcoin returns and COVID-19 metrics (vaccination rates, hospitalizations, testing statistics) was constructed. Predictive models, trained with and without COVID-19 features, were optimized using GA over 31 independent runs, allowing robust statistical assessment. Performance metrics (R2, RMSE, MAE) were statistically compared through distribution overlaps and Mann-Whitney U tests. Permutation Feature Importance (PFI) analysis quantified individual feature contributions. Results indicate that COVID-19 indicators significantly improved model performance, particularly in capturing extreme market fluctuations (R2 increased by 40%, RMSE decreased by 2%, both highly significant statistically). Among COVID-19 features, vaccination metrics, especially the 75th percentile of fully vaccinated individuals, emerged as dominant predictors. The proposed methodology extends existing financial analytics tools by incorporating public health signals, providing investors and policymakers with refined indicators to navigate market uncertainty during systemic crises.

Petr Boriskov, Vadim Putrolaynen, Andrei Velichko et al.

In this study a new method for analyzing synchronization in oscillator systems is proposed using the example of modeling the dynamics of a circuit of two resistively coupled pulse oscillators. The dynamic characteristic of synchronization is fuzzy entropy (FuzzyEn) calculated a time series composed of the ratios of the number of pulse periods (subharmonic ratio, SHR) during phase-locking intervals. Low entropy values indicate strong synchronization, whereas high entropy values suggest weak synchronization between the two oscillators. This method effectively visualizes synchronized modes of the circuit using entropy maps of synchronization states. Additionally, a classification of synchronization states is proposed based on the dependencies of FuzzyEn on the length of embedding vectors of SHR time series. An extension of this method for analyzing non-relaxation (non-spike) type signals is illustrated using the example of phase-phase coupling rhythms of local field potential of rat hippocampus. The entropy-statistical approach using rational fractions and pulse signal forms makes this method promising for analyzing biosignal synchronization and implementing the algorithm in mobile digital platforms.

Hanif Heidari, Andrei Velichko, Murugappan Murugappan et al.

Entropy is a fundamental concept in the field of information theory. During measurement, conventional entropy measures are susceptible to length and amplitude changes in time series. A new entropy metric, neural network entropy (NNetEn), has been developed to overcome these limitations. NNetEn entropy is computed using a modified LogNNet neural network classification model. The algorithm contains a reservoir matrix of N=19625 elements that must be filled with the given data. The contribution of this paper is threefold. Firstly, this work investigates different methods of filling the reservoir with time series (signal) elements. The reservoir filling method determines the accuracy of the entropy estimation by convolution of the study time series and LogNNet test data. The present study proposes 6 methods for filling the reservoir for time series. Two of them (Method 3 and Method 6) employ the novel approach of stretching the time series to create intermediate elements that complement it, but do not change its dynamics. The most reliable methods for short time series are Method 3 and Method 5. The second part of the study examines the influence of noise and constant bias on entropy values. Our study examines three different time series data types (chaotic, periodic, and binary) with different dynamic properties, Signal to Noise Ratio (SNR), and offsets. The NNetEn entropy calculation errors are less than 10% when SNR is greater than 30 dB, and entropy decreases with an increase in the bias component. The third part of the article analyzes real-time biosignal EEG data collected from emotion recognition experiments. The NNetEn measures show robustness under low-amplitude noise using various filters. Thus, NNetEn measures entropy effectively when applied to real-world environments with ambient noise, white noise, and 1/f noise.

Andrei Velichko

Edge computing is a fast-growing and much needed technology in healthcare. The problem of implementing artificial intelligence on edge devices is the complexity and high resource intensity of the most known neural network data analysis methods and algorithms. The difficulty of implementing these methods on low-power microcontrollers with small memory size calls for the development of new effective algorithms for neural networks. This study presents a new method for analyzing medical data based on the LogNNet neural network, which uses chaotic mappings to transform input information. The method effectively solves classification problems and calculates risk factors for the presence of a disease in a patient according to a set of medical health indicators. The efficiency of LogNNet in assessing perinatal risk is illustrated on cardiotocogram data obtained from the UC Irvine machine learning repository. The classification accuracy reaches ~91% with the ~3-10 kB of RAM used on the Arduino microcontroller. Using the LogNNet network trained on a publicly available database of the Israeli Ministry of Health, a service concept for COVID-19 express testing is provided. A classification accuracy of ~95% is achieved, and ~0.6 kB of RAM is used. In all examples, the model is tested using standard classification quality metrics: precision, recall, and F1-measure. The LogNNet architecture allows the implementation of artificial intelligence on medical peripherals of the Internet of Things with low RAM resources and can be used in clinical decision support systems.

Andrei Velichko, Hanif Heidari

Measuring the predictability and complexity of time series using entropy is essential tool de-signing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the res-ervoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice.

Hanif Heidari, Andrei Velichko

In the age of neural networks and Internet of Things (IoT), the search for new neural network architectures capable of operating on devices with limited computing power and small memory size is becoming an urgent agenda. Designing suitable algorithms for IoT applications is an important task. The paper proposes a feed forward LogNNet neural network, which uses a semi-linear Henon type discrete chaotic map to classify MNIST-10 dataset. The model is composed of reservoir part and trainable classifier. The aim of the reservoir part is transforming the inputs to maximize the classification accuracy using a special matrix filing method and a time series generated by the chaotic map. The parameters of the chaotic map are optimized using particle swarm optimization with random immigrants. As a result, the proposed LogNNet/Henon classifier has higher accuracy and the same RAM usage, compared to the original version of LogNNet, and offers promising opportunities for implementation in IoT devices. In addition, a direct relation between the value of entropy and accuracy of the classification is demonstrated.

Andrei Velichko

This study presents a neural network which uses filters based on logistic mapping (LogNNet). LogNNet has a feedforward network structure, but possesses the properties of reservoir neural networks. The input weight matrix, set by a recurrent logistic mapping, forms the kernels that transform the input space to the higher-dimensional feature space. The most effective recognition of a handwritten digit from MNIST-10 occurs under chaotic behavior of the logistic map. The correlation of classification accuracy with the value of the Lyapunov exponent was obtained. An advantage of LogNNet implementation on IoT devices is the significant savings in memory used. At the same time, LogNNet has a simple algorithm and performance indicators comparable to those of the best resource-efficient algorithms available at the moment. The presented network architecture uses an array of weights with a total memory size from 1 to 29 kB and achieves a classification accuracy of 80.3-96.3%. Memory is saved due to the processor, which sequentially calculates the required weight coefficients during the network operation using the analytical equation of the logistic mapping. The proposed neural network can be used in implementations of artificial intelligence based on constrained devices with limited memory, which are integral blocks for creating ambient intelligence in modern IoT environments. From a research perspective, LogNNet can contribute to the understanding of the fundamental issues of the influence of chaos on the behavior of reservoir-type neural networks.

Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen et al.

In the present paper, we report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in oscillatory neural networks. Based on these results, we further select an adequate SPICE model to describe the modes of operation of coupled oscillator circuits. Physical mechanisms influencing the time of forward and reverse electrical switching, that determine the applicability limits of the proposed model, are identified. For the resistive coupling, it is shown that synchronization takes place at a certain value of the coupling resistance, though it is unstable and a synchronization failure occurs periodically. For the capacitive coupling, two synchronization modes, with weak and strong coupling, are found. The transition between these modes is accompanied by chaotic oscillations. A decrease in the width of the spectrum harmonics in the weak-coupling mode, and its increase in the strong-coupling one, is detected. The dependences of frequencies and phase differences of the coupled oscillatory circuits on the coupling capacitance are found. Examples of operation of coupled VO2 oscillators as a central pattern generator are demonstrated.

Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen et al.

We explore a prototype of an oscillatory neural network (ONN) based on vanadium dioxide switching devices. The model system under study represents two oscillators based on thermally coupled VO2 switches. Numerical simulation shows that the effective action radius RTC of coupling depends both on the total energy released during switching and on the average power. It is experimentally and numerically proved that the temperature change dT commences almost synchronously with the released power peak and T-coupling reveals itself up to a frequency of about 10 kHz. For the studied switching structure configuration, the RTC value varies over a wide range from 4 to 45 mkm, depending on the external circuit capacitance C and resistance Ri, but the variation of Ri is more promising from the practical viewpoint. In the case of a "weak" coupling, synchronization is accompanied by attraction effect and decrease of the main spectra harmonics width. In the case of a "strong" coupling, the number of effects increases, synchronization can occur on subharmonics resulting in multilevel stable synchronization of two oscillators. An advanced algorithm for synchronization efficiency and subharmonic ratio calculation is proposed. It is shown that of the two oscillators the leading one is that with a higher main frequency, and, in addition, the frequency stabilization effect is observed. Also, in the case of a strong thermal coupling, the limit of the supply current parameters, for which the oscillations exist, expands by ~ 10 %. The obtained results have a universal character and open up a new kind of coupling in ONNs, namely, T-coupling, which allows for easy transition from 2D to 3D integration. The effect of subharmonic synchronization hold promise for application in classification and pattern recognition.

Andrei Velichko, Petr Boriskov

The study presents an oscillator circuit for a spike neural network with the possibility of firing rate coding and sigmoid-like activation function. The circuit contains a switching element with an S-shaped current-voltage characteristic and two capacitors; one of the capacitors is shunted by a control resistor. The circuit is characterised by a strong dependence of the frequency of relaxation oscillations on the magnitude of the control resistor. The dependence has a sigmoid-like form and we present an analytical method for dependence calculation. Finally, we describe the concept of the spike neural network architecture with firing rate coding based on the presented circuit for creating neuromorphic devices and artificial intelligence.

Andrei Velichko

This paper presents a new method for evaluating the synchronization of quasi-periodic oscillations of two oscillators, termed "chimeric synchronization". The family of metrics is proposed to create a neural network information converter based on a network of pulsed oscillators. In addition to transforming input information from digital to analogue, the converter can perform information processing after training the network by selecting control parameters. In the proposed neural network scheme, the data arrives at the input layer in the form of current levels of the oscillators and is converted into a set of non-repeating states of the chimeric synchronization of the output oscillator. By modelling a thermally coupled VO2-oscillator circuit, the network setup is demonstrated through the selection of coupling strength, power supply levels, and the synchronization efficiency parameter. The distribution of solutions depending on the operating mode of the oscillators, sub-threshold mode, or generation mode are revealed. Technological approaches for the implementation of a neural network information converter are proposed, and examples of its application for image filtering are demonstrated. The proposed method helps to significantly expand the capabilities of neuromorphic and logical devices based on synchronization effects.

Andrei Velichko, Vadim Putrolaynen, Maksim Belyaev

In the circuit of two thermally coupled VO2 oscillators, we studied a higher order synchronization effect, which can be used in object classification techniques to increase the number of possible synchronous states of the oscillator system. We developed the phase-locking estimation method to determine the values of subharmonic ratio and synchronization effectiveness. In our experiment, the number of possible synchronous states of the oscillator system was twelve, and subharmonic ratio distributions were shaped as Arnold's tongues. In the model, the number of states may reach the maximum value of 150 at certain levels of coupling strength and noise. The long-range synchronization effect in a one-dimensional chain of oscillators occurs even at low values of synchronization effectiveness for intermediate links. We demonstrate a technique for storing and recognizing vector images, which can used for reservoir computing. In addition, we present the implementation of analog operation of multiplication, the synchronization based logic for binary computations, and the possibility to develop the interface between spike neural network and a computer. Based on the universal physical effects, the high order synchronization can be applied to any spiking oscillators with any coupling type, enhancing the practical value of the presented results to expand spike neural network capabilities.