Nikita Gordienko

Nikita Gordienko, Peng Gang, Yuri Gordienko et al.

Machine learning techniques are presented for automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine). A new image dataset of these carved Glagolitic and Cyrillic letters (CGCL) was assembled and pre-processed for recognition and prediction by machine learning methods. The dataset consists of more than 4000 images for 34 types of letters. The explanatory data analysis of CGCL and notMNIST datasets shown that the carved letters can hardly be differentiated by dimensionality reduction methods, for example, by t-distributed stochastic neighbor embedding (tSNE) due to the worse letter representation by stone carving in comparison to hand writing. The multinomial logistic regression (MLR) and a 2D convolutional neural network (CNN) models were applied. The MLR model demonstrated the area under curve (AUC) values for receiver operating characteristic (ROC) are not lower than 0.92 and 0.60 for notMNIST and CGCL, respectively. The CNN model gave AUC values close to 0.99 for both notMNIST and CGCL (despite the much smaller size and quality of CGCL in comparison to notMNIST) under condition of the high lossy data augmentation. CGCL dataset was published to be available for the data science community as an open source resource.

Nikita Gordienko, Yuriy Kochura, Vlad Taran et al.

Automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine) was demonstrated by means of capsule deep learning neural network. It was applied to the image dataset of the carved Glagolitic and Cyrillic letters (CGCL), which was assembled and pre-processed recently for recognition and prediction by machine learning methods (https://www.kaggle.com/yoctoman/graffiti-st-sophia-cathedral-kyiv). CGCL dataset contains >4000 images for glyphs of 34 letters which are hardly recognized by experts even in contrast to notMNIST dataset with the better images of 10 letters taken from different fonts. Despite the much worse quality of CGCL dataset and extremely low number of samples (in comparison to notMNIST dataset) the capsule network model demonstrated much better results than the previously used convolutional neural network (CNN). The validation accuracy (and validation loss) was higher (lower) for capsule network model than for CNN without data augmentation even. The area under curve (AUC) values for receiver operating characteristic (ROC) were also higher for the capsule network model than for CNN model: 0.88-0.93 (capsule network) and 0.50 (CNN) without data augmentation, 0.91-0.95 (capsule network) and 0.51 (CNN) with lossless data augmentation, and similar results of 0.91-0.93 (capsule network) and 0.9 (CNN) in the regime of lossless data augmentation only. The confusion matrixes were much better for capsule network than for CNN model and gave the much lower type I (false positive) and type II (false negative) values in all three regimes of data augmentation. These results supports the previous claims that capsule-like networks allow to reduce error rates not only on MNIST digit dataset, but on the other notMNIST letter dataset and the more complex CGCL handwriting graffiti letter dataset also.

Olga Barkova, Natalia Pysarevska, Oleg Allenin et al.

Particularly close attention is being paid today among researchers in social science disciplines to aspects of learning in the digital age, especially for the Digitally Native Generation. In the context of museums, the question is: how can rich learning experiences be provided for increasingly technologically advanced young visitors in museums? Which high-tech platforms and solutions do museums need to focus on? At the same time, the software games business is growing fast and now finding its way into non-entertainment contexts, helping to deliver substantial benefits, particularly in education, training, research, and health. This article outlines some aspects facing Digitally Native learners in museums through an analysis of several radically new key technologies: Interactivity, Wearables, Virtual Reality, and Augmented Reality. Special attention is paid to use cases for application of games-based scenarios via these technologies in non-leisure contexts and specifically for educational purposes in museums.

Yuri Gordienko, Sergii Stirenko, Yuriy Kochura et al.

The new method is proposed to monitor the level of current physical load and accumulated fatigue by several objective and subjective characteristics. It was applied to the dataset targeted to estimate the physical load and fatigue by several statistical and machine learning methods. The data from peripheral sensors (accelerometer, GPS, gyroscope, magnetometer) and brain-computing interface (electroencephalography) were collected, integrated, and analyzed by several statistical and machine learning methods (moment analysis, cluster analysis, principal component analysis, etc.). The hypothesis 1 was presented and proved that physical activity can be classified not only by objective parameters, but by subjective parameters also. The hypothesis 2 (experienced physical load and subsequent restoration as fatigue level can be estimated quantitatively and distinctive patterns can be recognized) was presented and some ways to prove it were demonstrated. Several "physical load" and "fatigue" metrics were proposed. The results presented allow to extend application of the machine learning methods for characterization of complex human activity patterns (for example, to estimate their actual physical load and fatigue, and give cautions and advice).

Nikita Gordienko

The new method is proposed to monitor the level of currently accumulated fatigue and estimate it by the several statistical methods. The experimental software application was developed and used to get data from sensors (accelerometer, GPS, gyroscope, magnetometer, and camera), conducted experiments, collected data, calculated parameters of their distributions (mean, standard deviation, skewness, kurtosis), and analyzed them by statistical methods (moment analysis, cluster analysis, bootstrapping, periodogram and spectrogram analyses). The hypothesis 1 (physical activity can be estimated and classified by moment and cluster analysis) and hypothesis 2 (fatigue can be estimated by moment analysis, bootstrapping analysis, periodogram, and spectrogram) were proposed and proved. Several "fatigue metrics" were proposed: location, size, shape of clouds of points on bootstrapping plot. The most promising fatigue metrics is the distance from the "rest" state point to the "fatigue" state point (sum of 3 squared non-normal distribution of non-correlated acceleration values) on the skewness-kurtosis plot. These hypotheses were verified on several persons of various age, gender, fitness level and improved standard statistical methods in similar researches. The method can be used in practice for ordinary people in everyday situations (to estimate their fatigue, give tips about it and advice on context-related information).