Andrey Somov

Melaku N. Getahun, Oleg Y. Rogov, Dmitry V. Dylov et al.

Retinal vascular segmentation, a widely researched topic in biomedical image processing, aims to reduce the workload of ophthalmologists in treating and detecting retinal disorders. Segmenting retinal vessels presents unique challenges; previous techniques often failed to effectively segment branches and microvascular structures. Recent neural network approaches struggle to balance local and global properties and frequently miss tiny end vessels, hindering the achievement of desired results. To address these issues in retinal vessel segmentation, we propose a comprehensive micro-vessel extraction mechanism based on an encoder-decoder neural network architecture. This network includes residual, encoder booster, bottleneck enhancement, squeeze, and excitation building blocks. These components synergistically enhance feature extraction and improve the prediction accuracy of the segmentation map. Our solution has been evaluated using the DRIVE, CHASE-DB1, and STARE datasets, yielding competitive results compared to previous studies. The AUC and accuracy on the DRIVE dataset are 0.9884 and 0.9702, respectively. For the CHASE-DB1 dataset, these scores are 0.9903 and 0.9755, respectively, and for the STARE dataset, they are 0.9916 and 0.9750. Given its accurate and robust performance, the proposed approach is a solid candidate for being implemented in real-life diagnostic centers and aiding ophthalmologists.

Ren Yang, Radu Timofte, Jing Liu et al.

This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at a fixed bit-rate. Besides, the quality enhancement of Tracks 1 and 3 targets at improving the fidelity (PSNR), and Track 2 targets at enhancing the perceptual quality. The three tracks totally attract 482 registrations. In the test phase, 12 teams, 8 teams and 11 teams submitted the final results of Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of video quality enhancement. The homepage of the challenge: https://github.com/RenYang-home/NTIRE21_VEnh

Anton Smerdov, Bo Zhou, Paul Lukowicz et al.

Proper training and analytics in eSports require accurately collected and annotated data. Most eSports research focuses exclusively on in-game data analysis, and there is a lack of prior work involving eSports athletes' psychophysiological data. In this paper, we present a dataset collected from professional and amateur teams in 22 matches in League of Legends video game with more than 40 hours of recordings. Recorded data include the players' physiological activity, e.g. movements, pulse, saccades, obtained from various sensors, self-reported aftermatch survey, and in-game data. An important feature of the dataset is simultaneous data collection from five players, which facilitates the analysis of sensor data on a team level. Upon the collection of dataset we carried out its validation. In particular, we demonstrate that stress and concentration levels for professional players are less correlated, meaning more independent playstyle. Also, we show that the absence of team communication does not affect the professional players as much as amateur ones. To investigate other possible use cases of the dataset, we have trained classical machine learning algorithms for skill prediction and player re-identification using 3-minute sessions of sensor data. Best models achieved 0.856 and 0.521 (0.10 for a chance level) accuracy scores on a validation set for skill prediction and player re-id problems, respectively. The dataset is available at https://github.com/smerdov/eSports Sensors Dataset.

Anton Smerdov, Evgeny Burnaev, Andrey Somov et al.

The emerging progress of eSports lacks the tools for ensuring high-quality analytics and training in Pro and amateur eSports teams. We report on an Artificial Intelligence (AI) enabled solution for predicting the eSports player in-game performance using exclusively the data from sensors. For this reason, we collected the physiological, environmental, and the game chair data from Pro and amateur players. The player performance is assessed from the game logs in a multiplayer game for each moment of time using a recurrent neural network. We have investigated that attention mechanism improves the generalization of the network and provides the straightforward feature importance as well. The best model achieves ROC AUC score 0.73. The prediction of the performance of particular player is realized although his data are not utilized in the training set. The proposed solution has a number of promising applications for Pro eSports teams and amateur players, such as a learning tool or a performance monitoring system.

Anton Smerdov, Andrey Somov, Evgeny Burnaev et al.

Current research in eSports lacks the tools for proper game practising and performance analytics. The majority of prior work relied only on in-game data for advising the players on how to perform better. However, in-game mechanics and trends are frequently changed by new patches limiting the lifespan of the models trained exclusively on the in-game logs. In this article, we propose the methods based on the sensor data analysis for predicting whether a player will win the future encounter. The sensor data were collected from 10 participants in 22 matches in League of Legends video game. We have trained machine learning models including Transformer and Gated Recurrent Unit to predict whether the player wins the encounter taking place after some fixed time in the future. For 10 seconds forecasting horizon Transformer neural network architecture achieves ROC AUC score 0.706. This model is further developed into the detector capable of predicting that a player will lose the encounter occurring in 10 seconds in 88.3% of cases with 73.5% accuracy. This might be used as a players' burnout or fatigue detector, advising players to retreat. We have also investigated which physiological features affect the chance to win or lose the next in-game encounter.

Anton Smerdov, Anastasia Kiskun, Rostislav Shaniiazov et al.

eSports is the rapidly developing multidisciplinary domain. However, research and experimentation in eSports are in the infancy. In this work, we propose a smart chair platform - an unobtrusive approach to the collection of data on the eSports athletes and data further processing with machine learning methods. The use case scenario involves three groups of players: `cyber athletes' (Monolith team), semi-professional players and newbies all playing CS:GO discipline. In particular, we collect data from the accelerometer and gyroscope integrated in the chair and apply machine learning algorithms for the data analysis. Our results demonstrate that the professional athletes can be identified by their behaviour on the chair while playing the game.

Anton Stepanov, Andrey Lange, Nikita Khromov et al.

eSports industry has greatly progressed within the last decade in terms of audience and fund rising, broadcasting, networking and hardware. Since the number and quality of professional team has evolved too, there is a reasonable need in improving skills and training process of professional eSports athletes. In this work, we demonstrate a system able to collect heterogeneous data (physiological, environmental, video, telemetry) and guarantying synchronization with 10 ms accuracy. In particular, we demonstrate how to synchronize various sensors and ensure post synchronization, i.e. logged video, a so-called demo file, with the sensors data. Our experimental results achieved on the CS:GO game discipline show up to 3 ms accuracy of the time synchronization of the gaming computer.

Alexander Korotin, Nikita Khromov, Anton Stepanov et al.

eSports is a developing multidisciplinary research area. At present, there is a lack of relevant data collected from real eSports athletes and lack of platforms which could be used for the data collection and further analysis. In this paper, we present a sensing system for enabling the data collection from professional athletes. Also, we report on the case study about collecting and analyzing the gaze data from Monolith professional eSports team specializing in Counter-Strike: Global Offensive (CS:GO) discipline. We perform a comparative study on assessing the gaze of amateur players and professional athletes. The results of our work are vital for ensuring eSports data collection and the following analysis in the scope of scouting or assessing the eSports players and athletes.

Anton Smerdov, Evgeny Burnaev, Andrey Somov

Today's competition between the professional eSports teams is so strong that in-depth analysis of players' performance literally crucial for creating a powerful team. There are two main approaches to such an estimation: obtaining features and metrics directly from the in-game data or collecting detailed information about the player including data on his/her physical training. While the correlation between the player's skill and in-game data has already been covered in many papers, there are very few works related to analysis of eSports athlete's skill through his/her physical behavior. We propose the smart chair platform which is to collect data on the person's behavior on the chair using an integrated accelerometer, a gyroscope and a magnetometer. We extract the important game events to define the players' physical reactions to them. The obtained data are used for training machine learning models in order to distinguish between the low-skilled and high-skilled players. We extract and figure out the key features during the game and discuss the results.

Roman Ibrahimov, Evgeny Tsykunov, Vladimir Shirokun et al.

We report on the teleoperation system DronePick which provides remote object picking and delivery by a human-controlled quadcopter. The main novelty of the proposed system is that the human user continuously gets the visual and haptic feedback for accurate teleoperation. DronePick consists of a quadcopter equipped with a magnetic grabber, a tactile glove with finger motion tracking sensor, hand tracking system, and the Virtual Reality (VR) application. The human operator teleoperates the quadcopter by changing the position of the hand. The proposed vibrotactile patterns representing the location of the remote object relative to the quadcopter are delivered to the glove. It helps the operator to determine when the quadcopter is right above the object. When the "pick" command is sent by clasping the hand in the glove, the quadcopter decreases its altitude and the magnetic grabber attaches the target object. The whole scenario is in parallel simulated in VR. The air flow from the quadcopter and the relative positions of VR objects help the operator to determine the exact position of the delivered object to be picked. The experiments showed that the vibrotactile patterns were recognized by the users at the high recognition rates: the average 99% recognition rate and the average 2.36s recognition time. The real-life implementation of DronePick featuring object picking and delivering to the human was developed and tested.

Boris B. Velichkovsky, Nikita Khromov, Alexander Korotin et al.

Using highly interactive systems like computer games requires a lot of visual activity and eye movements. Eye movements are best characterized by visual fixation - periods of time when the eyes stay relatively still over an object. We analyzed the distributions of fixation duration of professional athletes, amateur and newbie players. We show that the analysis of fixation durations can be used to deduce the skill level in computer game players. Highly skilled gaming performance is characterized by more variability in fixation durations and by bimodal fixation duration distributions suggesting the presence of two fixation types in high skill gamers. These fixation types were identified as ambient (automatic spatial processing) and focal (conscious visual processing). The analysis of computer gamers' skill level via the analysis of fixation durations may be used in developing adaptive interfaces and in interface design.

Nikita Khromov, Alexander Korotin, Andrey Lange et al.

We present a comparative study of the players' and professional players' (athletes') performance in Counter Strike: Global Offensive (CS:GO) discipline. Our study is based on ubiquitous sensing helping identify the biometric features significantly contributing to the classification of particular skills of the players. The research provides better understanding why the athletes demonstrate superior performance as compared to other players.