Svetlana Yanushkevich

Mansooreh Montazerin, Elahe Rahimian, Farnoosh Naderkhani et al.

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a compact deep learning framework referred to as the CT-HGR, which employs a vision transformer network to conduct hand gesture recognition using highdensity sEMG (HD-sEMG) signals. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. CT-HGR can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the CT-HGR framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the CT-HGR is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed CT-HGR framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The CT-HGR achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image.

Kenneth Lai, Mohammed Almekhlafi, Svetlana Yanushkevich

This paper presents an approach for detecting unilateral arm paralysis/weakness using kinematic data. Our method employs temporal convolution networks and recurrent neural networks, guided by knowledge distillation, where we use inertial measurement units attached to the body to capture kinematic information such as acceleration, rotation, and flexion of body joints during an action. This information is then analyzed to recognize body actions and patterns. Our proposed network achieves a high paretic detection accuracy of 97.99\%, with an action classification accuracy of 77.69\%, through knowledge sharing. Furthermore, by incorporating causal reasoning, we can gain additional insights into the patient's condition, such as their Fugl-Meyer assessment score or impairment level based on the machine learning result. Overall, our approach demonstrates the potential of using kinematic data and machine learning for detecting arm paralysis/weakness. The results suggest that our method could be a useful tool for clinicians and healthcare professionals working with patients with this condition.

Kenneth Lai, Svetlana Yanushkevich

The impacts of mass migration, such as crisis induced by climate change, extend beyond environmental concerns and can greatly affect social infrastructure and public services, such as education, healthcare, and security. These crises exacerbate certain elements like cultural barriers, and discrimination by amplifying the challenges faced by these affected communities. This paper proposes an innovative approach to address migration crises in the context of crisis management through a combination of modeling and imbalance assessment tools. By employing deep learning for forecasting and integrating causal reasoning via Bayesian networks, this methodology enables the evaluation of imbalances and risks in the socio-technological landscape, providing crucial insights for informed decision-making. Through this framework, critical systems can be analyzed to understand how fluctuations in migration levels may impact them, facilitating effective crisis governance strategies.

Mackenzie Wallich, Kenneth Lai, Svetlana Yanushkevich

Accelerometry has been extensively studied as an objective means of measuring upper limb function in patients post-stroke. The objective of this paper is to determine whether the accelerometry-derived measurements frequently used in more long-term rehabilitation studies can also be used to monitor and rapidly detect sudden changes in upper limb motor function in more recently hospitalized stroke patients. Six binary classification models were created by training on variable data window times of paretic upper limb accelerometer feature data. The models were assessed on their effectiveness for differentiating new input data into two classes: severe or moderately severe motor function. The classification models yielded Area Under the Curve (AUC) scores that ranged from 0.72 to 0.82 for 15-minute data windows to 0.77 to 0.94 for 120-minute data windows. These results served as a preliminary assessment and a basis on which to further investigate the efficacy of using accelerometry and machine learning to alert healthcare professionals to rapid changes in motor function in the days immediately following a stroke.

Kenneth Lai, Vlad Shmerko, Svetlana Yanushkevich

In this paper, we study performance and fairness on visual and thermal images and expand the assessment to masked synthetic images. Using the SpeakingFace and Thermal-Mask dataset, we propose a process to assess fairness on real images and show how the same process can be applied to synthetic images. The resulting process shows a demographic parity difference of 1.59 for random guessing and increases to 5.0 when the recognition performance increases to a precision and recall rate of 99.99\%. We indicate that inherently biased datasets can deeply impact the fairness of any biometric system. A primary cause of a biased dataset is the class imbalance due to the data collection process. To address imbalanced datasets, the classes with fewer samples can be augmented with synthetic images to generate a more balanced dataset resulting in less bias when training a machine learning system. For biometric-enabled systems, fairness is of critical importance, while the related concept of Equity, Diversity, and Inclusion (EDI) is well suited for the generalization of fairness in biometrics, in this paper, we focus on the 3 most common demographic groups age, gender, and ethnicity.

Kenneth Lai, Svetlana Yanushkevich, Vlad Shmerko

This paper considers the adaptation of the e-coaching concept at times of emergencies and disasters, through aiding the e-coaching with intelligent tools for monitoring humans' affective state. The states such as anxiety, panic, avoidance, and stress, if properly detected, can be mitigated using the e-coaching tactic and strategy. In this work, we focus on a stress monitoring assistant tool developed on machine learning techniques. We provide the results of an experimental study using the proposed method.

Rahat Islam, Kenneth Lai, Svetlana Yanushkevich

In this paper, we investigate hand gesture classifiers that rely upon the abstracted 'skeletal' data recorded using the RGB-Depth sensor. We focus on 'skeletal' data represented by the body joint coordinates, from the Praxis dataset. The PRAXIS dataset contains recordings of patients with cortical pathologies such as Alzheimer's disease, performing a Praxis test under the direction of a clinician. In this paper, we propose hand gesture classifiers that are more effective with the PRAXIS dataset than previously proposed models. Body joint data offers a compressed form of data that can be analyzed specifically for hand gesture recognition. Using a combination of windowing techniques with deep learning architecture such as a Recurrent Neural Network (RNN), we achieved an overall accuracy of 70.8% using only body joint data. In addition, we investigated a long-short-term-memory (LSTM) to extract and analyze the movement of the joints through time to recognize the hand gestures being performed and achieved a gesture recognition rate of 74.3% and 67.3% for static and dynamic gestures, respectively. The proposed approach contributed to the task of developing an automated, accurate, and inexpensive approach to diagnosing cortical pathologies for multiple healthcare applications.

Peter Shmerko, Yumi Iwashita, Adrian Stoica et al.

Mission teams are exposed to the emotional toll of life and death decisions. These are small groups of specially trained people supported by intelligent machines for dealing with stressful environments and scenarios. We developed a composite model for stress monitoring in such teams of human and autonomous machines. This modelling aims to identify the conditions that may contribute to mission failure. The proposed model is composed of three parts: 1) a computational logic part that statically describes the stress states of teammates; 2) a decision part that manifests the mission status at any time; 3) a stress propagation part based on standard Susceptible-Infected-Susceptible (SIS) paradigm. In contrast to the approaches such as agent-based, random-walk and game models, the proposed model combines various mechanisms to satisfy the conditions of stress propagation in small groups. Our core approach involves data structures such as decision tables and decision diagrams. These tools are adaptable to human-machine teaming as well.

Jie Yang, Ziyi Liu, Mengchen Lin et al.

Wide dynamic range (WDR) images contain more scene details and contrast when compared to common images. However, it requires tone mapping to process the pixel values in order to display properly. The details of WDR images can diminish during the tone mapping process. In this work, we address the problem by combining a novel reformulated Laplacian pyramid and deep learning. The reformulated Laplacian pyramid always decompose a WDR image into two frequency bands where the low-frequency band is global feature-oriented, and the high-frequency band is local feature-oriented. The reformulation preserves the local features in its original resolution and condenses the global features into a low-resolution image. The generated frequency bands are reconstructed and fine-tuned to output the final tone mapped image that can display on the screen with minimum detail and contrast loss. The experimental results demonstrate that the proposed method outperforms state-of-the-art WDR image tone mapping methods. The code is made publicly available at https://github.com/linmc86/Deep-Reformulated-Laplacian-Tone-Mapping.

Olha Shaposhnyk, Noor Abid, Mouri Zakir et al.

This study investigates the generation of high-quality synthetic categorical data, such as survey data, using causal graph models. Generating synthetic data aims not only to create a variety of data for training the models but also to preserve privacy while capturing relationships between the data. The research employs Structural Equation Modeling (SEM) followed by Bayesian Networks (BN). We used the categorical data that are based on the survey of accessibility to services for people with disabilities. We created both SEM and BN models to represent causal relationships and to capture joint distributions between variables. In our case studies, such variables include, in particular, demographics, types of disability, types of accessibility barriers and frequencies of encountering those barriers. The study compared the SEM-based BN method with alternative approaches, including the probabilistic Gaussian copula technique and generative models like the Conditional Tabular Generative Adversarial Network (CTGAN). The proposed method outperformed others in statistical metrics, including the Chi-square test, Kullback-Leibler divergence, and Total Variation Distance (TVD). In particular, the BN model demonstrated superior performance, achieving the highest TVD, indicating alignment with the original data. The Gaussian Copula ranked second, while CTGAN exhibited moderate performance. These analyses confirmed the ability of the SEM-based BN to produce synthetic data that maintain statistical and relational validity while maintaining confidentiality. This approach is particularly beneficial for research on sensitive data, such as accessibility and disability studies.

Olha Shaposhnyk, Daria Zahorska, Svetlana Yanushkevich

Objective: This study investigates the potential of Large Language Models (LLMs) as an alternative to human expert elicitation for extracting structured causal knowledge and facilitating causal modeling in biometric and healthcare applications. Material and Methods: LLM-generated causal structures, specifically Bayesian networks (BNs), were benchmarked against traditional statistical methods (e.g., Bayesian Information Criterion) using healthcare datasets. Validation techniques included structural equation modeling (SEM) to verifying relationships, and measures such as entropy, predictive accuracy, and robustness to compare network structures. Results and Discussion: LLM-generated BNs demonstrated lower entropy than expert-elicited and statistically generated BNs, suggesting higher confidence and precision in predictions. However, limitations such as contextual constraints, hallucinated dependencies, and potential biases inherited from training data require further investigation. Conclusion: LLMs represent a novel frontier in expert elicitation for probabilistic causal modeling, promising to improve transparency and reduce uncertainty in the decision-making using such models.

Leonardo Queiroz, Kenneth Lai, Svetlana Yanushkevich et al.

In this study of the face recognition on masked versus unmasked faces generated using Flickr-Faces-HQ and SpeakingFaces datasets, we report 36.78% degradation of recognition performance caused by the mask-wearing at the time of pandemics, in particular, in border checkpoint scenarios. We have achieved better performance and reduced the degradation to 1.79% using advanced deep learning approaches in the cross-spectral domain.

Ziyi Liu, Jie Yang, Svetlana Yanushkevich et al.

Deep convolutional neural networks (DCNNs) have aided high dynamic range (HDR) imaging recently and have received a lot of attention. The quality of DCNN-generated HDR images has overperformed the traditional counterparts. However, DCNNs are prone to be computationally intensive and power-hungry, and hence cannot be implemented on various embedded computing platforms with limited power and hardware resources. Embedded systems have a huge market, and utilizing DCNNs' powerful functionality into them will further reduce human intervention. To address the challenge, we propose LightFuse, a lightweight CNN-based algorithm for extreme dual-exposure image fusion, which achieves better functionality than a conventional DCNN and can be deployed in embedded systems. Two sub-networks are utilized: a GlobalNet (G) and a DetailNet (D). The goal of G is to learn the global illumination information on the spatial dimension, whereas D aims to enhance local details on the channel dimension. Both G and D are based solely on depthwise convolution (D_Conv) and pointwise convolution (P_Conv) to reduce required parameters and computations. Experimental results show that this proposed technique could generate HDR images in extremely exposed regions with sufficient details to be legible. Our model outperforms other state-of-the-art approaches in peak signal-to-noise ratio (PSNR) score by 0.9 to 8.7 while achieving 16.7 to 306.2 times parameter reduction.

Ziyi Liu, Jie Yang, Mengchen Lin et al.

Currently, face detection approaches focus on facial information by varying specific parameters including pose, occlusion, lighting, background, race, and gender. These studies only utilized the information obtained from low dynamic range images, however, face detection in wide dynamic range (WDR) scenes has received little attention. To our knowledge, there is no publicly available WDR database for face detection research. To facilitate and support future face detection research in the WDR field, we propose the first WDR database for face detection, called WDR FACE, which contains a total of 398 16-bit megapixel grayscale wide dynamic range images collected from 29 subjects. These WDR images (WDRIs) were taken in eight specific WDR scenes. The dynamic range of 90% images surpasses 60,000:1, and that of 70% images exceeds 65,000:1. Furthermore, we show the effect of different face detection procedures on the WDRIs in our database. This is done with 25 different tone mapping operators and five different face detectors. We provide preliminary experimental results of face detection on this unique WDR database.

Arash Mohammadi, Yingxu Wang, Nastaran Enshaei et al.

The novel Coronavirus disease, COVID-19, has rapidly and abruptly changed the world as we knew in 2020. It becomes the most unprecedent challenge to analytic epidemiology in general and signal processing theories in specific. Given its high contingency nature and adverse effects across the world, it is important to develop efficient processing/learning models to overcome this pandemic and be prepared for potential future ones. In this regard, medical imaging plays an important role for the management of COVID-19. Human-centered interpretation of medical images is, however, tedious and can be subjective. This has resulted in a surge of interest to develop Radiomics models for analysis and interpretation of medical images. Signal Processing (SP) and Deep Learning (DL) models can assist in development of robust Radiomics solutions for diagnosis/prognosis, severity assessment, treatment response, and monitoring of COVID-19 patients. In this article, we aim to present an overview of the current state, challenges, and opportunities of developing SP/DL-empowered models for diagnosis (screening/monitoring) and prognosis (outcome prediction and severity assessment) of COVID-19 infection. More specifically, the article starts by elaborating the latest development on the theoretical framework of analytic epidemiology and hypersignal processing for COVID-19. Afterwards, imaging modalities and Radiological characteristics of COVID-19 are discussed. SL/DL-based Radiomic models specific to the analysis of COVID-19 infection are then described covering the following four domains: Segmentation of COVID-19 lesions; Predictive models for outcome prediction; Severity assessment, and; Diagnosis/classification models. Finally, open problems and opportunities are presented in detail.

Kenneth Lai, Svetlana Yanushkevich

The focus of this paper is dynamic gesture recognition in the context of the interaction between humans and machines. We propose a model consisting of two sub-networks, a transformer and an ordered-neuron long-short-term-memory (ON-LSTM) based recurrent neural network (RNN). Each sub-network is trained to perform the task of gesture recognition using only skeleton joints. Since each sub-network extracts different types of features due to the difference in architecture, the knowledge can be shared between the sub-networks. Through knowledge distillation, the features and predictions from each sub-network are fused together into a new fusion classifier. In addition, a cyclical learning rate can be used to generate a series of models that are combined in an ensemble, in order to yield a more generalizable prediction. The proposed ensemble of knowledge-sharing models exhibits an overall accuracy of 86.11% using only skeleton information, as tested using the Dynamic Hand Gesture-14/28 dataset

Svetlana Yanushkevich, Vlad Shmerko

Contemporary Epidemiological Surveillance (ES) relies heavily on data analytics. These analytics are critical input for pandemics preparedness networks; however, this input is not integrated into a form suitable for decision makers or experts in preparedness. A decision support system (DSS) with Computational Intelligence (CI) tools is required to bridge the gap between epidemiological model of evidence and expert group decision. We argue that such DSS shall be a cognitive dynamic system enabling the CI and human expert to work together. The core of such DSS must be based on machine reasoning techniques such as probabilistic inference, and shall be capable of estimating risks, reliability and biases in decision making.

Thomas Truong, Jonathan Graf, Svetlana Yanushkevich

Uni-modal identification systems are vulnerable to errors in sensor data collection and are therefore more likely to misidentify subjects. For instance, relying on data solely from an RGB face camera can cause problems in poorly lit environments or if subjects do not face the camera. Other identification methods such as electrocardiograms (ECG) have issues with improper lead connections to the skin. Errors in identification are minimized through the fusion of information gathered from both of these models. This paper proposes a methodology for combining the identification results of face and ECG data using Part A of the BioVid Heat Pain Database containing synchronized RGB-video and ECG data on 87 subjects. Using 10-fold cross-validation, face identification was 98.8% accurate, while the ECG identification was 96.1% accurate. By using a fusion approach the identification accuracy improved to 99.8%. Our proposed methodology allows for identification accuracies to be significantly improved by using disparate face and ECG models that have non-overlapping modalities.

Thomas Truong, Svetlana Yanushkevich

A major obstacle in radar based methods for concealed object detection on humans and seamless integration into security and access control system is the difficulty in collecting high quality radar signal data. Generative adversarial networks (GAN) have shown promise in data generation application in the fields of image and audio processing. As such, this paper proposes the design of a GAN for application in radar signal generation. Data collected using the Finite-Difference Time-Domain (FDTD) method on three concealed object classes (no object, large object, and small object) were used as training data to train a GAN to generate radar signal samples for each class. The proposed GAN generated radar signal data which was indistinguishable from the training data by qualitative human observers.

Kenneth Lai, Xinyuan Tu, Svetlana Yanushkevich

This paper addresses the problem of biometric identification of animals, specifically dogs. We apply advanced machine learning models such as deep neural network on the photographs of pets in order to determine the pet identity. In this paper, we explore the possibility of using different types of "soft" biometrics, such as breed, height, or gender, in fusion with "hard" biometrics such as photographs of the pet's face. We apply the principle of transfer learning on different Convolutional Neural Networks, in order to create a network designed specifically for breed classification. The proposed network is able to achieve an accuracy of 90.80% and 91.29% when differentiating between the two dog breeds, for two different datasets. Without the use of "soft" biometrics, the identification rate of dogs is 78.09% but by using a decision network to incorporate "soft" biometrics, the identification rate can achieve an accuracy of 84.94%.

Thomas Truong, Svetlana Yanushkevich

Detecting visual relationships between people and clothing in an image has been a relatively unexplored problem in the field of computer vision and biometrics. The lack readily available public dataset for ``worn'' and ``unworn'' classification has slowed the development of solutions for this problem. We present the release of the Relatable Clothing Dataset which contains 35287 person-clothing pairs and segmentation masks for the development of ``worn'' and ``unworn'' classification models. Additionally, we propose a novel soft attention unit for performing ``worn'' and ``unworn'' classification using deep neural networks. The proposed soft attention models have an accuracy of upward $98.55\% \pm 0.35\%$ on the Relatable Clothing Dataset and demonstrate high generalizable, allowing us to classify unseen articles of clothing such as high visibility vests as ``worn'' or ``unworn''.

Svetlana Yanushkevich, Shawn Eastwood, Kenneth Lai et al.

This paper aims at identifying emerging computational intelligence trends for the design and modeling of complex biometric-enabled infrastructure and systems. Biometric-enabled systems are evolving towards deep learning and deep inference using the principles of adaptive computing, - the front tides of the modern computational intelligence domain. Therefore, we focus on intelligent inference engines widely deployed in biometrics. Computational intelligence applications that cover a wide spectrum of biometric tasks using physiological and behavioral traits are chosen for illustration. We highlight the technology gaps that must be addressed in future generations of biometric systems. The reported approaches and results primarily address the researchers who work towards developing the next generation of intelligent biometric-enabled systems.