Stephan Sigg

Cristian J. Vaca-Rubio, Dariush Salami, Petar Popovski et al.

Since electromagnetic signals are omnipresent, Radio Frequency (RF)-sensing has the potential to become a universal sensing mechanism with applications in localization, smart-home, retail, gesture recognition, intrusion detection, etc. Two emerging technologies in RF-sensing, namely sensing through Large Intelligent Surfaces (LISs) and mmWave Frequency-Modulated Continuous-Wave (FMCW) radars, have been successfully applied to a wide range of applications. In this work, we compare LIS and mmWave radars for localization in real-world and simulated environments. In our experiments, the mmWave radar achieves 0.71 Intersection Over Union (IOU) and 3cm error for bounding boxes, while LIS has 0.56 IOU and 10cm distance error. Although the radar outperforms the LIS in terms of accuracy, LIS features additional applications in communication in addition to sensing scenarios.

Si Zuo, Vitor Fortes Rey, Sungho Suh et al.

Human activity recognition (HAR) from on-body sensors is a core functionality in many AI applications: from personal health, through sports and wellness to Industry 4.0. A key problem holding up progress in wearable sensor-based HAR, compared to other ML areas, such as computer vision, is the unavailability of diverse and labeled training data. Particularly, while there are innumerable annotated images available in online repositories, freely available sensor data is sparse and mostly unlabeled. We propose an unsupervised statistical feature-guided diffusion model specifically optimized for wearable sensor-based human activity recognition with devices such as inertial measurement unit (IMU) sensors. The method generates synthetic labeled time-series sensor data without relying on annotated training data. Thereby, it addresses the scarcity and annotation difficulties associated with real-world sensor data. By conditioning the diffusion model on statistical information such as mean, standard deviation, Z-score, and skewness, we generate diverse and representative synthetic sensor data. We conducted experiments on public human activity recognition datasets and compared the method to conventional oversampling and state-of-the-art generative adversarial network methods. Experimental results demonstrate that this can improve the performance of human activity recognition and outperform existing techniques.

Dariush Salami, Ramin Hasibi, Stefano Savazzi et al.

RF-sensing, the analysis and interpretation of movement or environment-induced patterns in received electromagnetic signals, has been actively investigated for more than a decade. Since electromagnetic signals, through cellular communication systems, are omnipresent, RF sensing has the potential to become a universal sensing mechanism with applications in smart home, retail, localization, gesture recognition, intrusion detection, etc. Specifically, existing cellular network installations might be dual-used for both communication and sensing. Such communications and sensing convergence is envisioned for future communication networks. We propose the use of NR-sidelink direct device-to-device communication to achieve device-initiated,flexible sensing capabilities in beyond 5G cellular communication systems. In this article, we specifically investigate a common issue related to sidelink-based RF-sensing, which is its angle and rotation dependence. In particular, we discuss transformations of mmWave point-cloud data which achieve rotational invariance, as well as distributed processing based on such rotational invariant inputs, at angle and distance diverse devices. To process the distributed data, we propose a graph based encoder to capture spatio-temporal features of the data and propose four approaches for multi-angle learning. The approaches are compared on a newly recorded and openly available dataset comprising 15 subjects, performing 21 gestures which are recorded from 8 angles.

Dariush Salami, Ramin Hasibi, Sameera Palipana et al.

We present Tesla-Rapture, a gesture recognition interface for point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g. Raspberry PI), XR hardware (e.g. HoloLens), or smart-phones. To tackle this issue, we developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on two datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios like a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.

Jing Ma, Si-Ahmed Naas, Stephan Sigg et al.

With the advance of machine learning and the internet of things (IoT), security and privacy have become key concerns in mobile services and networks. Transferring data to a central unit violates privacy as well as protection of sensitive data while increasing bandwidth demands.Federated learning mitigates this need to transfer local data by sharing model updates only. However, data leakage still remains an issue. In this paper, we propose xMK-CKKS, a multi-key homomorphic encryption protocol to design a novel privacy-preserving federated learning scheme. In this scheme, model updates are encrypted via an aggregated public key before sharing with a server for aggregation. For decryption, collaboration between all participating devices is required. This scheme prevents privacy leakage from publicly shared information in federated learning, and is robust to collusion between $k<N-1$ participating devices and the server. Our experimental evaluation demonstrates that the scheme preserves model accuracy against traditional federated learning as well as secure federated learning with homomorphic encryption (MK-CKKS, Paillier) and reduces computational cost compared to Paillier based federated learning. The average energy consumption is 2.4 Watts, so that it is suited to IoT scenarios.

René Mayrhofer, Vishwath Mohan, Stephan Sigg

Mobile device authentication has been a highly active research topic for over 10 years, with a vast range of methods having been proposed and analyzed. In related areas such as secure channel protocols, remote authentication, or desktop user authentication, strong, systematic, and increasingly formal threat models have already been established and are used to qualitatively and quantitatively compare different methods. Unfortunately, the analysis of mobile device authentication is often based on weak adversary models, suggesting overly optimistic results on their respective security. In this article, we first introduce a new classification of adversaries to better analyze and compare mobile device authentication methods. We then apply this classification to a systematic literature survey. The survey shows that security is still an afterthought and that most proposed protocols lack a comprehensive security analysis. Our proposed classification of adversaries provides a strong uniform adversary model that can offer a comparable and transparent classification of security properties in mobile device authentication methods.

Arne Brüsch, Ngu Nguyen, Dominik Schürmann et al.

Gait has been proposed as a feature for mobile device pairing across arbitrary positions on the human body. Results indicate that the correlation in gait-based features across different body locations is sufficient to establish secure device pairing. However, the population size of the studies is limited and powerful attackers with e.g. capability of video recording are not considered. We present a concise discussion of security properties of gait-based pairing schemes including a discussion of popular quantization schemes, classification and analysis of attack surfaces, discussion of statistical properties of generated sequences, an entropy analysis, as well as possible threats and security weaknesses of gait-based pairing systems. For one of the schemes considered, we present modifications to fix an identified security flaw. As a general limitation of gait-based authentication or pairing systems, we further demonstrate that an adversary with video support can create key sequences that are sufficiently close to on-body generated acceleration sequences to breach gait-based security mechanisms.

Stephan Sigg

This document summarises the results of several research campaigns over the past seven years. The main connecting theme is the physical layer of widely deployed sensors in Pervasive Computing domains. In particular, we have focused on the RF-channel or on ambient audio. The initial problem from which we started this work was that of distributed adaptive transmit beamforming. We have been looking for a simple method to align the phases of jointly transmitting nodes (e.g. sensor or IoT nodes). The algorithmic solution to this problem was to implement a distributed random optimisation method on the participating nodes in which the transmitters and the receiver follow an iterative question-and-answer scheme. We have been able to derive sharp asymptotic bounds on the expected optimisation time of an evolutionary random optimiser and presented an asymptotically optimal approach. One thing that we have learned from the work on these physical layer algorithms was that the signals we work on are fragile and perceptive to physical environmental changes. These could be obstacles such as furniture, opened or closed windows or doors as well as movement of individuals. This observation motivated us to view the wireless interface as a sensor for environmental changes in Pervasive Computing environments. Another use of physical layer RF-signals is for security applications. We are currently working to further push these mentioned directions and novel fields of physical prototyping. In particular, the calculation of mathematical operations on the wireless channel at the time of transmission appears to contain good potential for gains in efficiency for communication and computation in Pervasive Computing domains.

Le Ngu Nguyen, Stephan Sigg

Personal devices (e.g. laptops, tablets, and mobile phones) are conventional in daily life and have the ability to store users' private data. The security problems related to these appliances have become a primary concern for both users and researchers. In this paper, we analyse first-person-view videos to develop a personalized user authentication mechanism. Our proposed algorithm generates provisional image-based passwords which benefit a variety of purposes such as unlocking a mobile device or fallback authentication. First, representative frames are extracted from the egocentric videos. Then, they are split into distinguishable segments before a clustering procedure is applied to discard repetitive scenes. The whole process aims to retain memorable images to form the authentication challenges. We integrate eye tracking data to select informative sequences of video frames and suggest a blurriness-based method if an eye-facing camera is not available. To evaluate our system, we perform experiments in different settings including object-interaction activities and traveling contexts. Even though our mechanism produces variable graphical passwords, the log-in effort for the user is comparable with approaches based on static challenges. We verified the authentication challenges in the presence of a random and an informed attacker who is familiar with the environment and observed that the time required and the number of attempts are significantly higher than for the legitimate user, making it possible to detect attacks on the authentication system.

Muneeba Raja, Stephan Sigg

We present RFexpress! the first-ever network-edge based system to recognize emotion from movement, gesture and pose via Device-Free Activity Recognition (DFAR). With the proliferation of the IoT, also wireless access points are deployed at increasingly dense scale. in particular, this includes vehicular nodes (in-car WiFi or Bluetooth), office (Wlan APs, WiFi printer or projector) and private indoor domains (home WiFi mesh, Wireless media access), as well as public spaces (City/open WiFi, Cafes, shopping spaces). Processing RF-fluctuation at such edge-devices, enables environmental perception. In this paper, we focus on the distinction between neutral and agitated emotional states of humans from RF-fluctuation at the wireless network edge in realistic environments. In particular, the system is able to detect risky driving behaviour in a vehicular setting as well as spotting angry conversations in an indoor environment. We also study the effectiveness of edge-based DFAR emotion and activity recognition systems in real environments such as cafes, malls, outdoor and office spaces. We measure radio characteristics in these environments at different days and times and analyse the impact of variations in the Signal to Noise Ratio (SNR) on the accuracy of DFAR emotion and activity recognition. In a case study with 5 subjects, we then exploit the limits of edge-based DFAR by deriving critical SNR values under which activity and emotion recognition results are no longer reliable. In case studies with 8 and 5 subjects the system further could achieve recognition accuracies of 82.9\% and 64\% for vehicular and stationary wireless network edge in the wild (non-laboratory noisy environments and non-scripted, natural individual behaviour patterns).

Dominik Schürmann, Arne Brüsch, Stephan Sigg et al.

Secure spontaneous authentication between devices worn at arbitrary location on the same body is a challenging, yet unsolved problem. We propose BANDANA, the first-ever implicit secure device-to-device authentication scheme for devices worn on the same body. Our approach leverages instantaneous variation in acceleration patterns from gait sequences to extract always-fresh secure secrets. It enables secure spontaneous pairing of devices worn on the same body or interacted with. The method is robust against noise in sensor readings and active attackers. We demonstrate the robustness of BANDANA on two gait datasets and discuss the discriminability of intra- and inter-body cases, robustness to statistical bias, as well as possible attack scenarios.

Stephan Sigg, Kai Kunze, Xiaoming Fu

Ubiquitous sensing is tightly coupled with activity recognition. This survey reviews recent advances in Ubiquitous sensing and looks ahead on promising future directions. In particular, Ubiquitous sensing crosses new barriers giving us new ways to interact with the environment or to inspect our psyche. Through sensing paradigms that parasitically utilise stimuli from the noise of environmental, third-party pre-installed systems, sensing leaves the boundaries of the personal domain. Compared to previous environmental sensing approaches, these new systems mitigate high installation and placement cost by providing a robustness towards process noise. On the other hand, sensing focuses inward and attempts to capture mental activities such as cognitive load, fatigue or emotion through advances in, for instance, eye-gaze sensing systems or interpretation of body gesture or pose. This survey summarises these developments and discusses current research questions and promising future directions.