Jeroen Famaey

Nabeel Nisar Bhat, Rafael Berkvens, Jeroen Famaey

In recent years, channel state information (CSI) at sub-6 GHz has been widely exploited for Wi-Fi sensing, particularly for activity and gesture recognition. In this work, we instead explore mmWave (60 GHz) Wi-Fi signals for gesture recognition/pose estimation. Our focus is on the mmWave Wi-Fi signals so that they can be used not only for high data rate communication but also for improved sensing e.g., for extended reality (XR) applications. For this reason, we extract spatial beam signal-to-noise ratios (SNRs) from the periodic beam training employed by IEEE 802.11ad devices. We consider a set of 10 gestures/poses motivated by XR applications. We conduct experiments in two environments and with three people.As a comparison, we also collect CSI from IEEE 802.11ac devices. To extract features from the CSI and the beam SNR, we leverage a deep neural network (DNN). The DNN classifier achieves promising results on the beam SNR task with state-of-the-art 96.7% accuracy in a single environment, even with a limited dataset. We also investigate the robustness of the beam SNR against CSI across different environments. Our experiments reveal that features from the CSI generalize without additional re-training, while those from beam SNRs do not. Therefore, re-training is required in the latter case.

Filip Lemic, Jakob Struye, Jeroen Famaey

Full-immersive multiuser Virtual Reality (VR) envisions supporting unconstrained mobility of the users in the virtual worlds, while at the same time constraining their physical movements inside VR setups through redirected walking. For enabling delivery of high data rate video content in real-time, the supporting wireless networks will leverage highly directional communication links that will "track" the users for maintaining the Line-of-Sight (LoS) connectivity. Recurrent Neural Networks (RNNs) and in particular Long Short-Term Memory (LSTM) networks have historically presented themselves as a suitable candidate for near-term movement trajectory prediction for natural human mobility, and have also recently been shown as applicable in predicting VR users' mobility under the constraints of redirected walking. In this work, we extend these initial findings by showing that Gated Recurrent Unit (GRU) networks, another candidate from the RNN family, generally outperform the traditionally utilized LSTMs. Second, we show that context from a virtual world can enhance the accuracy of the prediction if used as an additional input feature in comparison to the more traditional utilization of solely the historical physical movements of the VR users. Finally, we show that the prediction system trained on a static number of coexisting VR users be scaled to a multi-user system without significant accuracy degradation.

Filip Lemic, Jakob Struye, Thomas Van Onsem et al.

The advancement of Virtual Reality (VR) technology is focused on improving its immersiveness, supporting multiuser Virtual Experiences (VEs), and enabling users to move freely within their VEs while remaining confined to specialized VR setups through Redirected Walking (RDW). To meet their extreme data-rate and latency requirements, future VR systems will require supporting wireless networking infrastructures operating in millimeter Wave (mmWave) frequencies that leverage highly directional communication in both transmission and reception through beamforming and beamsteering. We propose the use of predictive context-awareness to optimize transmitter and receiver-side beamforming and beamsteering. By predicting users' short-term lateral movements in multiuser VR setups with Redirected Walking (RDW), transmitter-side beamforming and beamsteering can be optimized through Line-of-Sight (LoS) "tracking" in the users' directions. At the same time, predictions of short-term orientational movements can be utilized for receiver-side beamforming for coverage flexibility enhancements. We target two open problems in predicting these two context information instances: i) predicting lateral movements in multiuser VR settings with RDW, and ii) generating synthetic head rotation datasets for training orientational movements predictors. Our experimental results demonstrate that Long Short-Term Memory (LSTM) networks feature promising accuracy in predicting lateral movements, and context-awareness stemming from VEs further enhances this accuracy. Additionally, we show that a TimeGAN-based approach for orientational data generation can create synthetic samples that closely match experimentally obtained ones.

Laura Rayón Ropero, Jasper De Laet, Filip Lemic et al.

Facial Emotion Recognition is a critical research area within Affective Computing due to its wide-ranging applications in Human Computer Interaction, mental health assessment and fatigue monitoring. Current FER methods predominantly rely on Deep Learning techniques trained on 2D image data, which pose significant privacy concerns and are unsuitable for continuous, real-time monitoring. As an alternative, we propose High-Frequency Wireless Sensing (HFWS) as an enabler of continuous, privacy-aware FER, through the generation of detailed 3D facial pointclouds via on-person sensors embedded in wearables. We present arguments supporting the privacy advantages of HFWS over traditional 2D imaging, particularly under increasingly stringent data protection regulations. A major barrier to adopting HFWS for FER is the scarcity of labeled 3D FER datasets. Towards addressing this issue, we introduce a FLAME-based method to generate 3D facial pointclouds from existing public 2D datasets. Using this approach, we create AffectNet3D, a 3D version of the AffectNet database. To evaluate the quality and usability of the generated data, we design a pointcloud refinement pipeline focused on isolating the facial region, and train the popular PointNet++ model on the refined pointclouds. Fine-tuning the model on a small subset of the unseen 3D FER dataset BU-3DFE yields a classification accuracy exceeding 70%, comparable to oracle-level performance. To further investigate the potential of HFWS-based FER for continuous monitoring, we simulate wearable sensing conditions by masking portions of the generated pointclouds. Experimental results show that models trained on AffectNet3D and fine-tuned with just 25% of BU-3DFE outperform those trained solely on BU-3DFE. These findings highlight the viability of our pipeline and support the feasibility of continuous, privacy-aware FER via wearable HFWS systems.

David E. Ruiz-Guirola, Samer Nasser, Bikramjit Singh et al.

Ambient Internet of Things (A-IoT) targets energy harvesting (EH), battery-less devices as a simple connectivity solution for extensive ultra-low-power deployments. These devices typically face intermittent energy availability, making uplink reports increasingly susceptible to access collisions and energy outages. In this paper, we build upon the cellular standardization of A-IoT and examine the paging-triggered contention-based random access (CBRA) framework for uplink reporting. We analyze the effects of energy availability and collisions on these systems and introduce an EH-aware access control mechanism. In this mechanism, the reader broadcasts an access probability in the paging message, which helps regulate the number of devices attempting random access. Results show that, unlike the baselines, the proposed method scales well under dense deployments by keeping collisions nearly constant, improving access efficiency, and substantially reducing the number of paging rounds required for successful reporting. These results highlight the importance of lightweight reader-side access control for reliable and resource-efficient reporting in A-IoT environments.

Xinlei Liu, Andrey Belogaev, Jonathan Oostvogels et al.

The latency gap between wired and wireless networks poses a challenge in the adoption of wireless technologies in latency-sensitive scenarios. The gap is especially notable in multi-hop communication typical for industrial sensor networks and robotic swarms. The main reason behind it is that commonly used wireless protocols rely on store-and-forward routing and costly overhead procedures to avoid interference. This article introduces RF-Zero-Wire, an RF-based symbol-synchronous communication protocol. Instead of relaying the whole frame per hop in a store-and-forward manner, nodes concurrently relay the frame symbol by symbol, without the need for tight time synchronization. Based on data collected in real-world experiments, we reveal that the inevitable carrier frequency offsets (CFOs) introduced by imperfect crystal oscillators cause a beating effect under concurrent symbol transmissions. This is characterized by periodic constructive and destructive interference, which significantly affects reliability. Subsequently, a thorough simulation study shows how the beating problem can be overcome with error correction codes. RF-Zero-Wire allows achieving an end-to-end latency of less than 1ms for a small 4-byte frame transmitted across 5 hops. Moreover, latency is shown to increase only by 0.16% per extra hop for 16-byte frames, which is negligible compared to the over 100% per-hop latency increase observed in store-and-forward protocols. The trade-offs between network reliability and CFO range, communication distance, node density, and achievable data rate are studied in large-scale experiments based on simulation.

Xinlei Liu, Andrey Belogaev, Jeroen Famaey

Ambient Internet of Things (A-IoT) devices, as a critical enabler of future green IoT networks, have attracted broad interest from both industry and academia due to their ability to operate without batteries and with low maintenance costs. To accommodate their dynamic and constrained energy budget, an ultra-low-power connectivity protocol is required. Due to the severely limited transmit power of A-IoT devices, multi-hop connectivity is an interesting paradigm to extend their range. However, commonly used protocols for multi-hop communication may not be suitable for A-IoT due to excessive overhead related to channel access procedures, coordinated routing, and tight time synchronization requirements. This paper presents a novel network connectivity protocol based on symbol-synchronous transmissions, which allows battery-less relay nodes to participate in the forwarding process in an ad-hoc manner, without the need for synchronization or coordination. This allows them to adapt their duty cycle to the available harvested energy. Simulation results show that the proposed protocol can ensure high reliability in data packet delivery while significantly reducing the energy consumption of each relay node. We also investigate the relationship between wake-up probability and network density. For example, a 400-node network in a 625 m2 area can achieve a packet error rate below 1 % with an average awake time of 6 % per node, achieving an energy consumption reduction of 88 % compared to the baseline approach.

Jakob Struye, Filip Lemic, Jeroen Famaey

Extended Reality is a revolutionary method of delivering multimedia content to users. A large contributor to its popularity is the sense of immersion and interactivity enabled by having real-world motion reflected in the virtual experience accurately and immediately. This user motion, mainly caused by head rotations, induces several technical challenges. For instance, which content is generated and transmitted depends heavily on where the user is looking. Seamless systems, taking user motion into account proactively, will therefore require accurate predictions of upcoming rotations. Training and evaluating such predictors requires vast amounts of orientational input data, which is expensive to gather, as it requires human test subjects. A more feasible approach is to gather a modest dataset through test subjects, and then extend it to a more sizeable set using synthetic data generation methods. In this work, we present a head rotation time series generator based on TimeGAN, an extension of the well-known Generative Adversarial Network, designed specifically for generating time series. This approach is able to extend a dataset of head rotations with new samples closely matching the distribution of the measured time series.

Nabeel Nisar Bhat, Maksim Karnaukh, Jakob Struye et al.

Person identification plays a vital role in enabling intelligent, personalized, and secure human-computer interaction. Recent research has demonstrated the feasibility of leveraging Wi-Fi signals for passive person identification using a person's unique gait pattern. Although most existing work focuses on sub-6 GHz frequencies, the emergence of mmWave offers new opportunities through its finer spatial resolution, though its comparative advantages for person identification remain unexplored. This work presents the first comparative study between sub-6 GHz and mmWave Wi-Fi signals for person identification with commercial off-the-shelf (COTS) Wi-Fi, using a novel dataset of synchronized measurements from the two frequency bands in an indoor environment. To ensure a fair comparison, we apply identical training pipelines and model configurations across both frequency bands. Leveraging end-to-end deep learning, we show that even at low sampling rates (10 Hz), mmWave Wi-Fi signals can achieve high identification accuracy (91.2% on 20 individuals) when combined with effective background subtraction.

Nabeel Nisar Bhat, Maksim Karnaukh, Stein Vandenbroeke et al.

This article presents mmHSense, a set of open labeled mmWave datasets to support human sensing research within Integrated Sensing and Communication (ISAC) systems. The datasets can be used to explore mmWave ISAC for various end applications such as gesture recognition, person identification, pose estimation, and localization. Moreover, the datasets can be used to develop and advance signal processing and deep learning research on mmWave ISAC. This article describes the testbed, experimental settings, and signal features for each dataset. Furthermore, the utility of the datasets is demonstrated through validation on a specific downstream task. In addition, we demonstrate the use of parameter-efficient fine-tuning to adapt ISAC models to different tasks, significantly reducing computational complexity while maintaining performance on prior tasks.

Nabeel Nisar Bhat, Rafael Berkvens, Jeroen Famaey

In recent years, Joint Communication and Sensing (JC&S), has demonstrated significant success, particularly in utilizing sub-6 GHz frequencies with commercial-off-the-shelf (COTS) Wi-Fi devices for applications such as localization, gesture recognition, and pose classification. Deep learning and the existence of large public datasets has been pivotal in achieving such results. However, at mmWave frequencies (30-300 GHz), which has shown potential for more accurate sensing performance, there is a noticeable lack of research in the domain of COTS Wi-Fi sensing. Challenges such as limited research hardware, the absence of large datasets, limited functionality in COTS hardware, and the complexities of data collection present obstacles to a comprehensive exploration of this field. In this work, we aim to address these challenges by developing a method that can generate synthetic mmWave channel state information (CSI) samples. In particular, we use a generative adversarial network (GAN) on an existing dataset, to generate 30,000 additional CSI samples. The augmented samples exhibit a remarkable degree of consistency with the original data, as indicated by the notably high GAN-train and GAN-test scores. Furthermore, we integrate the augmented samples in training a pose classification model. We observe that the augmented samples complement the real data and improve the generalization of the classification model.

Ken Mendes, Filip Lemic, Jeroen Famaey

Radio Environmental Maps (REMs) are a powerful tool for enhancing the performance of various communication and networked agents. However, generating REMs is a laborious undertaking, especially in complex 3-Dimensional (3D) environments, such as indoors. To address this issue, we propose a system for autonomous generation of fine-grained REMs of indoor 3D spaces. In the system, multiple small indoor Unmanned Aerial Vehicles (UAVs) are sequentially used for 3D sampling of signal quality indicators. The collected readings are streamlined to a Machine Learning (ML) system for its training and, once trained, the system is able to predict the signal quality at unknown 3D locations. The system enables automated and autonomous REM generation, and can be straightforwardly deployed in new environments. In addition, the system supports REM sampling without self-interference and is technology-agnostic, as long as the REM-sampling receivers features suitable sizes and weights to be carried by the UAVs. In the demonstration, we instantiate the system design using two UAVs and show its capability of visiting 72 waypoints and gathering thousands of Wi-Fi data samples. Our results also include an instantiation of the ML system for predicting the Received Signal Strength (RSS) of known Wi-Fi Access Points (APs) at locations not visited by the UAVs.