Hirley Alves

Pedro H. J. Nardelli, Mauricio de Castro Tomé, Hirley Alves et al.

This paper assesses the communication link from smart meters to aggregators as (unlicensed) secondary users that transmit their data over the (licensed) primary uplink channel. The proposed scenario assumes: (i) meters' and aggregators' positions are fixed so highly directional antennas are employed, (ii) secondary users transmit with limited power in relation to the primary, (iii) meters' transmissions are coordinated to avoid packet collisions, and (iv) the secondary links' robustness is guaranteed by an outage constraint. Under these assumptions, the interference caused by secondary users in both primary (base-stations) and other secondary users can be neglected. As unlicensed users, however, meter-aggregator links do experience interference from the mobile users of the primary network, whose positions and traffic activity are unknown. To cope with this uncertainty, we model the mobile users spatial distribution as a Poisson point process. We then derive a closed-form solution for the maximum achievable throughput with respect to a reference secondary link subject to transmit power and outage constraints. Our numerical results illustrate the effects of such constraints on the optimal throughput, evincing that more frequent outage events improve the system performance in the scenario under study. We also show that relatively high outage probabilities have little effect on the reconstruction of the average power demand curve that is transmitted from the smart-meter to the aggregator.

Mauricio C. Tomé, Pedro H. J. Nardelli, Hirley Alves

This paper studies a specific low-power wireless technology capable of reaching a long range, namely LoRa. Such a technology can be used by different applications in cities involving many transmitting devices while requiring loose communication constrains. We focus on electricity grids, where LoRa end-devices are smart-meters that send the average power demanded by their respective households during a given period. The successfully decoded data by the LoRa gateway are used by an aggregator to reconstruct the daily households' profiles. We show how the interference from concurrent transmissions from both LoRa and non-LoRa devices negatively affect the communication outage probability and the link effective bit-rate. Besides, we use actual electricity consumption data to compare time-based and event-based sampling strategies, showing the advantages of the latter. We then employ this analysis to assess the gateway range that achieves an average outage probability that leads to a signal reconstruction with a given requirement. We also discuss that, although the proposed analysis focuses on electricity metering, it can be easily extended to any other smart city application with similar requirements, like water metering or traffic monitoring.

Eslam Eldeeb, Dian Echevarría Pérez, Jean Michel de Souza Sant'Ana et al.

Many emerging Internet of Things (IoT) applications rely on information collected by sensor nodes where the freshness of information is an important criterion. \textit{Age of Information} (AoI) is a metric that quantifies information timeliness, i.e., the freshness of the received information or status update. This work considers a setup of deployed sensors in an IoT network, where multiple unmanned aerial vehicles (UAVs) serve as mobile relay nodes between the sensors and the base station. We formulate an optimization problem to jointly plan the UAVs' trajectory, while minimizing the AoI of the received messages. This ensures that the received information at the base station is as fresh as possible. The complex optimization problem is efficiently solved using a deep reinforcement learning (DRL) algorithm. In particular, we propose a deep Q-network, which works as a function approximation to estimate the state-action value function. The proposed scheme is quick to converge and results in a lower AoI than the random walk scheme. Our proposed algorithm reduces the average age by approximately $25\%$ and requires down to $50\%$ less energy when compared to the baseline scheme.

Pedro H. J. Nardelli, Hirley Alves, Antti Pinomaa et al.

This paper investigates the possibility of building the Energy Internet via a packetized management of non-industrial loads. The proposed solution is based on the cyber-physical implementation of energy packets where flexible loads send use requests to an energy server. Based on the existing literature, we explain how and why this approach could scale up to interconnected micro-grids, also pointing out the challenges involved in relation to the physical deployment of electricity network. We then assess how machine-type wireless communications, as part of 5G and beyond systems, will achieve the low latency and ultra reliability needed by the micro-grid protection while providing the massive coverage needed by the packetized management. This more distributed grid organization also requires localized governance models. We cite few existing examples as local markets, energy communities and micro-operator that support such novel arrangements. We close the paper by providing an overview of ongoing activities that support the proposed vision and possible ways to move forward.

Eslam Eldeeb, Hirley Alves

Mission critical applications, such as UAV-assisted IoT networks require risk-aware decision-making under dynamic topologies and uncertain channels. We propose meta-conservative quantile regression (M-CQR), a meta-offline distributional MARL algorithm that integrates conservative Q-learning (CQL) for safe offline learning, quantile regression DQN (QR-DQN) for risk-sensitive value estimation, and model-agnostic meta-learning (MAML) for rapid adaptation. Two variants are developed: meta-independent CQR (M-I-CQR) and meta-CTDE-CQR. In a UAV-based communication scenario, M-CTDE-CQR achieves up to 50% faster convergence and outperforms baseline MARL methods, offering improved scalability, robustness, and adaptability for risk-sensitive decision-making. Code is available at https://github.com/Eslam211/MA_Meta_ODRL

David E. Ruíz-Guirola, Carlos A. Rodríguez-López, Samuel Montejo-Sánchez et al.

Reducing energy consumption is a pressing issue in low-power machine-type communication (MTC) networks. In this regard, the Wake-up Signal (WuS) technology, which aims to minimize the energy consumed by the radio interface of the machine-type devices (MTDs), stands as a promising solution. However, state-of-the-art WuS mechanisms use static operational parameters, so they cannot efficiently adapt to the system dynamics. To overcome this, we design a simple but efficient neural network to predict MTC traffic patterns and configure WuS accordingly. Our proposed forecasting WuS (FWuS) leverages an accurate long-short term memory (LSTM)- based traffic prediction that allows extending the sleep time of MTDs by avoiding frequent page monitoring occasions in idle state. Simulation results show the effectiveness of our approach. The traffic prediction errors are shown to be below 4%, being false alarm and miss-detection probabilities respectively below 8.8% and 1.3%. In terms of energy consumption reduction, FWuS can outperform the best benchmark mechanism in up to 32%. Finally, we certify the ability of FWuS to dynamically adapt to traffic density changes, promoting low-power MTC scalability

Kaiqiang Lin, Muhammad Asad Ullah, Hirley Alves et al.

The integration of subterranean LoRaWAN and non-terrestrial networks (NTN) delivers substantial economic and societal benefits in remote agriculture and disaster rescue operations. The LoRa modulation leverages quasi-orthogonal spreading factors (SFs) to optimize data rates, airtime, coverage and energy consumption. However, it is still challenging to effectively assign SFs to end devices for minimizing co-SF interference in massive subterranean LoRaWAN NTN. To address this, we investigate a reinforcement learning (RL)-based SFs allocation scheme to optimize the system's energy efficiency (EE). To efficiently capture the device-to-environment interactions in dense networks, we proposed an SFs allocation technique using the multi-agent dueling double deep Q-network (MAD3QN) and the multi-agent advantage actor-critic (MAA2C) algorithms based on an analytical reward mechanism. Our proposed RL-based SFs allocation approach evinces better performance compared to four benchmarks in the extreme underground direct-to-satellite scenario. Remarkably, MAD3QN shows promising potentials in surpassing MAA2C in terms of convergence rate and EE.

Eslam Eldeeb, Hirley Alves

IoT has a significant role in the smart campus. This paper presents a detailed description of the Smart Campus dataset based on LoRaWAN. LoRaWAN is an emerging technology that enables serving hundreds of IoT devices. First, we describe the LoRa network that connects the devices to the server. Afterward, we analyze the missing transmissions and propose a k-nearest neighbor solution to handle the missing values. Then, we predict future readings using a long short-term memory (LSTM). Finally, as one example application, we build a deep neural network to predict the number of people inside a room based on the selected sensor's readings. Our results show that our model achieves an accuracy of $95 \: \%$ in predicting the number of people. Moreover, the dataset is openly available and described in detail, which is opportunity for exploration of other features and applications.

Eslam Eldeeb, Mohammad Shehab, Hirley Alves

In many massive IoT communication scenarios, the IoT devices require coverage from dynamic units that can move close to the IoT devices and reduce the uplink energy consumption. A robust solution is to deploy a large number of UAVs (UAV swarm) to provide coverage and a better line of sight (LoS) for the IoT network. However, the study of these massive IoT scenarios with a massive number of serving units leads to high dimensional problems with high complexity. In this paper, we apply multi-agent deep reinforcement learning to address the high-dimensional problem that results from deploying a swarm of UAVs to collect fresh information from IoT devices. The target is to minimize the overall age of information in the IoT network. The results reveal that both cooperative and partially cooperative multi-agent deep reinforcement learning approaches are able to outperform the high-complexity centralized deep reinforcement learning approach, which stands helpless in large-scale networks.

Mauricio de Castro Tomé, Pedro. H. J. Nardelli, Hirley Alves

This paper provides an in-depth analysis of the event-based metering strategy proposed by Simonov et al. This strategy is an alternative to the traditional periodic (time-based) metering where the power demand is averaged in fixed time periods (e.g. every 15 minutes). The event-based approach considers two thresholds that trigger an event, one related to the (instantaneous) power demanded, other to the accumulated energy consumed. The original work assumed these thresholds fixed for the measurements. Our present contribution relaxes this assumption by proposing a method to set the thresholds from the percentage of the peak power consumption over the period under analysis. This approach, in contrast to the time-based and the fixed thresholds, better captures the actual power demanded when different households with diverse power demand profiles are studied. In this sense, our method provides a more efficient way to store electricity demand data while maintaining the estimation error (in relation to the real-time power demand) under acceptable values. Numerical examples are presented to illustrate the advantage and possible drawbacks of the proposed method.

Prasoon Raghuwanshi, Onel Luis Alcaraz López, Neelesh B. Mehta et al.

Efficient Random Access (RA) is critical for enabling reliable communication in Industrial Internet of Things (IIoT) networks. Herein, we propose a deep reinforcement learning based distributed RA scheme, entitled Neural Network-Based Bandit (NNBB), for the IIoT alarm scenario. In such a scenario, the devices may detect a common critical event, and the goal is to ensure the alarm information is delivered successfully from at least one device. The proposed NNBB scheme is implemented at each device, where it trains itself online and establishes implicit inter-device coordination to achieve the common goal. Devices can transmit simultaneously on multiple orthogonal channels and each possible transmission pattern constitutes a possible action for the NNBB, which uses a deep neural network to determine the action. Our simulation results show that as the number of devices in the network increases, so does the performance gain of the NNBB compared to the Multi-Armed Bandit (MAB) RA benchmark. For instance, NNBB experiences a 7% success rate drop when there are four channels and the number of devices increases from 10 to 60, while MAB faces a 25% drop.

Pedro H. J. Nardelli, Hirley Alves

This letter introduces a method to manage energy storage in electricity grids. Starting from the stochastic characterization of electricity generation and demand, we propose an equation that relates the storage level for every time-step as a function of its previous state and the realized surplus/deficit of electricity. Therefrom, we can obtain the probability that, in the next time-step: (i) there is a generation surplus that cannot be stored, or (ii) there is a demand need that cannot be supplied by the available storage. We expect this simple procedure can be used as the basis of electricity self-management algorithms in micro-level (e.g. individual households) or in meso-level (e.g. groups of houses).

Eslam Eldeeb, Mohammad Shehab, Hirley Alves et al.

Semantic and goal-oriented (SGO) communication is an emerging technology that only transmits significant information for a given task. Semantic communication encounters many challenges, such as computational complexity at end users, availability of data, and privacy-preserving. This work presents a TinyML-based semantic communication framework for few-shot wireless image classification that integrates split-learning and meta-learning. We exploit split-learning to limit the computations performed by the end-users while ensuring privacy-preserving. In addition, meta-learning overcomes data availability concerns and speeds up training by utilizing similarly trained tasks. The proposed algorithm is tested using a data set of images of hand-written letters. In addition, we present an uncertainty analysis of the predictions using conformal prediction (CP) techniques. Simulation results show that the proposed Semantic-MSL outperforms conventional schemes by achieving 20 % gain on classification accuracy using fewer data points, yet less training energy consumption.

Eslam Eldeeb, Hirley Alves

Reinforcement learning (RL) has proved to have a promising role in future intelligent wireless networks. Online RL has been adopted for radio resource management (RRM), taking over traditional schemes. However, due to its reliance on online interaction with the environment, its role becomes limited in practical, real-world problems where online interaction is not feasible. In addition, traditional RL stands short in front of the uncertainties and risks in real-world stochastic environments. In this manner, we propose an offline and distributional RL scheme for the RRM problem, enabling offline training using a static dataset without any interaction with the environment and considering the sources of uncertainties using the distributions of the return. Simulation results demonstrate that the proposed scheme outperforms conventional resource management models. In addition, it is the only scheme that surpasses online RL with a 10 % gain over online RL.

Rumeshika Pallewela, Eslam Eldeeb, Hirley Alves

This paper introduces the minimum error entropy (MEE) criterion as an advanced information-theoretic loss function tailored for deep learning applications in wireless communications. The MEE criterion leverages higher-order statistical properties, offering robustness in noisy scenarios like Rayleigh fading and impulsive interference. In addition, we propose a less computationally complex version of the MEE function to enhance practical usability in wireless communications. The method is evaluated through simulations on two critical applications: over-the-air regression and indoor localization. Results indicate that the MEE criterion outperforms conventional loss functions, such as mean squared error (MSE) and mean absolute error (MAE), achieving significant performance improvements in terms of accuracy, over $20 \%$ gain over traditional methods, and convergence speed across various channel conditions. This work establishes MEE as a promising alternative for wireless communication tasks in deep learning models, enabling better resilience and adaptability.

Eslam Eldeeb, Mohammad Shehab, Hirley Alves

Task-oriented semantic communication is an emerging technology that transmits only the relevant semantics of a message instead of the whole message to achieve a specific task. It reduces latency, compresses the data, and is more robust in low SNR scenarios. This work presents a multi-task-oriented semantic communication framework for connected and autonomous vehicles (CAVs). We propose a convolutional autoencoder (CAE) that performs the semantic encoding of the road traffic signs. These encoded images are then transmitted from one CAV to another CAV through satellite in challenging weather conditions where visibility is impaired. In addition, we propose task-oriented semantic decoders for image reconstruction and classification tasks. Simulation results show that the proposed framework outperforms the conventional schemes, such as QAM-16, regarding the reconstructed image's similarity and the classification's accuracy. In addition, it can save up to 89 % of the bandwidth by sending fewer bits.

Eslam Eldeeb, Houssem Sifaou, Osvaldo Simeone et al.

Reinforcement learning (RL) has been widely adopted for controlling and optimizing complex engineering systems such as next-generation wireless networks. An important challenge in adopting RL is the need for direct access to the physical environment. This limitation is particularly severe in multi-agent systems, for which conventional multi-agent reinforcement learning (MARL) requires a large number of coordinated online interactions with the environment during training. When only offline data is available, a direct application of online MARL schemes would generally fail due to the epistemic uncertainty entailed by the lack of exploration during training. In this work, we propose an offline MARL scheme that integrates distributional RL and conservative Q-learning to address the environment's inherent aleatoric uncertainty and the epistemic uncertainty arising from the use of offline data. We explore both independent and joint learning strategies. The proposed MARL scheme, referred to as multi-agent conservative quantile regression, addresses general risk-sensitive design criteria and is applied to the trajectory planning problem in drone networks, showcasing its advantages.

Yaya Etiabi, Eslam Eldeeb, Mohammad Shehab et al.

Accurate indoor localization remains challenging due to variations in wireless signal environments and limited data availability. This paper introduces MetaGraphLoc, a novel system leveraging sensor fusion, graph neural networks (GNNs), and meta-learning to overcome these limitations. MetaGraphLoc integrates received signal strength indicator measurements with inertial measurement unit data to enhance localization accuracy. Our proposed GNN architecture, featuring dynamic edge construction (DEC), captures the spatial relationships between access points and underlying data patterns. MetaGraphLoc employs a meta-learning framework to adapt the GNN model to new environments with minimal data collection, significantly reducing calibration efforts. Extensive evaluations demonstrate the effectiveness of MetaGraphLoc. Data fusion reduces localization error by 15.92%, underscoring its importance. The GNN with DEC outperforms traditional deep neural networks by up to 30.89%, considering accuracy. Furthermore, the meta-learning approach enables efficient adaptation to new environments, minimizing data collection requirements. These advancements position MetaGraphLoc as a promising solution for indoor localization, paving the way for improved navigation and location-based services in the ever-evolving Internet of Things networks.

Eslam Eldeeb, Hirley Alves

The rapid growth of heterogeneous and massive wireless connectivity in 6G networks demands intelligent solutions to ensure scalability, reliability, privacy, ultra-low latency, and effective control. Although artificial intelligence (AI) and machine learning (ML) have demonstrated their potential in this domain, traditional online reinforcement learning (RL) and deep RL methods face limitations in real-time wireless networks. For instance, these methods rely on online interaction with the environment, which might be unfeasible, costly, or unsafe. In addition, they cannot handle the inherent uncertainties in real-time wireless applications. We focus on offline and distributional RL, two advanced RL techniques that can overcome these challenges by training on static datasets and accounting for network uncertainties. We introduce a novel framework that combines offline and distributional RL for wireless communication applications. Through case studies on unmanned aerial vehicle (UAV) trajectory optimization and radio resource management (RRM), we demonstrate that our proposed Conservative Quantile Regression (CQR) algorithm outperforms conventional RL approaches regarding convergence speed and risk management. Finally, we discuss open challenges and potential future directions for applying these techniques in 6G networks, paving the way for safer and more efficient real-time wireless systems.

Eslam Eldeeb, Hirley Alves

Reinforcement learning (RL) has been a promising essence in future 5G-beyond and 6G systems. Its main advantage lies in its robust model-free decision-making in complex and large-dimension wireless environments. However, most existing RL frameworks rely on online interaction with the environment, which might not be feasible due to safety and cost concerns. Another problem with online RL is the lack of scalability of the designed algorithm with dynamic or new environments. This work proposes a novel, resilient, few-shot meta-offline RL algorithm combining offline RL using conservative Q-learning (CQL) and meta-learning using model-agnostic meta-learning (MAML). The proposed algorithm can train RL models using static offline datasets without any online interaction with the environments. In addition, with the aid of MAML, the proposed model can be scaled up to new unseen environments. We showcase the proposed algorithm for optimizing an unmanned aerial vehicle (UAV) 's trajectory and scheduling policy to minimize the age-of-information (AoI) and transmission power of limited-power devices. Numerical results show that the proposed few-shot meta-offline RL algorithm converges faster than baseline schemes, such as deep Q-networks and CQL. In addition, it is the only algorithm that can achieve optimal joint AoI and transmission power using an offline dataset with few shots of data points and is resilient to network failures due to unprecedented environmental changes.

Eslam Eldeeb, Hirley Alves

Offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL, such as safety concerns, expensive data collection, extended training intervals, and high signaling overhead caused by online interactions with the environment. In this work, we propose an offline MARL algorithm for radio resource management (RRM), focusing on optimizing scheduling policies for multiple access points (APs) to jointly maximize the sum and tail rates of user equipment (UEs). We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE). Our simulation results demonstrate that the proposed offline MARL framework outperforms conventional baseline approaches, achieving over a 15\% improvement in a weighted combination of sum and tail rates. Additionally, the CTDE framework strikes an effective balance, reducing the computational complexity of centralized methods while addressing the inefficiencies of independent training. These results underscore the potential of offline MARL to deliver scalable, robust, and efficient solutions for resource management in dynamic wireless networks.

Sankani Sarathchandra, Eslam Eldeeb, Mohammad Shehab et al.

Age-of-information (AoI) and transmission power are crucial performance metrics in low energy wireless networks, where information freshness is of paramount importance. This study examines a power-limited internet of things (IoT) network supported by a flying unmanned aerial vehicle(UAV) that collects data. Our aim is to optimize the UAV flight trajectory and scheduling policy to minimize a varying AoI and transmission power combination. To tackle this variation, this paper proposes a meta-deep reinforcement learning (RL) approach that integrates deep Q-networks (DQNs) with model-agnostic meta-learning (MAML). DQNs determine optimal UAV decisions, while MAML enables scalability across varying objective functions. Numerical results indicate that the proposed algorithm converges faster and adapts to new objectives more effectively than traditional deep RL methods, achieving minimal AoI and transmission power overall.

Eslam Eldeeb, Mohammad Shehab, Hirley Alves

The age of information (AoI) is used to measure the freshness of the data. In IoT networks, the traditional resource management schemes rely on a message exchange between the devices and the base station (BS) before communication which causes high AoI, high energy consumption, and low reliability. Unmanned aerial vehicles (UAVs) as flying BSs have many advantages in minimizing the AoI, energy-saving, and throughput improvement. In this paper, we present a novel learning-based framework that estimates the traffic arrival of IoT devices based on Markovian events. The learning proceeds to optimize the trajectory of multiple UAVs and their scheduling policy. First, the BS predicts the future traffic of the devices. We compare two traffic predictors: the forward algorithm (FA) and the long short-term memory (LSTM). Afterward, we propose a deep reinforcement learning (DRL) approach to optimize the optimal policy of each UAV. Finally, we manipulate the optimum reward function for the proposed DRL approach. Simulation results show that the proposed algorithm outperforms the random-walk (RW) baseline model regarding the AoI, scheduling accuracy, and transmission power.

Eslam Eldeeb, Mohammad Shehab, Hirley Alves

The current random access (RA) allocation techniques suffer from congestion and high signaling overhead while serving massive machine type communication (mMTC) applications. To this end, 3GPP introduced the need to use fast uplink grant (FUG) allocation in order to reduce latency and increase reliability for smart internet-of-things (IoT) applications with strict QoS constraints. We propose a novel FUG allocation based on support vector machine (SVM), First, MTC devices are prioritized using SVM classifier. Second, LSTM architecture is used for traffic prediction and correction techniques to overcome prediction errors. Both results are used to achieve an efficient resource scheduler in terms of the average latency and total throughput. A Coupled Markov Modulated Poisson Process (CMMPP) traffic model with mixed alarm and regular traffic is applied to compare the proposed FUG allocation to other existing allocation techniques. In addition, an extended traffic model based CMMPP is used to evaluate the proposed algorithm in a more dense network. We test the proposed scheme using real-time measurement data collected from the Numenta Anomaly Benchmark (NAB) database. Our simulation results show the proposed model outperforms the existing RA allocation schemes by achieving the highest throughput and the lowest access delay of the order of 1 ms by achieving prediction accuracy of 98 $\%$ when serving the target massive and critical MTC applications with a limited number of resources.