Nader Mokari

Maryam Ansarifard, Nader Mokari, Mohammadreza Javan et al.

In this paper, we develop a hierarchical aerial computing framework composed of high altitude platform (HAP) and unmanned aerial vehicles (UAVs) to compute the fully offloaded tasks of terrestrial mobile users which are connected through an uplink non-orthogonal multiple access (UL-NOMA). To better assess the freshness of information in computation-intensive applications the criterion of age of information (AoI) is considered. In particular, the problem is formulated to minimize the average AoI of users with elastic tasks, by adjusting UAVs trajectory and resource allocation on both UAVs and HAP, which is restricted by the channel state information (CSI) uncertainty and multiple resource constraints of UAVs and HAP. In order to solve this non-convex optimization problem, two methods of multi-agent deep deterministic policy gradient (MADDPG) and federated reinforcement learning (FRL) are proposed to design the UAVs trajectory, and obtain channel, power, and CPU allocations. It is shown that task scheduling significantly reduces the average AoI. This improvement is more pronounced for larger task sizes. On one hand, it is shown that power allocation has a marginal effect on the average AoI compared to using full transmission power for all users. Compared with traditional transmission schemes, the simulation results show our scheduling scheme results in a substantial improvement in average AoI.

Narges Gholipoor, Ali Nouruzi, Shima Salarhosseini et al.

In this paper, we propose a joint radio and core resource allocation framework for NFV-enabled networks. In the proposed system model, the goal is to maximize energy efficiency (EE), by guaranteeing end-to-end (E2E) quality of service (QoS) for different service types. To this end, we formulate an optimization problem in which power and spectrum resources are allocated in the radio part. In the core part, the chaining, placement, and scheduling of functions are performed to ensure the QoS of all users. This joint optimization problem is modeled as a Markov decision process (MDP), considering time-varying characteristics of the available resources and wireless channels. A soft actor-critic deep reinforcement learning (SAC-DRL) algorithm based on the maximum entropy framework is subsequently utilized to solve the above MDP. Numerical results reveal that the proposed joint approach based on the SAC-DRL algorithm could significantly reduce energy consumption compared to the case in which R-RA and NFV-RA problems are optimized separately.

Mohammad Akbari, Mohammad Reza Abedi, Roghayeh Joda et al.

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.

Mohammad Parvini, Mohammad Reza Javan, Nader Mokari et al.

This paper investigates the problem of age of information (AoI) aware radio resource management for a platooning system. Multiple autonomous platoons exploit the cellular wireless vehicle-to-everything (C-V2X) communication technology to disseminate the cooperative awareness messages (CAMs) to their followers while ensuring timely delivery of safety-critical messages to the Road-Side Unit (RSU). Due to the challenges of dynamic channel conditions, centralized resource management schemes that require global information are inefficient and lead to large signaling overheads. Hence, we exploit a distributed resource allocation framework based on multi-agent reinforcement learning (MARL), where each platoon leader (PL) acts as an agent and interacts with the environment to learn its optimal policy. Existing MARL algorithms consider a holistic reward function for the group's collective success, which often ends up with unsatisfactory results and cannot guarantee an optimal policy for each agent. Consequently, motivated by the existing literature in RL, we propose a novel MARL framework that trains two critics with the following goals: A global critic which estimates the global expected reward and motivates the agents toward a cooperating behavior and an exclusive local critic for each agent that estimates the local individual reward. Furthermore, based on the tasks each agent has to accomplish, the individual reward of each agent is decomposed into multiple sub-reward functions where task-wise value functions are learned separately. Numerical results indicate our proposed algorithm's effectiveness compared with the conventional RL methods applied in this area.

Amirhossein Shaghaghi, Abolfazl Zakeri, Nader Mokari et al.

In this paper, we propose a Zero-Touch, deep reinforcement learning (DRL)-based Proactive Failure Recovery framework called ZT-PFR for stateful network function virtualization (NFV)-enabled networks. To this end, we formulate a resource-efficient optimization problem minimizing the network cost function including resource cost and wrong decision penalty. As a solution, we propose state-of-the-art DRL-based methods such as soft-actor-critic (SAC) and proximal-policy-optimization (PPO). In addition, to train and test our DRL agents, we propose a novel impending-failure model. Moreover, to keep network status information at an acceptable freshness level for appropriate decision-making, we apply the concept of age of information to strike a balance between the event and scheduling based monitoring. Several key systems and DRL algorithm design insights for ZT-PFR are drawn from our analysis and simulation results. For example, we use a hybrid neural network, consisting long short-term memory layers in the DRL agents structure, to capture impending-failures time dependency.

Moslem Forouzesh, Paeiz Azmi, Nader Mokari et al.

In this paper, we investigate the physical layer security in downlink of Power Domain Non Orthogonal Multiple Access based heterogeneous cellular network in presence of multiple eavesdroppers. Our aim is to maximize the sum secrecy rate of the network. To this end, we formulate joint subcarrier and power allocation optimization problems to increase sum secrecy rate. Moreover, we propose a novel scheme at which the eavesdroppers are prevented from doing Successive Interference Cancellation, while legitimate users are able to do it. In practical systems, availability of eavesdroppers Channel State Information is impractical, hence we consider two scenarios: 1 Perfect CSI of the eavesdroppers, 2 imperfect CSI of the eavesdroppers. Since the proposed optimization problems are nonconvex, we adopt the well known iterative algorithm called Alternative Search Method. In this algorithm, the optimization problems are converted to two subproblems, power allocation and subcarrier allocation. We solve the power allocation problem by the Successive Convex Approximation approach and solve the subcarrier allocation subproblem, by exploiting the Mesh Adaptive Direct Search algorithm. Moreover, in order to study the optimality gap of the proposed solution method, we apply the monotonic optimization method. Moreover, we evaluate the proposed scheme for secure massive connectivity in 5G networks. Numerical results highlight that the proposed scheme significantly improves the sum secrecy rate compared with the conventional case at which the eavesdroppers are able to apply SIC.

Moslem Forouzesh, Paeiz Azmi, Nader Mokari et al.

Information-theoretic secrecy, in particular the wiretap channel formulation, provides protection against interception of a message by adversary Eve and has been widely studied in the last two decades. In contrast, covert communications under an analogous formulation provides protection against even the detection of the presence of the message by an adversary, and it has drawn significant interest recently. These two security topics are generally applicable in different scenarios; however, here we explore what can be learned by studying them under a common framework. Under a similar but not identical mathematical formulation, we introduce power optimization problems for each of the secrecy and the covert communications scenario, and we exploit common aspects of the problems to employ similar tools in their respective optimizations. Moreover, due to the practical limitations, we assume only channel