Monireh Abdoos

Pouria Yazdani, Arash Rezaali, Monireh Abdoos

Multi-agent reinforcement learning (MARL) has emerged as a promising paradigm for adaptive traffic signal control (ATSC) of multiple intersections. Existing approaches typically follow either a fully centralized or a fully decentralized design. Fully centralized approaches suffer from the curse of dimensionality, and reliance on a single learning server, whereas purely decentralized approaches operate under severe partial observability and lack explicit coordination resulting in suboptimal performance. These limitations motivate region-based MARL, where the network is partitioned into smaller, tightly coupled intersections that form regions, and training is organized around these regions. This paper introduces a Semi-Centralized Training, Decentralized Execution (SEMI-CTDE) architecture for multi intersection ATSC. Within each region, SEMI-CTDE performs centralized training with regional parameter sharing and employs composite state and reward formulations that jointly encode local and regional information. The architecture is highly transferable across different policy backbones and state-reward instantiations. Building on this architecture, we implement two models with distinct design objectives. A multi-perspective experimental analysis of the two implemented SEMI-CTDE-based models covering ablations of the architecture's core elements including rule based and fully decentralized baselines shows that they achieve consistently superior performance and remain effective across a wide range of traffic densities and distributions.

Mohammad Mahdi Maghouli, Mohamadreza Fereydooni, Monireh Abdoos et al.

With the advent of new technologies, using various formats of digital gadgets is becoming widespread. In today's world, where everyday tasks are inevitable without technology, this extensive use of computers paves the way for malicious activity. As a result, it is important to provide solutions to defend against these threats. Malware is one of the well-known and widely used means utilized for doing destructive activities by malicious attackers. Producing malware from scratch is somewhat difficult, so attackers tend to obfuscate existing malware and prepare it to become an unrecognizable program. Since creating new malware from an old one using obfuscation is a creative task, there are some drawbacks to identifying obfuscated malwares. In this research, we propose a solution to overcome this problem by converting the code to an image in the first step and then using a semi-supervised approach combined with contrastive learning. In this case, an obfuscation in the malware bytecode corresponds to an augmentation in the image. Hence, by utilizing meaningful augmentations, which simulate some obfuscation changes and combine them to generate complex ambiguity procedures, our proposed solution is able to construct, learn, and detect a wide range of obfuscations. This work addresses two issues: 1) malware classification despite the data deficiency and 2) obfuscated malware detection by training on non-obfuscated malwares. According to the results, the proposed method overcomes the data shortage problem in malware classification, as its accuracy is 90.1% when just 10% of data is used for training the model. Moreover, training on basic malwares without obfuscation achieved 96.21 percent accuracy in detecting obfuscated malware.

Mahnoosh Mahdavimoghaddam, Amin Nikanjam, Monireh Abdoos

Nowadays, cooperative multi-agent systems are used to learn how to achieve goals in large-scale dynamic environments. However, learning in these environments is challenging: from the effect of search space size on learning time to inefficient cooperation among agents. Moreover, reinforcement learning algorithms may suffer from a long time of convergence in such environments. In this paper, a communication framework is introduced. In the proposed communication framework, agents learn to cooperate effectively and also by introduction of a new state calculation method the size of state space will decline considerably. Furthermore, a knowledge-transferring algorithm is presented to share the gained experiences among the different agents, and develop an effective knowledge-fusing mechanism to fuse the knowledge learnt utilizing the agents' own experiences with the knowledge received from other team members. Finally, the simulation results are provided to indicate the efficacy of the proposed method in the complex learning task. We have evaluated our approach on the shepherding problem and the results show that the learning process accelerates by making use of the knowledge transferring mechanism and the size of state space has declined by generating similar states based on state abstraction concept.