Noha Ibrahim

Ihor Vitenki, Noha Ibrahim, Sihem Amer-Yahia

Reinforcement learning (RL) policies are typically trained for fixed objectives, making reuse difficult when task requirements change. We study inference-time policy reuse: given a library of pre-trained policies and a new composite objective, can a high-quality policy be constructed entirely offline, without additional environment interaction? We introduce lever (Leveraging Efficient Vector Embeddings for Reusable policies), an end-to-end framework that retrieves relevant policies, evaluates them using behavioral embeddings, and composes new policies via offline Q-value composition. We focus on the support-limited regime, where no value propagation is possible, and show that the effectiveness of reuse depends critically on the coverage of available transitions. To balance performance and computational cost, lever proposes composition strategies that control the exploration of candidate policies. Experiments in deterministic GridWorld environments show that inference-time composition can match, and in some cases exceed, training-from-scratch performance while providing substantial speedups. At the same time, performance degrades when long-horizon dependencies require value propagation, highlighting a fundamental limitation of offline reuse.

Ricardo Pedro Querido Andrade Silva, Nassim Bouarour, Dina Fettache et al.

Quiz design is a tedious process that teachers undertake to evaluate the acquisition of knowledge by students. Our goal in this paper is to automate quiz composition from a set of multiple choice questions (MCQs). We formalize a generic sequential decision-making problem with the goal of training an agent to compose a quiz that meets the desired topic coverage and difficulty levels. We investigate DQN, SARSA and A2C/A3C, three reinforcement learning solutions to solve our problem. We run extensive experiments on synthetic and real datasets that study the ability of RL to land on the best quiz. Our results reveal subtle differences in agent behavior and in transfer learning with different data distributions and teacher goals. This was supported by our user study, paving the way for automating various teachers' pedagogical goals.

Noha Ibrahim, Frédéric Le Mouël, Stéphane Frénot

A computing infrastructure where everything is a service offers many new system and application possibilities. Among the main challenges, however, is the issue of service substitution for the application execution in such heterogeneous environments. An application would like to continue to execute even when a service disappears, or it would like to benefit from the environment by using better services with better QoS when possible. In this article, we define a generic service model and describe the equivalence relations between services considering the functionalities they propose and their non functional QoS properties. We define semantic equivalence relations between services and equivalence degree between non functional QoS properties. Using these relations we propose semantic substitution mechanisms upon the appearance and disappearance of services that fits the application needs. We developed a prototype as a proof of concept and evaluated its efficiency over a real use case.