Erdi Sayar

Hürkan Şahin, Van Huyen Dang, Erdi Sayar et al.

Reinforcement learning (RL) often struggles in real-world tasks with high-dimensional state spaces and long horizons, where sparse or fixed rewards severely slow down exploration and cause agents to get trapped in local optima. This paper presents a fuzzy logic based reward shaping method that integrates human intuition into RL reward design. By encoding expert knowledge into adaptive and interpreable terms, fuzzy rules promote stable learning and reduce sensitivity to hyperparameters. The proposed method leverages these properties to adapt reward contributions based on the agent state, enabling smoother transitions between fast motion and precise control in challenging navigation tasks. Extensive simulation results on autonomous drone racing benchmarks show stable learning behavior and consistent task performance across scenarios of increasing difficulty. The proposed method achieves faster convergence and reduced performance variability across training seeds in more challenging environments, with success rates improving by up to approximately 5 percent compared to non fuzzy reward formulations.

Artun Sel, Mehmet Koruturk, Erdi Sayar

This paper presents a novel method for estimating larger Region of Attractions (ROAs) for continuous-time nonlinear systems modeled via the Takagi-Sugeno (TS) framework. While classical approaches rely on a single TS representation derived from the original nonlinear system to compute an ROA using Lyapunov-based analysis, the proposed method enhances this process through a systematic coordinate transformation strategy. Specifically, we construct multiple TS models, each obtained from the original nonlinear system under a distinct linear coordinate transformation. Each transformed system yields a local ROA estimate, and the overall ROA is taken as the union of these individual estimates. This strategy leverages the variability introduced by the transformations to reduce conservatism and expand the certified stable region. Numerical examples demonstrate that this approach consistently provides larger ROAs compared to conventional single-model TS-based techniques, highlighting its effectiveness and potential for improved nonlinear stability analysis.

Erdi Sayar, Zhenshan Bing, Carlo D'Eramo et al.

Multi-goal robot manipulation tasks with sparse rewards are difficult for reinforcement learning (RL) algorithms due to the inefficiency in collecting successful experiences. Recent algorithms such as Hindsight Experience Replay (HER) expedite learning by taking advantage of failed trajectories and replacing the desired goal with one of the achieved states so that any failed trajectory can be utilized as a contribution to learning. However, HER uniformly chooses failed trajectories, without taking into account which ones might be the most valuable for learning. In this paper, we address this problem and propose a novel approach Contact Energy Based Prioritization~(CEBP) to select the samples from the replay buffer based on rich information due to contact, leveraging the touch sensors in the gripper of the robot and object displacement. Our prioritization scheme favors sampling of contact-rich experiences, which are arguably the ones providing the largest amount of information. We evaluate our proposed approach on various sparse reward robotic tasks and compare them with the state-of-the-art methods. We show that our method surpasses or performs on par with those methods on robot manipulation tasks. Finally, we deploy the trained policy from our method to a real Franka robot for a pick-and-place task. We observe that the robot can solve the task successfully. The videos and code are publicly available at: https://erdiphd.github.io/HER_force