Matin Hashemi

Sajad Mohaghegh, Mohammad Amin Ramezan Dehnavi, Golnoosh Abdollahinejad et al.

The paper presents the PCGPT framework, an innovative approach to procedural content generation (PCG) using offline reinforcement learning and transformer networks. PCGPT utilizes an autoregressive model based on transformers to generate game levels iteratively, addressing the challenges of traditional PCG methods such as repetitive, predictable, or inconsistent content. The framework models trajectories of actions, states, and rewards, leveraging the transformer's self-attention mechanism to capture temporal dependencies and causal relationships. The approach is evaluated in the Sokoban puzzle game, where the model predicts items that are needed with their corresponding locations. Experimental results on the game Sokoban demonstrate that PCGPT generates more complex and diverse game content. Interestingly, it achieves these results in significantly fewer steps compared to existing methods, showcasing its potential for enhancing game design and online content generation. Our model represents a new PCG paradigm which outperforms previous methods.

Amir Amirinezhad, Saber Salehkaleybar, Matin Hashemi

We study the problem of experiment design to learn causal structures from interventional data. We consider an active learning setting in which the experimenter decides to intervene on one of the variables in the system in each step and uses the results of the intervention to recover further causal relationships among the variables. The goal is to fully identify the causal structures with minimum number of interventions. We present the first deep reinforcement learning based solution for the problem of experiment design. In the proposed method, we embed input graphs to vectors using a graph neural network and feed them to another neural network which outputs a variable for performing intervention in each step. Both networks are trained jointly via a Q-iteration algorithm. Experimental results show that the proposed method achieves competitive performance in recovering causal structures with respect to previous works, while significantly reducing execution time in dense graphs.

Alireza Amirshahi, Matin Hashemi

This paper presents a novel ECG classification algorithm for real-time cardiac monitoring on ultra low-power wearable devices. The proposed solution is based on spiking neural networks which are the third generation of neural networks. In specific, we employ spike-timing dependent plasticity (STDP), and reward-modulated STDP (R-STDP), in which the model weights are trained according to the timings of spike signals, and reward or punishment signals. Experiments show that the proposed solution is suitable for real-time operation, achieves comparable accuracy with respect to previous methods, and more importantly, its energy consumption is significantly smaller than previous neural network based solutions.