Chao Wen

Chao Wen, Jacqueline Staub, Adish Singla

Large language and multimodal models have shown remarkable success on various benchmarks focused on specific skills such as general-purpose programming, math word problem-solving, and visual question answering. However, it is unclear how well these models perform on tasks that require a combination of these skills. In this paper, we curate a novel program synthesis benchmark based on the real-world tasks in the XLogoOnline visual programming environment. Each task requires a combination of different skills such as spatial planning, basic programming, and logical reasoning. Our evaluation shows that current state-of-the-art models like GPT-4V and Llama3-70B struggle to solve these tasks, achieving only 20% and 2.35% success rates, respectively. Next, we develop a fine-tuning pipeline to boost the performance of models by leveraging a large-scale synthetic training dataset with over 80,000 tasks. Moreover, we showcase how emulator-driven feedback can be used to design a curriculum over training data distribution, through which a fine-tuned Llama3-8B drastically outperforms GPT-4V and Llama3-70B models. Finally, we provide an in-depth failure analysis to understand the limitations of different models. We will publicly release the benchmark for future research on program synthesis in visual programming.

Tesfaye Hailemariam Yimer, Chao Wen, Xiaozhuo Yu et al.

When driving,it is vital to maintain the right following distance between the vehicles to avoid rear-end collisions. The minimum safe distance depends on many factors, however, in this study the safe distance between the human-driven vehicles and a fully autonomous vehicle at a sudden stop by an automatic emergency brake was studied based on the human driver ability to react in an accident, the vehicles' braking system performance, and the speed of vehicles. For this approach, a safe distance car-following model was proposed to describe the safe distance between vehicles on a single lane dry road under conditions where both vehicles keep moving at a constant speed, and a lead autonomous vehicle suddenly stops by automatic emergency braking at an imminent incident. The proposed model then finally was being tested using MATLAB simulation, and results showed that confirmed the effectiveness of this model and the influence of driving speed and inter-vehicle distance on the rear-end collision was also indicated as well compared with the two and three seconds rule of safe following distance. The three seconds safe distance following rules is safe to be applied for all speed limits; however, the two seconds can be used on speed limits up to 45 Km/hr. A noticeable increase in rear-end collision was observed according to the simulation results if a car follows a driverless vehicle with two seconds rule above 45 km/hr.