You Wu

Boshi Wang, Sewon Min, Xiang Deng et al. · deepmind, uw

Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). CoT explicitly encourages the LLM to generate intermediate rationales for solving a problem, by providing a series of reasoning steps in the demonstrations. Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance. In this paper, we show that CoT reasoning is possible even with invalid demonstrations - prompting with invalid reasoning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference. Further experiments show that other aspects of the rationales, such as being relevant to the query and correctly ordering the reasoning steps, are much more important for effective CoT reasoning. Overall, these findings both deepen our understanding of CoT prompting, and open up new questions regarding LLMs' capability to learn to reason in context.

Benita Wong, Joya Chen, You Wu et al.

A long-standing goal of intelligent assistants such as AR glasses/robots has been to assist users in affordance-centric real-world scenarios, such as "how can I run the microwave for 1 minute?". However, there is still no clear task definition and suitable benchmarks. In this paper, we define a new task called Affordance-centric Question-driven Task Completion, where the AI assistant should learn from instructional videos to provide step-by-step help in the user's view. To support the task, we constructed AssistQ, a new dataset comprising 531 question-answer samples from 100 newly filmed instructional videos. We also developed a novel Question-to-Actions (Q2A) model to address the AQTC task and validate it on the AssistQ dataset. The results show that our model significantly outperforms several VQA-related baselines while still having large room for improvement. We expect our task and dataset to advance Egocentric AI Assistant's development. Our project page is available at: https://showlab.github.io/assistq/.

You Wu, Xuehai Qian

This paper proposes RCP, a new reversible coherence protocol that ensures invisible speculative load execution (ISLE) with low overhead. RCP can be combined with processor mechanisms that eliminate the effects of speculative instructions on other instructions to achieve low overhead invisible speculative execution (ISE). ISE provides protection that is at least as strong as speculative privacy tracking (SPT) and stronger than speculative taint tracking (STT). RCP is designed by systematically extending the existing coherence protocol to incorporate speculative loads and states. The protocol is implemented in Gem5 and verified with Murphi. The results show that RCP based ISE incurs lower overhead than STT/SDO/SPT while providing similar/stronger protection.