Te-Yen Wu

Ruibo Liu, Jason Wei, Shixiang Shane Gu et al. · deepmind

Successful and effective communication between humans and AI relies on a shared experience of the world. By training solely on written text, current language models (LMs) miss the grounded experience of humans in the real-world -- their failure to relate language to the physical world causes knowledge to be misrepresented and obvious mistakes in their reasoning. We present Mind's Eye, a paradigm to ground language model reasoning in the physical world. Given a physical reasoning question, we use a computational physics engine (DeepMind's MuJoCo) to simulate the possible outcomes, and then use the simulation results as part of the input, which enables language models to perform reasoning. Experiments on 39 tasks in a physics alignment benchmark demonstrate that Mind's Eye can improve reasoning ability by a large margin (27.9% zero-shot, and 46.0% few-shot absolute accuracy improvement on average). Smaller language models armed with Mind's Eye can obtain similar performance to models that are 100x larger. Finally, we confirm the robustness of Mind's Eye through ablation studies.

Xuhai Xu, Mengjie Yu, Tanya R. Jonker et al.

Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses "when", "what", and "how" to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing 500+ end-users' preferences for AR-based explanations, and three workshops with 12 experts collecting their insights about XAI design in AR. XAIR's utility and effectiveness was verified via a study with 10 designers and another study with 12 end-users. XAIR can provide guidelines for designers, inspiring them to identify new design opportunities and achieve effective XAI designs in AR.

Weiye Xu, Yue Xu, Devin Murphy et al.

Battery-free RFID systems offer a scalable and maintenance-free approach to interaction. We present KnitID, a machine-knitted textile RFID antenna design that enables on-body authentication, localization, and interaction. Unlike prior antenna designs, KnitID achieves a compact antenna form factor (60mm by 8mm) by integrating magnet wire into the unique loop-over-loop structure of machine knitting. This structure reduces the size of conventional loop antennas by around 90\%, while also providing 30\% longer sensing ranges than standard dipole designs with similar size on the human body. The compact form factor creates new opportunities to embed multiple RFID tags across the human body, enriching backscatter signals and supporting a broader range of battery-free on-body interactions. To demonstrate this capability, we build an interactive sleeve to support wearer authentication, spatial localization, and interaction detection. Through technical evaluations, we show the feasibility of KnitID to provide diverse and battery-free interactions on knitted user interfaces.