Haotian Guan

Haotian Guan, Jinping Dong, Wei-Ning Lee

Ultrafast ultrasound blood flow imaging is a state-of-the-art technique for depiction of complex blood flow dynamics in vivo through thousands of full-view image data (or, timestamps) acquired per second. Physics-informed Neural Network (PINN) is one of the most preeminent solvers of the Navier-Stokes equations, widely used as the governing equation of blood flow. However, that current approaches rely on full Navier-Stokes equations is impractical for ultrafast ultrasound. We hereby propose a novel PINN training framework for solving the Navier-Stokes equations. It involves discretizing Navier-Stokes equations into steady state and sequentially solving them with test-time adaptation. The novel training framework is coined as SeqPINN. Upon its success, we propose a parallel training scheme for all timestamps based on averaged constant stochastic gradient descent as initialization. Uncertainty estimation through Stochastic Weight Averaging Gaussian is then used as an indicator of generalizability of the initialization. This algorithm, named SP-PINN, further expedites training of PINN while achieving comparable accuracy with SeqPINN. The performance of SeqPINN and SP-PINN was evaluated through finite-element simulations and in vitro phantoms of single-branch and trifurcate blood vessels. Results show that both algorithms were manyfold faster than the original design of PINN, while respectively achieving Root Mean Square Errors of 0.63 cm/s and 0.81 cm/s on the straight vessel and 1.35 cm/s and 1.63 cm/s on the trifurcate vessel when recovering blood flow velocities. The successful implementation of SeqPINN and SP-PINN open the gate for real-time training of PINN for Navier-Stokes equations and subsequently reliable imaging-based blood flow assessment in clinical practice.

Andong Wang, Bo Wu, Sunli Chen et al.

Learning commonsense reasoning from visual contexts and scenes in real-world is a crucial step toward advanced artificial intelligence. However, existing video reasoning benchmarks are still inadequate since they were mainly designed for factual or situated reasoning and rarely involve broader knowledge in the real world. Our work aims to delve deeper into reasoning evaluations, specifically within dynamic, open-world, and structured context knowledge. We propose a new benchmark (SOK-Bench), consisting of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving. To create such a dataset, we propose an automatic and scalable generation method to generate question-answer pairs, knowledge graphs, and rationales by instructing the combinations of LLMs and MLLMs. Concretely, we first extract observable situated entities, relations, and processes from videos for situated knowledge and then extend to open-world knowledge beyond the visible content. The task generation is facilitated through multiple dialogues as iterations and subsequently corrected and refined by our designed self-promptings and demonstrations. With a corpus of both explicit situated facts and implicit commonsense, we generate associated question-answer pairs and reasoning processes, finally followed by manual reviews for quality assurance. We evaluated recent mainstream large vision-language models on the benchmark and found several insightful conclusions. For more information, please refer to our benchmark at www.bobbywu.com/SOKBench.