Xiao Zhang

Nirat Saini, Navaneeth Bodla, Ashish Shrivastava et al.

We introduce InVi, an approach for inserting or replacing objects within videos (referred to as inpainting) using off-the-shelf, text-to-image latent diffusion models. InVi targets controlled manipulation of objects and blending them seamlessly into a background video unlike existing video editing methods that focus on comprehensive re-styling or entire scene alterations. To achieve this goal, we tackle two key challenges. Firstly, for high quality control and blending, we employ a two-step process involving inpainting and matching. This process begins with inserting the object into a single frame using a ControlNet-based inpainting diffusion model, and then generating subsequent frames conditioned on features from an inpainted frame as an anchor to minimize the domain gap between the background and the object. Secondly, to ensure temporal coherence, we replace the diffusion model's self-attention layers with extended-attention layers. The anchor frame features serve as the keys and values for these layers, enhancing consistency across frames. Our approach removes the need for video-specific fine-tuning, presenting an efficient and adaptable solution. Experimental results demonstrate that InVi achieves realistic object insertion with consistent blending and coherence across frames, outperforming existing methods.

Xiao Zhang, Xiaoping Xie, Shiquan Zhang

The embedded discontinuous Galerkin (EDG) method by Cockburn et al. [SIAM J. Numer. Anal., 2009, 47(4), 2686-2707] is obtained from the hybridizable discontinuous Galerkin method by changing the space of the Lagrangian multiplier from discontinuous functions to continuous ones, and adopts piecewise polynomials of equal degrees on simplex meshes for all variables. In this paper, we analyze a new EDG method for second order elliptic problems on polygonal/polyhedral meshes. By using piecewise polynomials of degrees $k+1$, $k+1$, $k$ ($k\geq 0$) to approximate the potential, numerical trace and flux, respectively, the new method is shown to yield optimal convergence rates for both the potential and flux approximations. Numerical experiments are provided to confirm the theoretical results.

Yanwei Zheng, Shaopu Feng, Bowen Huang et al.

Inspired by human-like behaviors for navigation: first searching to explore unknown areas before discovering the target, and then the pathfinding of moving towards the discovered target, recent studies design parallel submodules to achieve different functions in the searching and pathfinding stages, while ignoring the differences in reward signals between the two stages. As a result, these models often cannot be fully trained or are overfitting on training scenes. Another bottleneck that restricts agents from learning two-stage strategies is spatial perception ability, since the studies used generic visual encoders without considering the depth information of navigation scenes. To release the potential of the model on strategy learning, we propose the Two-Stage Reward Mechanism (TSRM) for object navigation that decouples the searching and pathfinding behaviours in an episode, enabling the agent to explore larger area in searching stage and seek the optimal path in pathfinding stage. Also, we propose a pretraining method Depth Enhanced Masked Autoencoders (DE-MAE) that enables agent to determine explored and unexplored areas during the searching stage, locate target object and plan paths during the pathfinding stage more accurately. In addition, we propose a new metric of Searching Success weighted by Searching Path Length (SSSPL) that assesses agent's searching ability and exploring efficiency. Finally, we evaluated our method on AI2-Thor and RoboTHOR extensively and demonstrated it can outperform the state-of-the-art (SOTA) methods in both the success rate and the navigation efficiency.