Yiming Gao

CV
h-index10
4papers
592citations
Novelty53%
AI Score45

4 Papers

36.1CVJul 8, 2024
MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions

Xuan Ju, Yiming Gao, Zhaoyang Zhang et al.

Sora's high-motion intensity and long consistent videos have significantly impacted the field of video generation, attracting unprecedented attention. However, existing publicly available datasets are inadequate for generating Sora-like videos, as they mainly contain short videos with low motion intensity and brief captions. To address these issues, we propose MiraData, a high-quality video dataset that surpasses previous ones in video duration, caption detail, motion strength, and visual quality. We curate MiraData from diverse, manually selected sources and meticulously process the data to obtain semantically consistent clips. GPT-4V is employed to annotate structured captions, providing detailed descriptions from four different perspectives along with a summarized dense caption. To better assess temporal consistency and motion intensity in video generation, we introduce MiraBench, which enhances existing benchmarks by adding 3D consistency and tracking-based motion strength metrics. MiraBench includes 150 evaluation prompts and 17 metrics covering temporal consistency, motion strength, 3D consistency, visual quality, text-video alignment, and distribution similarity. To demonstrate the utility and effectiveness of MiraData, we conduct experiments using our DiT-based video generation model, MiraDiT. The experimental results on MiraBench demonstrate the superiority of MiraData, especially in motion strength.

6.9DSApr 6
Subset Balancing and Generalized Subset Sum via Lattices

Yiming Gao, Yansong Feng, Honggang Hu et al.

We study the \emph{Subset Balancing} problem: given $\mathbf{x} \in \mathbb{Z}^n$ and a coefficient set $C \subseteq \mathbb{Z}$, find a nonzero vector $\mathbf{c} \in C^n$ such that $\mathbf{c}\cdot\mathbf{x} = 0$. The standard meet-in-the-middle algorithm runs in time $\tilde{O}(|C|^{n/2})=\tilde{O}(2^{n\log |C|/2})$, and recent improvements (SODA~2022, Chen, Jin, Randolph, and Servedio; STOC~2026, Randolph and Węgrzycki) beyond this barrier apply mainly when $d$ is constant. We give a reduction from Subset Balancing with $C = \{-d, \dots, d\}$ to a single instance of $\mathrm{SVP}_{\infty}$ in dimension $n+1$, which yields a deterministic algorithm with running time $\tilde{O}((6\sqrt{2πe})^n) \approx \tilde{O}(2^{4.632n})$, and a randomized algorithm with running time $\tilde{O}(2^{2.443n})$ (here $\tilde{O}$ suppresses $\operatorname{poly}(n)$ factors). We also show that for sufficiently large $d$, Subset Balancing is solvable in polynomial time. More generally, we extend the box constraint $[-d,d]^n$ to an arbitrary centrally symmetric convex body $K \subseteq \mathbb{R}^n$ with a deterministic $\tilde{O}(2^{c_K n})$-time algorithm, where $c_K$ depends only on the shape of $K$. We further study the \emph{Generalized Subset Sum} problem of finding $\mathbf{c} \in C^n$ such that $\mathbf{c} \cdot \mathbf{x} = τ$. For $C = \{-d, \dots, d\}$, we reduce the worst-case problem to a single instance of $\mathrm{CVP}_{\infty}$. Although no general single exponential time algorithm is known for exact $\mathrm{CVP}_{\infty}$, we show that in the average-case setting, for both $C = \{-d, \dots, d\}$ and $C = \{-d, \dots, d\} \setminus \{0\}$, the embedded instance satisfies a bounded-distance promise with high probability. This yields a deterministic algorithm running in time $\tilde{O}((18\sqrt{2πe})^n) \approx \tilde{O}(2^{6.217n})$.

47.3CVApr 10, 2024Code
InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models

Jiale Xu, Weihao Cheng, Yiming Gao et al.

We present InstantMesh, a feed-forward framework for instant 3D mesh generation from a single image, featuring state-of-the-art generation quality and significant training scalability. By synergizing the strengths of an off-the-shelf multiview diffusion model and a sparse-view reconstruction model based on the LRM architecture, InstantMesh is able to create diverse 3D assets within 10 seconds. To enhance the training efficiency and exploit more geometric supervisions, e.g, depths and normals, we integrate a differentiable iso-surface extraction module into our framework and directly optimize on the mesh representation. Experimental results on public datasets demonstrate that InstantMesh significantly outperforms other latest image-to-3D baselines, both qualitatively and quantitatively. We release all the code, weights, and demo of InstantMesh, with the intention that it can make substantial contributions to the community of 3D generative AI and empower both researchers and content creators.

9.6CVNov 25, 2024
NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model

Jinpeng Liu, Jiale Xu, Weihao Cheng et al.

We introduce NovelGS, a diffusion model for Gaussian Splatting (GS) given sparse-view images. Recent works leverage feed-forward networks to generate pixel-aligned Gaussians, which could be fast rendered. Unfortunately, the method was unable to produce satisfactory results for areas not covered by the input images due to the formulation of these methods. In contrast, we leverage the novel view denoising through a transformer-based network to generate 3D Gaussians. Specifically, by incorporating both conditional views and noisy target views, the network predicts pixel-aligned Gaussians for each view. During training, the rendered target and some additional views of the Gaussians are supervised. During inference, the target views are iteratively rendered and denoised from pure noise. Our approach demonstrates state-of-the-art performance in addressing the multi-view image reconstruction challenge. Due to generative modeling of unseen regions, NovelGS effectively reconstructs 3D objects with consistent and sharp textures. Experimental results on publicly available datasets indicate that NovelGS substantially surpasses existing image-to-3D frameworks, both qualitatively and quantitatively. We also demonstrate the potential of NovelGS in generative tasks, such as text-to-3D and image-to-3D, by integrating it with existing multiview diffusion models. We will make the code publicly accessible.