Yash Kant

ID-V2V restylizes video while preserving subject identity through a combination of diffusion-based video generation and identity-aware guidance!

Vista4D reshoots a dynamic scene from a single source video along novel camera trajectories and viewpoints, by grounding the video and target cameras in a 4D point cloud!

It is robust to real-world 4D reconstruction artifacts, and generalizes to dynamic scene expansion and 4D scene recomposition!

Go-with-the-Track unifies video compositing and motion control by conditioning a video diffusion model on multiple reference images and reference-anchored point tracks!

A single model handles keypoint-driven compositing, multi-reference camera control, and restylization, with point tracks establishing correspondences across generated frames!

We trained a high-resolution (1K) multi-view human generator on 3B human images and 2.5K studio captures!

Pippo outperforms all previous multiview methods! :)

Vid2Avatar-Pro creates photorealistic and animatable 3D human avatars from monocular videos!

Fillerbuster auto-completes missing regions in casually captured shot with a multi-view diffusion model!

SG-I2V enables zero-shot image animations relying solely on the knowledge present in a pre-trained image-to-video diffusion model!

We trained a spatially aware multi-view diffusion model that can generate many consistent novel views in a single forward pass given a text prompt / image!

SPAD outperforms MVDream and SyncDreamer, and enables generating 3D assets from text within 10 seconds!

We systematically evaluate several model merging methods within a unified experimental framework, focusing on compositional generalization.

We explore the impact of scaling the number of merged models and sensitivity to hyper-parameters, offering a clear assessment of the current state of model merging techniques.

We perform novel view synthesis from a single image by repurposing Stable Diffusion inpainting model, and depth based 3D unprojection. We outperform baselines (such Zero-1-to-3) on PSNR and LPIPS metrics.

Our 3D-aware inpainting model was trained on Objaverse on 96 A100 GPUs for two weeks!

We propose an end-to-end invertible and learnable reposing pipeline that allows animating implicit surfaces with intricate pose-varying effects. We outperform the state-of-the-art reposing techniques on clothed humans while preserving surface correspondences and being order of magnitude faster!

Housekeep is a benchmark to evaluate commonsense reasoning in the home for embodied AI. Here, an embodied agent must tidy a house by rearranging misplaced objects without explicit instructions.

To capture the rich diversity of real world scenarios, we support cluttering environments with ~1800 everyday 3D object models spread across ~270 categories!

We build a simple method to extract an object from a scene given 2D images, camera poses, a natural language description of the object, and a few annotated pixels of object and background.

We introduce an automated Annotation and Video Stream Alignment Pipeline (abbreviated ASAP) for aligning unlabeled videos of four different sports (Cricket, Football, Basketball, and American Football) with their corresponding dense annotations (commentary) freely available on the web. Our human studies indicate that ASAP can align videos and annotations with high fidelity, precision, and speed!

We propose a training scheme which steers VQA models towards answering paraphrased questions consistently, and we ended up beating previous baselines by an absolute 5.8% on consistency metrics without any performance drop!

We built a self-attention module to reason over spatial graphs in images. We ended up with an absolute performance improvement of more than 4% on two TextVQA bechmarks!