Shortcuts

Tutorials

Overview

Inferring 4D representations given 2D observations is challenging due to its under-constrained nature. With recent advances in differentiable rendering, visual correspondence and segmentation, we built an optimization framework that reconstructs dense 4D structures with test-time optimization, by minimizing the different between the rendered 2D images and the input observations.

The tutorials introduce a complete workflow of Lab4D. We’ll use the method and dataset from the following papers:

The tutorials assumes a basic familiarity with Python and Differentiable Rendering concepts.

Each of the tutorial can be executed in a couple of ways:

  • Custom videos: This option allows you to train a model on your own videos.

  • Preprocessed data: This option skips the preprocessing step and train models on the preprocessed data we provided.

  • Render-only: This option skips model training and allows you to render the pre-trained model weights we provided.

Content


© Copyright 2023, Gengshan Yang, Jeff Tan, Alex Lyons, Neehar Peri, Deva Ramanan, Carnegie Mellon University.

Built with Sphinx using a theme provided by Read the Docs.