Physically Based Video Editing

Bazin, Jean-Charles; Plüss (Kuster), Claudia; Yu, Guo; Martin, Tobias; Jacobson, Alec; Gross, Markus

Convincing manipulation of objects in live action videos is a difficult and often tedious task. Skilled video editors achieve this with the help of modern professional tools, but complex motions might still lack physical realism since existing tools do not consider the laws of physics. On the other hand, physically based simulation promises a high degree of realism, but typically creates a virtual 3D scene animation rather than returning an edited version of an input live action video. We propose a framework that combines video editing and physics-based simulation. Our tool assists unskilled users in editing an input image or video while respecting the laws of physics and also leveraging the image content. We first fit a physically based simulation that approximates the object’s motion in the input video. We then allow the user to edit the physical parameters of the object, generating a new physical behavior for it. The core of our work is the formulation of an image-aware constraint within physics simulations. This constraint manifests as external control forces to guide the object in a way that encourages proper texturing at every frame, yet producing physically plausible motions. We demonstrate the generality of our method on a variety of physical interactions: rigid motion, multi-body collisions, clothes and elastic bodies.

Physically Based Video Editing

 

Efficient and Reliable Self-Collision Culling using Unprojected Normal Cones

Tongtong Wang, Zhihua Liu, Min Tang, Roufeng Tong, and Dinesh Manocha

We present an efficient and accurate algorithm for self-collision detection in deformable models. Our approach can perform discrete and continuous collision queries on triangulated meshes. We present a simple and linear time algorithm to perform the normal cone test using the unprojected 3D vertices, which reduces to a sequence point-plane classification tests. Moreover, we present a hierarchical traversal scheme that can significantly reduce the number of normal cone tests and the memory overhead using front-based normal cone culling. The overall algorithm can reliably detect all (self) collisions in models composed of hundred of thousands of triangles. We observe considerable performance improvement over prior CCD algorithms.

Efficient and Reliable Self-Collision Culling using Unprojected Normal Cones

Eurographics 2017

Interactive Paper Tearing

Camille Schreck, Damien Rohmer, Stefanie Hahmann

We propose an efficient method to model paper tearing in the context of interactive modeling. The method uses geometrical information to automatically detect potential starting points of tears. We further introduce a new hybrid geometrical and physical-based method to compute the trajectory of tears while procedurally synthesizing high resolution details of the tearing path using a texture based approach. The results obtained are compared with real paper and with previous studies on the expected geometric paths of paper that tears.

Interactive Paper Tearing

Interpolations of Smoke and Liquid Simulations

Nils Thuerey

We present a novel method to interpolate smoke and liquid simulations in order to perform data-driven fluid simulations. Our approach calculates a dense space-time deformation using grid-based signed-distance functions of the inputs. A key advantage of this implicit Eulerian representation is that it allows us to use powerful techniques from the optical flow area. We employ a five-dimensional optical flow solve. In combination with a projection algorithm, and residual iterations, we achieve a robust matching of the inputs. Once the match is computed, arbitrary in between variants can be created very efficiently. To concatenate multiple long-range deformations, we propose a novel alignment technique. Our approach has numerous advantages, including automatic matches without user input, volumetric deformations that can be applied to details around the surface, and the inherent handling of topology changes. As a result, we can interpolate swirling smoke clouds, and splashing liquid simulations. We can even match and interpolate phenomena with fundamentally different physics: a drop of liquid, and a blob of heavy smoke.

Interpolations of Smoke and Liquid Simulations

Interactive Modeling and Authoring of Climbing Plants

Torsten Hädrich, Bedrich Benes, Oliver Deussen, Sören Pirk

We present a novel system for the interactive modeling of developmental climbing plants with an emphasis on efficient control and plausible physics response. A plant is represented by a set of connected anisotropic particles that respond to the surrounding environment and to their inner state. Each particle stores biological and physical attributes that drive growth and plant adaptation to the environment such as light sensitivity, wind interaction, and physical obstacles. This representation allows for the efficient modeling of external effects that can be induced at any time without prior analysis of the plant structure. In our framework we exploit this representation to provide powerful editing capabilities that allow to edit a plant with respect to its structure and its environment while maintaining a biologically plausible appearance. Moreover, we couple plants with Lagrangian fluid dynamics and model advanced effects, such as the breaking and bending of branches. The user can thus interactively drag and prune branches or seed new plants in dynamically changing environments. Our system runs in real-time and supports up to 20 plant instances with 25k branches in parallel. The effectiveness of our approach is demonstrated through a number of interactive experiments, including modeling and animation of different species of climbing plants on complex support structures.

Interactive Modeling and Authoring of Climbing Plants

Art-Directed Muscle Simulation for High-End Facial Animation

Matthew Cong, Kiran S. Bhat, Ronald Fedkiw

We propose a new framework for the simulation of facial muscle and flesh that so significantly improves the technique that it allows for immediate mainstream use of anatomically and biomechanically accurate muscle models as a bread and butter technique in a high-end production quality pipeline. The key idea is to create a blendshape system for the muscles that gives the precise directability and controllability required in a high-end production environment. The blendshape muscles are used to drive the underlying anatomically and biomechanically motivated simulation in a way that is unbound by the typical restrictions of a simulation system while still retaining the desirable degree of freedom richness that leads to high quality results. We show that we are able to target production quality facial shapes, whether from scans or an animation system, and illustrate that the resulting nonlinear simulation in-betweens are of higher quality than those obtained from traditional linear blendshapes. We also demonstrate the ability to selectively improve areas on a given blendshape using the results of a simulation, as well as the ability to edit muscle shapes and paths in order to produce directability for animator control. Then, we show how these techniques can be used to transition from one blendshape to another or even track and selectively modify an entire performance. The efficacy of our system is further demonstrated by using it to retarget animation onto new creature models given only a single static rest pose as input.

Art-Directed Muscle Simulation for High-End Facial Animation

Two-way coupling of fluids to reduced deformable bodies

Wenlong Lu, Ning Jin, Ronald Fedkiw

We propose a fully monolithic two-way coupling framework that couples incompressible fluids to reduced deformable bodies. Notably, the resulting linear system matrix is both symmetric and positive-definite. Our method allows for the simulation of interesting free-surface as well as underwater phenomena, enabling the use of reduced deformable bodies as full-fledged simulation primitives alongside rigid bodies and deformable bodies. Momentum conservation is crucial to obtaining physically correct and realistic-looking motion in a fluid environment, and we achieve this by following previous work to describe reduced deformable bodies using both a rigid frame and a reduced space deformation component. Our approach partitions forces and impulses between the reduced space and the rigid frame of the reduced deformable bodies using a projection scheme that cleanly accounts for momentum losses in the reduced space via corrections in the rigid frame, resulting in a new theoretical formulation for the momentum-conserving reduced deformable body. We demonstrate that robust and stable contact, collision, articulation, and two-way coupling with fluids are all attainable in a straightforward way using this new formulation. Compared with fully deformable objects, our framework consumes less memory and scales better in large scenes, while still nicely approximating the deformation effects.

Two-way coupling of fluids to reduced deformable bodies

Accurate Contact Modeling for Multi-rate Single-point Haptic Rendering of Static and Deformable Environments

Thomas Knott, Torsten Kuhlen

Common approaches for the haptic rendering of complex scenarios employ multi-rate simulation schemes. Here, the collision queries or the simulation of a complex deformable object are often performed asynchronously on a lower frequency, while some kind of intermediate contact representation is used to simulate interactions on the haptic rate. However, this can produce artifacts in the haptic rendering when the contact situation quickly changes and the intermediate representation is not able to reflect the changes due to the lower update rate. We address this problem utilizing a novel contact model. It facilitates the creation of contact representations that are accurate for a large range of motions and multiple simulation time-steps.We handle problematic convex contact regions using a local convex decomposition and special constraints for convex areas.We combine our accurate contact model with an implicit temporal integration scheme to create an intermediate mechanical contact representation, which reflects the dynamic behavior of the simulated objects. Moreover, we propose a new iterative solving scheme for the involved constrained dynamics problems.We increase the robustness of our method using techniques from trust region-based optimization. Our approach can be combined with standard methods for the modeling of deformable objects or constraint-based approaches for the modeling of, for instance, friction or joints. We demonstrate its benefits with respect to the simulation accuracy and the quality of the rendered haptic forces in multiple scenarios.

Accurate Contact Modeling for Multi-rate Single-point Haptic Rendering of Static and Deformable Environments

Divergence-Free SPH for Incompressible and Viscous Fluids

Jan Bender, Dan Koschier

In this paper we present a novel Smoothed Particle Hydrodynamics (SPH) method for the efficient and stable simulation of incompressible fluids. The most efficient SPH-based approaches enforce incompressibility either on position or velocity level. However, the continuity equation for incompressible flow demands to maintain a constant density and a divergence-free velocity field. We propose a combination of two novel implicit pressure solvers enforcing both a low volume compression as well as a divergence-free velocity field. While a compression-free fluid is essential for realistic physical behavior, a divergence-free velocity field drastically reduces the number of required solver iterations and increases the stability of the simulation significantly. Thanks to the improved stability, our method can handle larger time steps than previous approaches. This results in a substantial performance gain since the computationally expensive neighborhood search has to be performed less frequently. Moreover, we introduce a third optional implicit solver to simulate highly viscous fluids which seamlessly integrates into our solver framework. Our implicit viscosity solver produces realistic results while introducing almost no numerical damping. We demonstrate the efficiency, robustness and scalability of our method in a variety of complex simulations including scenarios with millions of turbulent particles or highly viscous materials.

Divergence-Free SPH for Incompressible and Viscous Fluids