Everybody Dance Now

Everybody Dance Now is an interactive video installation that encourages movement and play in young children and their caretakers. The project is part of the the Oregon Museum of Science and Industry’s (OMSI) exhibition AlegreMENTE | Happy Brain: Celebrating early connections, a traveling exhibition funded by the National Institutes of Health Science Education Partnership Award. AlegreMENTE premiered at OMSI on September 14, 2021.

The installation places real-time animated shadows of visitors into a cartoon forest scene, where they dance with human and animal characters. The characters are brought to life with motion capture animation, through which the 2D cartoon characters closely mimic human movement. This movement provides an invitation to visiting children and caretakers to dance along with the characters.

Visitors approach a dance floor in front of a video projection of this scene situated inside a semicylindrical structure, at the top of which is affixed an Azure Kinect depth camera that captures point clouds of visitors’ bodies. Those point clouds are rendered as real-time shadows of the visitors in the projected forest scene, allowing visitors to see their movements in the context of the animated characters. A particle system simulating elements like leaves and floating bubbles encourage further movement, as the particles react to contact with visitor shadows.

To inexpensively animate the cartoon characters that OMSI provided, Glowbox created a system using Brekel Body v3 motion capture software, Unity, and a depth camera that allowed Choreographer Cat Ross to see their movements bring the characters to life in real time. All these movements were designed so that children could mimic them. With young visitors as the focus for this installation, it was important that no visitor’s shadow appear smaller or larger depending on their position on the dance floor. We rendered the 3D point clouds that the Kinect camera captured of people as a flat orthographic projection to translate their shadows into the 2D space of the cartoon scene.

Additionally, we built a calibration system that “learns” obstructions to the Kinect camera and removes them from the rendered visitor shadows. This system was critical to the successful long-term operation of Everybody Dance Now, since the project will be installed at various museums over the course of ten years.