Film Mirror

Film Mirror is an interactive installation that allows participants to perform as iconic film characters. Using recent AI research in neural network-based pose detection and image synthesis, the installation maps the bodies of the participants into the bodies of film characters in real time, enabling anyone to creatively re-enact and re-interpret classic film moments. Film Mirror elevates films from static artifacts, finalized at the moment of their release, into dynamic source material that can be used to generate an infinite number of new works.

Behind the scenes, the installation uses two open-source deep neural network models to perform the scene transfer:

• DensePose, based on a research paper by Güler et al., is used to recognize the body pose maps of the film characters (during training) and participants (during inference).

• pix2pixHD, based on a research paper by Wang et al., is used to synthesize high-resolution images in the style of a film scene given an input body pose.

Film Mirror premiered at Ars Electronica Festival 2018 in Linz, Austria.