Michael P. Smith
Through synthesis of the ethnographic data collected, we generated a number of research insights. From these insights and core product qualities from competitive analysis, we generated three major themes for research findings. We believe the product should help film-makers with:
- Understanding spatial constraints
- Visualizing virtual objects
- Addressing business needs
The research report found below is the culmination of 10 weeks of research work done on the project. We presented the physical report to the Mixed Reality at Work (MRW) team at Microsoft. The findings in the report was used by the design and prototyping team to flesh out a functional prototype to demonstrate how mixed reality could be used to assist the filmmaking process. The outcome of the design and prototyping phase could be found on Surabhi Wadhwa's portfolio here.
Miniature 3D setPeople see the world in 3D. Current pre-visualization tools used on computers and phones only depict a 2D version of this 3D world. We see a huge opportunity in the form of a holographic tabletop miniature 3D model of the set that could be used in pre-production. We envision a Hololens application that would provide the director with a birds-eye view of the production setup in order to plan the movement, scale, and positioning of characters, objects within a scene.
Spatial location mappingWhile location scouting, production designers and location managers can scan a space using the Hololens or capture 360 video and photos of the location and from this render a 3D model of the indoor or outdoor space. This built to scale 3D model serves as a communication for the film crew by showcasing the spatial limitations of the set, leading to better alignment, planning, and organization during production.
Visualizing CGIAnother promising opportunity we identified is helping actors and the camera crew visualize computer generated characters and objects that would be added in post-production. Actors could wear the Hololens to rehearse with a CGI hologram and match eye-lines before shooting a scene. This application of the Hololens could also be used by the camera crew to pull focus on a CGI character or object and reduce the number of plate shots captured.
Investigate ways to assist filmmakers to externalize creative ideas and collaborate with stakeholders using mixed reality as a design intervention.
My contributionI acted as a design researcher on this team. I was responsible for outlining the recruitment strategy for the participant pool, responsible for creating the study guide for the participants and experts we spoke with, and recruiting the participants and experts for the ethnographic interviews.
I played an instrumental role in synthesizing the data. In addition, I was responsible for writing the majority of the content in the research report.
BackgroundTechnology has impacted the filmmaking industry in the past few years. From George Lucas to Robert Rodriguez, technology has been adopted to drive the industry in very different directions.
Recently, there has been a huge interest from filmmakers in immersive technologies. Steven Spielberg used the HoloLens while building the VR cinematic universe of Ready Player One, and James Clyne the director of photography used a VR headset to assist him in filming Solo.
Just to clarify, we used @oculus Rift, @htcvive, and @HoloLens during the filming of #ReadyPlayerOne, not just one headset. Each one was used at a different stage of production.— Girish Balakrishnan (@_MasterG_) March 18, 2018
When shooting mocap, Rift. When scouting virtual cameras, Vive. When on the practical set, Hololens https://t.co/62Ikvwd0d7
Film production processWe jumped into this by first understanding the film production process. What we learned is that filmmaking consists of five stages, but the majority of technology advancements have been focused around post-production transcendence.
We saw this as a potential opportunity to focus on improving the pre-production and production phase of filmmaking using mixed reality. Therefore we set out to speak with experts and participants that hold ownership over the pre-production and production phases of filmmaking - namely directors, producers, cinematographers, and VFX artists.
Research participantsDiversification of participants is key for our project. We looked for triangulation across different stakeholders. Pain points identified by multiple participants helped us narrow down areas where MR could be most effective as a design intervention.
We identified that it is important to speak to participants that work on the production of major motion picture films, as well as independent filmmakers such as art and documentary filmmakers.
Thomas FurnessThomas Furness is a professor at the University of Washington (UW) in the Department of Industrial & Systems Engineering where he teaches Virtual Interface Technology. He has been a pioneer of augmented reality technologies for over 50 years. He has collaborated with Peter Jackson (Director, Lord of the Rings) on the LOTR franchise and partnered with Douglas Trumbull (VFX, 2001: A Space Odyssey) on some VR related projects.
Sura KalyanSura is a trained VFX artist, director, and writer currently working in Los Angeles. He also worked as a Technical Director (TD) at DreamWorks Animation, and a VFX artist for Detective Pikachu, The Hedgehog, and MIB International.
Ryan WoodwardRyan has been in the film and animation industry under various roles such as animator, storyboard artist, concept artist, writing and directing for 23 years.
As a storyboard artist, he worked on Spider-Man 2 and 3, Cowboys and Aliens, Where the Wild Things Are, Iron Man 2, Snow White and the Huntsman, Thor 2, Captain America 2 and The Avengers.
Kevin PhilbinKevin Philbin is an independent film producer, and director of photography. Kevin has 9 years of experience running his own film production studio, Kevmo Productions. His work focuses on the production of cinematic shots at international locations. His list of clients include Flor De Caña Rum, The Nature Conservancy, Bike MS, Diamondback Bicycles, and many others.
Insights learnedBy cross-referencing the information gathered from primary and secondary research, we generated a total of nine different insights. From these nine insights, distinct patterns emerged. We decided the best way to contextualize the insights would be to group them under three distinct themes.
- Understanding Space
- Visualizing Virtual Objects
- Business Viability
It’s hard to verbally explain the geometrics, where’s everyone positioned. Just explaining the ideas to the DP takes a lot of time and effort.
My team once transformed a large downtown church into a police station, a prison warden’s office, and prison hallways; a basement, an upscale office, a Brazilian kitchen, and a small Brazilian church.
The analog style of having a physical model that everyone can look at, around the table is immensely helpful.
Previs is the ideation phase, nothing is solid. It’s a scratch pad for the ideas between the Director and his team.
They’re like ‘look out to the castle’ and you just think ‘well, how far away is it? Is it right here?’ and although you can ask all of those questions it never looks right.
Achieving your exact vision without enormous budgets is very difficult. Higher budget films are strangled by producers and execs.
The VFX industry is catered towards industry studios, not indie filmmakers and smaller studios. [...] most indie directors aren’t really aware of what technology can do for them.
Connecting back to MRW at Microsoft
- Build miniature 3D sets
- Spatial location mapping
- Visualize CGI