Click on the image above to view the whole behind-the-scenes gallery

Representing mobile augmented reality on film is a challenge! How do you convey what it feels like to play a game that embeds digital art and animation in a player’s real world environment—surrounding you in a totally immersive way—but is being viewed through your mobile phone screen?

Our first solution in an early teaser trailer for The Glass Ceiling Games was to screen record the mobile gameplay in various locations, then show it on film through the window of the phone screen…

But that just didn’t cut it for communicating how it feels to chase around a physical space in 360 degrees as you slingshot neon cartoon catcalls or machete unsolicited 🍆 pix (in the form of cartoon aubergines, hotdogs and other phallic foods).

We were inspired by the trailer for Harry Potter: Wizards Unite—also a mobile AR game—which represented the digital game world around real-life actors in a really exciting way:

Though we had to work out if we could do expensive Hollywood special effects on a shoestring budget...?

The first challenge was to understand how something like this is made. Shout out to our friend and Hollywood VFX artist, Dylan Owen, who kindly took time to explain the process of combining real world live action film footage and digital effects so we could have informed conversations with artists who might do that work.

Dylan explained that the stylised neon punk cartoon aesthetic of the fantasy game world would be ‘forgiving’ in direct contrast to real world footage—harder to make something look like it’s genuinely part of real world action, and that there were three steps:

1. Animation! We already had this covered as our 3D game assets are animated for gameplay. What we’d need to do is use the screen record function in Unity (the software engine we’ve built the game in) to render out footage of our animations for compositing with live action footage; but, we’d need to do it against a green screen that could be removed.
2. Tracking! Strap in here, this is where you build a digital copy of the real world camera you’ve used to shoot the real world footage, so it duplicates all the real world movement (important because our gameplay footage was shot handheld) and ‘pins’ the animation to it. You then look through that camera when rendering out the animation so that it has the same camera movement as the live action footage. Dylan explained Virtual Production is now starting to make this seem antiquated.
3. Compositing! Here you combine your animation sequences with your live action footage in a compositing software of choice and they should line up.

So once we had our heads around this we had to find someone to do it! That’s when we remembered one of our fellow UK Games Fund recipients, Cornwall-based Engine House—an award-winning animation studio.

We spoke to Tash and Jason, who said we ‘had them at feminist punk' and gave us some very clear guidance on how to shoot the live action footage, which included:

• Make sure the actor wears something that can be tracked like a badge. I mean, tell a feminist punk game to wear a pin badge!?
• Either put a green screen on the phone the actor’s using, so that gameplay footage can be added to it after, or shoot the gameplay footage live and they'll line the digitally composited animation up with it. We went the second route for authenticity.

Our next job was to find three shoot locations: a street scene for the gameplay footage, gig venue for the band footage and workspace for a talking head.

Leadworks warehouse, a local Plymouth venue, was recommended for the band shoot and on meeting Bronwen and Donna we instantly felt at home. The social enterprise supports grassroots creative endeavours and the Pride in Plymouth signage was up when we visited, which cemented it as the right place—the LGBTQIA+ community being an essential part of our identity at Story Juice, with important intersectional feminism crossovers. They also lent us some great pin badges!

Unexpected connections also arose from hiring Leadworks: a local band with a gig the night before our shoot offered the use of their kit to save our band transporting theirs. The lighting technicians setting up for the gig offered to programme the lights for us with the signature neon pink, blue and yellow of the brand—making an invaluable difference to the trailer aesthetic. They also left their haze machine! One of the staff at Leadworks, Michael Roberts, also happened to be a photography student who we hired to take the behind-the-scenes shots on the day, which are in the gallery at the top.

For the talking head we knew we wanted to use the new Market Hall digital and immersive space in Plymouth, which has great hotdesking rooms where we sometimes work, with brick walls that would make an undistracting backdrop for adding assets either side of the talking head. Real Ideas were generous in lending the space to shoot in as we’re members.

On the day, we shot the talking head first, then the band, who absolutely smashed it playing live along to the tracks without any hint a couple of them had just returned from a festival. It made us excited for a game launch party with them. Lyricist/singer Anna’s T-shirt also needs paying tribute to:

We scoped a few streets for the gameplay but ended up choosing somewhere else nearby on the day that was quieter and had more clear air for the compositing. Thanks to the weather gods for shining on us!

Actor Anna played the game live for some of the shoot and then acted the rest, imagining the digital assets popping up all around her. And surviving the heat in that coat.

Once the footage was shot, Director Amanda Bluglass and DOP/editor, Jake Cauty, got their heads together on the edit, which was great from the start and helped us work out which bits of the script to lose or represent slightly differently.

Once the edit was set, we sent the footage to Engine House, who made the VFX process very easy, when it obviously isn’t. They added especially cool things we didn’t even ask for, like light bouncing off walls and floors!

Then it went back to Jake for a final edit to add SFX and make it the polished film you see:

We’re really proud of it, but if there’s one learning from the Kickstarter process, it’s that we should’ve got gameplay footage into the first 30 seconds and added more of it throughout. You live, you learn.

Read Jake’s brilliant blog on shooting and editing tips.

And our awesome interview with Amanda about what it’s like being a woman in a male-dominated role.

Huge thanks to the team that made this film happen:

Writer: Hannah Wood, Story Juice

Director: Amanda Bluglass

DOP, Editing: Jake Cauty

VFX: Jason Robbins and Tash Price, Engine House

Soundtrack: Go Fallopian! Anna Wheatley, Ben Hudson, Jimi Stewart, Lucy Board

Actor: Anna Ray

Sound: Chris Baker

Production Assistant: Hannah Alexander, Story Juice

Production Assistant: Rose Thompson

Photographer: Michael Roberts

Lights and haze: Bubba InMedia

Production support: Arthur Hewes

Locations: Everyone at Leadworks and the Market Hall.