How to let the viewer lose himself in the story?
Jimmy Maidens VFX supervisor at Penrose Studios (on the left) and Yves Nougarède, in charge of programmation VR CITIA (on the right).
Interview of Jimmy Maidens, VFX supervisor at Penrose Studios
This interview took place during the Annecy Animation Film Festival (June 12-17 2017) where VR movies are now part of the selection of the official competition. They have their own dedicated category : VR@ANNECY.
Coming soon, other related articles: scroll down to know more.
I have been very happy to meet with Jimmy Maidens, VFX supervisor at the studio which produced « Allumette » and « The Rose and I », Penrose Studios.
In Annecy they were presenting their latest production: « Arden’s Wake ».
Sigrid Coggins : I’ve just seen Arden’s wake, and wow… I absolutely need to know ;-): what are your secrets?
Jimmy Maidens: I think that one secret is that everything that you think you know about [storytelling], you have to be ready to question every day, as you’re working. You have to make sure that you don’t come with preconceived notions, especially coming from film background. When you come to VR you have to make sure you don’t just make it like you make a film. You have to re-learn how to do things.
S.C: VR is about creating a new language…
J.M: Yes, for sure! I think, like everybody in the team, that one of the big things is making sure everybody’s on board with knowing that this is new, and that we don’t understand it yet! So that, when we come to work everyday, we build new things, and we try new things, and are ready to throw it out if it doesn’t work, until we find what works.
So… yes! Many scripts, many versions, lots of trials and errors, different paths to try to get there.
SC: What you are saying is that it’s more about trying than theorizing?
J.M: Yes, a lot of time I think we have ideas but there are always some people to say « …that won’t work because that’s not how you would do it » and you say « well, let’s try it before we say it won’t work.”
Or even if we think it works, before we say « this is how we are going to do », let’s try it, and then see.
Sometimes it works, sometimes it does not work, and you don’t know until you are in VR looking at it. With « Arden’s wake », we’ve been building tool suites to be able to make VR easier, to be able to be in VR when we’re working on it. Through the entire process, we are using a new tool that we call Maestro. So, when the team is looking at [the current work], you know, dailies, and when we review with the director how the scene is working and how the animation and everything is coming together, the whole team can put on VR headsets and go in there, and watch together in the project. He hits play, and everybody can watch and say « Oh, we need to do this » … or « the piece is too slow », or « too fast », or « this isn’t working, we need to think about this some more and go test ». All the team is in there, and you can point, you can see my head and hands and so I can point at things. And even though, in some cases, people are a hundred kilometers away, when you are in a virtual space, you are there!.. and you can all see in VR, and sort of look at a 3D screen.
Previously we always had to all gather around a 2D screen and talked about it, and then nobody can really see, you know, because… well, you’ve experienced VR and you know how it’s like when you’re in there: it’s different than when you are looking at the screen. That makes such a big difference in our production, in allowing us to get from where « Allumette » is to « Arden’s wake » to like, advance story and understand it faster. Part of what we are trying to do is become native VR thinkers, so that we can speak the language of VR, and understand VR. Right now it’s all little more a theory and when we’re working, we still spend most of our time on a 2D screen. We’re trying to spend more and more time in virtual reality while we work. Which I think is very important to understand it.
S.C: Do you develop any theory while discovering and doing VR? Do you have a few things you are sure of, about the way to tell stories in VR?
J.M: I think if I was to give any, it will probably change in our next piece!
I think the biggest thing is that you need to take your story, or whatever you’re working on, and view it in VR, as fast as you can!
Because things that you don’t think would work end up working, and things that you think would work, don’t work. And the faster you go in and check that, the sooner you can move forward. And in some cases, if you wait too long, and then you get in and see it, and you realize « Oh, I’ve spent a lot of work! » that you will have to redo, or you commit to, and then you can’t change it. So it’s harder for you to make those changes. What I mean getting in, like, every hour… as fast as possible.
That’s why we turn out trying to become native VR thinkers and to speak the language of VR, so that we are immersed in it. Just like we are now with over a 120 years for film: we understand it because it’s been around for so long, and it’s in our houses and on our phones, and it’s everywhere, and we’re just immersed in it! We’re not there with VR yet.
I think that the second one is what I said earlier, that is: there are guidelines. Don’t move the viewer, no cuts. Clearly, we’re doing cuts and fades and transitions now. We are moving the viewer a little bit. We’re trying to figure that out so we can tell bigger stories, not just in one space. There’s been a lot of tests – but generally speaking, if you move from here to here and if you start slow and speed up, and then slow down, it’s very bad, because the time it takes for the acceleration makes your ear and you eye disagreeing about what you are seeing. This all time. If you make that very short, for one moment you get that sickness, then it goes away because it’s like when you’re in a car and you’re moving at a speed, you don’t notice. That’s only when you step on the gas, and you hit the breaks that you really feel both your eyes and your ears feel. So, we start instantly, and then move you to steady pace, and never slow down, never speed up, and then stop.
And that helps reduce the sickness.
We still have a lot of work to do it better, but we implement things like that into the piece, and we’re still trying to find the pathway to lure people into the environments. « Allumette » will bring you into the boat, so you want to explore!
We like to have people stand up during the experiences: if you’re sitting in a chair or on the floor, you’re less likely to engage.
If you’re sitting in a chair or on the floor you would still see the whole experience, it just might not be as engaging. If you’re sitting on the floor, you couldn’t put your head underwater, to look under water because water would be lower, because we start wherever you are. We start to make sure that’s a good place.
S.C: Do you have a special way to write your script for VR?
J.M: We still write more like a traditional way : write the story, do a script, iterate on that – multiple times. And again, like other aspects, writing has to happen early, and then you need to go train, like, do very rough in VR, just to see if it’s going to work. And, as you find things that don’t work, then go back to the writing and write some more to fix it.
The team: we are in the same room, under the same roof. In some cases when somebody is not close by, we can go into VR and talk, and be in that same space with them so that we are understanding. That’s part of the native VR thinking, and the better we get it, the easier it will be to write the scripts and stories, so that makes sense. Because just like traditional film, it’s still searching an idea, and then write it down on paper. Even traditional storyboards: we try to move those into VR as soon as possible. As fast as we can, we try and get everything into VR, which is the key.
S.C: And what about the place of the spectator in your VR creations?
J.M: In our pieces so far, we’ve done more omnipresence, because one other thing is presence – being here – and storytelling. When you are reading a book and you vanish into the work, you’re not reading the book anymore: you’re in a sort of place until somebody tapes you on the shoulder… You’re not present in the book, because you’ve gone into your head, and you are someone else. That’s the same thing in movie theater, you turn your phone on and you pull out of the movie. But in VR you’re there and you don’t have to imagine: it’s like you get put in there, and that’s the presence, and it’s just harder to tell a story, and watch your story right now, because you are here, we are here! This does not feel like a story, so… how do you disengage from that so that you can lose yourself in the story?
And we’re also trying to understand how we can make you a character in the story and tell you the story at the same time. And what does that mean and how does that feel. We’re constantly experimenting with that, trying to find the right thing, and we don’t want to put anything out in the world until this is working and it does what we are trying to do!
We have found that the omnipresence works really well. Also, changing the scale separates you out from the character so you don’t feel like « oh she should look at me because I’m the same size » and things like that actually help to tell the story and let the viewer lose himself in the story. Maybe two years now we’ve been doing that, cinema is 120 years old, so we’re right at the beginning.
S.C: How many people are working with you?
J.M: We have an amazing team, we are around 20 to 40, it depends on where the project is. An amazing production, filmmakers, storytellers, artists. And engineers: having people to build the tools, that’s a very important piece of it. We have a lot of technology, that makes it happen and our engineers and our artists work very well together. That’s very important right now as we are at the early stages, making sure that our technology and art work together so the artists can bring their ideas. An amazing thing is that engineers also bring new ideas. I’ve always thought of engineers as just as creative as artists, in a different format, in its own way.
And it’s a technology having yet not separated the sector of art and of the engineers. It’s all together, and we need to build this together.
S.C: How do you see the future of VR economically?
J.M: That’s a great question and we’re still trying to find that exact place. It’s ongoing. We think this is just taking off now, so that’s the next year we will really see.
Last year was the first year that headsets were available to public, so this is the time to build and get ready.
S.C: So… you are optimistic?
J.M: We in Penrose absolutely believe that VR and AR, even if we think it’s going to turn into one, it’s gonna be the future, and it’s not going away! Most people love stories, and love to be in these worlds. We think of building our worlds with the stories and characters that populate these worlds.
In the future we will have the world of “Allumette”: you can just go there and stay and do other things in that world, see other stories, meet friends and people there to share with, and sing with. The first piece « Rose and I » is the world of a solar system; the worlds of « Arden’s wake » is in the sea… We think of these as worlds with many stories and they are there forever and we can always go back to them.
S.C: What is your next project?
Jimmy Maidens »: We have been very busy working on the next part of « Arden’s Wake », it will be longer than this one, together there will be more than 30 minutes.
Our plan is to release it it to the world by the end of the year!
Sigrid Coggins: Thank you so much, Jimmy Maidens, for having shared some of your secrets with the readers of VRStory!
To know more about the Penrose Studios, check out their website: http://www.penrosestudios.com/
A suivre, plusieurs articles #SigridCogginsASuiviPourvous !
Interview avec l’un des deux réalisateurs du doublement primé “Nothing happens”, Uri Kranot, qui a obtenu à Annecy le Prix Festivals Connexion – Région Auvergne-Rhône-Alpes / En partenariat avec Lumières Numériques & Pilon Cinéma (2017) et le Prix André-Martin pour un court métrage français (2017).
Rencontre avec l’équipe de réalisation de Real-Time Virtual Reality Experience for Björk’s, « Notget ».
Interview de Scot Stafford au sujet de Sonaria, production de Google Spotlight Stories.
Rencontre avec Yves Nougarède chargé du programme VR à Citia (en photo ici en compagnie de Jimmy Maidens).