Sundance in 360

A storytelling revolution is happening at Sundance. While in the popular press, the virtual reality and 360 immersive shorts screened as part of the festival’s New Frontier section are still overshadowed by traditional two-dimensional features, every year they become more numerous and more innovative, both in the technology they employ and the way in which they do so.

Among the artists showing work at New Frontier this week are Rose Troche, Lily Baldwin, and Saschka Unseld. Each one came to 360 storytelling through a different means: Troche is a writer, director,  and producer, whose first film, Go Fish, screened at Sundance in 1994, and whose television credits include The L Word and Six Feet Under; Baldwin is dancer and filmmaker; Unseld is an animator, director, cinematographer, photographer, and Pixar alumnus.

Here, the three friends discuss their new pieces, Troche’s If Not Love, which takes place in a gay dance club during a shooting, and Baldwin and Unseld’s multi-decade love story Through You. They challenge the restrictions placed on 360 filmmakers, from the label of “narrative” and whether such a thing is or isn’t possible in VR, to the question of cutting scenes.

ROSE TROCHE: What do you remember about your first experience with VR? Just give me a little history from the both of you about when VR blew your mind.

SASCHKA UNSELD: Mine was probably three years ago. It was when the [Occulus Rift] DK2 had just come out, so for the first time you could slightly move around in the space if you moved your head left or right. I read a huge amount of the crazy hype around it, and then I finally saw something. I don’t remember what it was, the only thing I remember about it was that I was really disappointed and it was really bad. I was still at Pixar and someone brought by a laptop, and I was like, “Oh, that’s horrible.” It didn’t blow me away at all. Then that evening I took the bus home and I was thinking back on what I saw in VR and the thing that, if you want to use your word, “blew me away,” was that I remembered the experience so differently. I remembered it more like something that had happened to me then a film I had watched, despite it being so bad and so crappy and looking so horrible.

TROCHE: We’re not going to name that piece. [laughs]

UNSELD: Exactly. So that was just interesting. My feeling of it afterwards was very different from a film. It was closer to a book than it was to a film.

TROCHE: Because you create those images in your mind when you read a book.

UNSELD: Exactly.

TROCHE: We used to say a movie could never come close to a book, but no one reads books anymore.

UNSELD: It was more that the experience, to a certain extent, happened inside of me more than with a film. That is what got me really intrigued by it: “What if it was something actually good and not so bad?” That’s what started it all for me.

TROCHE: Lily, what about you?

LILY BALDWIN: My first recollection is really walking past the tech-y lounges with these headsets and people talking about it, and me being kind of annoyed with the lines and not sure what to make of it. I was so focused on some projects I was doing, I didn’t quite realize the possibilities that it had or what it entailed. I dismissed it. I wasn’t excited about it at all, except the people that I really respected were talking about it. People were like, “Why aren’t you in this? You’d have something to say here—isn’t it right up your alley?” And I was like, “You don’t know my alley!” It also felt untouchable; I didn’t know how to grasp it. I was completely affronted by how fuzzy it was and the cumbersomeness of the medium itself. I didn’t want to stand with something strapped to my head and lose my location in space and environment and have people watch me having an experience that I didn’t even think was that amazing. It was more, for me, the conversations around the space. I connected with you, Rose, I think last Sundance. It was around your piece and those conversations. Saschka was just embarking, when I met Saschka, on his adventures at Occulus—he was leaving Pixar and I remember hearing those conversations. I knew how he was as a maker and I trusted what he would put his time into, so I was like, “Alright, I’ve got to check this out.” So it was basically the power of persuasion and influence of people I respect that inspired me to fall headfirst into this. I realized it’s exactly in line with who I am as a creative person in terms of not having to fit inside of parameters, not being interested in following protocol, and always wanting to peel off layers of skin and be less predictable in approaches to story and narrative. That’s all that I was trying to do [before]—stuff my own kind of visceral perspective having been a dancer into a two-dimensional space. Saschka’s inquiry really made me want to do it. It was him saying, “Let’s play, let’s mess around, let’s find something.” I didn’t know what was possible and what wasn’t possible. I fell into it without knowing what could be done.

TROCHE: I remember seeing you in 2015 and 2016 at New Frontier and I could see some of those feelings of, “I don’t want to be strapped into a space—the expectation of a viewer in that space.” I haven’t see Through You, but I’m really looking forward to seeing it. Lily, if you look at your work and just the nature of experimentation within your work, it seems like 360, interactive, and immersive experiences is really the place you should be at. Saschka, you’re obviously so accomplished in the space in just several years. I’m really intrigued by Through You and want to know what brought that about.

UNSELD: We did a small traditional film—”flatties,” as people have started to call them now.

TROCHE: “Flatties” remind me of “fatties,” and it reminds me of something derogatory. We need to call it something different. I liked to call it 2D.

UNSELD: [laughs] We met at South by Southwest about two and a half years ago, and we were intrigued by each other’s work—I loved Sleepover LA. We started to collaborate a bit on a small two-minute piece that we shot one day in New York. Then it came up, “You know what we should do? A thing in VR.” Dance is so fascinating and I like collaborating with people a lot. I always worry that me by myself, I’m doing the same thing I’m always doing and become too lazy and conservative in what I do, or too incestuous in my own thoughts. I was excited how to see the dance of our different views lead to something—what would it lead to? Pushing me outside of my comfort zone and seeing what I bring to her comfort zone and seeing where that would go. Then we just wrote something without even knowing if we’d ever have the money to do it.

TROCHE: Did you feel pushed outside of your comfort zone when you were working together? This is a question for both of you.

UNSELD: Yeah. In the best possible way. I don’t know how you feel about that, Lily.

BALDWIN: I wouldn’t say I ever felt outside my comfort zone. I felt contained, and that was uncomfortable in a good way. I get messy with everything I do—I like to try every option and go to an extreme and then tip over the edge of the extreme and see what happens. That is a fertile place. To look at the larger picture and stop and corral that, it made me antsy, but I trusted Saschka. It’s about trusting someone not opposite, but just with very different impulses. It’s just really nice to share the responsibility. It’s my first time co-directing ever, and I’m pretty opinionated, but I knew that I would only be able to work with someone equally opinionated.

TROCHE: Saschka, would you consider yourself equally as opinionated?

UNSELD: Yeah. I think we express our opinions in very different ways, but I think we’re both definitely as opinionated.

TROCHE: I’m waiting for that moment when I co-direct. I haven’t done it ever in my career. It’s a very interesting prospect.

UNSELD: I guess there must be co-directors who think exactly alike and make exactly the same decisions. Lily and I are both really alike in what we like and what we don’t, but we’re also very, very different in the way we think, and I find that really interesting. I think that collaboration makes me a better director and observer and writer because I can look at things in a different way. I understand now how Lily approaches things, and to get that in another creator and then take that with you afterwards is really fascinating.

TROCHE: To that end, what do you think you discovered in the collaboration? I started my own VR work collaborating with Morris May, but we didn’t collaborate on content, we had our roles. He had an idea to do first person perspective, and that was the conceit, and then I came up with what that was going to be. We didn’t ever really overlap. We didn’t ever come into a conflict. This new piece If Not Love is the most my own that I’ve done. I think what I’m fascinated by about your collaboration is you are really bouncing a script back and forth between each other. You’re having to come to a consensus about what it is that you’re after. Lily, this is your first VR piece. Saschka, this is your first VR live action piece. You’re an Emmy-winning VR maker, but in this other space in which you always have collaborators. Why VR for this piece and not something else? Don’t say, “Because I got a grant for VR.”

UNSELD: To be honest, I find “why VR?” a secondary question. For me, there is a project I find fascinating, or an idea of creating something or something I’m chasing, and then if I tell it in VR, I have to constantly ask myself, “What is the unique way in VR to tell it?” I start more with something I want to exist, and because I’m working in VR now, I then really, really make sure that it uniquely uses what VR can do, versus starting with “why VR?” I think it becomes uniquely VR. The question is answered in the end because we do stuff that we never could in a film or in a book, but somehow I find that secondary. We found a way of moving the camera secondary to “why VR?” We found a way of editing secondary to the story, and we found that in the process.

TROCHE: When I approach VR, and if we’re going to get technical about VR, I’m not really making VR, I’m making 360 video—it’s not an interactive experience, which I think is the definition.

UNSELD: Through You is the same.

TROCHE: One can argue it’s immersive because you’re able to have the agency to look around; you’re not just staring at a screen in a fixed position. Maybe that counts as immersion or interaction. For me, I always think, “This is a story to be told in 360, because of this, that, and that.” I did a piece [If Not Love] about a shooting at a club and there are many shots—more than ever before—where you have almost nothing else to look at—there’s only the north-facing camera. It goes against my philosophy of why you shoot 360. When you’re inside of a club and there’s so much to see, that’s when it really thrives and the medium comes to life in certain scenes in the piece. In The Misdemeanor, in the party in the music video I just did, I really built those so that there would be something around you.

I actually saw your test shoot, which I really loved, at the New Frontier Story Lab, and I remember thinking to myself, “Cutting in VR is actually amazing.” When you were laying out your piece and laying out the choreography, were you thinking about pulling us as viewers around the space? How did you go about that?

BALDWIN: Through You is a very different experience than our first test, just given the choices we made around narrative and location and the cast, but we did begin with the two of us—him as camera and me as subject, but then the two becoming a conversation. I would always dance and he would always move the camera, and then we would co-direct and perform together. That’s sort of how we began. Through You is a departure from that. I learned how to edit in VR with Saschka this month, and I don’t know why people think they can’t cut. I don’t get it. I love cutting, because it’s something that I can’t do as a performer.

TROCHE: I think it has to do with the complexity of cutting in VR. You can turn the sphere for a cutting point, but if you’re really just existing in the 180, cutting is much easier. If you’re existing in a 360 space, it gets more complex: Where do you want your viewer to be? Are you predicting where someone’s going to be? Are you collecting biodata to see where someone’s looking? If you’re laying out a piece where you’re doing something a little bit more traditional and everything’s in the forward-facing in that north-facing point, then you can cut away. It resembles traditional cinema in this way, but it’s really not. The more you populate the 360-degree space, the more difficult it is to cut within that.

UNSELD: Exactly, and even in [my 2016 film] Dear Angelica, there is a moment when I know you’re going to look at one point, and then there are moments when I let go of the control and moments when I grab the control again as a storyteller. Two years ago, it was all about people making up arbitrary rules: you can’t move the camera, you can’t cut. Over the last year, it’s been about people being like, “This is bullshit” and breaking them.

TROCHE: It’s so hilarious, to both of your points. When we started, I had a lunch with someone who was a brain scientist, and she was like, “It’s going to make you crazy if you cut in VR.” Approaching what I’m doing now, I was like, “Is it going to make you crazy that I’m moving in reverse time—that everything’s moving backwards? Is that going to be disorientating?” I’d sit on panels and people would definitively say, “You cannot do that. There is no narrative; you can’t do narrative in VR.” I think a lot of the technology is trying to push it towards traditional cinema. Saschka, testing Dear Angelica, did you guys do biometrics? Were you testing where someone was looking? Did you collect data on where someone’s looking to see what is the smoothest cut from one thing to the other?

UNSELD: No. We watched a lot of people. We could [have], but it’s such a scientific thing. You want to see people’s reactions and you need to understand why they do something. If it would just be cold, anonymous data of where someone looks it would be pointless because it would just show me the result of something not working, and not why it doesn’t work. So I need to observe the people, and I think I need to observe myself. It’s nearly this thing you do, but to the nth degree, when you edit a movie. At one point you sit in the edit room and you try to watch it as if you have no idea what the heck it is. And it’s really hard, but you need to do it; that’s the way I feel like ultimately you’ll see if something does or doesn’t work. In VR, it’s kind of the same—you try to pretend to be someone who sees it for the first time, the only difficulty is what kind of person is that? Is that someone who looks around a lot? Is that someone who doesn’t look around at all? But I think it’s going into it and pretending to be someone who hasn’t seen it before and erasing your mind, and then just seeing how it feels.

BALDWIN: I like people having long cords and not [being] attached to chairs. I get a more pure reaction when it’s just them in space. Also sound, as you know, is a huge focus driver. But these larger questions of, “Can you do narrative in VR?” It makes me chuckle, only because I don’t know how we’re defining narrative. I feel like narrative in itself has been so fractured in a wonderful way in two-dimensional space. So for me, the bigger question is are we defining narrative as linear time? Are we defining narrative as fixed focal range? What is story? And that’s something that I can personally ask creatively in all my projects. That’s something I learned a lot with Saschka: The way I see story is not how other people do, obviously. For me particularly, story is more about slices of moments than it is about consequence and action. It’s really fertile ground, but I’d for people to ask the question, “Can you do blank in VR?” and fill it with something other than narrative just to get more specific.

TROCHE: Would you call Through You a narrative piece?

BALDWIN: I would.

UNSELD Yes and no. Anything that exists on a time basis—that has a beginning, middle, and end, because you start watching it and then you’re in the middle of watching it and then it ends—anything linear, for me, is narrative. In a traditional sense of narrative, it’s still dance. They don’t talk about the relationship, they dance it out, so I think for some people, if you have a conservative perspective on what is narrative, it wouldn’t be a narrative piece. But we have a script and we have a progression and we have two characters go through a relationship drama, so of course it’s narrative.

TROCHE: I like that definition of narrative—if there’s a beginning, middle, and an end—that it has a timeline in some kind of way.

BALDWIN: Yeah, but there’s an experience to narrative. If I was put inside of a Rothko painting, I’ll have the start of an experience, the middle, and then end of it, and that will be an internal narrative inside of an experience.

TROCHE: I’ve stood in front of a Rothko and had a narrative.

UNSELD: Exactly. I would go so far as to say that a painting is narrative, because I have the moment where I approach it and see it for the first time, and then I spend time with it, discover details and things I haven’t seen, and then I have the moment when I decide to leave it. I, as a person, make anything a narrative experience because I experience things linearly. The biggest question for me, is will I go through a transformation? Will I be bored or not? Is it a good or bad narrative experience?

TROCHE: Let’s look at the questions they gave us—”People talk about VR as a tool for empathy…” I thought we left the empathy question behind. That’s a very 2016 question. Obviously I’ve done work to make people believe this notion. I think it’s been called an empathy machine because when you’re immersed in something, you can’t look away from it—you can’t look away from VR. You inherently have this private moment through it.

UNSELD: I think a lot of what I do is based on that first initial experience of VR, how I remembered it differently. With Through You, the thought was that there are all these memories we have that we carry with us—especially of people—that feel weirdly present, and you can sit in an empty room and feel like a person, the memory of a person, is still with you, and it influences how you think or how you behave, and kind of experiencing someone’s life story through that window of a memory. Like if we would be that memory sitting next to us watching someone’s life unfold while we—that memory—is still around the whole time. Maybe that answers the question, “Why VR?” as well.

Going back to empathy, I read an article a few months ago about how the word “sympathy” has fallen from grace—it became a very derogative term, and “empathy” is all the rage.

ROCHE: Sympathy feels like a performative word, whereas empathy is human. But you’re right.

UNSELD: In tradition it’s actually not, and my way to disrupt the empathy question is to say I think it’s sympathy and not empathy. In something like Henry, you perceive the other person as a real person that is next to you, and that is sympathy—sympathy is towards someone else, versus empathy, which is feeling like someone else. I think our perception of a person in VR is much more a sympathy relationship in the original meaning of the term.

ROCHE: Are you going for a quotable right now? I can imagine someone quoting that on Pinterest. You should have T-shirt on at Sundance that just says, “Sympathy, not empathy.” 

IF NOT LOVE AND THROUGH YOU ARE CURRENTLY ON VIEW AT THE SUNDANCE FILM FESTIVAL IN PARK CITY, UTAH. FOR MORE ON LILY BALDWIN AND SASCHKA UNSELD, YOU CAN VISIT THEIR WEBSITES.