Back in October I watched several of the XR offerings from the Vancouver International Film Festival Immersed exhibition, and I have been chewing on some thoughts. If the theme for my thoughts about the Biennale was tempo, the theme for VIFF was affordances.
Every medium has its particular qualities, things that it does and doesn’t do. These qualities offer a particular set of possibilities to a creator using the medium to communicate. Using these particular possibilities — the medium’s affordances — in an effective way is often key to creating good work. Works that exploit their medium’s affordances poorly may still be successful, but they do so despite themselves. There is an elegance to a work that uses its medium to the fullest, that could not have been implemented in any other medium.
XR is a young medium, and creators are still figuring out what its affordances are. Indeed, that process of learning is a lot of what makes this new medium exciting.
All of the pieces that were available to me on the Oculus Quest were three-degrees-of-freedom VR films. I think my takeaway from VIFF, and the Biennale before it, is that the essential affordance of 3DOF film is agency: the ability of the viewer to choose what to pay attention to, and how much attention to pay, within a set of interesting possibilities. The VIFF entries I watched were extremely uneven at exploiting this.
My favorite piece, easily, was Kowloon Forest. It describes itself as “a journey through the private lives of five strangers in Hong Kong,” but I think that doesn’t do it justice, because what made it really interesting to me was the way the piece was structured. Kowloon Forest is a series of vignettes shot in 360 degrees; each vignette places the viewer between two people or representations of people –a woman and her mirror, two friends, a man watching a mukbang video, and a man setting up a livestream. This meant that at every moment the viewer has two major loci of action to choose as their focus. Furthermore, each vignette took place in a really visually dense space, so there were ample things to look at in addition to the characters. I am always a sucker for that urban-palimpsest aesthetic that prevails in Hong Kong, but it is particularly well-suited to the 3DOF form. That visual richness, coupled with the feeling of discovery in exploring that space, was deeply satisfying.
Conversely, the most disappointing pieces were the ones that simply tried to replicate an existing form in VR. For example, Orpheus, a retelling of the myth of Orpheus and Eurydice, was essentially just a play with a proscenium stage, but modeled in computer-generated VR. There was a fully realized 360-degree environment, but none of it meant anything besides what was happening on the stage. (I understand from the blurb on the festival website that the full version of the performance involves some level of interactivity, but I don’t have any way to judge how that played out.)
I should concede that one can approach 3DOF as merely a dumb pipe to deliver more or less traditional theatre and film experiences. I think that’s legitimate, particularly in these times when traditional venues are unavailable, but I don’t think it’s very interesting. Also, the particularities of 3DOF are such that using it as a dumb pipe is perhaps more complicated than creators recognize.
The two Japanese-language pieces in the festival, GEIMU and VR Hero Sakura, illustrate this problem. Both are arguably merely ordinary films shot with a 180-degree camera. VR Hero Sakura is available in full on YouTube, and for my money not a lot is lost by watching it in a browser. Both pieces are about VR — they’re both stories of an immersive RPG gone wrong, but GEIMU aims for horror while VR Hero Sakura is comedy — but neither really tries to engage the distinctive affordances of the medium.
Translating conventional film techniques to 3DOF has unfortunate consequences for both pieces. Most obviously, close-ups shot at a distance appropriate to conventional film place the actor squarely inside the 3DOF viewer’s personal space, which is extremely uncomfortable. In GEIMU, that experience was effective, although I’m not sure if it was intentional. GEIMU tries to evoke a sense of horror and danger, and the perceived violation of my personal space served that end in a way that I’m not sure would have been possible in any other medium; even in live theatre I would have had the option to move back or otherwise react to the invasion of my space. In VR Hero Sakura, meanwhile, the invasion of my space was just off-putting. VR Hero Sakura also chose to place its subtitles into a space midway between the actors and the camera, which meant constantly refocusing between the text and the actors. This is tiring on the eyes, already a problem with VR headsets.
Similarly, neither piece did very much with the 180-degree field of vision. It’s not particularly useful to be able to look anywhere I want when the filmmakers have only given me one thing to look at. VR Hero Sakura in particular failed to take this opportunity in a way I still find baffling. One sequence, a melodramatic exchange between the protagonist and a monster, positions the viewer off to the side of the line between the two characters. The filmmakers decided to film this as alternating single shots. At any given moment, the character who’s not talking is not even within the 180-degree field of vision; you literally cannot look at them. What is the point of offering three degrees of freedom if you’re not going to let the viewer look at things that the story is telling them are interesting? 3DOF is a medium in which the viewer’s agency over their attention is the whole point; bringing in a filmmaker’s customary expectation of total control of the audience viewpoint is deeply counterproductive.
The documentaries in the program also mostly failed to exploit the affordances of 3DOF, largely for the same reasons the documentaries at the Biennale did: most of the time, there was one thing worth paying attention to, and then a lot of irrelevant things you could look at if you want. With the Wind and the Stars, for example, is an interesting documentary about a woman’s journey to launch her own airline, but I don’t think 3DOF adds much. The flying shots are fun in VR, and there are a couple of interesting moments when the camera is capturing semi-candid multi-person conversations, but mostly it’s interview material which is very single-point-of-focus.
Ecosphere: Raja Ampat had a different affordance challenge, which took me by surprise. The segment available to view online is about an ocean sanctuary in Indonesia, with a lot of underwater diving sequences, which initially seemed to me like a fantastic application of VR. What I had forgotten, however, is how much the current generation of VR headsets relies on the power of the human brain to interpolate sense data. I was using a Quest 1, and it just did not have the power to crisply convey hundreds of brilliantly colored, constantly moving fish. Everything seemed blurry and quickly tiring to watch, which wasn’t a problem for any of the other pieces aside from the previously mentioned subtitle issue.
By the Waters of Babylon was the best 3DOF documentary I’ve seen. It tells the story of a group of composers imprisoned at Theresienstadt during the Holocaust through the experiences of the Clarion Quartet performing a work by one of those composers, Viktor Ullmann, at Theresienstadt and elsewhere. The filmmakers really embraced the audience’s agency of attention, and the film combines live footage with computer-generated sequences that took excellent advantage of the expanded spaces that 3DOF can enable. A moment that the film returns to repeatedly as a sort of frame is a scene of the quartet playing the Ullmann piece. In a clever use of 3DOF, the quartet performs in a circle, with the viewer at the center. You can, at any point, choose which member of the quartet to watch, and none are the wrong choice. Similarly, there are several sequences that involve moving through the museum at Theresienstadt, which is an excellent application of the form; in those sorts of historical sites, everywhere and nowhere is the right place to look, and enabling the viewer to explore is perfect.
This, I think, is the hallmark of the 3DOF works that really work for me. Because the viewer can look anywhere, everywhere should be a good place to look, or should guide the viewer’s attention to a place that is. That’s a hard goal to achieve, but the works that do it achieve a level of engagement that is rare for me in a non-interactive medium.