Affordances of Virtual Reality: Thoughts on VIFF Immersed 2020

Back in October I watched several of the XR offerings from the Vancouver International Film Festival Immersed exhibition, and I have been chewing on some thoughts. If the theme for my thoughts about the Biennale was tempo, the theme for VIFF was affordances.

Every medium has its particular qualities, things that it does and doesn’t do. These qualities offer a particular set of possibilities to a creator using the medium to communicate. Using these particular possibilities — the medium’s affordances — in an effective way is often key to creating good work. Works that exploit their medium’s affordances poorly may still be successful, but they do so despite themselves. There is an elegance to a work that uses its medium to the fullest, that could not have been implemented in any other medium.

XR is a young medium, and creators are still figuring out what its affordances are. Indeed, that process of learning is a lot of what makes this new medium exciting.

All of the pieces that were available to me on the Oculus Quest were three-degrees-of-freedom VR films. I think my takeaway from VIFF, and the Biennale before it, is that the essential affordance of 3DOF film is agency: the ability of the viewer to choose what to pay attention to, and how much attention to pay, within a set of interesting possibilities. The VIFF entries I watched were extremely uneven at exploiting this.

My favorite piece, easily, was Kowloon Forest. It describes itself as “a journey through the private lives of five strangers in Hong Kong,” but I think that doesn’t do it justice, because what made it really interesting to me was the way the piece was structured. Kowloon Forest is a series of vignettes shot in 360 degrees; each vignette places the viewer between two people or representations of people –a woman and her mirror, two friends, a man watching a mukbang video, and a man setting up a livestream. This meant that at every moment the viewer has two major loci of action to choose as their focus. Furthermore, each vignette took place in a really visually dense space, so there were ample things to look at in addition to the characters. I am always a sucker for that urban-palimpsest aesthetic that prevails in Hong Kong, but it is particularly well-suited to the 3DOF form. That visual richness, coupled with the feeling of discovery in exploring that space, was deeply satisfying.

Conversely, the most disappointing pieces were the ones that simply tried to replicate an existing form in VR. For example, Orpheus, a retelling of the myth of Orpheus and Eurydice, was essentially just a play with a proscenium stage, but modeled in computer-generated VR. There was a fully realized 360-degree environment, but none of it meant anything besides what was happening on the stage. (I understand from the blurb on the festival website that the full version of the performance involves some level of interactivity, but I don’t have any way to judge how that played out.)

I should concede that one can approach 3DOF as merely a dumb pipe to deliver more or less traditional theatre and film experiences. I think that’s legitimate, particularly in these times when traditional venues are unavailable, but I don’t think it’s very interesting. Also, the particularities of 3DOF are such that using it as a dumb pipe is perhaps more complicated than creators recognize.

The two Japanese-language pieces in the festival, GEIMU and VR Hero Sakura, illustrate this problem. Both are arguably merely ordinary films shot with a 180-degree camera. VR Hero Sakura is available in full on YouTube, and for my money not a lot is lost by watching it in a browser. Both pieces are about VR — they’re both stories of an immersive RPG gone wrong, but GEIMU aims for horror while VR Hero Sakura is comedy — but neither really tries to engage the distinctive affordances of the medium.

Translating conventional film techniques to 3DOF has unfortunate consequences for both pieces. Most obviously, close-ups shot at a distance appropriate to conventional film place the actor squarely inside the 3DOF viewer’s personal space, which is extremely uncomfortable. In GEIMU, that experience was effective, although I’m not sure if it was intentional. GEIMU tries to evoke a sense of horror and danger, and the perceived violation of my personal space served that end in a way that I’m not sure would have been possible in any other medium; even in live theatre I would have had the option to move back or otherwise react to the invasion of my space. In VR Hero Sakura, meanwhile, the invasion of my space was just off-putting. VR Hero Sakura also chose to place its subtitles into a space midway between the actors and the camera, which meant constantly refocusing between the text and the actors. This is tiring on the eyes, already a problem with VR headsets.

Similarly, neither piece did very much with the 180-degree field of vision. It’s not particularly useful to be able to look anywhere I want when the filmmakers have only given me one thing to look at. VR Hero Sakura in particular failed to take this opportunity in a way I still find baffling. One sequence, a melodramatic exchange between the protagonist and a monster, positions the viewer off to the side of the line between the two characters. The filmmakers decided to film this as alternating single shots. At any given moment, the character who’s not talking is not even within the 180-degree field of vision; you literally cannot look at them. What is the point of offering three degrees of freedom if you’re not going to let the viewer look at things that the story is telling them are interesting? 3DOF is a medium in which the viewer’s agency over their attention is the whole point; bringing in a filmmaker’s customary expectation of total control of the audience viewpoint is deeply counterproductive.

The documentaries in the program also mostly failed to exploit the affordances of 3DOF, largely for the same reasons the documentaries at the Biennale did: most of the time, there was one thing worth paying attention to, and then a lot of irrelevant things you could look at if you want. With the Wind and the Stars, for example, is an interesting documentary about a woman’s journey to launch her own airline, but I don’t think 3DOF adds much. The flying shots are fun in VR, and there are a couple of interesting moments when the camera is capturing semi-candid multi-person conversations, but mostly it’s interview material which is very single-point-of-focus.

Ecosphere: Raja Ampat had a different affordance challenge, which took me by surprise. The segment available to view online is about an ocean sanctuary in Indonesia, with a lot of underwater diving sequences, which initially seemed to me like a fantastic application of VR. What I had forgotten, however, is how much the current generation of VR headsets relies on the power of the human brain to interpolate sense data. I was using a Quest 1, and it just did not have the power to crisply convey hundreds of brilliantly colored, constantly moving fish. Everything seemed blurry and quickly tiring to watch, which wasn’t a problem for any of the other pieces aside from the previously mentioned subtitle issue.

By the Waters of Babylon was the best 3DOF documentary I’ve seen. It tells the story of a group of composers imprisoned at Theresienstadt during the Holocaust through the experiences of the Clarion Quartet performing a work by one of those composers, Viktor Ullmann, at Theresienstadt and elsewhere. The filmmakers really embraced the audience’s agency of attention, and the film combines live footage with computer-generated sequences that took excellent advantage of the expanded spaces that 3DOF can enable. A moment that the film returns to repeatedly as a sort of frame is a scene of the quartet playing the Ullmann piece. In a clever use of 3DOF, the quartet performs in a circle, with the viewer at the center. You can, at any point, choose which member of the quartet to watch, and none are the wrong choice. Similarly, there are several sequences that involve moving through the museum at Theresienstadt, which is an excellent application of the form; in those sorts of historical sites, everywhere and nowhere is the right place to look, and enabling the viewer to explore is perfect.

This, I think, is the hallmark of the 3DOF works that really work for me. Because the viewer can look anywhere, everywhere should be a good place to look, or should guide the viewer’s attention to a place that is. That’s a hard goal to achieve, but the works that do it achieve a level of engagement that is rare for me in a non-interactive medium.

Tournedos: A User’s Guide

From 2005 to 2008, I kept a food review blog called Tournedos, mostly about the places I had lunch during law school.  (It was called Tournedos because I started law school at UC Hastings, so I was mostly reviewing restaurants in the Tenderloin.)  I’m now consolidating those reviews with the rest of my food writing here at Plausibly Deniable.

Tournedos was a little more systematic than the rest of my food reviews, so I want to explain the things I put in the header of each post.

Types of Post: There are two types of post — Impressions and Reviews. Impressions are exactly that: a reaction based on a single meal. They consist basically of where I went, what I ate, what I thought about it, and what it cost. Reviews, on the other hand, follow what I’m given to understand is the New York Times model; I only write one after I’ve been somewhere at least three times and eaten three different things. In a review, I try to cover a restaurant’s strengths and weaknesses more broadly.

Ratings: Like Homeland Security, I have a five-level rating system. Also like Homeland Security, only two of them get used much. It’s important to bear in mind that these ratings are calibrated to the assumption that you’re in the Tenderloin and need lunch nearby; I’d probably shift the ratings down a level if I were judging them on an absolute scale. The ratings are:

Strongly Recommended: This rating goes to restaurants that I think are worth experiencing of their own right, and I would even recommend a trip to eat at them.

Recommended: This rating goes to places I like, and will eat at again. If you asked for a restaurant in a given food genre, these are the ones I’d mention.

OK: This rating goes to places that were fine, but I’m in no hurry to return to.

Not Recommended: This rating goes to places that I probably won’t go back to. Not disgusting; just not good. These are restaurants that aren’t worth your time.

Avoid: This rating is reserved for places so bad that I had to abandon my meal.

Weekly Update 1/8/21

This week I posted three new pieces:

The Palace in Death – a three-paragraph dungeon fantasy campaign

Experiences That Have Shaped My Thinking: The National Security Decision Making Game – a story about the incentives underlying American politics

Coda to an Age of Heroes, Episode 2 – sleep, breakfast, and succession wars

Also, over on Twitter I had a thread talking about gamebooks I wrote back in the day, and a retweet of my Christmas revenge tragedy tweetstorm.

New year, new website

Hello, and welcome to the new and improved Plausibly Deniable! The first tranche of writings to be posted are listed below. I’ll be updating this blog with a summary of new material on a weekly basis, and on @plausiblyd when new material is posted.

Five Geek Social Fallacies – some thoughts on interpersonal dynamics

Erik and the Goblins – a cautionary tale

The Barbecue Wars – a setting element for GURPS Transhuman Space

In the City of Alago Dun – a small piece of an imaginary world

Coda to an Age of Heroes – my as-yet-unfinished improvised serialized fantasy novel

Failure Modes in LARP – reflections on LARPs than went okay

Jargon and Definitions – I get tired of certain kinds of argument

Sacrificed Chine of Beef – the flavors are divine

Squishy Yellow Elegy – I really like macaroni and cheese, ok?

Bachelor Cuisine: Box Macaroni and Cheese – ibid

The Autarchs of Cephlen – another small piece of a place that never was

Venice Biennale VR Expanded: observations on tempo

Last weekend I watched the 360-degree films (what they called 3DOF) that were part of the Venice Biennale VR Expanded exhibition. I have thoughts on a couple of topics.

The 3DOF works were available through a virtual exhibition hall within the VRChat platform, and I thought there were a lot of missed opportunities there. The experience began in a sort of antechamber where one could select a mask off of a table, and then board a gondola which whisks one away through a series of canals to the exhibition hall proper. The antechamber was nicely done, and I liked the masks; however, it struck me that one of VRChat’s more popular functions is the ability to pick up an entire new avatar inside a world, and it felt like if the designers wanted to go with a masquerade theme, they could have thought bigger.

I did not care for the gondola ride. It felt theme-parky, and the environment felt thin. It felt like it was signifying the Biennale’s home without representing it — gondolas! masks! canals! palazzos! There were several points at which I could see between the polygonal palazzos to an infinite expanse of water beyond, which was uncanny without even being surreal.

Eventually, the gondola arrived at the exhibition hall itself, which was minimalist. Essentially there was a dock for the gondola and some stairs, leading to a tasteful warehouse with a red carpet that pointed the attendee to a series of portals for each of the available pieces. I realize, on reflection, that I cannot remember seeing anything but the exhibition hall from the dock. In my memory the infinite expanse of water just goes to the horizon in all directions, which may illuminate the aforementioned uncanny feeling. I am reminded of Larry Niven’s description of hyperspace as a giant blind spot.

The other thing that was a bit offputting about the exhibition hall was that whenever I left one of the pieces, I respawned on the dock, not at the portal for the piece I had just left. I understand that this was probably technologically simplest; I’m not sure how hard it would be within VRChat to create that many different spawn points. Still, it disrupted the flow of the experience very effectively.

I did, however, very much enjoy the works being exhibited themselves. They were my first experience with 360-degree films, and there were all sorts of exciting formal novelties for me.

The most obvious difference between 360 films and regular films is the absence of the frame, and this has various downstream effects. It’s tricky to manipulate focus appropriately; on the one hand, you don’t want the viewer to miss important story information, but on the other hand if there’s only a single point of interest, what was the point of using 360? I thought several of the pieces that were centered around interviews were disappointing for this reason. What the subjects had to say was (mostly) interesting, but that was really all there was. I could look around, but why? It was a little interesting to experiment with placing the subject in different places within my field of view, but only a little. The freedom the format gave me to explore was basically useless, and actually distracting from what the creators appeared to be trying to do.

Conversely, while I really liked the fairy-tale-style adventure In The Land of the Flabby Schnook, I feel like I missed a lot of interesting environmental detail trying to keep up with the story. On at least two occasions I missed story developments because I was looking at some exciting bit of the world in the opposite direction.

The key, I think, is pacing. I thought Penggantian (“Replacements”) handled this really well. That piece is essentially a series of vignettes — almost snapshots, really — of a neighborhood in Jakarta spaced over decades, tracing the changes to one stretch of road. Very little really happened in any given segment, but because there was a limited time to look around and explore that static space, it felt like things were happening as I absorbed all of the things that had changed from the last segment. Furthermore, the pace of change was managed quite masterfully. In the early segments, very little changed. It felt slow and lazy, and even a little boring, which was appropriate to the point I think the filmmakers were trying to make about Jakarta’s transformations over the years. As it went on, however, more and more changed with each transition, and I felt increasing urgency to find all the interesting new details before they changed again. Then the second-to-last segment was relatively uncluttered, creating the sense of a pause and a moment to reflect. It was a really remarkable management of tempo for a piece with virtually no actual action.

4 Feet High also did a good job with pacing. Its story — a teenager adjusting to life at a new school — is conventional, but the filmmakers did a good job of adjusting the pace and style of the film to exploit the 360-degree format. At any moment, the story was focused on the main characters, but the pace of events was slow enough that I had time to glance around and explore the environment. The creators also did an excellent job of selecting visually interesting locations that rewarded those little excursions. I often had the feeling that I had missed something in the corner of my eye that would have been interesting, but wasn’t important; I think that feeling may be the sweet spot for this sort of pacing. The best environmental experiences have always made deft use of an inchoate sense of FOMO.

Another thing I thought 4 Feet High did well was its use of signposting. There are several moments during the film where animated effects appear, and to me they felt like small nudges indicating where I ought to orient my attention without demanding it. It was so gentle, and yet in combination with the pacing choices I think it was the only film in the exhibition where I never felt like I had made a poor choice about where to look.

I had very much the opposite experience with 1st Step. That piece adapted historical footage of the moon missions into an immersive environment. It is a really impressive technical achievement, but I felt almost constantly that I was looking in the wrong place, that the rocket was always behind me. The creators also seemed to like using fades to black, which had an almost diametrically opposed effect to what they do in traditional film. In a regular film, a fade to black is a pause, a moment to reflect on what you just saw; in the 360 environment, my instinct was invariably “crap, is something happening behind me? Did VRChat crash?”

It is, I think, a lesson of environment-oriented performances that one has to let go. There are so many possible interactions of viewer and viewed that a creator can’t control them all, and a viewer can’t encompass them all; you can only hope to channel the flocks of possibilities. The most successful works are the ones that roll with that truth, that point and nudge without trying to direct, that offer a buffet rather than a tasting menu.

The Man of Gold

In honor of the release of Danielle Goudeau’s Choice of the Petal Throne, I wanted to share something that’s been kicking around in my head a while.

I think my favorite part of The Man of Gold is at the end, when the protagonist must choose, as is traditional, between two love interests: on the one hand, the good-hearted girl with Aryan racial signifiers, and on the other, the tempestuous beauty who makes his heart race but is proooobably kind of evil, and the Imperial Prince steps in to say, in effect, “You have clearly forgotten that we live in a polygamous society! Marry them both!”

Damn right, Prince Eselne. Subvert those tropes like a boss.

Originally published on Google Plus

Skelebear

It is weird how children’s media and horror so often run up in close proximity. This morning my younger son and I were playing with an alphabet puzzle where one picture is, I think, supposed to be a bear standing up and holding an X-ray, but actually looks like a bear pulling open its belly to display its bones.

The idea of a skeletal grizzly bear stalking about in an ill-fitting sack of its own pelt is one of the more horrific ideas that has graced my brain lately.

Originally published on Google Plus

Uplift in heroic fantasy RPGs

I reread Startide Rising and The Uplift War recently, and it occurred to me that that interspecies dynamic would be an interesting way of handling the profusion of intelligent races in an “everything in the books”-style D&D campaign.  Sure, maybe bugbears and ogres are basically the same thing, but that’s just because the illithids thought the aboleth had too good a thing going with ogres to let the idea go unswiped.  If you have a problem with that, I’m sure the beholders will be happy to field your complaints about their client race.  Or their patrons, if you can find the right Outer Plane.

Originally published on Google Plus