Celia Wagar's CritPoints

Disclaimer: This has been on the backburner for a very long time. I’m sorry if some information is out of date, or if I’m missing something.

My original attitude towards VR was skeptical to dismissive. Anyone who knows me knows that I’m skeptical about anything promising “immersion” and pretty much the concept of immersion in general. I got to try a prerelease Oculus VR headset at a showcase in New York City and it was pretty much what I expected, a screen strapped to my face, while I controlled an FPS game like I’ve always done. To me, VR was another type of screen, I didn’t expect it to fundamentally change anything about video games.

First Gen VR devices changed my mind, they brought on motion controls, like the Wii and PS Move. I eventually got to try a Vive headset, with Valve’s The Lab, SuperHot VR, and a couple others, and I saw what the device could do, and it was pretty obvious that there was untapped potential there. VR seems like a way to make motion controls work much more feasibly than they ever had before, and that could lead to some legitimately new game genres!

The Shortcomings of Current VR Games

My trouble is that VR is also couched in this “immersive promise”. People want the holodeck and full-dive like SAO or Ready Player 1. VR Companies promise that VR is the future of immersion into a virtual world, where we can finally enter The Matrix, The Metaverse, Ready Player 1; You get the picture, right? Except, VR as it currently exists is horrible at all of those things. VR isn’t suited for you to walk around virtual worlds at all.

Getting an omni-directional treadmill to track your footsteps and all the other attachments to rig your body up is a huge-ass investment that still doesn’t work amazingly well, and VR otherwise is kind of terrible at actually moving around environments bigger than a single room. I played an FPS game where I could move with a thumb stick, and I found myself swaying a little bit as I moved. Other people get outright sick when moving around in VR independent of their own locomotion.

Rather than look at the limitations of currently existing VR and imagine new games, a lot of VR games are just trying to remake the same genres we have already: Open World First Person Shooters & Military Simulations. And VR isn’t currently well-suited to those games, because locomotion is such a massive problem.

Half-Life Alyx does its damnest to try to create a traditional first person experience in VR, and even it has to submit to allowing the player to teleport, both out of convenience, and so they don’t get motion sick. While advancements in the way VR games are presented will and have reduced motion sickness in future games; motion sickness is still a massive impediment to VR’s widespread adoption (along with the price tag). And frankly, remaking the genres we have, but with a screen strapped to your face is a massive waste of potential.

We could be doing a lot more with VR games that we aren’t already doing, by looking at them as the next generation of motion controls, rather than looking at them as if they’re genuinely this “immersive” virtual reality experience. VR games give us a frame with which to see our movements contextualized in space, and there’s a lot that that enables, thanks mostly to next-generation motion control devices, which have incredibly accurate tracking compared to the Wii and PS Move.

The Wii succeeded pretty much entirely on the back of Wii Sports, a game that was the best selling game ever during its time, both bundled and unbundled with the console. People bought into the Wii because they wanted to play Wii Sports, and I think that VR lacks a similar “Killer App”.

The Limitations and Strengths of VR

So what are the limitations of VR? You can’t reliably achieve personal locomotion across a range larger than a room (and how many people even have an empty room to play in?). In order to achieve locomotion, you need a sled of some type, or the locomotion needs to be achieved directly through walking, or using your hands. Many games have pass-through vision reminding you of your surroundings when you’re too close to an object.

VR is really strong at engaging gross motor skills, and it has some limited capacity for engaging some fine motor skills as well. In other words, VR is made for games where you move your hands in interesting ways, and newer controllers, like Valve’s Knuckle or the Oculus controller have some support for the position of your fingers, such as which gesture your hand is currently holding. They also have buttons you can press, and sticks or touch pads you can move.

Inspiration

So where should we try pulling inspiration from for VR? Probably from real-life sports! They do a lot of gross motor functions over in sports, so why aren’t we looking at that? Here’s a list of sports that I think engage some interesting motor skills (taken from https://en.wikipedia.org/wiki/List_of_sports):

• Tennis/ping pong/handball
• Archery
• Boxing
• Canoe/Kayaking
• Grappling
• Billiards
• Kite fighting
• Hacky-sack/keep up
• Volleyball
• Darts/Javelin
• Wheelchair Racing
• Air Hockey
• Foosball
• Rock Climbing
• Egg and Spoon Race
• 4-square
• Tetherball
• Yoyoing
• Marbles
• Skee ball
• Slaps
• Jenga
• Bocce Ball

Obviously most of these have been done in VR already in some form or another, but usually only as some sort of demo game or arcade experience, rather than really being a fleshed out experience with a single player campaign of some type. Instead of looking at these literally, look at how they engage motor skills, what motions players make, and how they contextualize these motions into game dynamics. For the most part, we’re not seeing people try to mix these different styles of gameplay together into a bigger product, rather than creating demo games of these in isolation.

And an overwhelming number of VR games are just silly physics simulators, like Surgeon Simulator VR. VR is already being used by real world surgeons to learn new surgical techniques, or prep for difficult surgeries before actually performing them. Trauma Center was a cult hit that showed how surgery could be a good framing device for touch and motion controls on the DS and Wii. It had an engaging and deep scoring system that rewarded accuracy and speed, with a variety of different tools. It played a bit like a rhythm game sometimes, but you could set your own pace, and there were a number of dynamic and precision challenges. This could be a good template for expanding beyond silly surgery simulator to a full-fledged game, but VR just isn’t quite hitting that mark yet.

Beyond this, how many everyday things involve interesting gross motor function?

• Sign language (the Valve Knuckle controllers can let people do sign language or gestures, Apple Vision Pro has external cameras that can capture gestures)
• Carving sculptures
• Drilling
• Shoveling & moving dirt around
• Deck shuffling
• Assembling/disassembling things quickly
• Threading a needle
• Cracking a whip
• Turning a crank
• Drawing paths or tracing lines accurately
• Catching bubbles
• Laser avoidance
• Flap your hands to fly
• Hold your hands out to glide

We’re getting parts of these in a number of games that currently exist

VR’s Successes

This isn’t to say that VR is completely creatively bankrupt. There are a number of interesting VR games that are trying things out, even if they’re awkward and not suited to the medium all the time.

I think the real “Killer App” for VR right now is Beat Saber more than anything else. But if you don’t want to play Beat Saber, what else is there for you on the system that isn’t better elsewhere? Regardless, Beat Saber is a super fun rhythm game that emphasizes what the platform is good at: Standing in place, moving your hands and head. You need to duck under some obstacles or move side to side, and you need to make clean directional slices through the note blocks as they approach you. Beat Saber is really fun to play, and really fun to watch other people play. It’s a rhythm game unlike any we’ve seen before. I’m kind of surprised it never came to arcades, considering how well it would fit that format.

The other “Killer App” is SuperHot VR. The “time moves when you move” mechanic is astonishingly good in VR, allowing you to wave your hand to make time move, and to see and avoid bullets in slow motion. The physics are implemented spectacularly, and you can block shots with physics objects like the hammer, even doubling up to block with one hand, and shoot with the other. This isn’t unheard of in VR, but the time scaling system helps it really shine and give players a level of precision. Throwing things is still pretty janky in SuperHot, like most other VR games, but that’s certainly an area to work on. Superhot constrains you to a small area, thus avoiding many of VR’s shortcomings.

I played Blade and Sorcery briefly on an oculus device, and found it to be pretty interesting. There are a number of magic spells you can select, there’s a robust first person movement system, and a basic inventory system. You can control fireballs you shot after you shoot them, which is one thing I’ve been looking for.

However Blade and Sorcery is trying to do a lot of stuff that I don’t think VR is great at, and it feels pretty janky to have the awkward first person tank controls on a joystick while you can free look around with your headset. However, attempting to platform in this game is pretty miserable, both because of the controls, which would be poor even for a conventional game, and because VR makes it difficult to tell which way you’re moving, because you could face off in another direction from your character’s orientation, and your only cue for which way the character is facing is which direction they move when you hold the joystick forward. It’s all made worse by a crappy mantling system, and the fact that the jump button is depressing a joystick. Moving a joystick and clicking the button inside at the same time is agony in any game.

Maybe the biggest success of VR is VRChat, which isn’t a game at all. It’s just a space where people can build ridiculous things, explore, and hang out. More than the corporate side of VR promising the work environment of the future, VRChat is a space people actually want to hang out in, a lot like Second Life (or Furcadia) before it.

VRChat is a social phenomenon all unto itself, being flexible enough to support a massive variety of different gizmos that users can interact with, and letting players dress up in a massive number of avatars, or use 3d modeling to make avatars for themselves, rigging them in many different, complex ways.

What Could VR be Doing?

To start with, VR games featuring physics should be rolling their own physics more and applying more constraints and smoothing onto their physics middlewares in order to create more reliable and consistent physics interactions. Zelda: Tears of the Kingdom has shown this is an achievable goal (good luck getting the secret sauce out of Nintendo though!), and games like Rocket League have done this for ages.

General purpose physics engines are wacky, and “physics game” has become a whole genre for this reason. Manually coding physics interactions, or applying constraints to physics objects can allow for the robustness of interaction that comes from physics simulation, without the wild unpredictability, meaning that players can reliably interact and master physics systems.

Physics is one of the most accessible ways to cultivate depth in video games. Physics is almost synonymous with depth in video games. Given that the strength of VR as a format is moving your head and hands in a 3d space, reliable physic interations are an absolutely essential component of almost any VR experience.

Locomotion

Finding more ways to locomote using the hands is another pretty obvious win for VR. A lot of my examples of the successes of VR were in regards to hand-based locomotion for a reason. Walking around in VR is a massive hurdle to the format right now, in large part because people get sick from it. (maybe it would help to keep the floor/ceiling still, while walls, NPCs, and objects glide over the still floor?) There’s footage of hundreds of people running into their surroundings with a headset on, and people realistically do not have a lot of space in their homes for VR play. VR games need to adapt to being played sitting down, or standing in place.

The Climb VR is not the most interesting gameplay-wise, but it shows off that locomotion using your hands to drag yourself along is surprisingly effective. Echo Arena takes this concept further and basically builds Ender’s Game in VR. And Hookshot VR takes this concept to its extreme and is probably really motion sickness inducing. Hookshot VR gives you a hookshot and challenges you to fly around maps, shooting at things and accelerating off geometric structures. Kayak VR: Mirage lets you paddle yourself around using an oar, and has a basic simulation of the physics of a kayak being pushed around by an oar in water. You can dip one side of your oar into the water when you’re traveling with velocity to redirect your kayak.

One of my favorite games is Getting Over it With Bennett Foddy, which is entirely based on physics. Where’s that in VR?

Alright, maybe VR is cool. (The game is Clamb VR)

Probably the most daring hand-based locomotion in VR is Gorilla Tag, a game where you play Tag with others and you move by literally flinging yourself through the trees with your hands like a Gorilla. This can be intensely sickening for people sensitive to Simulation Sickness, but it’s also a natural form of movement in VR that incorporates gross motor function into a game mode! I feel like this is the Wii Sports of VR (a game that is instantly intuitive and uses the technology directly and fully to make a game that is uniquely suited to the format), and it has a small audience building around it, a lot like other streamer games like Among Us. It lacks a lot of polish, it’s no Beat Saber, but it’s something.

What Gorilla Tag also shows is, you could make hand-based locomotion the entire game. Imagine one of these VR sword games, and you can simply grip at any time to drag yourself along, regardless of facing an actual surface or not, just pull on the air, a lot like a flick of a mouse.

Valve’s The Lab has a number of cute test games and shows off a bit of what VR environments can do. In particular I like the Shmup style bullet hell where you hold the ship in your hand, controlling it with your hand. You then have a variety of enemies that spawn and shoot bullets everywhere. Your ship is limited to staying inside of an orb. This could be an amazing new format for Shmup games if they worked a bit on readability. It can be tricky to judge distances and angles, but it’s a very natural and intuitive interface to simply move your hand. Additionally, the orb and 6DOF orientation of enemy ships can make the game more difficult, as it’s hard to rotate your hand arbitrarily, so you get all twisted up trying to aim and move.

H3 VR has a little Merry Poppins style helicopter rotor you can hold up to accelerate you in the direction it points, allowing you to move around. This shows how locomotion can be made possible in a way that is controlled with your hands, though it is likely sim-sickness inducing to most people.

VR has a bunch of these one-off experiments, which individually show what might VR locomotion work in an intuitive way, but we aren’t seeing the same investment from more serious developers.

There are shockingly few VR games where you operate a wheelchair. Wheelchairs, Kayaks, and Climbing show how you can give people control over their movement without requiring them to walk, and while inducing less motion sickness. Dragging yourself along the environment like you’re an ape traveling on their knuckles is also unexplored.

Many VR games structure themselves a lot like lightgun games in arcades, plotting you in the center of a space and asking you to shoot at things and grab things around you, then teleporting you to a new room, or further along in the same room when you’ve cleared out a group of enemies. Superhot VR is probably one of the more successful examples of this. Pistol Whip has your camera glide through environments as you shoot enemies. A way of making this less sickening could be standing on a sled or platform, giving you visual grounding, and interacting with things as you pass them by. This could be structured like a virtual Safari; offering players a choice in which routes they travel, and how fast they progress. Perhaps you have a lever that can speed up or slow down your carriage, and select which route is preferred in advance.

Overhead Games

VR games can also offer new takes on traditional genres, like RTS and Board Games. Obviously tabletop simulator works in VR, but it limits itself to simply replicating the function of physical board games. A number of other games like Demeo, are creating virtual turn based games in VR, and using the computer to automate tasks like calculating damage and other game mechanics, as many turn based video games have always done, but VR could be allowing us to mix turn based strategy and real-time strategy in ways we never have before. Pieces could move of their own accord. Pieces could have cooldowns or effects over time on the other pieces around them. We could instantiate objects from menus as necessary. A lot of lessons could be taken from Knights in the Nightmare and the way it has real-time turns, and obstacles for your cursor to avoid as you control your units.

RTS games in VR could allow you to draw patrol patterns for your units, forbidden zones, and otherwise find ways to utilize your ability to easily define spaces and lines as a tool for defining the AI’s behaviors and limitations.

I’ve always been a fan of God Games, like From Dust, Black and White, and Populous. This style of terraforming gameplay, especially as From Dust implemented it, could easily fit into VR. You could pick up and drop down land, move plants around, manually direct rivers. You could carve the landscape and raise mountains. You could build a pinball style game based on this, where your hands can poke the board to create weak gravity wells, or alter the terrain of the course. Imagine a game like Yoku’s Island Express in VR, where you can reach out and interact with the environment as the pinball travels around the board. You could more naturally bring back the bumping mechanics of some pinball games to enable things like Death Saves, without the awkward single button bumps of early pinball video games.

Manipulating Objects

You could even do silly stuff in VR, like having psychic powers. Point at something far away to highlight it, grab it, and then move it around like it was in your hand, like you have force powers. There could even be a “reel-in” gesture that pulls and snaps the object into your VR hand after it’s been force grabbed. You could have VR arms that extend further than your real arms, or which grow proportionally longer as you stretch them out, like you’re Mr. Fantastic. There are already VR games that let you web sling like Spiderman.

Imagine a Tron disc game, where you can toss a disc, and move or rotate your hand around to control its flight arc as it flies, then it’s rigged to bounce off of random objects like it’s Captain America’s shield and always return to you.

Bocce Ball is also ripe with possibilities, trying to propel something with physics and get it to land as close as possible to something else, or just barely within range of something else, or close to parallel, or perpendicular, or so on. Finding ways to systemitize these types of geometric and physical relationships between objects into game systems is a massive field of potential for VR games (and any game for that matter).

Imagine a game like Okami, where you can paint symbols over things to apply effects to them, or like Metal Gear Rising, where you’re challenged to find specific cross-cuts of objects. VR makes these types of interactions much simpler and less awkward by allowing you both the ability to visualize objects better in 3d space, and more direct analog control over the tools used to draw. The 3d Sculpture tools we have in VR could become the basis of a whole game.

VR games could experiment with stacking things and getting them to stay upright (with of course healthy physics constraints so the whole thing doesn’t explode when you put your hand near it). You could assemble block buildings, or assemble things with mechanisms based on the order you stack. You could reassemble a tool you wield in a bunch of different ways to get different effects, so you’re reconfiguring it all the time to get different effects out of it. Maybe you have an inventory of these multi-tools and you could swap parts between them? Tears of the Kingdom is currently having players do a lot of this all over the place, and is absolutely a great role model for this mechanistic style of gameplay.

Guns & Gestures

I know that a lot of people like the realism of aiming down the sights of a gun to shoot, and VR weapons simulations replicate a lot of the complexity that goes into real world firarms handling, but, I think that VR games should probably abstract a lot of these fiddly secondard skills away so they can focus more on the core interaction of aiming and firing. Any VR game that could reasonably have a laser pointer on something you aim should have one, or should highlight your designated target. Motion controls are fiddly in a way that’s exaggerated compared to real life or even Mouse controls. Physics constraints and additional aim assistance can help smooth out this extra fiddliness. Giving people reliable controls that test their skills helps make it easier to diversify VR games into meaningfully different gameplay experiences instead of the game of, “can I operate this gun and aim at a basic target?”

The Valve Knuckle controller’s ability to recognize the extension of each digit in your hand has allowed people to speak rudimentary sign language in VR, which is incredible! This could also be used as a form of gestural control, or as a control modifier for actions in much the same way as buttons. You could literally do Naruto jutsu hand signs and produce effects based on which digits you extend. You could do weapon selection by holding out rock✊, paper ✋, scissors ✌️, gun 👈, or the devil horns 🤘.

If methods can be developed for consistent gestural controls, then you could make a whole game based on using magic with swishes and flicks of your hand. You could develop a dancing game. You could go full Minority Report and have a whole menu manipulation game.

It should be obvious that the untapped possibility space and the expressive potential for VR games is really really massive, and I believe the failure of VR as a consumer product is due to a failure to recognize that this potential even exists, as well as a failure to leverage VR games into full consumer products instead of shovelware or tech demos. The platform holders: Valve and Sony, are too moored to preconceptions of the games we know to really step up to the plate in delivering anything new that isn’t just a tech demo, and smaller players are scared to invest in a platform that doesn’t have widespread support already.

AR: Augmented Reality

One of the big motivators in finishing this article was the Apple Vision Pro. Apple’s new AR glasses have pass-through vision, letting you see your surrounding environment, and much like the Kinect, are operated through hand gestures, as well as vision-tracking. It has no games, and it wasn’t touted as a gaming device in any way, but it was really surprising to see Apple take a stab at this market, and I expected it to represent the possible dawn of a new age of legitimacy for the technology.

AR has a lot of potential to more or less make a game like the original trailers for Pokemon Go, where you explore the real world and see additional information overlaid on top of it, and maybe throw some pokeballs around. Imagine extra information about places hanging around in virtual space everywhere you go, like Dark Souls soapstone messages. You can go watch the anime Dennou Coil for all sorts of ideas as to what’s possible in AR; Virtual pets, currency, windows into extradimensional spaces, AR-only glyphs. You could hold up your hand like a gun and play laser tag with your friends.

Unfortunately, despite the image of competence that Apple projects, Apple Vision pro was a complete flop, for a variety of reasons. The biggest among these probably being: It has no games, and motion controls or gestures are a miserable way to interact with user interfaces of any kind. We have keyboards, mice, and capacitive touchscreens for a reason.

Apple Vision Pro didn’t bring legitimacy to VR/AR. It launched with a massive price tag of $3,500 USD (about 7 times the price of another headset) and failed to meet even reduced sales targets. The headsets would heat up and the front panel would crack in half. Apple didn’t bring a new level of competence and mastery to VR technology, it just brought a massive markup and a walled ecosystem lacking a killer app or user experience.

My prediction was that the Apple Vision Pro would be decidedly an enthusiast device, as the price tag keeps it impractical for any purpose. But I also expected that this would raise the bar for AR and VR devices as other manufacturers copy or reverse engineer apple features, drive down the manufacturing costs, and spur a new wave of interest into the technology. This didn’t end up being true. Despite the price tag and Apple name behind it.

Conclusion

My overall prediction remains, of course, that VR will never catch on unless it has more software that demonstrates the strengths of the format, showing people something they want that they cannot get anywhere else; that isn’t simply a gimmick. In other words, it needs to be useful in a unique way that justifies the purchase price. This is a hard and fast law of any video game system. VR simply does not have good enough games to make it worth it to anyone besides tech enthusiasts. And in the case of Apple Vision Pro, this proved to not only be true, but even tech enthusiasts quickly discarded theirs, asking for refunds after they were done showing off for their audiences.

VR is a platform, like a console, or like Steam. People ONLY buy a console or install an app store when there is a game (or application) they want on it. Most VR games are not bringing people into VR.

A lot of current VR games are segregated into either AAA publishers trying to release a VR version of the game they already have, indies releasing basically demos, or indies making first person gun simulators. Here’s a quick google search:

Almost all of these games are doing stuff we’ve seen before, that would probably be better with traditional mouse and keyboard controls. VR has perhaps moved beyond chore simulators, but developers still aren’t capitalizing on the format. In order for VR to catch on, VR is going to need games that aren’t just moving around with an analog stick and pointing at things.

We can’t expect mass adoption of this inconvenient device (or fun and deep games to be made) if our sole influence is the Star Trek Holodeck and AAA video games.