Over a hundred years ago, most scientific evidence pointed toward an impending invention which would change the world, encapsulated in the paradoxical expression “perpetual motion.”  Ultimately that invention proved to be impossible because of the brand-new scientific discovery that energy cannot be created nor destroyed.

Nowadays, a similarly profitable fantasy builds on a similarly paradoxical expression: “virtual reality” (VR).  Turns out Nature says VR won’t succeed either, because VR will inevitably induce “simulator sickness,” as it always has.

The Industrial Revolution started with steam, allowing fuel (coal) to do the work of many men.  As the technology improved, more and more power became available. Part of that power came from burning more coal. Another part came from improved mechanical efficiency, that is by recovering and reusing waste heat, force and momentum.  Many tinkerers were convinced that by using clever mechanical trickery, such as lifting weights over here in order to drop them on lever-arms over there, engines could in fact “recover” more energy than went in.  Evidence made this hypothesis reasonable, because the trend of recovered energy had been rising upward steadily for decades. Hopefully it could pass 100%.

The idea behind perpetual motion was that if the trick worked — that if a machine could essentially harvest its own momentum to keep itself running forever — then even a tiny excess of power could be amplified and scaled, and no one would need to burn actual fuel any more.

Back then physics and physicists didn’t really exist, but thoughtful people ever since da Vinci have known perpetual motion was a fantasy. A hundred years ago, they proved it scientifically by finding a deeper principle at work, one which absolutely limited the amount of energy in play. The new science said that energy is not created, not destroyed, and certainly not free.  The total energy must be “conserved” (kept fixed).  No free lunch from Nature.  But optimistic tinkerers kept trying anyway, until the US Patent office stopped allowing applications altogether, killing the “technology” for good.

Virtual Reality or Unreal Virtuality?

That fantasy repeats itself with so-called “virtual reality.” According to the evidence, VR gets better every year.  An extrapolation of that trend would let VR replace the boring physical world we’re usually stuck in, literally creating whole new universes (or metaverses) and whole new streams of revenue, almost out of nothing. Free reality.

I know VR cannot work because I happen to know how nervous systems work. New technology won’t fix that mismatch, but at least new research explains it.  That research explains both human and machine learning in the same terms; neuroscience and data science account for both as signal bandwidth. So formerly fuzzy questions about how brains work now have mathematically absolute answers.  In the case of VR, as with creating energy, it turns out there are absolute limits on what brains can and can’t do, limits not provable before.

Make Sense of the World

Unique Insights from 2,500+ Contributors in 90+ Countries

I agree to receive emails and other content from Fair Observer. I understand that I may repeal my consent at any time. You can review our Privacy Policy and Terms of Use for further information..

There are many ways to prove that VR makes people sick; two will do for now.  One involves how different senses mix together in the brain.  The other involves how much time a brain takes to mix and make sense of them.

Vertebrate brains evolved 500 million years ago to do exactly one thing, a task which even now is far more difficult than memory or speech: making 3D pictures out of tiny input pulses (a computational process called “tomography”).  Our everyday experience bears this out. The sensory inputs into our bodies (and outputs from nerves into the brain) come from eyeballs, eardrums, taste and smell receptors, and especially from millions of vibration-sensors spread throughout the body. Airborne sound hits ears and skin together, and our brains combine them into a single unified experience so solid and believable that we know for sure the world exists, even behind us, even when we can’t see it. Lived sensory experience is unified by the hardware of our brain: that’s how brains work and what they do. Neuroscientists call the process “sensory fusion.”

Obviously, a brain fabricating a single unified experience is the opposite of fabricating two inconsistent, competing experiences, which is what VR forces on our brains.  For example, a gamer’s eyes may be convinced that he is flying high-G rolls inside a fighter plane aloft, because VR is so good at creating visual illusions, making every visible cue consistent with all the rest….looming, moving, twisting, occluding, dropping, all synchronized so the visual world makes 3D sense.

But vision isn’t everything to brains, not even half. In the gamer’s case, all the other senses agree that the body is not moving or flying, but sitting in a chair. Neural signals from the inner ear, the legs, the gut, the spine all confirm no barrel-rolls, no upside-down, no special forces pulling or pushing.  No jet engine sounds rattling the body, just injected in the ears.  In this configuration roughly half the brain is convinced the body is quite still, the other half convinced it’s flying hard and fast.  A brain can’t hold such a deep contradiction for very long, so “simulator sickness” makes the gamer nauseous. That problem hasn’t changed in 40 years, and won’t, ever, because brains can only feel one reality at a time, and the real reality is always centered in your gut, regardless of what the eyeballs say.

Vision and motion

Another insoluble problem with VR is how fast it responds to self-motion. In the regular real world (no VR yet), every time you move your body, neck, head, or eyeballs, the image into your eyeballs (and onto your retina) changes with that motion.  To make its picture of the world, the brain anticipates the physical shift before it moves its muscles, and uses that anticipation to predict what it will see. The brain uses an interactive process of continual exploration and zooming (neuroscience buzzword: “sensory contingencies”). Because the brain makes plans, then sends pulses. And then the head and eyes begin to move. The brain therefore creates internal expectations long before any motion could be visible from outside.

But at best VR can measure your self-motion from the outside, after the fact.  It can’t measure things which haven’t happened yet. (Even access to your brainwaves would not solve this problem, since even brain waves are merely delayed traces of yet smaller and more subtle processes). So even an ideal VR response would be fatally delayed, relative to how your eyes and brain normally work.  What VR shows your eyeballs is not exactly what would come from a real world, but milliseconds slower, and only approximate. The faster you move your head and eyes, the more weirdly a fake world slips under them.

Make Sense of the World

Unique Insights from 2,500+ Contributors in 90+ Countries

I agree to receive emails and other content from Fair Observer. I understand that I may repeal my consent at any time. You can review our Privacy Policy and Terms of Use for further information .

The core problem is not with VR, but with brains themselves because their task is nearly impossible already. It’s clear most humans see the world in high-resolution (HDMI or better in space, seamless motion in real time). But synthesizing high-resolution 3-D moving images is hard even for supercomputers and MRI machines. It’s even harder for the brain to synthesize so much data (teravoxels) if it gets a million pulses per second of input from two jiggling spheres of jelly (the eyeballs). That’s about a million data points synthesized for each single input pulse.  It’s a miracle that Nature can leverage such internal fakery, then erase the artifacts so perfectly the result seems not merely realistic, but absolutely real. Unfortunately for VR, that miracle is utterly dependent on the 3-D world actually being there. There is no mathematical way to make a consistent world-image from partial, delayed, corrupted data injected into only part of a brain’s input stream, while ignoring all the rest. Our brains need real-live 3D data like our lungs need air, and no amount of hype will change that fact.

*[The articles in this column present a set of permanent scientific truths that interlock like jigsaw pieces. They span physics, technology, economics, media, neuroscience, bodies, brains and minds, as quantified by the mathematics of information flow through space and time. Together, they promote the neurosafe agenda: That human interactions with technology do not harm either the nervous system’s function, nor its interests, as measured by neuromechanical trust.]

The views expressed in this article are the author’s own and do not necessarily reflect Fair Observer’s editorial policy.