As data scientists, we’re lucky. Data scientists, in the broadest sense, are the very first human beings whose conceptual language — in particular dimensionality-reduction, compression and statistical validation — can explain human bodies and minds, and hence “consciousness,” in neutral, actionable terms. Our brains are information processors, and data scientists know information theory. We have the golden ticket.

I came to data science after a job at Bell Labs, a Caltech Physics Ph.D. and a postdoc at the National Institutes of Health math research branch. I worked at one tech startup after another as a coder and software architect, first calling myself a “specialist in statistical algorithms,” and later chief algorithm officer, while still writing my own queries and code in real languages like Java, SQL and Python — not Matlab or Mathematica. I made a couple big splashes, and the last two startups were acquired. Those “exits” bought me time to work on cooler stuff.

In retrospect, I was probably destined for data science. I grew up in Silicon Valley, in Menlo Park. Learning to program BASIC at age 9 on an old-school teletype with an acoustic modem, tracing down nano-amp current-leaks in homemade circuit boards, measuring atomic nuclei as a laser-jock at Bell Labs, finding statistical gold in neural spike-trains at Caltech — I was always gathering data and making sense of it myself.

From this physics/brain/data perspective I see the sword of technology cuts both ways. Information theory tells us how fragile brains are, and Big Data tells us effective ways to exploit that sensitivity in grabbing human attention. The problem is, attentional economies are zero-sum: Every time my app captures attention or influences behavior, some live human at the other end loses focus or autonomy. Yes, algorithms can influence nervous systems, but it’s not fair, and bad for them. That’s the dilemma I want us data scientists to solve.

We Ruled Tech

I was the child of two brilliant nerds, and the older brother of one. Dad was a nuclear physicist, mom too for a while, before becoming a reporter and environmentalist. My kid brother Ed filed a corporate patent in high school, inventing a way to measure hundreds of sensors with three wires. With me he was a brilliant co-conspirator, collaborator and technical colleague. Our parents gave Ed and me a lot of help and a lot of freedom, so we rode bikes everywhere, bought old radios to fix at garage sales, and climbed trees.

Silicon Valley wasn’t called Silicon Valley then. Santa Clara and Fremont were mostly orchards. Start-ups and stock options and venture capital weren’t a thing yet. Life was more three-dimensional: We built things with our hands, bought from surplus stores (not screens), rode actual bikes on real asphalt, and were unreachable unless our parents knew whose house to call. Tech was rare: long-distance calls were expensive, no answering machines, only a couple TV channels (none on Sunday morning except “Agriculture USA” out of Nebraska), no cable, no internet, no Google, nothing wireless except walkie-talkies. It was easier to work on major projects without distraction. What was fun about making TV jammers, “librarian tormentors,” blinking tie clips and such was actually making them. We had to select the transistors, read resistor color-codes, make our own circuit boards and measure voltages and currents all day long, with analog voltmeters and a 50-pound 1949 Tektronix 512 vacuum-tube oscilloscope with a small round green screen (bandwidth limit 1 MHz). We took and analyzed our own data on the fly, and when our circuits finally worked, it felt like we had beaten Mother Nature.

Back then we ruled tech, it didn’t rule us. You would build a gadget to do something cool and simple, and it did. No software, no dependencies — just wires and voltages. For example, Ed’s and my piece de resistance was hacking the public-address system at Menlo-Atherton High School. This was a perfect “MacGyver” project made from a cassette-tape player hidden under the eaves, two wires from it covertly spliced into the amplifier’s co-ax cable, and a remote on-switch dropped down a drain pipe for access from the hall below.

We had one shot, and it worked: Our bootleg announcement canceling final exams aired during the second period on the last day of classes my sophomore year. That technology was analog, simple and do-it-yourself. But with digital tech today, the knowledge-to-complexity ratio of man vs. machine has been inverted, and you buy it instead of building it yourself. Now tech can understand you, it can advise you, it can anticipate you and it can trick you. For these abilities, data scientists can take credit.

Back then, data science wasn’t called “data science” either. I called myself a “specialist in statistical algorithms” when I first returned to the valley. There weren’t billboards up and down Bayshore Freeway recruiting machine learning and data science specialists, like now. Then, the job made you one of the few people in the building who actually knew what’s going on. You run the queries, you vet the data, you find the trends and correlations, you build the predictive models, you live and die by the math, and you have the CEO’s ear in ways marketing guys never can. If data is the lifeblood of a company, then a data scientist is an oracle.

Data Oracles

Of course, people don’t always listen to oracles, even if they should. Execs see patterns in random noise, or fail to believe statistics. One CEO told me to lie to his investors (I didn’t). Several ignored proof of crippling product problems; more than once I was the messenger who got shot. Of course, the customer needs take priority. Of course, deep architectures take too long to build. Of course, the laws of information processing can’t answer everything, but the answers they do give are always right. (“Ye canna change the laws of mathematics, Captain!”)

This is where our collective faith in mathematics can help us solve our dilemma. The most important principle I internalized in 20 years of data science, from my first successful commercial algorithm (video car detection) through my most impactful (the PreFix debugger that saved Microsoft) and most elaborate (an auto-bidder against Google’s AdWords “auction”), is dimensionality reduction. In information-limited scenarios — which means almost always — raw data should be fit as gracefully as possible to low-dimensional continuous models.

For example, the algorithmic trick called singular value decomposition (SVD) transforms one high-dimensional vector into another by going through a narrow, low-dimensional chokepoint. The narrower the choke-point, the better the signal-to-noise ratio and the better the answers. It’s kind of like Occam’s razor: The best models are the most compact. Dimensionality-reduction gets much more complicated when the low-dimensional data-manifold is curved or variable, but the basic idea to wring out noise by judicious compression is essential.

Ideally you know the structure you’re looking for in advance, like “assuming” a day has 24 hours or a week has seven days. Never, ever use data to relearn things you know in other ways — that’s a waste of data. For example, MRI machines know in advance you exist in three dimensions, as a prior assumption hardwired in place before scanning data even arrives. Such prior assumptions aren’t cheating or laziness, they’re mathematically essential to get anywhere with limited data. It’s called Bayes Theorem. I first heard it evangelized 25 years ago at Caltech, and now it’s common currency.

If you’re with me so far, you already have the keys to understanding the brain and how to live. That’s the cool project I’ve been working on the last couple years, with my partner Criscillia. Here are the results in three very compressed paragraphs. They are justified at length in this peer-reviewed article, “Sensory Metrics of Neuromechanical Trust.”

Brains Are 3D

All brains evolved to map 3D space, assembling spatial maps from discontinuous pulses sent by distributed sensors. Doesn’t matter if it’s a visual map from retinal pulses, or a proprioceptive map from mechanoreceptor pulses, in both cases the data is high-dimensional and discontinuous, yet must represent continuous 3D space. That’s a functional definition of the technology called real-time tomography (like MRI), a very data-hungry and processing-hungry operation. 

The conclusion that brains are 3D simulation engines seems obvious in hindsight, but in one stroke it promotes raw, real-time sensorimotor experience as the highest, purest function of a brain, and demotes anything quantized. Memory, language, categories, decisions and cognition are all coarse, low-bandwidth hacks, ill-suited to the exquisitely speedy circuitry they run on.

We did not evolve to “think” — we evolved to move, sense and feel, which is still our primary activity even if we’re oblivious. Our minds are actually continuous, unified and far more powerful than we’ve been taught. I’m sharing these insights with you because I want you to understand how our minds work and use that knowledge to begin healing the world.

Natural Inputs Are Good, Artificial Ones Bad

Look around you: at your fingernails, the room, beyond the window. Do the numbers: using an HDMI pixel-pitch of 0.2mm as a benchmark, your visual field exhibits teravoxel resolution with predictive latency at zero and sub-millisecond phase precision, fed by only a million pulses per second, i.e. about a megabyte of input decompressed millions-fold into a vision far surpassing anything Virtual Reality can even dream about.

Whatever magnificent engine is synthesizing that real time, hyper-resolution, hyper-consistent image from a few photons through two tiny pupil-holes, the self-tuning strategies of that engine evolved for fractal, continuous, multi-sensory, interactive inputs. Think trees and turbulence, not discontinuous images, events or words. Evolution hyper-optimized brains not just for 3D space in general, but for specific continuous patterns found in nature.

That is the statistical contract and informational nutrition each brain needs. As one might expect, such a high-performance system doesn’t do well when most of its data arrives in the wrong format. Digital inputs deviate from the natural contract in every possible way: They alternate smoothness with discontinuity, they fracture sensory input, they arrive on flat screens, they teleport through space and time and they are specially sculpted to catch our eyes and hold our interest. Technology is neither natural nor random — it is so sophisticated that it can captivate and even hack our brains.

Go Back to Basics

Brains work fine with the right input. You’re a vertebrate, so your most important systems lie along your midline — swallowing, breathing, reproduction. Learn to feel and flex your spine with yoga, Pilates, dance or Feldenkrais. Go for high-entropy grace and flexibility, not low-entropy reps and cardio. Go “forest bathing” in quiet and natural places with trees, vistas, wind or water. When you can, be close to people in proximity, ideally without remote distractions, and soak up the micro-expressions and nano-gestures of human resonance.

Use those nonverbal channels at work as much as possible. When you must, use digital channels, respect their glitches, biases and limited sensory bandwidth. Don’t expect low-latency replies, assume misunderstandings are innocent, don’t do emotions via text. Mobile is better than text, landline better than mobile, video better than only audio, but proximity still beats all, million-fold.

The tragedy of technological damage is best explained not in the language of capitalism, materialism, civilization, human biology or even carbon-based life. The most efficient language is mathematics, and it describes the inevitable collision between creatures’ information-foraging habits and the natural results of their material productivity. We make things we find attractive, and now that’s all we see. This could happen anywhere in the universe.

The triumph of the data scientist is that we can understand our minds as ultra-high-performance representational engines. The tragedy of the data scientist is twofold. First, doing computer work damages our own sensory needs. We must sit down indoors for hours, stare at screens, communicate with text and think through code and numbers.

Second, our profession damages the outside world when it manipulates people by aggregating revenue from micro-thefts of human attention and autonomy. Worse, the results turn children into addicts and tip elections. We, as data scientists, are both blessed and cursed: We will be the first to know what humankind is made of and the first to offer help, but also the first to understand our crimes. Let us apply our brains to solve them.

*[The articles in this column present a set of permanent scientific truths that interlock like jigsaw pieces. They span physics, technology, economics, media, neuroscience, bodies, brains and minds, as quantified by the mathematics of information flow through space and time. Together, they promote the neurosafe agenda: That human interactions with technology do not harm either the nervous system’s function, nor its interests, as measured by neuromechanical trust.]

The views expressed in this article are the author’s own and do not necessarily reflect Fair Observer’s editorial policy.