The Metaverse Is Already Here

It's fair to say that reality has a lot of problems. War. Famine. Disease. Taxes. Unwanted accumulations of pet hair. But if you look at the world through the eyes of some of the world's biggest tech companies, a deeper, more fundamental problem reveals itself: There just aren't enough holograms.

From the earliest days of the modern technology revolution—the postwar rise of computers and connections that would eventually give us the internet and email and iPhones and Farmville—technologists and sci-fi writers have dreamed of a world with holograms: information and three-dimensional virtual objects floating in space around you, or entirely new spaces out of the digital ether that you can explore and interact with.

This was a multidisciplinary head-scratcher. It's easy to think of holograms as just an advanced display technology, like televisions and computer monitors. But it goes much deeper than that. To interact seamlessly with objects in three-dimensional space, even in the simplest way—say, turning your head to look at something from a different angle—requires the display not only to acquire information about the physical properties of your environment but to track what you are seeing and how, and then adapt accordingly.

The same is true of sound, which varies subtly based on factors such as the mass and texture of objects in your room as well as the tilt and location of your head. Your eyes and ears are sensors, detecting a vast amount of information about the world around you, which your brain then decodes, processes, and synthesizes in real time. Add touch, and the sensory measurement challenges grow broader still. To create a world rich with virtual interaction, you'd need technology to track and measure the breadth of human perception.

In a way, this is a philosophical problem as much as a technological challenge. What does it mean to see, hear, touch, connect, communicate—to interact with the reality around us? What even is the nature of reality? What even is, like, existence, man? Feel free to take copious bong rips before proceeding.

In any case, we don't yet have this sort of hologram. But in October 2021, Facebook chief Mark Zuckerberg announced that he was committed to solving the hologram problem once and for all.

He didn't call it the hologram problem, because no one calls it that. Instead, he said that Facebook—the social-media behemoth that in two decades has gone from a website where you could post party photos and use a "poke" button on your college crush to a sprawling and controversial cultural-political online ecosystem, widely viewed as both a radical threat to democracy and a good place to sell an old couch—was going to devote its considerable resources to building something called the metaverse, and would change its corporate name to Meta.

In a lengthy presentation that doubled as a demo reel of products, some coming soon, others hypothetical, Zuckerberg showed off his company's vision for the metaverse, a network of virtual and quasi-virtual places to work, play, buy, sell, build businesses, and connect with friends. There would be video games and meetings and workouts and other activities of every sort.

Zuckerberg cast his vision as empathic, human, liberating, and science-fictional. In the metaverse, you could take the form of a photorealistic version of yourself, interacting with a photorealistic version of your apartment, or you could have a staff meeting in space while inhabiting the persona of a dancing robot. You could ride a wave as a cartoon version of yourself or virtually punch a fantasy creature in a magical boxing ring. There would even be polygonal misbehaving pets, presumably without the accumulations of unwanted pet hair.

Throughout every experience, there would be a lot of holograms: shiny boxes glowing with virtual light and information display cards floating in midair. You yourself would be represented in many of the demo's hypothetical situations by a kind of hologram: an "avatar," or hyper-customizable profile image, which might look like you, or might look like a lobster, or might switch back and forth depending on the circumstance.

You could sense the nerdy thrill in Zuckerberg's hammy, stilted delivery. Finally, the dream of a hologram-rich experience was going to be realized. The presentation was bathed in tech-industry idealism, with Zuckerberg enthusiastically describing a world that was not only more immersive but more human, better able to connect people and let them live and work freely.

But there was something more than a little bit cynical about the whole affair, and not only in the invocations of easy payment transaction systems. Of course Facebook—now Meta—wants to sell stuff, and to make it easy for other people to use its platform to sell stuff. But throughout the presentation, Zuckerberg also kept coming back to the importance of rules. His remarks included studiously vague mentions of regulators and policy makers and shared norms of responsible corporate behavior. All of which, if he gets his way, will reflect Meta's business interests.

The company is hardly the only one declaring its intention to build the metaverse. Most obviously, software behemoths such as Microsoft and the video game production company Epic have announced major efforts. But brands that seemingly have nothing to do with technology—producers of dish detergent, for example—have also clamored to create metaverse experiences as marketing exercises. Holograms, sadly, won't wash your plates and glassware. But they might sell you soap.

Since the dawn of the internet, the titans of tech have aspired to disrupt old industries, from taxicabs to pizza delivery to conference calls to house hunting to saying "yo" to your friends. (Really. In 2014, there was an app called Yo that had exactly one function: to send the word yo to someone else. It received $1.5 million in venture funding.)

But the metaverse is something more. It's an effort to disrupt the nature of reality itself—to replace it, or at least augment it, with a digital simulacra of reality, one that is more malleable and more personalized. It offers a vision of a brighter world connected by freewheeling sensory technologies that facilitate human interaction and experience, designed and controlled from the ground up by users who are empowered to shape and share the spaces around them.

At the same time, the metaverse offers a vision of a highly mediated reality that is even more friendly to ungainly forms of digital commerce and advertising, one whose highest-profile proponents seek to preemptively manage and manipulate government intervention by welcoming it from the very beginning. It is a competition, in other words, between two rival, seemingly contradictory visions of social technology, one about liberation, one about control.

As it turns out, both visions are (mostly) prescient.

From Virtual Reality to Cyberpunk

The first thing to understand about the metaverse is that it's not just virtual reality, or VR—that thing where you strap on goofy-looking electronic goggles and "jack in," then grope around the air looking like an idiot to anyone who can see you in the real world. The metaverse, as the venture capitalist Matthew Ball wrote in a lengthy online primer on the subject, is an interlinked mesh of technologies, including omnipresent networking capability, digital payment processing, portable computing, and shareable virtual identities, which can encompass not only your avatar but your digital wallet, your social networks, and various other forms of personal information.

But the metaverse is not not virtual reality, either. Its users will almost certainly wear goofy-looking goggles at least some of the time. So to understand where the metaverse comes from, you have to understand how VR came to be, including the idiosyncratic outsider culture it spawned.

Just as there's no definitive starting point for the internet, virtual reality crept into the world slowly, an iterative product of many different lines of research. But if we must pick a single point, let's start in the 1960s, with the legendary computer scientist Ivan Sutherland and a device that became known as the Sword of Damocles.

In 1963, Sutherland developed Sketchpad, a primitive but revolutionary digital drawing program. With this software, a human hand could trace light that would appear on screen. It was the earliest program ever to employ a graphical interface.

Graphical interfaces are commonplace now: You use them every single day on devices ranging from phones to cars to refrigerators. But in the early days of computing, they didn't exist. Users interacted with computers through abstruse code and paper punch cards, which were sometimes handled by trained human intermediaries, so that most people, even scientists who relied on computer calculations, never interacted directly with computers at all. Sketchpad showed not only that people could interact directly with the digital world but that they could do so to produce art, not just equations.

In the late 1960s, Sutherland followed Sketchpad with yet another revolutionary graphical interaction device, this one arguably even more ahead of its time. It was a set of goggles—a head-mounted display hung from a ceiling armature that connected to a computer. The system tracked user movement, and people who looked through the goggles could see extremely basic (but at the time unprecedented) computer-generated images floating in the room. If you moved your head, you could see around the image from many sides and angles. Line-drawn cubes floated in midair with a seemingly dimensional presence. It looked and felt like the cube was really there.

Technically, this was a form of augmented reality, also known as mixed reality, which blended the real and the virtual. But these distinctions barely existed back then. The point was that as early as 1969, Sutherland had invented a primitive yet functional system for viewing—you guessed it—holograms.

The Sword of Damocles was not the viewing mechanism. It was the heavy motion tracking system that hung precariously on the armature over the viewer's head. Hence the name, a reference to the ancient Roman parable about a court member who wished to rule like the king. The king agreed to switch places for a day, seating him on a golden couch and tasking servants with pleasing his every whim. But the king also hung a sword above the subject's head, reminding him that with awesome power came an omnipresent threat.

When Sutherland ushered early VR technology into the world, there was no awesome power yet. But there was a sense of unlimited possibility about the radical liberation the technology could bring about. For within Sutherland's virtual floating cube was the potential for a world in which reality, or at least the perception of reality, could be altered at a whim.

With computer networks growing rapidly through the '70s and '80s, a new idea took hold: Not only could you reshape reality for yourself, but you could share that new reality with other people. VR didn't just let you augment the existing world; it let you create entirely new ones, enabling both individual freedom and communal connection.

In the years after Sutherland's invention, the idea of VR would inspire and enthrall a real-world Silicon Valley community of inventors, visionaries, quacks, cranks, and entrepreneurs. The effort was tied up with nascent hacker culture and its radical ideals, which mixed and matched space-age engineering processes with various flavors of libertarianism, syndicalism, communalism, posthumanism, New Age hippie-dippie woo, psychedelia, techno-spiritual transcendence, radical environmentalism, and whatever else happened to be coursing through the California counterculture at the moment. Timothy Leary called virtual reality "digital LSD." Others compared it to a shared lucid dream, a world without rules, whose only limit was imagination.

As Jaron Lanier, founder of the first virtual reality company and one of the medium's most influential figures, recounts in his 2017 memoir, Dawn of the New Everything (Henry Holt and Co.), some people in the scene even threw "VR parties" in which rooms were decorated to look like polygonal virtual environments that might eventually exist. People were so desperate for a shared world filled with holograms that they hung cardboard props in party spaces to imagine how they would someday feel.

But the actual technology was slow to develop. VR concepts were mostly explored and popularized in science fiction, which wasn't bound by technological constraints.

In 1984, William Gibson published Neuromancer, a neo-noir mashup of hard-boiled detective fiction, art-rock sneer, and early hacker mythos—think Blade Runner by way of the proto-punk band Television—that helped define the mood and themes of the genre we now know as cyberpunk. The book's most enduring contribution to popular culture was a single word: cyberspace. The more important part of Gibson's neologism was not the often-abused prefix cyber but the term that it modified: space. Virtual reality wasn't just a vibe, a style, or an attitude. It was a place you and other people could go.

The idea of virtual reality as a communal environment, which had now been given a name, persisted and proliferated throughout the decade. The sci-fi series Star Trek: The Next Generation featured an infinitely modifiable virtual reality playroom that could accommodate complex role-playing games, historical exploration, exotic-environment relaxation, and other full-on sensory experiences, no goofy goggles required.

Naturally, it was called the Holodeck: a place the crew of the starship Enterprise could go to experience holograms together.

From Concept to Prototype

In the early 1990s, the core concepts of VR began to meld with another new technology: the World Wide Web. Computer networks had been around for years in the form of Usenet and other online message boards, but the user base was comparatively small, made up of a self-selected bunch of academics, computer geeks, and enthusiasts. The web brought connection to ordinary homes across the world, and it did so through an intuitive visual interface—browser windows that displayed hypertext, hyperlinks, and images.

The early web was rather primitive. But from its inception, tech-savvy thinkers could see its potential. It could connect everyone to a shared universe of information and communication. And what if, instead of simply connecting people through basic text, links, and images, it connected them through representations of shared digital space—a vast and manipulatable layer of virtual reality, accessible to anyone with a connection, on top of the physical universe we all already share.

Meta, on top of universe. You might call it…a metaverse.

Indeed, that is precisely what Neal Stephenson did in his prophetic, funny, zany, incredibly influential 1992 sci-fi novel Snow Crash, which described a sprawling digital otherworld in which people interacted through avatars—graphical stand-ins for their physical selves—of various quality, depending on how much people paid for their connections.

Snow Crash wasn't just a primer on the metaverse. Among other things, it was an elaborate satire of anarchist politics, in which a shriveled government gives way to a fully privatized society. The metaverse was an escape from that reality, a world with its own chaotic culture and jargon—a world that, before long, people in our own reality would start trying to build.

As science fiction explored the potential for VR, the underlying technology was gradually catching up. In the early 1980s, Lanier founded VPL Research, a company fully devoted to virtual reality.

Lanier was (and is) a showman and a polymathic oddball. The son of a science columnist for science fiction magazines, he grew up in a sci-fi-style dome house he designed himself as an adolescent and built with his father, and he was an obsessive player and collector of obscure instruments. In the early 1980s, he built a video game, Moondust, that played more like an art-school thesis. It featured an unusual, naturalistic control scheme and, in a video-game first, music that responded organically to the user controls and graphics on screen.

Lanier was a fixture in Silicon Valley's virtual reality counterculture scene, giving regular "VR talks" that, as he recalled in Dawn of the New Everything, dealt with "ideas about early childhood, cephalopod cognition, and how humanity would destroy itself unless art got more and more intense into the future." He imagined virtual reality as a tool for connection and creation, a kind of post-language space built from interactive symbols and infinitely malleable digital environments. Reshaping reality in VR would be like playing music. "Imagine that someday there will be user interfaces for creating fresh stuff in VR that works as well, and as quickly, as musical instruments do today," he said during one such talk reprinted in his book.

Inevitably, VPL built goofy-looking goggles. It also created gloves meant to let people manipulate objects in virtual space, and it designed software and sensory tracking devices. Its immersive demos wowed people: The October 1987 cover of Scientific American featured one of VPL's virtual reality gloves next to its graphical representation, along with the words "The next revolution in computers."

But VPL's products were cumbersome and expensive—an affordable VR set never brought to market was intended to be priced at $75,000. And in the early 1990s, Lanier left the company over internal disputes about its direction.

It wasn't until 2003 that someone actually put something like the complete idea of the metaverse into practice, with a virtual gathering place called Second Life. Even then, it was a strictly flat-screen affair, rather than the immersive VR simulator that Stephenson, Lanier, and others had imagined. Second Life featured a gamelike interface in which user-controlled avatars moved through sprawling virtual environments. Avatars could attend events, give talks, hang out with friends and strangers, and generally conduct themselves as citizens of a shared virtual country. There was commerce and camaraderie, education and entertainment.

It wasn't exactly Neal Stephenson's metaverse, but it employed some of the same language and concepts. And in his book, Lanier had kind words for its self-directed ethos, praising its emphasis on user buy-in and an "economy that valued individual worth." It wasn't utopian, but it was idealistic, and it stemmed from an essentially optimistic worldview that saw individual cooperation and coordination, self-direction and self-organization, as the key to human advancement.

Second Life was a sort of prototype, an imperfect mass demo of a bottom-up, people-powered, commerce-and-community-oriented virtual gathering space.

Like Lanier's VPL, Second Life received some fanfare in the press. At times the company boasted as many as a million users. Notably, the creators denied that they had created a game, since it contained no top-down objectives. It was a social space whose users would define its purposes. Instead of "players" it had "residents," because Second Life was an online society.

But it turned out that what many people actually wanted was games. And that's where the metaverse as we know it today really took shape.

From Fortnite to Ferrari

Nearly 20 years later, the actual metaverse—or something pretty close—has begun to emerge. Virtual reality headsets aren't as ubiquitous as iPhones, but you can get a decent one for a few hundred dollars and a great one for a little more than the price of a high-end gaming computer.

It has become clear, however, that the metaverse, especially in its current early stage, isn't strictly dependent on VR—not when we have smartphones, superfast mobile networks, online payment processors, cryptocurrency, and, perhaps most importantly, mass video game communities built around games like Fortnite and Pokemon Go. Thanks to such developments, the dream of a connected world, a shared digital reality, as fluid and beautiful and intuitive as an instrument, is finally taking shape, with or without goofy-looking goggles.

And so, of course, an awful lot of it is being used to sell virtual crap, watch virtual porn, and virtually shoot each other.

In 2021, Adult Video News reported the launch of JOI.CITY, the self-described "Metaverse of Erotica," a virtual cityscape filled with sexual entertainment. As virtual places go, it's fairly unoriginal; it looks like Pornhub rented out the Blade Runner set for a night. There's a neon-lit street scene with a couple of doors into online erotic clubs and some sexy advertisements plastered on the walls. And there are, of course, virtual erotic dancers, some apparently modeled on real-life porn stars, who will perform for a fee. In this part of the metaverse, even the strippers are holograms.

In 2022, a separate company, xxxNifty, announced that it would be launching "the first full Adult Metaverse." In a virtual environment dubbed the "Pink Tower," according to Adult Video News, players "can live and love by having sex with their own unique 3D character." In the coming metaverse, not only can you be a hologram, you can have virtual sex with one, as one.

Porn creators aren't the only ones looking to monetize the metaverse. Signal360 is an online publication that grew out of a corporate conference series sponsored by Procter & Gamble, a multinational consumer goods company that owns such brands as Downy, Febreze, and Tide. In March, the site published a video conversation about the metaverse between Raffaella Camera of Epic Games and NFT pioneer Eric Pulier of Vatom, which bills itself as an all-in-one metaverse platform for businesses. It was obvious what their vision of the new internet is: The metaverse is about advertising.

Camera's company is best known for its super popular online game Fortnite, a large-scale multiplayer shooter that has evolved into a virtual hangout for mostly younger players. Epic is one of the many companies attempting to build out the foundational tech of the metaverse, and Fortnite has hosted massive in-game events, from a virtual concert featuring rapper Travis Scott to the premiere of a trailer for the Christopher Nolan movie Tenet, that go beyond the scope of a traditional video game.

Unlike Second Life, Fortnite is very clearly a game, built on a top-down directed experience organized for users by corporate minders. The community, however, has evolved out of that shared, goal-driven experience.

Camera, however, mostly talked about the company's digital tools, a suite of software products and virtual object libraries designed to enable anyone to create a metaverse-type experience of their own. In theory, that could mean ordinary individuals pursuing their dreams. But in practice, it's brands—such as Balenciaga, the first high-fashion label to launch in Fortnite, and Ferrari, which has placed hyper-detailed digital mock-ups of its supercars inside Epic's games—that are taking advantage.

One of the core appeals of the metaverse, Pulier said, is the notion of a world without traditional advertising—"a world without annoyance." Instead, advertising would be immersive. It would be fun. People would want to participate.

All of this, it's worth reiterating, appeared on a website sponsored by a company that sells paper towels and eyeliner. The theory seemed to be that, in the metaverse, people would enjoy spending time experiencing an ad for mouthwash. (More than a week after the video went online, it had received just 97 views, including, presumably, the three times I watched it myself.)

It's hard not to suspect that Facebook—sorry, Meta—is after something similar.

In early 2022, the company let investors know that it would undershoot its expected revenue targets by about $10 billion. The problem was that Facebook is an app. It collects data about its users, then sells that data to brands that can use it to tailor and target their ads. Facebook has unique insights into its very large user base that make it extraordinarily valuable to advertisers. But apps run on hardware and are accessed through marketplaces such as Apple's app store. And in 2021, Apple started asking users if they want to turn off the in-app tracking that leads to more personalized ads. The majority of users turned off tracking.

Apple's control of the hardware and software ecosystem, it turned out, was the Sword of Damocles hanging over Zuckerberg's head.

Meta bills itself as a company devoted to connecting people. But its actual business model, the way it makes money, relies on tracking user behavior. Without tracking data, ad buys on its platform became much less valuable. So the company has turned to another technology that is, on the surface, about connection and communication but that is built in large part on advanced behavioral tracking systems. Virtual reality, remember, is a technology of tracking and measuring human perception.

If adopted by the masses, those systems would give Meta more user data than ever before. And this time, the company wants to control the hardware and software ecosystems so that a rival can never turn off the flow of data. Hence Meta's $2 billion acquisition of the VR headset maker Oculus VR and the deployment of a series of relatively affordable consumer-grade virtual reality headsets. And hence Zuckerberg's insistence that the metaverse, as he imagines it, will require not only norms but "new forms of governance."

For the last several years, Zuckerberg's company has faced intense scrutiny from lawmakers in Congress. Much of this scrutiny is misguided, predicated on misunderstandings of internet technology generally and of what Facebook does specifically. But Facebook's response has been increasingly focused on accommodation. In advertising and in hearings, the company's position has been that of course the internet needs new rules and regulations—and that Facebook is ready and willing to help lawmakers and bureaucrats write them.

And so, late in its metaverse presentation, when an employee explained that "the speed that new technologies emerged sometimes left policy makers and regulators playing catch-up," it was not hard to guess the endgame. By bringing lawmakers and regulators into the fold early, Meta is looking for a seat at the table, where it can push for rules that will benefit Meta.

Lanier, who now works at Microsoft, has been particularly critical of Meta's plans and of Facebook's business model, which he believes relies on stoking anger for engagement. "Listening to Mark Zuckerberg talk, it sounds like some megalomaniac took my stuff and filtered it through some weird self-aggrandizement filter. I mean, it's just the weirdest thing," he said on the New York Times podcast Sway in November 2021, shortly after Facebook's big metaverse presentation. "My thought of it always was that you would emerge and it would be like 100 million micro entrepreneurs doing their little thing here and there. And there wouldn't be some overlord."

VR pioneers envisioned a freewheeling world of musical holograms where people constructed their own realities without top-down direction. Meta wants the opposite: a mediated, regulated world where advertisers track your every move.

The Hologram Dream Lives

So is the metaverse a cynical ploy to co-opt the regulatory process and sell branded virtual tchotchkes? Well, yes. But it isn't only that. The old dream of a shared virtual space, liberated from the constraints of physical reality and built to inspire creativity, is very much alive.

Consider Roblox, a social gaming platform that has, for most of its existence since 2006, largely targeted children (although its user base has now started to age). Roblox isn't a game itself; it's an ecosystem for making games that you and others can play. The games can then be sold, and the company shares the revenue with creators.

Some of these games are fairly conventional action-adventure experiences, but many are just experiences, without traditional video game objectives. On Roblox, you can visit tropical islands, adopt virtual pets, manage a digitized pizza restaurant, and create secret or public virtual clubhouses to hang out with your friends. Like most of today's metaverse-type experiences, it doesn't require VR goggles—you can access it from a computer or a phone—but you can use a VR headset to add to the experience if you want.

And some Roblox users are moving beyond minigames and virtual worlds to other types of creation, like music. Roblox users have created their own sonic subgenre, dubbed Robloxcore, that features bouncy, digitally created sounds and computer-manipulated voices colliding against each other in what feels like a mashup of top-40 pop and old Nintendo game soundtracks. It's exactly what you'd imagine teen pop would sound like if it came from a world inside a computer.

It would be an exaggeration to say that Roblox represents a fully liberated, user-driven world of musical holograms. But it moves us closer to that ideal.

This essay has made light of holograms, because they are silly. But they're also important, because holograms represent something vital: the potential for mass personalization and individual control.

One of the key ideas of the metaverse is the preservation and customization of individual identity.

Some of this is aesthetic: A virtual environment, or a virtual body, can be modified and altered according to your whims and preferences. It can change based on context or mood. One day you might show up to work as a cartoon panda, the next as a giant walking redwood tree. You might appear as a man or as a woman or as something else entirely. You might add extra virtual limbs or a tail; among VR research's many enlightening discoveries is that humans have little trouble adapting to avatars with additional, non-human appendages.

But it's also about personal data: the architects of the metaverse want you to be able to transfer your digital identities, from your avatars to your credit card numbers to your grocery delivery service logins, across virtual realms.

The fundamental precept of the metaverse—and the essential idea of VR—is that people are different, they have different desires and preferences, sometimes even within themselves, and they should be allowed to exist in a world that respects that fact.

Sometimes such a world will produce never-before-imagined types of music and weird art happenings that attempt to bring about elevated collective consciousness. Sometimes it will serve up a virtual Ferrari or a cool jacket from a hot brand. And sometimes it will give us a seedy neon strip club with a holographic dancer.

Just as some people design their own geodesic dome houses and other people move into cookie-cutter subdivisions, some denizens of the metaverse will appear exclusively as unrecognizable posthuman entities and build vast, strange virtual worlds to inhabit, while others will choose avatars that look a lot like their real-life bodies and stick mostly to prefab spaces, designed by others for mass use.

Some metaverse experiences will be created from the bottom up by ad hoc, anarchistic virtual communities; others will be organized from the top down, but in ways that make the experience more immediately accessible, just as Facebook brought social media to the masses and smartphones brought virtual connectedness to our persons. These two visions of a hologram-built world will coexist, sometimes in tension with each other, sometimes in cooperation.

Indeed, these two visions already exist, in various forms, in the proto-metaverse that has already begun to arrive and in video games, social media, videoconferences, telework, and various meldings of those elements. Directed, gamelike experiences like Fortnite are already leading to self-organized communities, and nondirected toolkits like Roblox are already generating video games with the usual gamelike goals.

All of this is happening in something like a metaverse—really metaverses—with or without VR goggles. And all of this will continue to happen in the metaverses of the future, as they grow and change and become an even greater part of our lives, making the dream of a disrupted reality ever more real.

As this happens, there will be limits. So far, virtual reality is primarily a technology of sight and sound, with limited tactile feedback. You can't have a cocktail in the metaverse, or pet your dog. (Hence the lack of dog hair.) Like physical reality, virtual reality has its share of problems. But it also has holograms, and a liberatory potential. As ever, human beings will make of it what we can.