Most drones don’t kill. Instead, they like to watch. We usually think of the small, unmanned aerial vehicles in terms of grainy overhead shots of desert explosions, but less than 5 percent of the U.S. overseas drone arsenal consists of those lethal Predators and Reapers. The remainder are mostly Peeping Toms engaged in overhead reconnaissance and surveillance.

Drones particularly like to shoot video. Thousands upon thousands of hours of it, most of which will never be viewed by human eyes. The National Geospatial Intelligence Agency, which analyzes much of the military’s drone footage, has more than 416,000 hours’ worth of it in digital storage, with more added all the time.

The military is planning for a future that relies more on drones than it does on manned planes. The next generation of jet fighters may be the last one with human beings in the cockpit. The next model of surveillance aircraft is already being designed as “pilot optional.”

But the military’s insatiable appetite for robot planes is no match for the market in domestic drones that’s poised for takeoff. A drone revolution is coming, and in only a few short years you’ll be able to look up and see it with your own eyes. In fact, you won’t be able to miss it. And it won’t be able to miss you.

Drones will take flight in a host of commercial industries, from agriculture to logistics. They’ll be deployed by SWAT teams, border patrol agents, and traffic cops. If you can imagine a task being performed right now with a set of human eyes, there’s probably a drone sitting on the runway waiting to do the job. Drones work without pay, don’t eat or sleep, and in the not-too-distant future they may be able to use solar power to stay aloft for hours. And they’ll do most of this work—taking off, gathering intelligence, transmitting signals, landing—entirely on their own.

This technological progress will come at a price. If you thought the debate over drones in combat was intense, wait until their flying eyes are on you around the clock. Profound moral dilemmas about privacy, profit, and autonomous machines await us. You won’t be able to escape the drones’ gaze. But maybe you won’t want to.

Domestic Drone Boom

There are about 6,700 unmanned aerial vehicles in the entire military fleet. If you take out the small hand-held Raven drones flown by the Army for surveillance and reconnaissance on the battlefield, that leaves about 1,300 robot planes. Only a fraction are armed for combat; most take pictures and a few haul gear.

The Federal Aviation Administration (FAA) estimates that 20 years from now, there will be 30,000 drones airborne over the United States. Some of them will be as big as a Predator, about the size of a single-engine Cessna. But there will also be drones the size of hummingbirds, and probably even as small as moths.

If all goes according to current plan, the drone revolution will go fully airborne on September 30, 2015. By that day, the FAA is required by law to integrate unmanned aerial vehicles into national airspace. Until then, flying drones domestically for commercial reasons is totally off-limits.

The first for-profit applications will be in agriculture. Drones will take over crop dusting. (In Japan they already have.) With long-range cameras and precision sensors, including night vision and thermal imaging, they’ll monitor fields for waste-product runoff into waterways and track air pollution from industrial facilities. They’ll herd livestock, like aerial sheepdogs. Big agribusinesses will use drones to detect whether farmers are using their seeds without permission.

The Association for Unmanned Vehicle Systems International, the de facto drone trade group, estimates that “precision agriculture and public safety” will account for about 90 percent of the new domestic drone market in its first few years of airspace. The public safety category includes federal, state, and local law enforcement agencies, currently the main drone customers. To date, their flights have been beset by controversy, and there is a movement building to ground domestic police drones.

The private market hasn’t provoked such controversy, but that picture may look very different in 2020. Starting then, analysts predict, transportation companies will be using drones to deliver cargo—think UPS and FedEx planes without pilots. Traffic helicopters will be replaced by swarms of small hover-drones, probably weighing just a few pounds, that may be powered by solar energy to stay aloft for hours. Media organizations will use drones to cover breaking news. (Get ready for lots and lots of footage of high-speed chases.) These early adopters will be joined by the novel, mom-and-pop drones that have attracted much of the media attention to date, such as the famed prototypes for a TacoCopter and a Burrito Bomber.

Eventually, commercial aviation will consider going pilotless. If drones are good enough for UPS, why not U.S. Airways? Passenger trepidation may hold the airlines back, but the technology won’t. Many of the commercial planes you fly aboard today can already take off and land themselves. And of course they’re all equipped with an autopilot. Turning passenger jets into drones is no great feat of engineering, and it would save a fortune in pilot and air crew salaries.

The drone association estimates that the first three years of integration into national airspace will have an economic impact of more than $13.6 billion, made up of sales, good-paying manufacturing jobs, and a network of suppliers that will spring up to keep the aircraft fitted with ever-more sophisticated avionics, sensors, and assorted technical wonders. More than 70,000 jobs will be created over that time, the trade group predicts, increasing to 104,000 by 2025.

Too Many People

One group is certainly not going to profit from the drone boom: pilots. There will be lots of people building drones and maintaining them on the ground. But flying them? Not so much. Indeed, there are already too many. 

Imagine the prototypical lethal drone strike. You probably envision a young pilot sitting in a mocked-up cockpit, hand on a control stick, seeing what the drone sees on a video screen. But he’s just one of a few hundred people it takes to fly that single mission.

In a typical lethal drone strike, more than 100 analysts are monitoring communications, signals, and imagery. When you include the technicians making sure the drone is in contact with a communications satellite, the maintenance crew on the ground, and the lawyers (yes, lawyers monitor every lethal strike), the typical killer flight involves anywhere from 150 to 200 people. For a surveillance-only flight by the Global Hawk, which doesn’t carry any weapons, there can be up to 300 people involved. It takes a huge number of people to fly a drone. 

But when drones become domestic big time, there will be hordes of drones in the sky. With 30,000 of them buzzing around—the same number of commercial flights per day in the United States—how will they keep from running into each other? How could any fledgling drone company afford even a few dozen people to man every robot flight?

To keep the drones aloft, human beings will have to hand over most of the control and navigation tasks to computers and, ultimately, to the drones themselves. Tomorrow’s private drone operators won’t be yanking joysticks or pushing rudder pedals. They’ll be sitting at a computer, or holding a tablet or a mobile phone, from where they’ll select a mission profile from a pre-set menu, then hit a button and let the drone go to work.

Actually, several drones will go at once to work together as a team. They’ll take off, fly to their job sites, communicate with each other, perform their tasks, and then fly home and land, all while figuring out on their own the most efficient routes. The robots’ human assistant won’t be a pilot in any traditional sense. In fact, it won’t matter if he’s ever flown in an airplane.

The technology to manage multiple drones simultaneously exists today. A California-based software maker called DreamHammer has been licensing a system called Ballista to the U.S. government since 2011. The interface is startlingly simple: You point and click on a three-dimensional map, and the drones, which are represented by little airplane icons, do the rest. They know where to go, the fastest way to get there, and, crucially, how to avoid running into each other. “Today, drones require many people to operate one,” says DreamHammer CEO Nelson Paez. “Ballista allows them to focus on their mission and not on the complexity of operating vehicles.”

Drones have come a long way from their early ancestors. The first truly remote-controlled vehicle was built in 1898 by the famed inventor Nikola Tesla, who tried to sell a radio-controlled motorboat to the U.S. military. Washington laughed him off. As the historian P.W. Singer writes in his 2009 book Wired for War, Tesla learned an important lesson about humanity’s willingness to accept these new, automated devices: “what was technically possible mattered less than whether it was bureaucratically imaginable.”

It wasn’t until the mid-1990s, with the emergence of the Predator, that the military finally embraced the concept of unmanned aerial vehicles. The Predator began as a surveillance drone, then was armed to hunt and kill terrorists shortly after the 9/11 attacks.

Unsettling as it may sound, the truth is that operating drones is often best left to computers. And as more drones fill the sky, it will be essential to automate their flights. Human pilots are too prone to error, too easily distracted, and too slow to react to unexpected interruptions and dangers. Thanks to a biological response known as the neuromuscular lag, it takes us about half a second to recognize a threat—say, a potential mid-air collision with a flock of birds—and another half-second to take action to avoid it. An autonomous drone can do this faster and more reliably.

That was the conclusion reached last year by the Defense Science Board (DSB), a committee of civilian advisers to the Pentagon. In a report, the DSB urged the military to “more aggressively use autonomy” in its drone missions, in order to reduce the large number of people it takes to fly the drones and to unleash the full potential of the technology. But the board also recognized that the idea of quasi-conscious robots can scare the hell out of people.

“Unfortunately, the word ‘autonomy’ often conjures images in the press and the minds of some military leaders of computers making independent decisions and taking uncontrolled action,” the board wrote. “While the reality of what autonomy is and can do is quite different from those conjured images, these concerns are—in some cases—limiting its adoption.”

The military’s top drone operators echo that sentiment. “We have the potential to achieve greater and greater degrees of autonomy,” says retired general Dave Deptula, who was the Air Force’s first deputy chief of staff for intelligence, surveillance, and reconnaissance, and who now advises companies that develop drone technology. “But that brings with it huge policy issues. We’re not ready today, and may never be, to hit a button and say, OK, come on back after you’ve delivered your bombs and tell us what you hit.”

But drones need to be autonomous, to some degree, if they’re going to efficiently perform the myriad tasks engineers have planned for them. The Defense Science Board advised the Pentagon to mount a public relations effort, and to choose words designed to reassure anxious humans. “It should be made clear that all autonomous systems are supervised by human operators,” the board wrote, “at some level.” 

Law Enforcement Takes to the Skies

Most of the tasks for which drones have already been used in this country demonstrate the other main application of flying robots: law enforcement and regulatory surveillance.

The Customs and Border Protection agency has a small fleet of drones monitoring illegal immigration and drug trafficking on the border with Mexico. (The comprehensive immigration bill passed by the Senate in June calls for drones “to maintain continuous surveillance of the Southern border.”) The Environmental Protection Agency uses drones to monitor animal feedlots, watching to see if pig and cow manure ends up in rivers. The Department of Homeland Security has flown drones as lookouts on raids and during chases by the FBI, the Secret Service, and the Texas Rangers.

State and local governments are also taking to the skies. In 2011, police in Nelson County, North Dakota, made the first arrest with the assistance of a drone when an unarmed Predator equipped for high-altitude surveillance led officers to three fugitives. Law enforcement and public safety agencies across the country have expressed interest in surveillance drones for aiding SWAT raids, assisting search and rescue operations, finding missing persons, following fleeing suspects, and fighting wildfires.

The FAA has authorized drone flights by police in Miami-Dade County; North Little Rock, Arkansas; Ogden, Utah; Seattle; and the State of Texas. The City of Herrington in Kansas and the Mississippi Department of Marine Resources have also gotten the FAA’s OK, as have Eastern Gateway Community College in Steubenville, Ohio; the University of Connecticut; and Utah State University, which are all using drones for research.

According to the FAA, more than 300 entities have received official clearance for domestic drone flights ahead of the 2015 opening of the national airspace. Some, including private operators, are already using the aircraft overseas. A team from the University of Maryland, for instance, has deployed a fleet of small reconnaissance drones to monitor rhino poaching in South Africa. 

The terrorist bombings of the Boston Marathon in April 2013 triggered renewed calls for deploying unmanned aircraft. “It sure would be nice to have a drone up there” to track the then-fugitive bomber Dzhokhar Tsarnaev, Sen. Lindsey Graham (R-S.C.) told The Washington Post. But so far, surveillance drones within the United States have only led armed officers to a suspect. To date, there have been no reported lethal drone strikes in America.

That hasn’t stopped a preventative backlash. In early February, just a few weeks before Sen. Rand Paul (R-Ky.) made national headlines with a 13-hour filibuster centered around the possible use of drone strikes against U.S. citizens, the college town of Charlottesville, Virginia, became the first city in the nation to pass a resolution restricting drones. The two-year moratorium on using the aircraft for surveillance of citizens was based on a resolution drafted by the Rutherford Institute, a civil liberties advocacy group.

The Charlottesville moratorium sparked a nationwide movement. A few days after it passed, Seattle Mayor Mike McGinn banned the city’s police department from flying a pair of 3.5-pound helicopter drones, which were returned to the manufacturer. Cities in California and Illinois followed suit with their own anti-drone proposals. Bills to restrict drone flights soon were proposed in more than a dozen state legislatures. To date, 43 states have introduced more than 90 bills and resolutions proposing to regulate official use of drones.

The measures are mostly symbolic, since state and local governments cannot regulate U.S. airspace. And many of them stop short of prohibiting drones for search and rescue operations or looking for missing children. Other proposals are filled with exemptions, including a bill in Texas that would allow the media to fly drones to cover breaking news events.

But the efforts to control the drones reveal a broad anxiety about a technology that most people still associate with death and destruction. We’ve had very little time to get to know these aircraft, yet they’re on the verge of ubiquity.

Privacy in the Drone Era

Once the skies open up, police will start putting up drone blimps and other high-altitude surveillance platforms to watch over entire cities. “You can’t keep the tides from coming in. We’re going to have more visibility and less privacy,” New York Mayor Michael Bloomberg said in a radio interview earlier this year. “I don’t see how you stop that. And it’s not a question of whether I think it’s good or bad. I just don’t see how you could stop that because we’re going to have [drones].”

There’s no reason to think these flying security cameras will have any more deterrent effect on crime than cameras on the sides of buildings do. More likely, they’ll be used as part of everyday policing and what the military calls “situational awareness.” These law enforcement drones, like their battlefield cousins in Iraq and Afghanistan, will be watching. And conceivably watching everyone, though a mere fraction of the millions of minutes of video they collect will ever be viewed by human eyes.

Drones present novel legal quandaries. In a 2012 case that tested the limits of law enforcement surveillance, the Supreme Court ruled unanimously that police needed to obtain a warrant before placing a GPS tracker on a suspect’s car, which is considered property and is thus protected by the Fourth Amendment. But do the police also need a warrant to follow a suspect with a drone as he walks down a public street? “It may be that achieving the same result [tracking] through electronic means, without an accompanying trespass, is an unconstitutional invasion of privacy,” Justice Antonin Scalia wrote for the majority, “but the present case does not require us to answer that question.” The Court kept its ruling narrowly focused on one kind of technology, sidestepping thorny questions that might address the legal limits of drone activities.

It’s also not clear whether a human being has to be doing the watching in order for overhead surveillance to constitute a search. Automated drones over Iraq and Afghanistan use software to keep track of the comings and goings of individuals on the ground. They can be programmed to alert a human operator only when the targets deviate from a pattern of normal behavior, or when someone new shows up on the scene. Police will probably use the same kinds of software to have drones keep an eye on high-crime zones and send an alert when they spot trouble. That looks a lot like everyday policing. And the cops don’t need a warrant to patrol the streets.

Rand Paul’s March filibuster raised the hypothetical scenario of a president ordering a drone strike on an American terrorist suspect sitting in a cafe in the United States. Attorney General Eric Holder, in a glib response, assured the senator that the president does not have “the authority to use a weaponized drone to kill an American not engaged in combat on U.S. soil.” Nothing in Holder’s statement prohibits a police department from using an armed drone to fire upon an armed suspect. It’s possible to imagine the use of a small, lethal drone to kill a hostage-taker or end an armed standoff, acts not fundamentally different from using human snipers. What’s far more likely, though, is that law enforcement will keep increasing their use of drones with non-lethal force. A SWAT team responding to a hostage standoff will arm a drone with tear gas, for example. Such a “strike” would be more legally defensible.

Most research on drone engineering today has focused on making the robots smaller, so that a camera-equipped drone the size of a spider could crawl through an open window or under a door and beam video to the armed police outside. The real money is in smaller drones that last longer on tinier batteries, not aerial brutes with bigger guns and bombs. 

Drones Abroad

Today, American drones watch suspected terrorists congregating in overseas camps. They watch over U.S. forces, alerting them to the presence of enemy forces nearby. Armed with extraordinarily powerful and sensitive cameras, the drones can observe anything and everything at once in their field of vision, which can encompass dozens of square miles, and, if the camera is strong enough, an entire city.

The drones are also workhorses. The military has begun to transport cargo in Afghanistan using unmanned helicopters. Unlike their human counterparts, the drones don’t take breaks. They don’t need food. And they don’t sleep. In the past 20 years, the Army and Air Force’s unmanned aerial fleets have logged more than 1 million hours of flying time—each. The Air Force’s drones racked up more than 500,000 hours in 2010 alone. Today, at any given time, the Air Force is operating at least 57 unmanned combat air patrols around the world. 

But the U.S. is hardly alone in its appetite for drones. According to the Government Accountability Office (GAO), the number of countries that have acquired unmanned aerial vehicles since 2005 has nearly doubled, from 40 to 76. Most of these countries are using the aircraft for surveillance exclusively. Only the United States, the United Kingdom, and Israel are known to have used armed drones in combat.

Building a drone isn’t absurdly difficult, but arming it with a missile is much harder than outfitting it with a camera. The guidance systems and associated technology required to weaponize a drone are complex and available mostly to countries with sophisticated militaries. (The United States only figured out how to successfully arm a Predator with a Hellfire missile in 2001.)

Although the GAO warned that “the United States likely faces increasing risks as countries of concern and terrorist organizations seek to acquire [unmanned aerial vehicle] technology,” we don’t yet need to fear Iran or North Korea, or even China, sending drones over American soil. For starters, they’d have to base the drones close enough to enter U.S. airspace, where they’d be swiftly shot down. Also, those countries lack the constellation of global communications satellites that let American pilots bounce signals from an air-conditioned trailer in Nevada to a drone over North Waziristan. There’s some risk that a terrorist could equip a drone with chemicals or a lethal weapon, but there are plenty of easier, lower-tech ways to inflict damage.

Of all the countries intent on acquiring drones, China concerns U.S. officials the most. The Defense Science Board finds it “worrisome” that “China has ramped up research in recent years faster than any other country.” Five years ago, at China’s biggest air show, visitors saw a crude computer animation of a Chinese drone, looking suspiciously like a Predator, firing a missile at what looked suspiciously like a U.S. carrier group floating off Taiwan. Even if China can’t send drones over San Francisco, or even Hawaii, it could use them to monitor or attack neighboring countries such as Japan and to harass or kill its own citizens.

Analysts believe that China is in the advanced stages of developing a so-called medium altitude, long endurance (MALE) drone, a class to which the Predator belongs. A MALE drone would allow for 24-hour surveillance orbits in the country. The total number of drones in China today is not known, but it is presumed low.

The barriers to entry into the armed-drone club won’t remain high forever. Many military strategists take it as a given that drones and other robotic forces will be a common feature of future wars between nations. In theory, using a drone to kill your enemy shouldn’t present a challenge to the law of armed conflict any more than the lethal use of a manned aircraft does: A country has the legal right to use force in war zones, and even to do so preemptively in self-defense. Under international law, the operations must target legitimate combatants, and the attacking country must take precautions to minimize collateral damage to civilians and ensure that the use of force is proportional to the threat. The kind of weapon used—so long as it’s not prohibited by laws or treaties—is incidental.

Where it is used, however, is not. Opponents of the Obama administration’s targeted killing program, which has sent drones to kill U.S. citizens abroad, object that many lethal strikes have occurred in places where the United States is not at war, namely Pakistan, Yemen, and the Horn of Africa. How can a country be in compliance with the laws of war if it’s using force in a country where there is no recognized armed conflict, or if the host government hasn’t given official permission to intervene?

These basic questions, which could easily be applied to cruise missiles or teams of commandos, are not resolved under international law. Considering that the United States is the only country with a long and observable track record of using drones to kill people, it will probably set the standards that other nations follow—whether those actions are technically legal or not.

In May, President Barack Obama defended the use of drone strikes outside recognized combat zones under the September 2011 Authorization to Use Military Force, which gives the commander in chief broad latitude to attack the Taliban, al Qaeda, and associated forces wherever they congregate. “Beyond the Afghan theater, we only target al Qaeda and its associated forces,” Obama said. “Even then, the use of drones is heavily constrained.”

The president said the strikes are only carried out after “consultation with partners” in other countries and with “respect for state sovereignty,” targeting only “terrorists who pose a continuing and imminent threat to the American people” and who cannot be captured, and only when there’s a “near-certainty that no civilians will be killed or injured.”

In effect, Obama argued that the drones comport with all the requirements of the international law of armed conflict. But he and his administration know that this isn’t a universally shared assessment. 

“The exponential rise in the use of drone technology in a variety of military and non-military contexts represents a real challenge to the framework of established international law,” said Ben Emmerson, the United Nations’ Special Rapporteur on Counterterrorism and Human Rights, when announcing an inquiry earlier this year “into the civilian impact, and human rights implications of the use [of] drones and other forms of targeted killing for the purpose of counter-terrorism and counter-insurgency.”

The muddy nature of these armed conflicts—Who is legally a terrorist? Who is an insurgent?—has complicated the development of rules for using drones. “The world is facing a new technological development which is not easily accommodated within the existing legal frameworks, and none of the analyses that have been floated is entirely satisfactory or comprehensive,” Emmerson said.

Washington does not get much sympathy with its claim of requiring special latitude to fight a uniquely stateless enemy. “This analysis is heavily disputed by most States, and by the majority of international lawyers outside the United States of America,” Emmerson said. Yet “the plain fact is that this technology is here to stay, and its use in theatres of conflict is a reality with which the world must contend.”

The Future of Drones

The way that the military has struggled with questions about killer robots exposes thorny moral questions about launching drones that can act on their own. Resolving these questions will have vast implications for our future robot cropdusters, cargo-carriers, and city watchers. 

Ask any military officer trying to make drone policy today, and he or she will insist that a robot will never, under any circumstances, be given the authority to decide who to kill. A human being will always be touching the loop, even if he’s not entirely in it. 

Yet the Defense Department’s own research into autonomous systems suggests otherwise. For example, under the Persistent Close Air Support Program, engineers are looking for ways to speed up the process of sending tactical air support to assist ground forces. It takes about a half-hour now to call in an air strike, and the goal is to whittle that down to six minutes. To do that, drones will have to be programmed to respond with some degree of independence to threats on the ground. There isn’t time to wait for a human to direct fire.

And even if a breathing person is required to make the final call to fire a missile, it’s hard to imagine him overruling the robot. After all, it will be the drone that “sees” the threat on the ground, processes the images and sound, and coordinates the action, all faster than the human being can. If the relevant officer is effectively just following the drone’s lead, isn’t the robot really the one in charge?

That’s the question we’ll all be asking when drones finally take off in America.