Technology

The Decentralized Web Is Coming

Amazon, Google, Facebook, and Twitter are in the federal government's crosshairs, but the technology necessary to undermine their dominance may already exist.

|

HD Download

Google handles 88 percent of search traffic in the United States. Facebook has more than 2.4 billion active monthly users worldwide. Half of all U.S. online retail is projected to go through Amazon by 2021.

Both Democrats and Republicans have called for breaking up the tech giants, holding them legally liable for what others say on their platforms, and imposing new regulations that would stop them from misusing their customers' personal information. But there's also a growing movement, which includes some of the web's early pioneers, to come up with technological ways to counter Facebook, Amazon, Twitter, and Google.

The goal is to build a better, more decentralized web.

"There are so many different possible ways of decentralizing the internet, and what's lacking is the legal right to interoperate and the legal support to stop dirty tricks from preventing you from exercising that legal right," says Cory Doctorow, a science fiction author and tech journalist who's been thinking and writing about the web since Tim Berners-Lee introduced it to the public in the early 1990s.

Berners-Lee and other web pioneers intended for their creation to be decentralized and open-source. "The cyber-utopian view was not merely that seizing the means of information would make you free, but that failing to do so would put you in perpetual chains," says Doctorow.

There are many theories about why the web became centralized. Doctorow largely blames the abuse of intellectual property law to defeat the decentralized "free software" movement championed by the programmer and activist Richard Stallman. Stallman helped create the popular open-source operating system Linux after freely modifying Unix, Bell Labs' proprietary system.

But the Digital Millennium Copyright Act, passed in 1998, became an impediment to the open and permissionless approach to software development. The law was intended to prevent duplication of copryrighted works and was eventually applied to all software. Breaking "digital locks" to learn from, interact with, and improve upon the code of dominant web platforms became a federal crime. It's standard practice for today's tech companies to shield their proprietary code from would-be competitors by wielding the power of an increasingly expansive intellectual property regime.

"And so this thicket of exclusive rights around products that can be invoked to prevent new entrants for making add-ons, compatible products, or even competing products is a really important change in the landscape," says Doctorow. "One that has made it very hard for new entrants to emerge and I think is in large part responsible for the concentration in the industry."

Despite these legal and political challenges, innovators are attempting to create new decentralized ecosystems of web services.

Mitra Ardron is the head of decentralization at the Internet Archive, a digital repository of more than 50 petabytes of images, movies, and texts—including more than 330 billion webpages.

"The archive's mission is to make all of mankind's knowledge available online forever to everyone for free, which is a pretty big vision, right?" says Ardron.

He says the history of the web is too important to be held in custody by a single organization. So he's overseeing a plan to migrate the Internet Archive's more than 50 million gigabytes of data to a distributed network maintained by users.

A beta version of this peer-to-peer network is already operating and publicly accessible.

"I think what [a more decentralized web] would look like is a world where servers were everywhere, that your internet router at home would also be a server," says Ardron.

Doctorow doesn't think the decentralized web can take off without government intervention. He agrees with Sen. Elizabeth Warren (D-Mass.) and Josh Hawley (R–Mo.) that the Federal Trade Commission should break up the tech giants.

"[The tech giants] got giant doing what we used to ban and that we stopped banning right when the tech industry started," says Doctorow, who argues that Ronald Reagan and the federal courts undermined the original intent of the Sherman Antitrust Act during the 1980s using the legal theories of former federal judge Robert Bork:

Every president since Reagan has expanded Bork's doctrine, allowing for even more aggressive market concentration, producing a country (and a world) where a handful of firms dominate virtually every industry, from telcoms to talent agencieswrestling to eyewear, to Big Tech

But in the October 2019 issue of Reason magazine, economist Thomas Hazlett argues that sweeping antitrust action has often entrenched existing players, largely due to the phenomenon of regulatory capture:

The late Nobel laureate George Stigler started as a "bust 'em up" guy: In 1952 he wrote an article in Fortune stating the "case against Big Business" and calling for the dissolution of General Motors. But through observation and analysis, Stigler's view progressed until he arrived at an antitrust policy that gave dynamic forces their due and put consumer interests at the center. He came to see government institutions as imperfect, and he posited in a 1971 paper the theory of "regulatory capture," whereby "regulation is acquired by the industry and is designed and operated primarily for its benefit."

Arguments about antitrust aside, the technological tools needed to bring about a more decentralized web may already exist, though they're not yet widely implemented.

"Web 3.0 has this wonderful set of trust baked into the Internet itself," says Molly Mackinlay, a former Google programmer and a current project lead of the InterPlanetary File System (IPFS), a communications protocol that's meant to replace the hypertext transfer protocol (HTTP) that most of us use to access the web now. While HTTP connects your computer to a particular server, IPFS scours the network for a piece of content, which gets assigned a unique ID marker and connects you to whomever happens to be hosting it.

Mackinlay wants a decentralized web that relies less on centralized servers and more on distributed storage networks—such as Filecoin, a cryptographic token that rewards users for storing data. This, she says, would be an effective way to sidestep the dangers of censorship and overregulation.

"That's a better, safer, more resilient world, which doesn't end up…susceptible to authoritarian manipulation and control," says Mackinlay.

Produced by Zach Weissmueller. Camera by Alexis Garcia, John Osterhoudt, and Weissmueller. Opening graphic by Lex Villena. Additional graphics by Meredith Bragg.

Photo credits: Preston Ehrler/ZUMA Press/Newscom, Stefani Reynolds/CNP/Polaris/Newscom, ITU Pictures (under CC Attribution 2.0 License), Jeremy Hogan/Polaris/Newscom. 

Subscribe to our YouTube channel.

Like us on Facebook.

Follow us on Twitter.

Subscribe to our podcast at iTunes.

NEXT: Prosecution for Rifle-UC-Rifle Gun Control Sticker

HD Download

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. This sounds like it could be a haven for illegal activity, content, and a great way to manipulate information. Sounds like fun.

    1. You’ve got things backwards here. The internet isn’t intrinsically supposed to prevent illegal activity any more than English is supposed to prevent illegal activity. Just because English facilitates criminals by allowing them to invent new words to describe their illegal activities is no reason to rigorously formalize/centralize the language.

      Reciprocally, just because you use the word ‘theft’ doesn’t make you a criminal or an accessory. The word ‘theft’ isn’t a crime. Standing on a publicly-funded street corner, next to a murderer doesn’t make you a murderer and hosting 1s and 0s that, if decrypted a certain way, demonstrates that child sex trafficking is happening, doesn’t make you an accessory or facilitating of child sex trafficking. Especially since, if you decrypt it just about any other way, it could mean just about anything else. The information itself is meaningless and it’s the crime and criminals that give it context, meaning, and (negative) value.

      1. My point is that it doesn’t change anything, but now no one is there to moderate my potentially shitty behavior. Not that I’m advocating for that, but let’s get realistic. It’s not like Facebook empowered dog fighters. Hosting a website with the dog fights would have been tough because hosting would be complicated to secure. Decentralized means if you got viewers and betters you have hosting. It opens up all kinds of new doors along with legislation that closes them for good people.

        1. My point is that it doesn’t change anything, but now no one is there to moderate my potentially shitty behavior.

          Either it doesn’t change anything and nobody is currently there to moderate your shitty behavior or it does change something and nobody will be able to going forward.

          You’re starting to sound like the disingenuous shitbags who insist that section 230, the 1A of the internet, doesn’t change anything about free speech but cannot be repealed at any costs or else free speech is lost.

          1. I am in favor of totally scrapping 230 and opening every web platform to liability for what they allow to be posted.
            Until then, I might as well have my livestream dog website that people use in an unfortunate manner to stream dog fights for betting purposes. 230, yo!

        2. “My point is that it doesn’t change anything, but now no one is there to moderate my potentially shitty behavior.”

          Why should anyone other than yourself be responsible for moderating your potentially shitty behavior?

  2. Just as it’s easier to go to war with a nation state than with guerillas, it’s easier for the government to regulate an internet controlled by a few tech giants. I’d be surprised if they ever actually went through with breaking up Facebook. The threat is the control.

    1. While I don’t like the tech giants I do give them credit for deleting or stopping the posting of illegal content. What’s going to happen on a decentralized internet? When you become the server then you’re hosting the illegal content. If you have the power to delete or prevent yourself from hosting what’s illegal then you also have the power to censor.

      1. If the technology is done correctly, you won’t know what data you’re hosting, because it will be both encrypted and fragmented to such a way where it’s merely 1s and 0s to the host. No single individual would have any knowledge as to what part of the whole the fragments of data they host contain.

        1. So you could be hosting all kinds of illegal things that could put you behind bars for the remainder of your life. We would first need revised laws saying users are not responsible for what they host.
          That will happen when, well, no. It won’t happen.
          Try telling the judge you were only hosting part of underage pictures.

          1. I can’t speak to that because again, if the technology is done correctly, no official, nor individual user would even KNOW what data they were hosting. And it would create some very tricky legal scenarios to prosecute.

            The following word fragment is part of a classified document I’m not legally allow to share here.

            “th”

            Have I committed a crime?

            1. Would you want to chance it?
              If I handed you a bag and asked you to carry it on a plane, would you?
              If what you’re hosting is a crime and you play a part in distributing it then you’re looking at an expensive legal battle. Any user who participates in hosting data that they aren’t knowledgeable of with current laws is making a poor decision.

              1. Would you want to chance it?
                If I handed you a bag and asked you to carry it on a plane, would you?M

                No, because that’s not analogous to what a properly done decentralized storage technology proposes.

                However, to Steel Man your argument, I do understand your concern and it shouldn’t be discounted out of hand. Even if you’ve got an unrecognizable part of a recognizable whole, that still might be concerning to people. And of course, some whole parts of data may end up being small enough where one person’s storage medium might unwittingly host the recognizable whole.

                Google Drive, Microsoft One Drive could conceivably be hosting recognizable whole parts of illegal content now, and they haven’t shut their servers down. One could be, at this moment, fully encrypting child pornography and hosting it on a google or MS Onedrive account and neither of those companies would know.

                Any decentralized storage technology would have to natively encrypt any stored data where only the owner or intended recipients can decrypt said data. Otherwise the technology is useless because of the very fears you’re pointing out.

            2. Have I committed a crime?

              technically, yes. “no part” of any classified document may be released. For the purposes as you used it, no.

      2. Then do not ban “illegal” information.

        Phone companies thrive in spite of plenty of illegal things being discussed and planned on their network.

        1. I can’t use my phone to call potentially millions of people to sell your social security number. If you can’t remove it from the decentralized internet then you’re going to have to lock your credit forever. What if the Equifax hack was posted for lulz?
          Decentralized internet is a cool idea, but humans can be horrific beings at times. The potential to inflict massive damage with no ability for damage control is there. Along with free audio books.

          1. The real SSN problem is that the whole concept of SSNs is completely insecure. You won’t fix that by fighting a decentralized internet.

            1. I agree about the SSN and that was just an example. My point is that if you can’t delete anything then think of the worst thing you can imagine. It will get posted over and over again. Laws WILL be made in regards to decentralized internet. It’s not going to be the wild west fantasy people think it is. When have we ever been given that much freedom?
              Laws will make it so the old internet is more beneficial. If you are capable of using it to send a picture of your penis then expect tons of government regulation.

          2. “I can’t use my phone to call potentially millions of people to sell your social security number.”

            It’s not as easy, but it is doable. People were committing identity theft before the internet was all that mainstream or people used smartphones all that often.

            “If you can’t remove it from the decentralized internet then you’re going to have to lock your credit forever. What if the Equifax hack was posted for lulz?”

            You assume it was not. You can always sue for damages. The “dark web” exists largely because our government WANTS it to exist.

      3. While I don’t like the tech giants I do give them credit for deleting or stopping the posting of illegal content. What’s going to happen on a decentralized internet?

        You prosecute the people responsible for illegal behavior.

        When you become the server then you’re hosting the illegal content. If you have the power to delete or prevent yourself from hosting what’s illegal then you also have the power to censor.

        Sure, and that’s a good thing. Most people don’t want to host child porn or terror plans, so they won’t. But that same mechanism–the rightful hatred of the crowds of odious things–will not also allow dissenters to be suppressed, since there is no central control. Sure, you could “censor” by refusing to host, and someone else could just as easily host instead.

        1. The problem is that if you know what you’re hosting you can censor. If you don’t know what you’re hosting then you could be hosting bad stuff.
          Also, what can I get you to accidentally host?

          1. You can be alerted to hosting bad content. Or you could refuse to host content without seeing it.

            1. If I have to review all content I host then I’m not going to host a damn thing along with most people. Decentralized internet will be as slow as dial up. It will have takers and no givers. If you force a ratio the non-centralized internet wins and you see ads for car loans.

              1. Decentralized internet will be as slow as dial up. It will have takers and no givers.

                In practice, you may be correct about this, hence the “incentive” angle some companies are putting in.

                I agree with your concerns and they shouldn’t be dismissed lightly because we saw what happened with Napster.

          2. “The problem is that if you know what you’re hosting you can censor. If you don’t know what you’re hosting then you could be hosting bad stuff.”

            Which falls under platform protections. Speaking as somebody who works for a large cell company…we do not know what you discuss on a call nor do we honestly care. If you plan a murder over the phone, nobody here would be liable for a tiny shred of it.

      4. Who cares about illegal content?
        Take kiddie porn. Go back and look at who gets busted. Is it the guy who is kidnapping 8-year-old and dealing them off to porn makers? Is it the porn-maker who engages in sex with underage kids and records it? Is the guy who sells this shit ever caught and punished?

        No. It’s usually some schlepp who has gone on line to a “chat room” and found a Federal Officer trying to deal kiddie porn.

        The guy they bust didn’t grab the kid. He didn’t video the kid having sex. He didn’t profit from the production and the rape. But he’s the low-hanging fruit – the dumbest guy in the chain.

        When was the last time you read about one of the kiddie-porn producers getting arrested?

        1. “When was the last time you read about one of the kiddie-porn producers getting arrested?”

          Last week when there was news about teens being charged with child pornography for “sexting” nude selfies.

  3. Hoo boy there is a lot to unpack here. There are concepts Doctorow is wrong about, even if his heart is in the right place. The technology to create a decentralized web is… as old as the internet. The “problem” as we’ll agree to identify it, is it’s a matter of crowds and how they tend to gravitate towards stable, reliable services and sources which have recognizable brands and institutional backing (even if those reliable services and sources admittedly don’t behaving in the best interest of their users or customers).

    Additionally, the 1998 law doesn’t stop any person or group from creating fully open, copyable or moddable software. Remember The Cathedral and the Bazaar? Yeah, few others do either. Not because it didn’t make good points, it’s just that it kind of missed the point.

    And while my history is rusty, Stallman to my knowledge had little to do with Linux development.

    1. Mackinlay wants a decentralized web that relies less on centralized servers and more on distributed storage networks—such as Filecoin, a cryptographic token that rewards users for storing data. This, she says, would be an effective way to sidestep the dangers of censorship and overregulation.

      This is an example. the technology is here to do this… now. The problem is it’s not really one of ‘technology’, it’s one of psychology. How do you get users to agree to store foreign, randomly accessible data and lend out precious bandwidth on their devices without some sort of incentive beyond trying to get everyone to buy into the semi-marxist concept of ‘hey man, data is free maaan’. Even if one agrees with that concept, most people don’t agree that their hard drive/memory is free, or the bandwidth required for connectivity.

      But we have similar technologies– at least in concept going this very instant. I’d like to direct your attention over to BitChute.

      1. Well, I think the incentive is the payment system, which didn’t really exist before cryptocurrencies (it was always centralized somehow).

        1. I’m sorry I wasn’t clear, but that was my point– the technology exists, so the novelty part is the incentive system they have to build into it– because the creators know just because they build it, people won’t just show up.

      2. You mean Bufferchute?

        1. Funny how decentralized datastore requiring dozens or hundreds (or thousands!) of points to be playing along at the same results in performance issues.

    2. “That’s a better, safer, more resilient world, which doesn’t end up…susceptible to authoritarian manipulation and control,” says Mackinlay.

      It’s ALWAYS open to manipulation. Always.

      1. It’s ALWAYS open to manipulation. Always.

        This is important. Especially when anything of value is at stake, always expect adversaries to be trying to break it, whether that is something as sophisticated as mathematical manipulation, or as old fashioned as tricking someone into giving up a password.

        1. Or someone fucking with data segments or fragments.

    3. “The technology to create a decentralized web is… as old as the internet. ”

      I think the most recent big technological development since the 80s has been multi core CPUs. Doesn’t this give any boost to the possibilities of a decentralized network?

      “The “problem” as we’ll agree to identify it, is it’s a matter of crowds and how they tend to gravitate,,,”

      That’s only half the problem. It’s not only crowds doing the gravitating. Companies tend to gravitate to each other for much the same reasons. Leads to unaccountable giganticism.

      1. That’s only half the problem. It’s not only crowds doing the gravitating. Companies tend to gravitate to each other for much the same reasons. Leads to unaccountable giganticism.

        Sure. But I’d call that a more symbiotic problem. Once a company forms, creating what I’m describing as a ‘stable, reliable resource’, if it’s successful at all, it will become big. However, ‘bigness’ is an ephemeral condition. I truly believe that Google/Youtube et. al. have already sewn the seeds of their decline.

  4. “Web 3.0 has this wonderful set of trust baked into the Internet itself,” says Molly Mackinlay, a former Google programmer and a current project lead of the InterPlanetary File System (IPFS), a communications protocol that’s meant to replace the hypertext transfer protocol (HTTP) that most of us use to access the web now. While HTTP connects your computer to a particular server, IPFS scours the network for a piece of content, which gets assigned a unique ID marker and connects you to whomever happens to be hosting it.

    Paging Pied Piper’s “middle-out” algorithm.

    1. “Let me ask you. How fast do you think you could jerk off every guy in this room? Because I know how long it would take me. And I can prove it.”

  5. ” holding them legally liable for what others say on their platforms,”

    The gross mendacity of this place never ceases to amaze. But, I guess ignorance is like that.

    How about we support “holding them responsible for the editorial choices they make.” Is that a distinction the authors here just cannot grok? Or is it more about what they don’t want to grok?

    Either way doesn’t matter, decentralized connectivity is coming, the more organizations try to create walled gardens the more they accelerate the process.

    1. “Either way doesn’t matter, decentralized connectivity is coming”

      I see laws forcing identity validation coming first. Shit like searchable IP’s that give you a name and address. Because, you know, to stop terrorism and to protect the kids.

      1. Searchable IPs make life easier, but they’re not necessary to find out who’s doing what.

        1. Years back Clinton passed that age check law for adult websites IIRC. Internet ID to post user generated content on platforms free from liability could be required. Would you be surprised?

          1. That’s actually a good question. If I’m going to answer honestly, yes, I would be surprised if anything ever went that far.

      2. Who remembers the ‘wild west’ days of BBSs? That was back when it was land lines and dial up.

        A decentralized web of the future would, in theory be just as personally identifiable. But you’d be just another needle in a sea of needles several orders of magnitude bigger than the old ways.

    2. “holding them responsible for the editorial choices they make.”

      What do you mean by that, exactly?

      1. “What do you mean by that, exactly?”

        Tough to be exact with a topic so broad.

        Hold them to the same standards applied to other publishers would seem decent start.

        We could also drop this charade of calling them platforms. Not when they plainly exert a tremendous amount of control over what appears, doesn’t appear, or get promoted/demoted. Whether for political or purely financial reasons they are not functioning as platforms.

        1. Hold them to the same standards applied to other publishers would seem decent start.

          Meaning what, exactly? In what way are they not currently?

        2. There’s no legal definition of “platform”, so I’m not sure how you can say they’re not functioning as such.

          1. I’m saying they are not functioning as such in the context of the way the author uses the term.

            But I’d be just fine if Weismuller, and all the others here stopped using the term which, as you note, is vague and not defined in the CDA.

            The term that is clearly defined in Section 230 is internet content provider. Defined as ” any person or entity that is responsible, in whole or in part, for the creation or development of information provided through the Internet or any other interactive computer service.”

            Take careful note of the words “in part” and “development:

            These entities that Reason likes to euphemise as ‘platforms’ are actually content providers, and should be described as such. So why aren’t they?

            Because internet content providers – unlike ‘interactive computer services’ have no special protections and are, as clearly stated in the law, to be treated like any other “publisher or speaker.”

            1. The terms are not mutually exclusive. Nothing in the text of 230 implies that. “The term “interactive computer service” means any information service, system, or access software provider that provides or enables computer access by multiple users to a computer server”. The same entity can function in either of these roles.

              “as clearly stated in the law, to be treated like any other “publisher or speaker.”

              Uh, yeah, “clearly stated”. Please provide a citation where this is clearly stated. Like, any citation in any part of the US code, or any court case. Take your time!

  6. Pied Piper has been working on a decentralized internet for a few years now.

  7. With all the other stuff that’s been in the news lately, the news that the Librem 5 has started shipping may have been buried:

    https://puri.sm/posts/first-librem-5-smartphones-are-shipping/

    I’m still partial to the e foundation’s beta for its ability to run Android apps and the fact that you can side load it on old model phones. (I see some compatible phones you can pick up unlocked for $85 or so.

    If you want a smart phone, there are certainly more options than getting screwed by Google or getting screwed by Apple.

    1. “If you want a smart phone, there are certainly more options than getting screwed by Google or getting screwed by Apple.”

      You have to buy a phone to get the software? I already have a smartphone, an Android, can’t I just download and install a new OS into it?

      1. What, you think you own that phone?

        1. Ideally what I do with my laptop I should be able to do with a smartphone. It just seems a lot riskier.

          1. I agree with the idea(l), unfortunately the reality is even worse than trying to run (emulate) MacOS on an x86 based system.

      2. You can do that with e foundation’s OS, yes.

        “I’m still partial to the e foundation’s beta for its ability to run Android apps and the fact that you can side load it on old model phones. (I see some compatible phones you can pick up unlocked for $85 or so.”

        —-Ken Shultz

        You need to use an unlocked phone that’s on their compatibility list, but their compatibility list is pretty extensive.

        https://gitlab.e.foundation/e/wiki/en/wikis/devices-list

        They’ll soon be launching a service where you can send your phone in and have it installed on your phone for you, too. Should come up and be available any day now. If you install it yourself, you need to come to terms with the possibility that you may brick your phone.

  8. “Berners-Lee and other web pioneers intended for their creation to be decentralized and open-source.”

    I understand Good Sir Tim is (re-)working this as we speak.

      1. Although, like CNN I’d like to be fair and balanced, Stallman denies ever “defending Epstein”, he says his remarks were mischaracterized– and given the jarring behavior of the media lo these last few years, I’d be willing to give him the benefit of the doubt. I’m guessing that his very much verifiable remarks about underaged sex, pornography and the like– defending such– somehow got woven into Epstein’s pedophile Island issue.

      2. OMG I’m shocked ok, totally not shocked at all that RMS would say or do something creepy.

      3. So MIT cut him loose, and have “ordered an investigation.”

        But I see no mention of them giving the money back.

  9. Those with the resources who would manipulate us to their advantage NEED to control public dialogue.

    Logically, we need two internet options.

    One secured by government whom we can hold accountable to move forward with our civic progress.

    Another for free speech.

    1. I would argue we need a third:

      One for Cat lolpics.

      1. What about pancake bunny?

  10. What I’d like to see a choice on censorship and control.
    If I want to read censored streams, I have the choice
    If I want to see potentially censored material so that I can make up my own mind, I have the choice
    If I want to use my own filtering system, I have the choice.

    A lot of people seem to think they should decide for me, and all other humans. I don’t want them to have their way.

    We can already do a lot, if we decide and persist. It would be helpful if government were part of the solution instead of being a big part of the problem.

    1. Once dialogue is censored, we all have lost the choice to recognize the truth, reality.

  11. I don’t think I’ll be able to stop worrying about this issue until the government gets more involved.

Please to post comments