Anthropic Co-Founder: 'The Most Powerful Technology Ever Built'
Jack Clark discusses Anthropic's regulatory fights, the possibility of recursive self-improvement, and how AI could reshape the economy.
Today's guest is Jack Clark. He's the co-founder of Anthropic—the artificial intelligence company behind Claude—and the head of its newly launched Anthropic Institute, a forum designed to think through the philosophical, political, and practical challenges that AI poses to society.
Nick Gillespie talks with Clark, a former tech journalist at Bloomberg and The Register, about his company's ongoing regulatory battles with the Trump administration; whether Anthropic should have held back Mythos, its superpowerful version, from general release; and what might happen if and when AI becomes fully capable of "recursive self-improvement," or the ability to build itself without human input.
They also talk about libertarian and market-based regulatory systems for the internet and tech sector and whether those would yield better outcomes than the ones we've seen so far from the European Union and other conventional governing bodies.
0:00—The promise of artificial intelligence
3:50—Overcoming AI hallucinations and sycophancy
5:08—Military applications of AI
9:38—Anthropic's red lines on mass surveillance and fully automated weapons
14:23—Lessons from the 1990s
16:12—Oversight and regulation of AI technology
23:07—Navigating politics through policy ambiguity
33:31—How does U.S. governance affect global AI policy?
36:42—What is recursive self-improvement?
43:35—How does Clark's literature and journalism background inform his views on AI?
45:43—The liberating aspects of technology
51:15—The effects of AI on employment
Producer: Paul Alexander
Audio Mixer: Ian Keyser
Transcript
This is an AI-generated transcript. Check all quotes against the audio for accuracy.
Nick Gillespie: This is The Reason Interview With Nick Gillespie. My guest today is Jack Clark. He's the co-founder of Anthropic, and he's one of the most influential figures in AI policy. We are going to talk about the government clampdown on Anthropic in particular and various things going on with that. We're gonna talk about the power of AI, the perils, the pitfalls, pretty much the whole ball of wax. So, Jack Clark, thanks for talking to Reason.
Jack Clark: Thanks for having me.
So let's start in a broader sense. Talk about what you see as the main kind of promise of AI. Because we're right now, we are definitely in a kind of technological dark cloud moment, where people are freaking out over AI. And you guys at Anthropic are freaking out on a certain level that we'll talk about. But just broadly, in a general way, what is the biggest, like, what's the biggest selling point of AI?
I'd split it into three different layers: There's stuff for the individual, stuff for what you might think of as the economy, and stuff for science. So, for the individual today, we have a universal teacher, which is something that people fantasized about building for decades and decades, and it just exists. You can go and spend $0 a month to talk to an expert teacher. And if you spend $20 a month, you can talk to, like, a really good teacher. This is incredible. So that's for the individual. For the economy - the economy - especially in the Western world, it is defined by back-office work and bureaucracy, or what David Graeber called, "Bullshit jobs". We now have a system that can deal with, navigate the bureaucracy and back-office stuff, do it better, in some cases get rid of it, and allow people to get back to the parts of the jobs which either use their skills or have more meaning.
Put a specific example on that.
Yeah, I'll give you an example. We worked closely with the makers of Ozempic to help them do work on the backend for how they processed and formatted results in their clinical trials. And we took it down from something on the order of two months of work to a week of work. I don't think anyone misses that. No one there is crying at their desk because they can't do that.
Yeah, I was the best back-office paper shuffler.
Yeah, damn it, that was me. That's what I did. So that and examples like that are found everywhere. There's someone I work with at Anthropic who got so annoyed at processing a certain type of invoice that they just fully automated that. They're not sad about it. Everyone's happy that that's automated. And then for science, we now have systems that, if you read the literature, are co-creating science at the frontier of biology, mathematics, physics, medicine. You are seeing the world's expert scientists co-author papers with Gemini, Claude, ChatGPT. This is a new era for scientific discovery.
Not Grok, right? Nobody serious uses Grok.
I may have seen it somewhere, but there are other names that come to mind as well.
Right. And this is also, you know, one of the examples that gets talked a lot about, which seems amazing, is that when you think of something like X-rays for tumors that might be cancerous or whatever, suddenly the entire world becomes a database, and you have computers training and looking at all of this and just coming up with better diagnostics and prognoses and things like that.
Much better diagnostics, much better prognoses. Also, on an individual level, we've now, you know, replaced WebMD, which always tells you you've got cancer and are going to die, with AI systems, which often tell you something a lot more reasonable, factually accurate, and also say, "Probably call the advice nurse and tell them these things."
What do you do? I'm forgetting the name of the disease that somebody made up that has gone through, like it's in every AI model now, Baxillism or something. But I mean, I guess just quickly before we move on to kind of more policy issues, how does the AI work to filter out kind of bullshit or mass delusion? Because this is always the problem. Right?
So there's a couple of things here. Years ago, AI systems used to hallucinate. They used to make stuff up a lot. And it's basically because we train AI systems to be like a contestant on Jeopardy!. They want to hit the buzzer. The moment you say something, they want to answer. It took a lot of work to say to them, "It's totally reasonable to hit the buzzer and say, 'I don't know,' or to not answer at all." It was kind of contra how we train them. So that took some time.
The more pernicious problem, which is still with us, is this issue of sycophancy, which is when AI systems are overly complimentary. When they walk in and they say, "It's amazing to be here on the world's greatest libertarian podcast. What a fantastic conversation we're having." Deeply unhelpful, completely opposite to what useful people in your life do to you. And we have to sort of train that out of the systems, where you want to make them have the form of, like a friend, something that will actually push back on you a little. Now, this introduces many, many, like, rich and interesting questions about what's appropriate or inappropriate there. But we sort of know it when we see it. We know it when we experience obsequiousness or sycophancy in our own life.
Talk a little bit about military applications of AI technology. Historically, certainly going back to people like Galileo and whatnot, technology, technological innovation, and defense, broadly kind of loosely defined, it's always been a big part. This was part of your early contentious intersection with the Trump administration. How is AI going to change kind of military and defense applications?
Well, there's one way where it's exactly like how it changes everything else. And there's another way that is unique to this, this kind of genre of work. So, often a joke about military organizations is they are logistics companies that have a sharp bit at the very end, right? The logistics stuff is the same back-office stuff I've talked about for everything else. AI can just be wildly helpful there and help automate and improve decision-making and back-office schlep, of which the military and other organizations famously do tons of.
I mean, this is Catch-22. The military is the bureaucratic system that exists to kind of punish or torment the people who it supposedly is serving.
My brother was in the British Army as well, so it sounds familiar. At the same time, AI can introduce new capabilities at the frontier. You know, the advancement of science creates new things that can be used by the military. An example outside of AI is hypersonic missiles. So we invented missiles. Now we've invented a much faster missile using advances in materials science, which is much harder to shoot down. Also, there's the example of things like drones and other stuff. I think that we can just generally expect AI to create new capabilities at the frontier of the military. And those capabilities, like previous technological creations ranging from nuclear weapons to hypersonics to cluster munitions to biological weapons, are going to be subjects of deep debate among militaries, among governments, and among citizens.
Is there a—I don't know—not first to market per se, and obviously AI has already kind of permeated the major powers and many minor powers, but is it going to be something like, "If we can get a nuclear bomb first, whoever we is, then we can stop the creation of further kind of technology"? Or are we going to replay the Cold War, where it's like, ok, the U.S. gets a nuclear weapon, then, you know, Russia and China and a variety of other countries do?
It's most helpful to think of AI as being almost equivalent to maybe electricity or oil, in the sense that it is both the thing and the production and management system around it. And that delivers this huge range of like commercial, civil benefits, but it has profound uses in conflict as well. And it also introduces profound weaknesses in conflict. Additionally, the most kind of refined forms of AI, really, really, very smart systems, may have properties that are akin to sort of handle-with-care technologies like nuclear weapons, where you need to find a way to proliferate their beneficial aspects but find ways to control that potentially very militarily decisive aspects.
And this was basically the crux of your, of Anthropic's confrontation or conflict with the Trump administration recently, right? Or can you explain that? Because this seems to be the essential backdrop to the current limitation on Anthropic's ability to kind of send its programs around to whoever it wants.
We've been longtime supporters of working with the intelligence and defense communities, and Anthropic has also always sought to talk about the full range of issues that lie ahead of us with this technology. At the moment, what we're grappling with is we have a technology, these models called Fable, that is intensely useful in a bunch of commercial ways, and it has some potentially national-security- or defense-relevant properties in cyber and bio. We've worked to find ways to make it hard for those capabilities to proliferate. And we're now in sort of daily discussion—and I obviously can't get into the specifics—with the government about what a sensible policy framework is for how you can both proliferate and export these models without proliferating and exporting the thing that seems more profoundly useful for national security.
If we take it back a step, I mean, it was, what, a few months ago when Pete Hegseth, the secretary of defense—I keep meaning, I don't want to call it whatever he wants it to be, the secretary of war, secretary of defense—but he was saying that Anthropic wouldn't allow for the way he wanted, or the Trump administration wanted, to deploy certain types of AI. And you guys, in the public narrative, were insisting, "No, certain kinds of kill decisions cannot be relegated to AI." That contradicts your corporate philosophy. And the Trump administration threw a shit fit about that, right?
We outlined two red lines. One is about the potential use of AI for domestic mass surveillance on Americans. And another is the use of AI for fully automated weapons. The reason we outlined this is not because we think that we are the ones that can define what's appropriate there. That's absurd. But because it's clearly an area where there needs to be a wider societal debate, there needs to be some notion of what we think is right as a set of individuals, citizens of this country. And we remain standing and ready to work with the U.S. government in all ways that can be helpful, while also we believe that that discussion about those red lines is something that is beneficial to have.
Where does that argument stand right now? Because the government wanted to put you guys on a list of, you know, bad corporate citizens, right? Is there any discussion still going on related to defense policies?
We're in regular discussion with the U.S. government, and it's important to note that we, you know, we are constantly finding ways to support the government and work with them. And I think that the broader thing that's useful to focus on here is that these models from us are not special. There are other models from other providers that are going to exhibit the same qualities. And our role, now that we've sort of outlined our views, we believe, is for there to be a broader discussion and debate had in the public sphere about this and what is appropriate and what makes sense.
Right. I mean, it's hard kind of sitting, reading the news. And it's like, ok, so you guys get into a fight with the Defense Department over the parameters of a contract that you guys had signed. And, you know, the secretary of defense and the president say, "No, you have to give us this. You know, we don't care about whatever contract there was." And then you release Mythos 5, right, which is—or you're bringing that to market—very powerful. And you say, "Hey, we're gonna pull that back, and we're going to release a slightly slimmed-down version for general release, Fable." And then the government says, "Yeah, you know what? You can't release that outside of the country," which is effectively saying that you really can't release it. How is that not just obviously payback from an administration that acts kind of like a criminal gang, where, you know, you pissed in the wrong—you know, you pissed in the punch bowl, and now this is what you get?
I can't talk too much about the specifics here. It's obviously a live discussion that we're having. But I would say this is a new technology. It has new capabilities. I actually think that these capabilities are deeply surprising, and they would be surprising to any administration. And here's my view of where I have huge empathy for people who work in the administration is: If you've spent years with a national security community that is developing and fielding capabilities around, say, cyber, these capabilities come from a whole bunch of contractors you work with; they come from the system you understand. And then here are these companies that are building something called AI. AI is a civilian technology. And one day, the AI systems that have been sold as a civilian technology, are used by civilians, and are, like, wildly successful within the American economy, start popping out things that your colleagues in the intelligence agency say, "Hang on, these things look like the things that we work on over here." How do you deal with that? I actually think this is, like, a really challenging question. And what we're doing right now is the messy way of dealing with it, which is you're saying, basically, "This is a lot. We need to figure out how to slow this down and get our arms around it." But it is a debate that would be had under any admin. And I think what we are gonna need is some kind of policy framework for how you bring these systems out without bringing out national security capabilities, or controlling those capabilities such that we can be comfortable with their proliferation.
Are there policy or even, you know, political or philosophical lessons to be learned from the '90s? Because when encryption, and particularly something like Phil Zimmermann's "Pretty Good Privacy," or PGP, when that got disseminated and suddenly everybody can encrypt everything, and people freaked out. And the Clinton administration, at the time, freaked out. It was like, "No, this is a munition. We're gonna regulate it as if it's a munition." And we still have these conversations about encryption, right? You know, where it's like, "Oh, the only people who want to encrypt things are criminals, obviously." Are there lessons from the '90s that we should be reminding ourselves about?
And there are also lessons from the late '90s or early 2000s, where Sony made a console called the PlayStation with something called the Cell processor. And I believe that we briefly wanted to control that because it was an incredibly powerful supercomputer inside a gamer's machine. And the lesson I take from it is your first contact with something which blurs the line between civilian and national security borders is always messy. And when things emanate from the non-defense or national security world, your initial response is, "Let's take a minute and figure this out, and let's control it." And then your longer-term response is, "Ok, we now need to balance, like, individual sovereignty and the ability to sort of, like, protect and secure the nation." And in both the case of computing and encryption, we've ended up in places where people have broad access as individuals to private encryption and broad access as individuals to private computation. And we've also figured out ways to kind of secure or manage some of the more national-security-relevant aspects of this. So I think that's roughly what's going to happen in the longer term.
So if we talk about Anthropic, you guys were about to release Mythos—and please correct me if I'm getting, you know, the names wrong and things like that—but you're going to release Mythos. You kind of do internal security checks on it. And, you know, your main kind of hacker in residence says, like, "Holy shit, this is too powerful." So you kind of pull it back, and then you release Fable, which has many of the capacities, but it stops short of, like, the full range of Mythos. You, as a private company, made the decision to do that. Isn't that the way this should be governed? Like, why should you then have, you know, whether it's Pete Hegseth or Howard Lutnick or Donald Trump, who doesn't, you know, who doesn't even use email—why are they the people who get to say what you can and cannot release?
It's never going to be only companies making the judgment calls about a technology this powerful. It is going to be a shared, like, exercise between not just companies and government, but also, you know, other parts of society, like the broader scientific community. We're in the messy part right now. But I think that the job of companies, as we do, is to communicate about what they see and run basically different forms of release experiments. And each release experiment generates data that we, as a community, can use to figure out what correct is. And the role of government often has been to look at what industry does and then figure out what the de facto norms and standards are. And if any of those norms or standards are sufficiently sensible and good for society, that you should enforce them. Seatbelts might be a good example here. Or we should probably ensure that people don't put CFCs in fridges because that seems dangerous. For a bit, that's sort of the role. That's where I think that we'll end up. And our role, as I see it, is the benefit of being on the frontier is we get to see this stuff early and we get to talk to the world about it. And our responsibility is to share what we see, and then we'll figure out some system.
Mark Andreessen, you know, venture capitalist, creator of Netscape, the poster, you know, internet poster boy of 1995, because Netscape, I guess, was generally considered the first pure internet IPO. He talked about the reason why he and many people in Silicon Valley supported Donald Trump this past election was because the Biden administration had said, you know, independent of defense stuff, it's like, "Hey, you know, we're putting a lot of borders or, you know, boundaries on AI." Trump said he wasn't going to. First off, was that accurate in your experience? Was the Biden administration generally less receptive to kind of the proliferation and development of AI than the Trump administration?
This wasn't really my experience. I'd say that there was—any different administration has a different set of interests and priorities. And I'd say that the Biden administration was just sort of somewhat more focused on aspects of maybe safety and security challenges of AI systems. The previous Trump administration had been focused on some of the security aspects of cloud computing, for instance, and that had been carried through by the Biden admin. The Trump admin has been focused, I think, rightfully, on, "Ok, economic security is national security. How do we both, like, push the sector forward, but also sort out things like the power stuff and also find ways to ensure that we actually successfully build semiconductors in America, which is something that started under Biden?" And then there was perhaps, at least initially, a bit less emphasis on some of the security stuff. And then, as the security stuff has arrived, well, they've raised the emphasis of it. So I, you know, for context, before founding Anthropic, I also worked at OpenAI. So I've worked on AI policy under the end of the Obama admin, the whole of the first Trump admin, the Biden admin, and then the Trump admin. And my experience is there's actually, like, tons of continuity between all of these admins around, like, cloud computing, supply chain, and this question of how you test and evaluate models. And then there are just different forms of emphasis along the way.
But we are now in an era where the government seems to have, you know, moved to a new level of regulatory—flexing regulatory muscle over AI.
We're in the era which many people working in AI have long assumed would arrive, which is the AI systems got not just scientifically interesting at domains that you might care about, but practically interesting. Where we are now is a world where we now know that AI models are going to have cyber properties that are genuinely useful. That just presents a new challenge that we have to deal with. And it, like, makes total sense to me that the government changes its attitude in response to that because you've gone from the theoretical or academic to the practical in here and now.
So one thing that Anthropic has been very clear about over the course of its existence is that it kind of believes in government regulation, right? I mean, you guys are not, you're not, you know, the Wired masthead circa 1996 or something, saying, you know, like, "You rotting hulks of old governments, go fuck yourself." What's wrong, like, right now, so you're facing, you know, somebody like Bernie Sanders, who says, like, "These are very nice AI companies you have here, and we want to take a huge chunk out of them and spend it the way that we see fit and regulate you." You have a Trump administration in the name of, well, it's certainly not free markets, but national security, saying, "Ok, we want to control this, and we're telling you now you cannot export these sorts of things." What is wrong with this? This isn't just the world that you dreamed of having, where the government was taking a very, you know, strong interest in regulating what you do and how you distribute it.
I mean, we have this joke at Anthropic where, you know, when I was a teenager, I worked in a bookshop and I thought to myself, "I have to get, like, a real job because I'm like extremely bored," and boy, did the monkey's paw curl on that one. And now I know I'm really seeing the fruits of that. You know, what do you think the response of the world is to probably the most powerful technology ever built by the species? It's not going to be nothing. It's going to be a highly politicized response. It's actually appropriate that the way you govern the most powerful technology that I believe may ever be built by our species can't be nothing. It can't be: We use only the existing rules and norms that we have, and we apply no new governance technology to it. I just can't reconcile that—
I mean, there's a lot packed into the idea that this is more transformative than fire or the wheel or, you know, technologies that developed without, you know, kind of top-down regulation, or, you know, pre-civilization.
The most powerful technology developed in the modern industrial era.
No, no, no. I'm not trying to keep a-dancin' or anything. But there's a lot riding on that question. But then, ok, let's assume that. Then what's wrong with the way the Trump administration is treating Anthropic right now?
Well, the way I'd put it is, like, there are all of these specifics to this situation, which I can't really talk about. It's a live situation. But the broader thing is the government has said, "Oh, these systems have national security properties, and we need to figure out how to deal with them." That's good. That's a good thing. You know, Bernie Sanders and others on the left are saying, "If the AI companies are right, they might create, like, vast, vast amounts of wealth and also massively centralize that wealth into a small number of entities. What are we going to do about that?" That's a good thing. Now—
That's interesting though that he was saying that 10 years ago and 100 years ago when he was, like, in his teens…
Maybe now the evidence has arrived more. But my broader point is, like, I think if you look at the history of policy debates and how change happens, it always starts off with a load of people having different ideas, radically different ideas of what you should do, getting very interested in it, there being forms of like disagreement and debate. And then, you know, there's a phrase in policy of, I think it's called the art of muddling through, and what you actually end up doing is typically you end up muddling through. You don't do any of the things that are like wildly, wildly extreme. You come up with regimes—
Are there any elected officials now? Because it's also, you know, the Trump administration is one part of the federal government. Congress, you know, and obviously its constituent parts are others. Are there elected officials that you think are articulating the best way of having these discussions? You know, you talked about the public sphere, so that includes government, but it also includes, you know, media and things like that. But are there elected officials? Because it seems to me the Sanders, you know, model is insane. The Trump administration, we're it's saying, like, you have great, you know—I mean, if Sanders is saying we want to take, you know, a 50 percent one-time tax on whatever that means, and then spend that money, but Trump has already shown that he's willing to say to companies, like, "You have to pay me some kind of, you know, vig or some kind of protection money in order to export or import things." Neither of those seem very good ideas. Are there people out there in Congress who you think are talking more sensibly about what this—
I mean, there's an AI policy bill working its way through Congress now from Obernolte and Trahan that has a bunch of, like, sensible ideas in it around transparency, third-party testing, and verification. It's a long bill, a thicket of stuff. We don't agree with everything in it, but it's got a broad set of things where you look at it and you're like, "This seems very reasonable." There are bills working their way through Congress around export controls, you know, preserving national security for not just us, but Western countries, by working on control of compute that we might otherwise ship to China. Broadly sensible. The Trump administration put out an executive order a few weeks ago on how it might approach some of the national security questions of AI systems. The architecture described in that executive order of testing and validation, coming up with shared frameworks, broadly sensible. I think that eventually we'll actually get to some system like that outlined there. So, yes, for like situations that happen that are, like, have a lot of heat and light attached to them. But I think if you look beyond them, there's the policy machine across both the admin and Congress and the intelligence community is processing this technology and coming up with relatively sensible, implementable ideas here. And if you look at the states, many states have implemented basic, what you might think of as transparency bills that will be applied to AI systems, which functionally means that my AI systems now get regulated as much as, like, a packet of potato chips. Which is better than nothing. It means I have to put some ingredient details on it and talk about how I made the chips.
Made. They're also saying you can sell them, or they're not regulating the number of calories or the percentage of fat or carbohydrates—
Yeah, they aren't doing that. A lot of this, our philosophy, is you need to start with mandated transparency from the companies about how they've tested their systems and what they've done. And then you need some form of third-party validation, that someone checks the homework of the person. And through that, you're going to build this giant base of evidence for society of, are there any things that we actually need to mandate? Like, it seems likely to me that at some point on cyber or bio, you might develop some very clear thresholds that become mandated, similar to how if you want to go and buy explosives in the world today, you can buy certain forms of explosives as a civilian like you or I. Others, you need to be a construction worker with a license, and others, you need to be a member of the military. Broadly reasonable. Has some liberty elements, but we found our way to something sensible there.
But you don't want to have to deal with 50 state laws, right? I mean, isn't there something?
You want there to be a federal approach. That's why we're excited about, you know, some of the bills working their way through Congress, because otherwise you will end up with a patchwork, which no one wants.
Or California. I mean, this happened for auto emissions and things like that. One state gets to de facto set a national standard, and it might be more ideologically extreme one way or the other than it would be from the national government.
The whole way I'd put this is AI is an incredibly powerful technology. Pressure is building up in the system about what to do about it. And the longer that the answer is nothing, the higher the chance that different parts of the policy system say, "Nothing ain't good enough. We're going to do something."
So, you know, one of the things to think about is, you know, this is something where libertarians and socialists—and I'm thinking particularly of the socialist historian Gabriel Kolko, who wrote a famous book-length treatment of the Progressive Era and railroads—and the progressive story about railroads is that railroads, you know, were these dominant, super-powerful, politically connected monopolies or cartels, and finally progressives gained enough power where they regulated the railroads and everything was good. Kolko, a socialist, said, "No, that's actually wrong," and that it was the railroads, when they reached a certain point of power and they also realized that, you know, the market it's gonna get tougher and tougher to have profits, they went to the government and said, "Hey, we need to be regulated." There was a moment with social media, or the tech sector more generally, in 2018, where Mark Zuckerberg said exactly that to Congress, where Tim Cook of Apple said, "You know what? The tech sector isn't working anymore. We need regulation." How, from a kind of libertarian or a Kolkoian socialist perspective, how can you assure me that this isn't what's happened? There is, you know, there's a half-dozen big AI companies who are suddenly saying, "You know what? It would be a damn good thing for the citizens of America if we had national regulation, you know, that preempted all of these problems." How is that not just locking in your market position right now?
Well, I've said the same thing for 10 years. I've testified in Congress for 10 years. I said the same thing for 10 years. We started saying this the moment we started Anthropic, when we were a company with no product, no revenue, nothing. We've always—
And now you have—generally, it's usually put around a billion-dollar valuation, right? I mean, you're private, but—
I think it's sometimes reported as a trillion dollars.
A trillion. Excuse me. Yeah, that's a minor difference. I'm a lit major. I'm not a math guy.
These things happen. These things happen.
Thank you.
Yeah. So look, I absolutely—I totally understand this frame. I'd say, for us, look at just our, like, consistency of action, statements, and beliefs over time. You know, Dario, I've worked with Dario, the CEO of Anthropic, for many years. We were both at OpenAI together. He said the same things for 10 years as well. And the approach which we've always had is we believed and continue to believe this will be the most powerful and consequential technology that gets built in this century, for sure. And the answer to how you deal with it must be that we need to create new tools for us to govern and manage it, and we need to integrate it with our existing laws, and we need to integrate it with some kind of system by which it can't only be companies making decisions about it because the technology will be too powerful and too influential.
Can you allay my worries that—and this is something that Zuckerberg said to Congress that—"I will help you write the regulations to control social media, but you realize that means that there will never be another Facebook. But is that the price you're willing to pay?" And obviously, it got a little bit squirrely for a lot of different reasons, but if Anthropic and OpenAI and a couple of other companies get to write the rules, does that mean, ok, these are the five firms or whatever that are just gonna dominate forever?
I mean we've shared our ideas for regulations, but I really welcome and encourage people like you and others to write your own take on it. I think you've seen people like Dean Ball, who was briefly in this admin. He does a huge amount of public writing on what he thinks sensible policy approaches. Oren Cass has done similar work.
Not a free market— I mean he…
The point I'm making is, the more people you have outside the companies articulating a view here, the better it is for everyone. Because you want a big pile of different ideas around how to approach this. And then some of the ideas, there might be commonality. Do those ones. If you can build me a table that has, like, companies, libertarians, like, free-market maximalists—and there are some ideas where everyone agrees, maybe those are sensible ideas to do. And if there are ideas that only the companies propose, be really, really, like, skeptical of them because there's probably a reason the others didn't propose it.
Talk about when we're dealing with issues of governance. Ok, so the U.S. is one thing. The E.U. is something else. And I think, generally speaking, certainly most people in America would believe this. I think many Europeans would. Their approach to technological governance has been bad. It's, like, tamped it down, and Europe is further behind than it was 30 years ago.
They've done regulations ahead of their own capacity to have the thing itself.
And then you have places like China, which are kind of like, you know, "We don't care." How does U.S. governance end up affecting global governance because the EU is gonna do what it's gonna do, and China is gonna do what it's gonna do? How does this play out in the best possible scenario?
The best possible scenario is something that happens millions of times a day, which is cars get exported between all of those markets—maybe with the exception of the U.S. in the car case—to one another, planes take off from airports in all of those places and land in the other ones. And you somehow have confidence—oh, food gets exported. You have confidence in the cars, planes, and food. Why is that? It's because you have some notion of shared standards, shared testing, shared means by which you can validate that the thing has the safety properties needed for it to be exported and imported. And rather than having some kind of global governance regime, you instead have a bunch of standards bodies in different countries, which basically have reciprocality. And I think that this has just worked amazingly well for commerce. You know, I buy a lot of baby toys now, as I have a 3-year-old and a 7-month-old. It's great to have certain labeling standards because my wife likes to buy organic food and also doesn't want toys with lead in them. I understand, but there's actually like labeling schemes that let me do that. So I think that's the…
A lot of those are voluntary, right? I mean you know if you buy something at IKEA, you know, a great Swedish company, it's less likely to be painted with lead paint.
Many of these are voluntary, like the energy staffing from the utility industry. But then some of these get codified as just hard standards, you know, certain hard standards for automotive and airline security, which just get baked in. And then there's a lot of stuff where it's voluntary and we compete, but that's just how the world figures this out. It figures out what is the core set of things, which just has to be true.
I am worried in the sense of, you know, when you were talking about, you know, commerce, and, you know, one of the great success stories of the post–World War II era was a general reduction in tariffs and trade exemptions, as well as capital requirements, that capital can roam more freely around the world. And that seems to be coming, if it's not coming to an end, very much under attack by this administration, by a Republican administration, as well as Democrats like Elizabeth Warren or Bernie Sanders types. So that worries me because Trump has made it clear, like, he doesn't want particular vegetables or people from certain countries coming here. And there seems to be broad support for that. So I worry about that. But, if we may, let's talk. I want to ask you about recursive self-improvement, which is a big focus of material coming out of Anthropic. Can you explain what that is and why this is kind of the big thing that we should be talking more about?
So AI moves forward by people gathering resources, like computing data, and using relatively well-understood algorithms to train deep neural networks on those resources. And you end up with these amazing predictive machines, generative models, universal teachers. Well, these systems have started to get really good at also writing code for training AI systems. They've started to get good at proposing ideas for how you might train AI systems. There may come a point where we as people can step back entirely from their forward development. And it's sort of a case of wind them up and watch them go. We give them the resources and we say, "Ok, Claude 10, build Claude 11," and Claude builds the architecture, does the research, de-risks it, does the training run. Claude 11 comes out. It's better in every way than Claude 10. We don't see that today, but I think that there's real potential for this to arrive. Certainly this decade, but my, if I had to bet the year, it would be 2028 towards the end. And that's based on reading a lot of the scientific literature. And also, we published information…
2028 is like, I mean, you said you have a 7-month-old. I have a 6-month-old. Like, they'll be, like, a couple of years older.
They'll be a couple years old.
I mean, 2028 isn't even two years away.
It's quite close.
Yeah.
Yeah. AI moves quickly, and what the implication is of that is we should expect AI systems to move even further in the future and grow even more powerful than they have so far. We should expect the progress we've felt in the last five or six years to actually maybe be compressed into the next two or three years and then compressed again.
Are there other examples of that, you know, at the surface level, where a technology has accelerated, like, just the improvements of the technology have accelerated that much?
There's basically no example. The only good example is evolution of biological creatures. And it takes millions of years, but, you know, you can make better dogs. Just takes you—it takes you many generations. Probably the best analogy I have is: Imagine a 3D printer that could print a finer-resolution print head than the one it had. That would be amazing. And it would yield, like, an amazing, like, burst in productivity of what you could manufacture and make. That's kind of what we're talking about here.
Do you put hedges on that? How do you put breaks—how do you build in breaks? Because this is part of the doomerism and the apocalypticism is that AI it realizes it's not just that it doesn't need humans or shareholders, but they're in the way.
Well, it's a harder problem, but of similar form to what we're currently dealing with things like cyber and bio, which is you have some property of the system that you're worried about. Well, you need to know how to measure it. You need to know how to potentially control it, you know, make it so that the system can sometimes not be capable of AI R&D. And you need to probably demand transparency out of the places which are building this because you, as society, might say, "This is great, and it's going to drive loads of strategic and important stuff in our economy and national security. And we probably don't need it to go thousands and thousands of times faster than any human can hope to govern it." So you need some measurement property of it. A lot of the work that I'm doing at the Anthropic Institute, which is kind of a think tank with a supercomputer inside Anthropic, is studying AI research and development within Anthropic and, you know, figuring out the measures that we might use to spot recursive self-improvement if it were happening. And eventually you want to share those measures publicly and socialize them with policymakers similar to bio or cyber and say, "Hey, we're going to need to figure out, like, a framework and a set of thresholds around this, because this is how we can make science go orders of magnitude faster than today." It's how we could intentionally bring benefits to today that will seem like miracles: novel, novel therapies, incredible, incredible advances in materials science. And the same would be true of national-security-relevant capabilities. And you're going to need to figure out what the control layer is on that.
Is there a broad buy-in from other major AI companies to kind of this type of sharing of standards and transparency? Because it's also, you know, like, "I've got my, you know, black box over here. And so, you know, I'm not giving you the Coca-Cola formula."
So we said recently in a post about recursive self-improvement, we think it would be really good for the world to have the option to potentially slow down the pace of progress. If you hit something like RSI, you might want to be able to intentionally slow that down to give us time to adapt.
Yeah. And who is we in that? Because, like, this, and I'm not trying to be a, you know, a dick libertarian, although I fail at that on an hourly basis, but to be, like, "Who's we, Kemosabe?" Because, you know, Anthropic so far has acted like a good corporate citizen, right? You delay the launch of something that you think is too powerful. You know, ok, so that's good. But then what about other companies who are, you know, part of a government that is like, "No, this is, we've got to go for broke right now"?
I think all we can do is put these ideas out there, socialize them, and I believe that if it's truly important, the world will find ways to come together to coordinate around that data. I think that the world has done this in the past, and it may need to do it in the future.
What are the best examples of that?
I mean, CFCs is one of the best examples. We were like, "Oh, ok, all right. No one actually wants the hole in the ozone layer. Right. Manufacturers, scientists, governments. Let's just stop this. Right." It's great. That was, like, a great, amazing example of global coordination.
And it's interesting that very few people talk about that.
Very few people talk about it.
The big loss was that McDonald's started wrapping things in wax-coated paper rather than using their clamshell foam container.
And, you know, the world cried a single tear, but, you know, came through it.
Can we talk a bit about your background? Because the work that you're doing at, you know, the kind of think tank and policy tank and whatnot at Anthropic—you went to university at East Anglia in England, which is famous for its creative writing program. You studied creative writing and literature. You worked as a journalist, a tech journalist at Bloomberg and other places. And, oh God, is it The Register, right? Which is just a wonderful, you know—
It's like a libertarian tech rag. It's just biting the hand that feeds IT.
Yeah, no, it's just wonderful. I've been reading it out forever. How does your background—one of the things that people say a lot, or I hear a lot, is that Silicon Valley needs more philosophers and philosophers need more tech understanding. But how does your background inform the way that you think about this stuff?
I think if you're a journalist and you have a background in reading widely, you recognize that all technologies have political and social elements. All technologies ultimately come along and they change how people behave and they change the structure of the world. And I think probably Silicon Valley spends a lot of time, rightfully, focused on building stuff because it's really fucking hard to build stuff. Like, you put a lot of effort in.
We kind of take it for granted really, over the past 50 years. It's pretty amazing.
And some of the lessons of the last 15 years is, in that total focus on building, they sometimes forget that they're also shipping a social and political technology into the world. Social media is a great example where we ran an uncontrolled experiment on hundreds of millions, billions of people around the world.
Is the world better off, though, because of social media?
In many ways it is, but I feel like we could have better information and we could have dealt with some of the issues, like, say, teen anorexia or suicide, better if we just shared a bit more information as it was going through. Now, who knows? But I'm just saying that's a frame that you can use. You don't stop the technology, but maybe there are ways we could have been better prepared to deal with it. Smartphones, we're just getting our hands around them similarly. So that background, I think, caused me to see technology as fundamentally something that would change the kind of political economy of society. I read an article in 2012 called "Facebook, Google: Rise of the New Feudalism." And it was just by studying and touring data centers from large companies, studying the types of software that were being used to organize and analyze that data and early machine learning, and realizing, "Hang on a second, this is, like, a system by which you can gather huge amounts of data around the world. You can have people working on your platforms. You can analyze the people. You can exercise unusual control. This looks a lot like something that we used to do called feudalism." And it's not a specific critique of these companies. It's an observation that technology has political qualities.
So prior to this, I think, the start of this conversation, you mentioned you read a lot of the Beats. And in that kind of formulation of technology, in particular, makes me think of William Burroughs. And William Burroughs, broadly throughout his work, saw systems of power or systems of control that enslaved people. And the paradigmatic example, it's in his first book, Junky—everything is kind of like heroin, where it takes over everything that you do, and your entire world is structured around that. Is that kind of how you see technology, or what about the liberating aspects of this? Because when people who make arguments about the new feudalism, the things that they kind of just kind of wave away are the fact not just that ultra-poverty, like extreme poverty, has basically vanished over the past 20 years, but that the rise of a global middle class, people who have disposable income relative to the countries they're in, in a way that, until Homi Kharas, the Brookings Institution demographer, says it was around 2017, for the first time in human history, the majority of people are living at the middle-class or higher level. That has political implications, right? Because middle-class people are less likely to put up with as much shit as serfs. So, when we talk about the new feudalism, is that a way of smuggling in an understanding of technology as ultra-deterministic that we should not accept at face value?
No, it's an observation that the technology can have these properties, and it's sort of the job of the world to closely look at, I think, what the distribution is and figure out if there are ways you spin the dial. Smartphones are a great example, where smartphones, they allow you to start businesses, they allow you to communicate, they allow you to—things like photography has had myriad amazing benefits, not least of all how many photos I take of my kids, which actually fills me with joy. It's great. And I sometimes wake up in the middle of the night, and I look at Instagram Reels, and I waste 45 minutes of my life. And I'm like, "Whoa, what's going on here? Why did I do that?" The world gives me the tool, and it lets me use it however I like. And also we know that the tool within itself contains multiple different forms of political reality, ranging from sovereignty to things that look like control.
Do you believe in progress either with a small p or a capital P—or how does that factor into the way that you conceptualize technology and AI?
Define progress here.
Well, that's, I'm asking, like, how do you—do you believe in progress, or how do you define progress? And is a technology good because it increases, I don't know, you know, it makes everybody wealthier? Or is technology good because it makes individuals freer to pursue their interests?
I fundamentally think that technology is a tool by which humans can greater enhance our understanding of the world and our understanding of ourselves and each other. And I think that the human species needs a lot of powerful technology to get through this century. We need to improve our energy systems. We need to improve government. We need to improve all kinds of stuff around us. And digital technology has been especially helpful for some of this. And AI is going to help it expand into the physical world and make things like medicine, energy, material science go incredibly fast. And I think that will lead to more material abundance for everyone. And I think abundance allows you to create liberty because it allows you to create more time for people.
But then everybody, you know, when you have more liberty, you have more choices, and then you have more responsibility. And we've kind of massified existential angst, right?
In some sense, but in other senses, I think that technology can be very empowering for community. I mean, you know, I lived in Oakland from 2013 onwards, and there was this period in Oakland, California, where there was an amazingly vibrant and diverse DIY punk scene. You know, permissionless, like, art communities, creating an amazing sense of community. How did it work? Flyers on Instagram, which said, like, there's a show at X. And it used to be flyers on the street pole, but now it had just transitioned to this new technology and was using it to not atomize people, but to de-atomize people and to bring people together. So it always contains both.
Hasn't that continued apace? Or rather, you know, it seems to me a lot of times when people say, "Well, the problem with capitalism broadly or technology is that it atomizes people." It's that, well, you like this particular DIY punk scene. I might've liked it too, or I like Burning Man, but everything that allows you to pursue your bliss or create that community allows people the right of exit to be like, "Yeah, thanks, but no thanks. I'm going to hang out over here." And we often mistake the ability of other people to live how they want to live with the end of the world.
Yeah. I mean, I think that one of the challenges of AI systems is you want to make these amazing capabilities as generally and broadly available as possible, where there's few, like, sort of intermediating aspects, and these systems include within themselves properties that are genuinely really scary and frightening and that actually need to be somehow controlled. And setting that border is basically like the grand political policy discussion of our current era. Like that's what we're—that's what this whole thing is about. And it's why it's so messy. Because the nice thing about AI is you work on problems which humans have never had satisfying answers to. This is one of them.
As we move to a close of our conversation, talk about the employment effects. One of the things that gets talked a lot about, which seems distinct in AI, is that in the past, it was always automation or, you know, technology was going to wipe out the crap jobs, and in many ways it has. And I say this as somebody in high school, I worked in factories, and, like, factory jobs have disappeared, not because of free trade, but mostly because of automation. If you've ever worked in a factory, you're like, thank God. I mean, it's like, who wouldn't rather have a robot do something better and in a painless way? But what people talk about AI is, "Oh, you know, the problem with this is it's coming for upper-middle-class, educated people." Is that accurate?
There are two contradictory things to hold in mind here. Number one, the data today doesn't show this, but the data today, the starting point of the current AI revolution, is also the starting point of COVID, which introduces a giant confounder in all of the economic data sets you can use. Because in COVID, there was massive over-hiring in some sectors. There's the move to work from home, all of these huge changes in the economic environment, and then AI comes along. Parsing this out is difficult. We can see some weakness in early graduate hiring in some sectors. That is the only thing that the evidence actually shows you.
I mean, it's kind of a bad time historically to be graduating as a computer science major right now, as opposed to 10 years ago—
Yeah, it may be. It may be the case. And when we look within Anthropic, we're hiring more people with lots and lots of experience than we did before, because the returns on intuition are much greater than before, because now you don't do the schlep work to run your experiments. You just—
What do you mean by returns on information, things like expertise?
Yeah. If you're an experienced researcher and you have a ton of ideas for experiments to run, well, previously we needed to also give you an engineering team so that you and the engineers could run the experiments. Now Claude runs the experiments, so actually let's hire way more people with, like, senior intuition than we did before, because we don't need to scale these or engineers around them.
So do you worry about a kind of reserve army of unemployed, recent college grads? Or is it that, as you were saying, like the data so far, it's like there seems to be a softening of hiring recent college grads or younger people into certain positions, but overall, the sector is growing.
Overall, the sector is growing, but the confounding factor here is that the software engineering sector is part of, like, the white-hot center of the American economy that's been growing more than anything else. So it's all confounded. But this question of pedagogy and skill formation is really important. How people get good at stuff is they start working on stuff. The implication of what I've talked about is we may have slightly fewer people starting work, and we may be taking more of the people who've been working. So we recently announced something for Anthropic called Claude Corps, where we're taking 1,000 early college graduates around America, and we're basically paying to embed them in nonprofits and other organizations and teaching them to help use AI skills to help those things improve their systems. And the idea here is twofold: One, proliferate the benefits of AI to organizations that might otherwise not access them; and two, give these people some experience. And I think from that, we will be able to run this natural experiment and see what happens in the economy.
Now, the second thing, which I mentioned, is almost contradictory. When I talk to many of my colleagues, they say, "I cannot reconcile the future economy with like the normalcy of today's economy. Everything I'm working on and the pace of the technology advancement seems to imply to me, a non-economist, you know, machine-learning researcher, the potential for my own unemployment or my own redundancy." And it's almost a belief that many of my colleagues hold. Either you could say this is just a belief they hold and it's wrong, or you could say these people have correctly called for the advance of a very, very profound technology. They have correctly called how far it would advance, how much more quickly than anyone expected, for multiple years. So maybe there's something to that as well. And so something I'm starting to do with governments is basically saying, "How are we prepared for unprecedented things to happen to the economy that are induced not by a recession, but by actually the arrival of, like, a new, very productive technology?" Because it's something we haven't dealt with so much.
Yeah. Although this is where, you know, I get, and I, you know, only time will tell. I remember reading an economic analysis of Elizabethan England where, because the, you know, early emanations, or premonitions, of the Industrial Revolution were happening and there was, you didn't need as many workers, and people were also aggregating in cities for the first time. And so Queen Elizabeth and King James did this too, where they were like, "OK, we're going to limit the amount of work that anybody can do, and you have to be an apprentice, and if you're an apprentice, to a tailor, you can only sew this many buttons, etc." And it never works. It just creates more distortion and creates a sluggish—
I am acutely conscious that everyone who predicts specific things about the shape of the future economy tends to be horribly embarrassed a few years down the line. All I'm saying, and what I say to governments is, "You should plan out for a world of many different scenarios. And I sort of expect that AI might yield more extreme scenarios than ones we've had in the past." Like, it might yield far above-trend GDP growth, and that GDP growth might be accompanied by a spike in unemployment that you typically only see during a recession. That'd be weird. And I actually just think governments generally maybe could do more work to invest in the scenario planning, including for the weird cases, so we know what to do.
And obviously, it's partly, it's, you know, what your job is right now is to be talking to governments, and also Anthropic is embroiled with governments in a very profound way. And I'm not saying this just out of pure libertarian cant. I say this as somebody who, you know, goes to the DMV and who, you know, deals with government bureaucracies and things like that in a place like New York, which has a mediocre, at best, government. I mean, what kind of faith do you have that governments, of all entities, are going to be able to be the backstop for any kind of social problems that come out of this?
I'm an optimist, and I don't think it's going to be every government. But I've interacted with government systems in America which have been good. You know, I paid my student loans off in England using a government system that was actually fine to use, to my surprise. I was like, "Oh, this is just like doing good online banking. Good job, guys." You know, you've been to Estonia, which has a highly digitized government that works extremely well. Governments can do this. Not all of them are going to, but some can, and we will work to try and find ways to help them do it.
All right. I think we will end it there. Jack Clark of Anthropic, thanks so much for talking to Reason.
Thanks very much.
