Pie thieves
AI CEOs want to take your stuff, a horse "rides" an astronaut, age verification part deux
A blessed day to you! Welcome to another issue of Endnotes. I just want to say that it feels like things are rapidly coming to a head. The drumbeat of negative stories and reports on AI companies is relentless. I am actually having a hard time keeping up, there are so many. Reality doesn't matter, so who knows what will actually happen, or when, but it feels like... something. Anyway, the Nasdaq keeps making new highs. Melt-up summer, baby! I am excited to find out if OpenAI makes it to IPO in Q3. In this issue:
- Where the AI revenue comes from
- What's a synonym for "ride"?
- John Oliver on AI chatbots
- Age verification: Still good, actually
- Links
Yell at me over on Mastodon if you want. Have a wonderful time, my dears!
Where the AI revenue comes from
There's a shocking detail in this story from The Verge about how the free ride for users is over because AI companies need to start earning revenue off their massive capital investments:
To avoid a write-down of these assets, major AI model providers would ideally generate a return on invested capital (ROIC) of about 25 percent, Sommer said. (That’s about what Amazon, Microsoft, and Google tend to earn on their overall capital investments.) On the other hand, if the returns fall below 12 percent, institutional capital loses interest — there’s better money elsewhere, Sommer said. Below 7 percent, you’re in write-down territory, which is “an unmitigated disaster for all of the investors in this technology,” Sommer said.
To reach that bare minimum of 7 percent, Gartner forecasts that large AI companies would need to earn cumulatively close to $7 trillion in AI-driven revenue through 2029, which is close to $2 trillion per year by the end of the period. In order to achieve “historic returns,” the providers would need to earn nearly $8.2 trillion in the same period. (emphasis mine)
Seven trillion dollars is an insane amount of revenue. To put it in perspective, in 2025, Meta ($200b), Alphabet ($114b), Apple ($416b), and Microsoft ($281b) combined took in $1.23 trillion in revenue, whereas to break even, generative AI investments would have to make $2 trillion just in 2029.
But let's play along for a second. Let's say they pull it off and generate $7 trillion in revenue. Where would it come from? Under what scenario are American corporations going to suddenly pay AI companies $2 trillion a year for their services?
There are two options.
One is that AI makes the pie bigger and everyone gets richer. That is, firms pay AI service providers $2 trillion/year because AI makes them more productive and therefore more profitable. Unfortunately, as far as anyone can tell, that's just not happening.
The second is that the pie doesn't get bigger and AI companies take someone else's piece, to the tune of $2 trillion. And that someone? You. They want to take it from you, a worker. They want your employer to turn your salary into tokens, replace you with Claude.
That's what Sam Altman and Dario Amodei want. That's what Satya Nadella and Elon Musk and Marc Andreeson and Andy Jassy and Jensen Huang and Mark Zuckerberg want. Explicitly. They want to take away your rent money. They want to take food out of the mouths of your children. They want to take away vacations and new clothes from your family so they can add more lucre to the huge, stinking piles they sleep on every night in their exquisite caves, deep inside mountains, dreaming slimy dreams.
To be clear, I don't think it will work. I don't think you can replace human workers with linear algebra, and I expect this whole thing to end badly.
But I also think we should remember what they want to do to us, and to our families and friends. They want to grind us up and turn us into food for their pets, fertilizer for their landscaping. We should never forget that.
What's a synonym for "ride"?
There's an old Gary Marcus post from 2022 about how AI image generators can't really understand anything and can't produce anything that's not already in their training data. As an example, he prompts for a horse riding an astronaut. Inevitably, the models produce an astronaut riding a horse.
It's a way of illustrating that the models don't "understand" anything, they are just doing mathematical calculations on language. They see "horse" adjacent to "astronaut" and "ride". Almost no one draws horses riding people, so math produces the statistically-likely result from the training data, which is a person riding a horse:
I like to try this out with image generation models from time to time to see how it's going, so I tried it with Google's Nano Banana recently and OH NO!!
Technically correct! That horse is, in fact, "riding" the astronaut.
John Oliver on AI Chatbots
"Last Week Tonight" did their main segment this week on AI chatbots. It's good!
Age verification: Still good, actually
I want to say a bit more about age verification for apps and websites. In case you missed it, the EU released an open-source app that member countries can use to implement age verification requirements. I think it's a good thing, actually.
The immediate push-back I got on Mastodon was largely from Americans. That makes sense, as I'm an American and lots of my online social circle is American, but I live in Europe and the app is for EU countries. It's a completely different social and political context.
There is no Heritage Foundation here, pushing to use age verification as a Trojan horse to censor sex work and LGBTQ+ topics. There is no Citizens United, allowing corporations to spend unlimited amounts of money on elections. And the legislative process remains intact: Across the union, governments are formed and dissolved and reformed, parties come and go, laws are passed and amended and amended and amended again.
The app released by the EU is meant to be a tool that solves the technical problem of "how to verify someone's age without violating their privacy," as the right to privacy and data protection are enshrined by law in EU countries. With the technical problem solved, it is up to EU countries to decide how (or if) they want to implement age verification, and that will be done through each country's democratic lawmaking process.
I explain all this because I think Americans have given up on the idea that government can do anything good or beneficial for the right purposes, whereas Europeans have not. While messy and imperfect, EU countries still have regulatory states, and I think it is good they are finally taking a crack at regulating US platforms that have done a significant amount of harm around the world, not just to children but to everyone.
I don't have a strong opinion on what age verification rules should look like, but with a technical tool available that makes it possible to verify someone's age without violating their privacy, I think it opens up a lot of positive possibilities. For example, rather than banning children from using platforms like Instagram or Snapchat, you could ban the platforms from doing the kind of surveillance of underage users that they do on adults. Or someone could create an app that only verified children are allowed to use. It could be an opportunity to create more and safer spaces for children, rather than keep them out of existing spaces.
Platforms also lose plausible deniability: "We didn't know that was a child we were using in our A/B testing on suicidal thoughts," uh, yeah you did. You now absolutely know who is a child, and are required to treat those users differently.
I know that a lot of hired guns are involved in lobbying around age verification, but that doesn't mean tech companies want it to happen. It means they don't think they can stop it, and they want to control the process. They would love it if it never happened at all, but since it will, they want to stay involved and poison it from within. And to do that, they would prefer that the concerned, educated, knowledgeable people like you, my dear readers, abandon the field. Don't do them the favor.
Links
- Here's a good blog post about quitting your job because "AI" turned it into a soulless grind. Many such cases!
- Maine Governor Janet Mills, who is currently running for Senate, vetoed the data center moratorium passed last week by the legislature. Good luck on your Senate run, ma'am!
- The Dutch central bank is switching its cloud computing infra from AWS to Lidl (their cloud is called STACKIT). Seems like it's part of a larger deal with the Dutch government.
- As reported by Ed Zitron a week ago and ignored by everyone, GitHub Copilot is moving to usage-based billing. Of course, people are melting down.
- From Microsoft researchers: "LLMs corrupt your documents when you delegate." They found that frontier models "corrupt an average of 25% of document content by the end of long workflows."
- A guy's AI-powered coding agent deleted all his app's data and the backup. Then he asked Claude to explain itself and posted the agent's "confession", indicating that he actually has no idea how any of this works. Here is his full post on X.