MARIA BUSTILLOS: So speaking of things that absolutely suck, let’s talk about ChatGPT.
DAVID ROTH: It’s hard to know where to start with this, but I think it’s good for me to get on the record and say that I hate this shit 1) in general and 2) in every single more particular example I’ve seen, at least where the text-generation/writing stuff is concerned.
I am aware that AI is apparently very good at reading x-rays and I think that’s super, but it’s also not really my lookout. And obviously the general objection to/distaste for AI-generated writing can be either principled or just grouchy. But line by line I think it’s undeniable that it sucks.
Maria: The loathing I feel for “generative AI” is right up there with “move fast and break things” and also the animated pancake Zuckerberg man of the “metaverse.”
David: Yeah, that comparison makes sense to me. With both of them they’re unpleasant and uncanny but also like…unclean. The soul recoils from it. A prospective future that is just right in your face about how little you will matter in it.
Maria: And then a bunch of giant dipwads go online and are all “Wow, this new technology is going to revolutionize media!”
David: “You can go into the computer for work. But no legs.”
Maria: Dead-eyed grinning like they’re either satanists in Hereditary, or Olive Garden servers grating cheese over your Never Ending Pasta Bowl.
David: These are people who are obviously not worth listening to in the least, but they also are both impossibly and immorally rich and whose whims either still or until very recently dictated what did and didn’t happen in a whole big sector of the economy.
But I think at this point it’s a pretty safe rule that if a personal friend of Elon Musk is trying to sell you on something, you just don’t make eye contact, say “sorry” and keep walking. They do not want anything good for you. There’s no reason to overthink it.
Maria: Do you sense this dichotomy, on the ChatGPT thing… like people are either 100% on board the suicide train of “this is going to replace Google and Everything Else” or, alternatively, gathering up pitchforks and torches?? (I am option “b” on this)
David: That’s a good question. I don’t really think any of the AI stuff works very well yet and most of the people pushing it are so obviously charlatans that I am skeptical of it just for that reason. But also I feel like you can sort of see things like Google (or Facebook, or social media in general) fracturing in these very serious ways that are unrelated to this, and it is not really clear that there’s much appetite to fix those things, either because those businesses are already built or just because it’s hard.
And there is this tide of sludge that’s always coming in and seeping through the cracks already. I want a site where I can do web searches that work, but if no one else is into it then maybe something this obviously bullshit could overtake it.
Maria: I was all set to ignore AI chatbots entirely, until this ghastly Tyler Cowen business? Because CJR’s Mathew Ingram noticed that guy committing the worst, most elementary journalistic sins regarding sourcing and accountability.
David: The Cowen story really got me.
Maria: To fill our readers in: Cowen is this high-profile libertarian economics prof at George Mason University—the kind of guy who goes on podcasts with Bari Weiss so they can tell each other how brilliant they are—and he has been a big cheerleader for ChatGPT.
David: Classic “healthy culture” stuff. This is all exactly what you look for, there.
Maria: So in February, on his blog, Cowen published a bunch of fake quotes from 17th-century polymath Francis Bacon, painting him as a critic of the printing press (which Bacon was not), and these fake quotes immediately found their way onto Google, whereupon they were returned as real Bacon quotes in Google search results. As of this writing, they’re still there:
A bunch of Cowen’s readers noticed this, including Ingram, and they figured the fake quotes must have been generated by ChatGPT. So that’s how I learned about it.
I emailed Cowen and asked him why he’d posted these fake Francis Bacon quotes—and under his own byline, no less. He shined me, but he also deleted his post, with no public explanation whatsoever; it’s still up at the Wayback Machine, though.
David: Again, just what you want if you’re drawing it up at home. Accountability, rigor, excellence.
Maria: I found this clown’s apparent refusal to own up or acknowledge his failure in the slightest so flabbergasting that I wound up interrogating ChatGPT myself about the whole fiasco, and I could not believe the torrent of lies that thing spewed up. (By now tons of people have reported on the same sort of thing—made-up citations, they’ve had fake articles attributed to them, and so on.)
I wrote about it at The Nation. And still, inexplicably, Tyler Cowen didn’t quit his job or change his name or get plastic surgery and move to like Bolivia.
[See Appendix I to this post: Knowledge Is Power and France Is Bacon]
David: It’s a perfect confluence of all the different types of cretinousness and recklessness in play here. That this absolute donkey is on a lifetime sinecure provided he doesn’t suddenly get into labor unions is perfect; that he’s enough of a chucklehead to look at “as Francis Bacon once asked, why do we park on the driveway and drive on the parkway” and then just run it up there in his post as fact is also perfect.
And that the only consequence is that it left a gross and untreated stain on the Google results for a long-dead philosophical titan feels… like vandalism, but also sort of perfect.
Maria: I’m tellin ya. This guy is a tenured prof.
David: Cowen’s job is to find new ways to say “actually, your boss is right,” so I wouldn’t think of him as someone who is going to know anything about Francis Bacon. I, as someone whose job is to find new ways to say “the Mets have made me upset again” am not that person either.
But he should for sure hold himself to a higher standard if he really wants to make use of this technology. He’s going to have to, if he’s going to be relying on an application that sometimes just fucking LIES when it doesn’t know an answer.
Maria: I was obsessed with the Essays, or Counsels, when I was a kid. This is partly why I noticed the Cowen mess in the first place, because I thought not a chance in hell Bacon, a man who loved books, would have written in opposition to the printing press; it really stuck out. Plus he was a super elegant prose stylist and the fakes were just the worst, globbiest argle-bargle.
David: I am also wondering why this [AI] technology is being talked about as the future when it is still in the “periodically making up weird shit” stage.
Is it going to grow out of that? Make up more convincing shit?
Maria: No!!
David: Obviously it’s never going to write like Francis Bacon. Cowen’s fabricated quotes were so cheesy. Just like bog-standard Reddit stuff. “As Sir Francis Bacon once said, ‘failure is just success that hasn’t happened yet, keep calm and chive on.’”
Maria: Srsly.
Real Bacon: Read not to contradict and confute, nor to believe and take for granted, nor to find talk and discourse, but to weigh and consider. Some books are to be tasted, others to be swallowed, and some few to be chewed and digested: that is, some books are to be read only in parts, others to be read, but not curiously, and some few to be read wholly, and with diligence and attention.
Fake Bacon: Lastly, we must use the help of printing rightly, and not suffer it to do us harm. And that is, by taking care that those things which are most solid and true, and of most weight and worth for the instruction of life and the increase of power, be printed in the best manner, and in the most correct editions; and that the rest either not be printed at all, or be printed more sparingly and more meanly.
David: So after you pointed out that he had credulously run up some nonsense that a bot handed him, Cowen just removed his post, never acknowledged it, and ignored you.
Maria: Yes. But the really harrowing thing is, all these apps like ChatGPT and Bard and LaMDA are right this minute belching out this cosmic firehose of bad writing and untruths and terrible machine attempts at “comedy.” This giant ocean of textual barf that is flooding all over.
David: Yeah this is something I was thinking about, after you sent me the examples of AI attributing a kids’ joke to me in a column I did not write [see BONUS post below]. Like how far downstream is this by now.
Is it being cited in high school term papers? Is it being handed in AS those term papers?
Maria: Oh yes, I think so. Because, I kept going… I couldn’t resist interrogating ChatGPT on all these different things, things I knew about, to test what it would do, and the results were basically some combination of gross, dumb, and fake. I couldn’t believe it. THEN I was shocked to the depths of my soul when it accused you and me, David, of making a desperately unfunny “joke”, about Jim Jones being a Wilco fan, in our review of The Master from years ago.
Literally, this.
[See Appendix II: Epic Flail]
It’s not just that that never happened, nor that it’s in poor taste or generally awful, it’s something neither of us would ever get within a mile of saying.
Anyway. The true extent of the debacle cannot be known.
David: This is the part where the haplessness of it at this moment starts to feel more menacing to me. How do you make any of this stop happening when it is built to just generate and generate. Which, like a lot of things here is the sort of metaphor for metastatic capitalism that would seem a bit too “on the nose” to use if it were not literally happening.
Maria: This is going to sound a little pollyannaish, but I am in hopes that real readers and writers can fix this mess.
David: Tell me why I should be hopeful. I would like to hear it.
Maria: OK, here’s the thing: I have yet to see a single paragraph written by an AI that didn’t just gnarl me out with its flatness, woodenness and lack of insight, humor, sensitivity, fun, or anything I like about writing.
David: Same. It’s awful and for all the ways it’s supposed to be improving at a geometric rate I have never seen one that was any different, let alone better. In a way it’s kind of heartening that it can’t convincingly sound like something that an actual writer wrote.
Maria: But it’s not just us, I know a lot of people who love and understand the good things about writing, they’re real things. I have loved your writing, for example, David, from that very first Puppy Bowl piece I read at the Awl.
Wit is absolutely not going to be possible to generate with brute-force computation. LLMs like ChatGPT literally just guess what is the most likely next word. That’s all they do, all they can do. They don’t—they can’t—judge or reason, they can’t surprise, they weren’t even meant to try. So the leap between “generating plausible-sounding language” and “sharing real ideas” isn’t just big, it’s unleapable, it’s not in the same universe.
Consequently, the real thing will remain obviously identifiable.
David: The connection that we had at The Awl wasn’t just you and me becoming friends because we loved each others’ blogs, but the way that commenters seemed to all kind of catch the same broader drift. There were a lot of different things in conversation there that I want to believe can’t be algorithmically replicated.
Maria: That’s right, the reason we all connected on there was that all of us wanted to know true things, and have fun, which machines can’t.
David: I think your point about the inability to capture the spirit of anything is good, and not just because it flatters me and my worldview. And I think that people still want that. I think, contra Jonah Peretti’s dreams of AI-generated listicles, that they want that even from content they don’t really put much thought into reading.
Maria: Yes.
David: Like I wouldn’t want an AI-generated post full of advice on what to do in Cincinnati over the weekend. Even if it was pulling from a robust pool of source text or whatever, I do not take restaurant advice from algorithms, that’s a pretty hard and fast rule for me.
Because I think the people doing this AI—my smartest friend from when I was little is in this business now—wouldn’t tell you that it’s going to replace anything. Right now they’re just trying to see what it can do.
Maria: Yes! I have a friend who works on it too, a natural language processing pioneer in California. A wonderful guy, a loon, also very brilliant, anyway I was yelling at him last week about how pitiful ChatGPT is.
And he goes, “I wouldn’t call it pitiful. What’s remarkable is how fast we’ve moved the goalposts. Two years ago the level of conversation was so poor that I wouldn’t even bother going back and forth [in a conversation] more than once or twice. But now it’s an infuriating idiot who only got into this discussion because his dad was rich and he skated through life on gentleman’s Cs. That’s progress!” (It’s not.)
I definitely have to go yell at him some more because it’s just… there’s no there there. The hype on this thing is so exactly like the bullshit of Elizabeth Holmes. Remember when all the real hematologists were like fuck you, that’s not how blood works, Elizabeth Holmes! But still, back then I knew a ton of really really smart people who were taken in by the magic one-drop blood machine, it was ridiculous, insane—
David: Yeah that Holmes comp is right on in that regard.
Maria: And I got in these huge arguments with people just like, “OK fine, but WHY do you think this works, what evidence is there?? Have you seen ANY evidence.” And like… nothing.
David: The people saying that it’s going to replace things like writing, or writers, are hustlers. In this case also hustlers who seem weirdly into the idea of replacing people with things, and especially into the idea of replacing writers with anything else.
Maria: The other thing it reminds me of is the scanning machines at the supermarket; they actually cost the supermarket a huge amount of money, I read somewhere, because the shrinkage goes up by a lot; people scan like a squash and then toss a filet steak into their shopping bags, I guess?
But more and more it strikes me that the scanners are there mainly to scare people into thinking they can be “replaced” by a machine, so they’ll be more docile.
David: This is a ridiculous thing but I feel like self-checkout is scabbing somehow. I hate it!
Maria: Same!! You can’t do pleasantries! Whatever is the point of buying Weetabix if you can’t joke around with the cashier??
Whatever, people still want to know true things and have some laughs, and that’s why Peretti should not have shut down BuzzFeed News. He should have read about loss leaders instead.
David: Also, while we both know that it isn’t exactly easy, it is possible to make money and build an audience with blog posts. Maybe or definitely not at the scale that venture investors demand, but also those demands are insane and those people should have nothing to do with this work.
I saw a bio for one of those Twitter Blue grindset guys today that said he was “scaling [the name of his athletic sock startup] to $100m.” Which made me laugh, but also that is like the minimum number that makes those guys pay attention, and not everything is that kind of business. Not every business should be.
Maria: That is so true it should be preserved in medieval calligraphy.
Another thing, I was just looking to see, you know… Twitter was more or less breaking even on revenues of like four and a half billion, before Musk saddled the company with all this debt. Thousands of people had jobs there, they were raising families, doing their thing… and breaking even, on that colossal sum, was somehow reckoned to be a huge business failure. Sustainability itself was unsustainable; that is, insufficient to appease the markets.
Public companies need to keep reporting growth, or else risk being taken over when their stock price drops. But logically, for people’s lives, for the quality of society, that formula no longer makes sense. For a lot of reasons unrelated to “growth”, Twitter was one of the most valuable media properties in the world until last November.
The structure of the public markets is responsible, but also one can’t help thinking that business is not real comfortable with and has never been conformable to the notion that people need and want information that money can’t control.
So yes, as you say, the only answer is for the people who produce media to own and operate the platforms we publish on. Though it’s very difficult, in the case of Popula and Brick House we can only afford to do this on a very small scale—and Defector should be a beacon to all, as it is to me.
The other option, which works in a number of different countries, is publicly-owned, state-funded media, but in the US that is a nonstarter, at the moment, because the fash are literally coming after even libraries now.
David: What I wonder about is where this ends. I agree with my whole being that business is never going to accept anything but control over the information it sells, whether that comes through hegemonic corporate-culture stuff like you see at the New York Times or something more oafish and overt.
But also it feels in some ways like the solution that a lot of the big and formerly big media companies are working on now is just… not providing information. Or doing less and less and less of it, cutting ten percent of the workforce every 18 months and seeing if somehow that makes things better. The business model is fucking Zeno’s Paradox.
And I don’t think that’s tenable, and I imagine they know it’s not tenable. You really can starve a newspaper to death.
Maria: So easily. How bitterly do I laugh to remember how certain I was, twenty-five years ago—when it was just Hampsterdance and a few loons on Compuserve and Prodigy—that the good outcome for the internet was inevitable, just because now anyone could talk with anyone, anywhere.
I completely underestimated the rapacity of the forces that were going to gang up and try to extinguish that first blinding flash of possibility. But I also believe we have another chance, now that we understand how much opposition there really still is to a free press, and a free, safe, democratic world.
David: And I think part of the reason that the authoritarian/anti-intellectual Silicon Valley types are enamored with AI is that they think you could just have a robot tell you what’s happening. No bias, no WOKE SHIT. Just the facts.
But if you want to actually know anything, if you want to read something that is worth reading, you still need a person to do that. You obviously need people for that.
Maria: And how!!! I just love when they yammer about “Truth,” it’s just so invincibly stupid. I really feel sorry for this man, Elon Musk, and for Tucker Carlson, whom I can’t see without remembering his bow tie and how Jon Stewart humiliated him on TV all those years ago. But. It’s not okay to call your thing “Truth” without evidence! ya dopes!
lol
David: Believing that you are the only rational being in the universe is such sad 15-year-old shit.
Maria: It’s so embarrassing!!
David: To become…A Mind Lord….the blade of reason must be kept sharp.
APPENDIX I
APPENDIX II
BONUS!!!!!!!!!!!!!!!!
