The AI moratorium letter only fuels AI hype. It repeatedly presents speculative, futuristic risks, ignoring the version of the problems that are already harming people. It distracts from the real issues and makes it harder to address them. The letter has a containment mindset analogous to nuclear risk, but that’s a poor fit for AI. It plays right into the hands of the companies it seeks to regulate. By @sayashk@mastodon.social and me. https://aisnakeoil.substack.com/p/a-misleading-open-letter-about-sci
Remote
Arvind Narayanan
@randomwalker@mastodon.social
mastodon 4.8.0-nightly.2026-10-06I'm a computer science professor at Princeton. I write about AI hype & harms, tech platforms, algorithmic bias, and the surveillance economy.
I've been studying decentralized social media since the late 2000s, so I'm excited to use and write about Mastodon at the same time.
Check out this symposium on algorithmic amplification that I'm co-organizing: https://knightcolumbia.org/events/optimizing-for-what-algorithmic-amplification-and-society
10180 Followers
95 Following
30 Posts
Joined April 01, 2018
Substack: AI Snake Oil:
Book: Fairness and machine learning:
The "ChatGPT has a liberal bias" paper has at least 4 *independently* fatal flaws:
– Tested an older model, not ChatGPT.
– Used a trick prompt to bypass the fact that it actually refuses to opine on political q's.
– Order effect: flipping q's in the prompt changes bias from Democratic to Republican.
– The prompt is very long and seems to make the model simply forget what it's supposed to do.
By @sayashk@mastodon.social and me, summarizing our analysis and a separate one by Colin Fraser. https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias
Replying to
Responses to some frequent comments:
-
I'm certainly not suggesting that algorithmic feeds should be imposed on everyone! Choice is great. I recognize that many, perhaps most current Mastodon users like chronological feeds.
-
"Reverse chronological" is an algorithm, albeit a simple one. It's currently the only option. Chronological feeds are not normatively neutral. There is, unfortunately, no neutral way to design social media. https://mastodon.social/@randomwalker/109308664849924122
Replying to
Update: it turns out that lots of people have similar views and Simon willison is exploring building something along these lines.
https://fedi.simonwillison.net/@simon/109289663684761988
Replying to
- What do I mean by chronological feeds don't scale? A few things.
-
There's a lot of social pressure to follow people (especially people you know). Old-timers here are comfortable with following a small set of people, but most newcomers aren't. For those people, pretty soon the feed becomes a firehose.
-
Even in a mostly-chronological feed, some ranking would be really nice. Not necessarily by popularity, but if there are 15 posts by the same person I don't want those to be the first 15.
Replying to
Amusingly, this thread is getting boosted a lot today, a couple of weeks after I first posted it. That's another difference between chronological feeds and Twitter's algorithm (which heavily emphasizes immediacy).
But—and this is the point of the thread—imagine all the interesting things you could do with a tunable algorithm. You could even let the recency preference depend on the type of content! You could customize it to show you news only if recent, but educational content regardless of age.
Replying to
OpenAI mitigates ChatGPT’s biases using fine tuning and reinforcement learning. These methods affect only the model’s output, not its implicit biases (the stereotyped correlations that it's learned). Since implicit biases can manifest in countless ways, OpenAI is left playing whack-a-mole, reacting to examples posted on social media.
Replying to
- As an academic, I use(d) Twitter to keep track of new research that's relevant to my interests. I found this much easier to do after I (reluctantly) switched to the algorithmic feed. Again, I recognize that this might not be everyone's experience, but I know I'm not the only one.
Replying to
- "Mastodon doesn't need to become popular." Sure. But like it or not, it's getting more popular, and many of the newcomers have a different culture and expectations. Eugen Rochko: "People who are arriving now have as much right to be here and bring their own culture as the ones who came before them." https://mastodon.social/@Gargron/109323118267580967
Again, all I'm suggesting is choice, and I thought Mastodon is all about choice.
Anthropomorphizing AI is dangerous: it causes emotional harms and it can derail policy debates. AI developers and journalists need to stop enabling this tendency, and we need research on how people interact with chatbots to create better guardrails. We also come up with a more nuanced message than “don’t anthropomorphize AI”. Perhaps the term anthropomorphize is so broad and vague that it has lost its usefulness when it comes to generative AI. https://aisnakeoil.substack.com/p/people-keep-anthropomorphizing-ai By @sayashk@mastodon.social and me.
Replying to
We looked at > 100 papers on flaws of specific predictive optimization applications. Seven recurring clusters emerged. Given the technical architecture of predictive optimization (vs other forms of automated decision making) we show that this is exactly what we should expect. That made us ask: does *every* predictive optimization application have these flaws? We picked 8 case studies based on prevalence and impact on people. Among these, the answer seems to be yes. https://predictive-optimization.cs.princeton.edu/
Replying to
The key move in our paper is defining the category of predictive optimization and showing what's wrong with it. Contesting these applications one by one (criminal risk prediction, resume screening, ...) leave us playing whack-a-mole as they proliferate (we counted about 50, probably missed many more). On the other hand, critiques that apply to *all* automated decision making are too broad to effectively challenge any particular system. We offer a middle ground. https://predictive-optimization.cs.princeton.edu/
Kirkus Reviews, which provides early book reviews to the publishing industry, has given AI Snake Oil a very positive "starred" review, which we're told is rare and kind of a big deal. Honored and grateful! https://kirkusreviews.com/book-reviews/arvind-narayanan/ai-snake-oil/
Preorder:
https://www.amazon.com/Snake-Oil-Artificial-Intelligence-Difference/dp/069124913X
https://bookshop.org/p/books/ai-snake-oil-what-artificial-intelligence-can-do-what-it-can-t-and-how-to-tell-the-difference-arvind-narayanan/21324674
More preorder links at the bottom of this post
https://www.aisnakeoil.com/p/ai-snake-oil-is-now-available-to
Coauthored by @sayashk@mastodon.social, published by @princetonupress@mastodon.social.
Replying to
This is the latest in the AI Snake Oil book blog by @sayashk and me. Writing this blog alongside the book has been really fun. I'll probably do something like this for all future books! Thank you to everyone who subscribed. https://aisnakeoil.substack.com/
Replying to
These flaws don’t have tech fixes. So predictive optimization *fails on its own terms*. Any one of these flaws should make us question a decision making system. Together, they severely challenge legitimacy of predictive optimization applications.
We present a rubric with 27 questions for activists to contest these systems. But the burden should be on those who exercise power to justify their use. To this end, our rubric can also help developers analyze their applications. https://predictive-optimization.cs.princeton.edu/rubric.pdf
Replying to
Given the harms of these decision making systems, this feels like the most important topic I’ve ever worked on. Of course, the paper’s impact remains to be seen. We’ll continue to workshop it for a few months before finalizing it. We would be grateful for feedback. 🙏
We're also very interested in expanding our inventory of predictive optimization systems (currently around 50 https://docs.google.com/spreadsheets/d/1eOIKLbT7sCIe0OMHiA6VNPmhqcpurF4f8qSJ1Dbnqro/edit?usp=sharing) If you know of any we've missed, please let us know. https://predictive-optimization.cs.princeton.edu/
At @knightcolumbia@mastodon.online I'm co-organizing what I think is the first symposium on the topic of algorithmic amplification on social media — April 28/29 (NYC and online). I'm told we have over 600 registrations already. We'll be moving to a waitlist for in-person participation soon. Register to hear from ~30 leading thinkers on the topic: https://knightcolumbia.org/events/optimizing-for-what-algorithmic-amplification-and-society
We're at over 1,000 registrations for the @knightcolumbia@mastodon.online algorithmic amplification symposium this Friday & Saturday. We're lucky to have an all-star cast of speakers. In-person registration is closed/waitlisted, but you can still register to attend online: https://knightcolumbia.org/events/optimizing-for-what-algorithmic-amplification-and-society
Replying to
OpenAI responded to the criticism by saying they'll allow researchers access to Codex. But the application process is opaque: researchers need to fill out a form, and the company decides who gets approved. It is not clear who counts as a researcher, how long they need to wait, or how many people will be approved. Most importantly, Codex is only available through the researcher program “for a limited period of time” (exactly how long is unknown).
https://aisnakeoil.substack.com/p/openais-policies-hinder-reproducible
Replying to
I plan to publish two follow-up essays in a few months. I’m super grateful to
@knightcolumbia@mastodon.online for the opportunity — this type of writing is hard to do because the traditional paper publication route isn’t available. I enjoyed writing this and I hope you enjoy reading it!
Replying to
I introduce three stylized models of information propagation (subscription, network, algorithmic) and discuss how virality operates. Turning to algorithmic recommendations, I give a bit of history, an overview of the main modern ideas, and a case study of Facebook's Meaningful Social Interaction metric. But the major platforms’ algorithms are far more similar than they are different — they’re all flavors of engagement optimization — a point I previously made about TikTok. https://knightcolumbia.org/blog/tiktoks-secret-sauce
Replying to
@nafnlaus Fair point about the name. We use it because it has some history behind it (https://aisnakeoil.substack.com/p/introducing-the-ai-snake-oil-book) and is a bit of a recognized brand at this point, which might be helpful when our book comes out. But perhaps we're weighted down by that baggage. We'll discuss it. Thanks! CC @sayashk
Replying to
@tg9541 The three classes are automated decision making about people, ML, and prediction. Simulation and existing rules are explicitly not predictive optimization. Explained in detail in Section 2.1 and Table 1 of the paper; hope that helps!
Replying to
@tuliotec@mastodon.social With modern mobile OSes, surreptitious eye tracking is not technically possible. It's also a legal risk that IMO outweighs the benefits.
But face analysis has been used for recommendations https://twitter.com/MarcFaddoul/status/1232014908536938498
Other biometrics like gait and activity recognition are also used, but not for recommendations AFAIK.
Use of location data is ubiquitous, of course.
Hope that helps!