Talin

@ewhac Code is not an asset; code is a liability. Being able to spew out code faster means that you gain liabilities at an increased rate. I recently heard someone refer to the process of writing code as "enbugging".

@cstross I switched to Vivaldi a few months ago and I am pretty happy with it.

@cstross@wandering.shop LLMs are replacing every aspect of civilization - institutions, process knowledge, organizational competence - with civilization fan-fiction.

@Nickiquote "Great Men" are most often monsters.

@bendelarre I tend to think of AI as the new court astrologers - entities designed to tell the powerful what they want to hear. Court astrologers weren't stupid, they had math, even Galileo did it, and he's a hero of science. This doesn't make them any less wrong. Consider also that "plausibility" is what reinforcement learning trains AIs to do. Answers are evaluated not on truth or usefulness, but whether a human reviewer believed it. This makes the errors ever so much harder to spot.

@cstross@wandering.shop Because so many people boosted my post, I want to explore this topic in more detail. First, a thought experiment: suppose we had cheaply scalable mind control, even if partially effective (LLMs are not mind control but they touch some of the same wires). Democracy would end, since the controllers could sway elections at will. The so-called free market would instantly become a command economy. Science, and any other activity requiring independent thought, would be a dead letter.

@cstross@wandering.shop As LLMs improve, they will get better at chasing their fitness function, that is, at convincing people that what they say is true. This is what LLMs have in common with YGBM. Those who know history know that the invention of cheap printing sparked of centuries of bloody religious conflict. In this regard, LLMs are like "hold my beer".

@ewhac https://en.wikipedia.org/wiki/Lysenkoism

@cstross@wandering.shop In the book _Rainbows End_, Vinge proposes a form of mind control called "YGBM" technology, which stands for "You Gotta Believe Me", a way to get affected individuals to accept arbitrary propositions as truth. What this has in common with LLMs is technologically enhanced plausibility.

@cstross@wandering.shop Now, we know from the earliest days of machine learning that algorithms are capable of exploiting any loophole in their fitness function with seemingly godlike competence. Unfortunately, we can't make a fitness function for intelligence, truthfulness, or integrity. What we can do, at great expense, is make a fitness function for plausibility. When an LLM goes through its reinforcement learning, it's behavior is rewarded based on whether some human reviewer believed the result.

@cstross@wandering.shop Also, if someone were to write such a book I would absolutely buy it. There are vast troves of transgender fan-fiction online, a lot of it bad, and a surprising amount of MtF fiction (on places like nifty and fictionmania) is apparently written by subs (not judging!) for whom becoming a woman is the ultimate humiliation fantasy -- and has little relation to the experience of actual transgender people. I'd pay to read something that was actually good.

@kottke Those who DO read history are condemned to suffer from endless nostalgia.

@cstross@wandering.shop A modern version of this tale would have to take into account that the protagonist would be used as a political football by many different factions, both pro- and anti-transgender. As a general rule, transgender is hard to write about because even mundane details can read as porn. Although some authors have done it (Delaney, Banks, etc.)