lnicola

@nixCraft@mastodon.social @alexmu@social.vivaldi.net Model distillation is not scrapping and your source understandably doesn't even mention the latter. Trying to equate them is bad reporting and muddies the waters. Scraped content can be protected by copyright, while LLM outputs aren't. Anthropic presents distillation as an "attack" because they're worried they'll fall behind the Chinese. This is in line with their previous policy (calling their model a "cyberweapon" and so on). They're fishing for regulation and protectionism.

@nixCraft@mastodon.social @alexmu@social.vivaldi.net So imagine you're a tech news tabloid using Claude to write articles. You want to save money so you ask Claude for 1000 topics from the past, then you ask GLM, DeepSeek and Kimi to write articles on them in your preferred style, and use Claude to rate which one is best. To Anthropic, this is indistinguishable from distillation, which they've just called an attack.

@alexmu@social.vivaldi.net @nixCraft@mastodon.social It doesn't involve scrapping copyrighted content (like downloading Harry Potter and training on it), it harms no-one because you're paying for your usage and you respect the rate limits (unlike hacking around OAuth to use your subscription with another harness, which is against their terms of use) so it's not an attack, and they can't even tell if you're doing something "wrong" (distilling a model) or something harmless like evaluating other models.

@alexmu@social.vivaldi.net @nixCraft@mastodon.social In addition, while scrapping is easy to define (exhaustively retrieving all known URLs from a server), I've shown that model distillation is indistinguishable from very common tasks like evaluating other models or take-home assignments. Scrapping has clear, direct, downsides (resource consumption), while distillation has none and is indistinguishable from a permitted workload. The only thing they have in common is what, doing remote calls to a server? 2/3

@alexmu@social.vivaldi.net @nixCraft@mastodon.social So if you feel they're the same thing, the onus is on you to explain why. They're different things with different purposes, working mechanisms and downsides. 3/3

@alexmu@social.vivaldi.net @nixCraft@mastodon.social Not sure if you saw my whole thread, but scrapping is bad mainly because of three reasons: 1. copyright concerns of the website owners, 2. unauthorized access to insecure web apps, 3. CPU and bandwidth consumption issues, especially with less optimized apps like Forgejo. 1 doesn't apply because Anthropic holds no copyright. 2 is not an issue because it's all authorized. 3 doesn't apply because you can distill a model at very reasonable rate limits. 1/3

@alexmu@social.vivaldi.net @nixCraft@mastodon.social I don't really see your point. Anthropic did not mention scrapping, so why would you, as a journalist, bring it up instead of using the correct term. It's like bringing up your neighbour's dog that bit you when reporting on an article about a new cat disease. Yes, cats and dogs can be pets, but there's no closer relationship. And if you think web scrapping is legal and almost harmless, may I refer you to all the complaints about "AI scraperd"?

@alexmu@social.vivaldi.net @nixCraft@mastodon.social I did not say it's illegal or against copyright, Like Cloudflare puts it, "content theft", "degraded site performance" (https://www.cloudflare.com/learning/ai/how-to-prevent-web-scraping/) It also "wastes application resources, skews analytics, compromises user accounts, and forces developers to build and maintain brittle, custom security logic" (https://www.cloudflare.com/products/bot-mitigation/) I think they're valid concerns, regardless of legality. None of these applies to Anthropic, so can you explain why scrapping is relevant at all?

@alexmu@social.vivaldi.net @nixCraft@mastodon.social Where does bias come into this? If it reported that Anthropic claims that usage indistinguishable from legitimate as an "attack", would it be biased for or against LLMs?