I actually have the tech to make website screenshots in the search engine dismiss their GDPR notices, but I have zero inclination to do so.
Remote
Marginalia
@marginalia@mastodon.social
mastodon 4.8.0-nightly.2026-10-06I've built the indie internet #searchengine #marginaliasearch. Working full time on this project. Kind of a librarian of the weird corners of the web.
1605 Followers
8 Following
39 Posts
Joined March 31, 2023
Website, blog, etc.:
Search Engine:
Encyclopedia:
I wrote a blog post, about the arduous joy of learning new things.
SIMD intrinsics are feeling more like a targeted accessibility hate crime the more I have to use them. 😅
The way I manage my email inbox is that I wait until it's so full of emails I should respond it makes me want to curl up in a ball and cry, a process which usually takes about a week.
Replying to
@ml@mas.to Haha, well, I'm kinda doing that, except I look at the number of requests to known adtech websites that they send instead.
There's even a nifty viewer thing for the data.
Did some statistics, and it seems posts from newly registered accounts on HN are nearly 10x more likely to use EM-dashes, arrows, and similar typography than established accounts.
Search engine is going offline for a short bit in solidarity with GitHub.
(also for a bit of server maintenance).
I low key want to turn my website into Myst, but the puzzles are also there as CAPTCHAS, to keep the AI scrapers out.
You want to read my recipe for chicken soup do you, well I hope you've made some coffee and brought your notebook and remembered to write down the hint in in the margins of the page about my cool stick collection.
All you need to understand the state of LinkedIn and HN the last few months can be found in this openclaw template:
New blog post, on the meandering route to getting a somewhat viable NSFW filter in Marginalia Search.
LinkedIn Park - In Hindsight It Doesn't Even Affect Our KPIs.
This will be very funny to a small number of people.
Ethics in AI is a trolley problem where you don't know who is tied to what track, and the AI CEOs make ridiculous claims that everyone in the world is tied to one of the tracks and you'd better switch onto that track because the tracks and the trolley are about to turn into Voltron and go pew-pew-pew and then fly to the moon and bounce it like a basketball and shoot lasers and blow up the sun and walk away from the explosion without looking in slow motion looking *awesome*.
I know I was gonna take time off but I couldn't help myself, got inspired and started working on an overhaul of the site viewer. Plan is to add more functionality for showing recent updates to a website, with inline back-links.
Very early mockups with contrast issues:
So I had to disable Anubis a while back because it was too much of an accessibility issue for browsers with no JS support.
Now I've deployed a query-rate based mitigation that makes you wait for a few seconds before you can proceed if the query rate is too high.
A bit annoying, but it works with any browser and will disable itself when the bots go away.
If programmer speed and efficiency was truly such a significant competitive factor, we wouldn't be packing them like sardines in noisy and stuffy open floor plan offices.
Replying to
@GroupNebula563@mastodon.social @Haste@mastodon.social FWIW the idea is pretty good for getting rid of bots.
I wouldn't try to block a bot or reject connections, that looks like something to be circumvented, an indicator for the scrapers to use a residental proxy and pretend to be chrome;' it's much better to serve a "Welcome to nginx!"-page or similar.
Replying to
@ploum@mamot.fr Biggest victim so far of AI is arguably online discussion, especially programming discussion. It's a real double whammy where on the one hand you have the bots butting in with non-sequitur insights, but it seems like anyone who drinks the AI Kool-Aid completely loses the ability to have interesting ideas or build interesting projects, likely because those things come as a result of the work that the LLM is now doing. I've said it before, but AI above anything else makes you boring.
The word processor and its consequences have been a disaster for humanity
Did some local LLM-based labeling of the HN comment corpus, tasking the model to classify how likely it is that each comment is:
* Using AI-like grammar (e.g. it's not X, it's Y)
* Using AI-like markup (e.g. em-dashes)
* Trying to shill something
* Trying to influence public opinion
Still a bit work in progress, but here are some preview data.
Which one of these would you say is correct?
Argument for "is": In English "data" is treated as an uncountable noun.
Argument for "are": In Latin, "data" is the plural of "datum".
Working on a NSFW filter and trying to tune it to not be an erasure machine as there is some vocabulary overlap between the contexts of porn and queer culture.
Question for the LGBT-community:
Wth are the "bisexual papers"?
Stumbled upon this (according to the classifier) ambiguously NSFW bigram that if nothing else sounds like something someone leaked, revealing some dark secret. Have I... dug too deep? Seen things not meant for my eyes?
(Screenshot is NSFW in the most literal sense)
Replying to
@ploum The fact that Anthropic is eyeing an IPO in 2026 is really all you need to know to understand their recent theatrics.
> [...] in that empty space, I begin to wonder: Where did all the people go? What are they all doing behind the artifice they’re showing here? Why am I wasting my time and energy wading through these shallow yet unbounded seas of nothing?
Have very much been feeling this bit lately. The social web is an evaporating ocean, leaving only an inhospitable salty death zone full of bots, shills, and influence campaigns, immitating the people that used to be around.
Replying to
@stepheneb@ruby.social
Yes! There is so much that happens when you learn new skills. You discover things about your own potential and taste, but there's also perceptual and conceptual changes that can be downright trippy to experience.
Replying to
@ploum@mamot.fr It's insane how bad it's gotten over the course of the last few months. You also see a huge amount of accounts that are very obviously run by AI agents in almost every thread, often drowning out any real discussion.
I've almost completely abandoned ship for lobste.rs, which suffers from the same tedious AI discussion being rehashed ad nauseum, but you can at least filter it out, and the participants still seem mostly like human beings.
Replying to
@drwho@masto.hackers.town @zwol@masto.hackers.town @reykjalin@social.treehouse.systems @b0rk@social.jvns.ca Though that last bit was largely considered a dead end, it was neat that it was possible to express queries in such a way but it didn't really help in any meaningful way (because the "embedding function" was just a tokenizer).
Replying to
@drwho@masto.hackers.town @zwol@masto.hackers.town @reykjalin@social.treehouse.systems @b0rk@social.jvns.ca
Vector search is as good as the vector representation you're using, that's the bit doing the magic. Something like HNSW makes it viable at scale, but that's just a vector index. It's the vector transformation that matters.
It's conceptually much older than the models we're using today. It was even fashionable for classic keyword based search as well (where you translate each term to a 1 bit index and do binary cosine similariy = bitwise AND + popcnt).
Replying to
@zwol@masto.hackers.town @reykjalin@social.treehouse.systems @b0rk@social.jvns.ca Seems fairly similar to BERT, which is basically a predecessor to modern LLMs. Unsure of how much data you'd need to train something like that, transformer models in general are very resource hungry for training so they're kinda beyond what you can play with without datacenter GPU access.
Bardiche man dreams of that time he scored four touch downs in a single game of peasantball.
Unicorn swears this has never happened before.
Cat has PTSD flash backs to the dog riots of 1208.
Putting this on my fridge.
Been replacing the graph representation for the link graph with a CSR implementation. It sure is a lot better than what I was doing before.
Biggest improvement to date is the website similarity calculation, which used to take literal days to finish, and now finishes in an hour.
Memory locality is pretty good.
Replying to
Replying to
@nikolasdi@mastodon.social Did Latin and antique philosophy a few years back. Neat cocktail of influences.
Replying to
@ploum I think many of these behavior addictions are generally closer to maladaptive soothing behaviors.
We learn to grab the phone when we feel discomfort, because it makes us feel less uncomfortable.
This quickly becomes a loop as it all but ensures we don't constructively address whatever discomforts or anxieties we have, which means we have more of that, and engage in more of the soothing behavior.
Replying to
@chatcontrol
Everything in the European Union, nothing outside the European Union, nothing against the European Union.
Il Duce and Stalin both would be applaud this utter obliteration of the very concept of privacy.
Replying to
@zwol@masto.hackers.town @drwho@masto.hackers.town @reykjalin@social.treehouse.systems @b0rk@social.jvns.ca It's a neat way of expressing a query mathematically, but it's not actually very good or useful. Though it's an interesting historical footnote, as it did kind of predict vector search 40 years before the fact.