Greg K-H

@hanno It is real, see my interview in the Register today about this very problem: https://www.theregister.com/2026/03/26/greg_kroahhartman_ai_kernel/

@joshbressers @deftpunk @wdormann @Viss the "announcement of a public web site and exploit" was not sent to the kernel security team. If you look at the timeline they published, they show what they sent the kernel security team and when, which seems to be correct to me.

@brauner@mastodon.social @axboe@fosstodon.org You most certianly deserve a crate of whatever you want!

@aho maybe, but at least one reporter insisted it wasn't LLM generated, which of course does not actually make it true, and pointed instead at a 10 year old presentation that they "happened" to find.

@corsac@mastodon.social @joshbressers@infosec.exchange @wdormann@infosec.exchange @Viss@mastodon.social Linux makes it very "easy", just update your kernel to the newest version. What's preventing that from happening for your systems?

@uecker@mastodon.social @icing@chaos.social @joshbressers@infosec.exchange @wdormann@infosec.exchange @Viss@mastodon.social There was no "embargo time". And again, Linux does not notify anyone because if we did, we would have to notify everyone. It's as if no one reads my long posts about this topic explaining it all...

@zmanion@infosec.exchange @joshbressers@infosec.exchange @wdormann@infosec.exchange @Viss@mastodon.social Why is linux-distros somehow "special" enough to get these types of announcements and not everyone else? How exactly would you explain that to your favorite government entity?

@joshbressers@infosec.exchange @deftpunk@fosstodon.org @wdormann@infosec.exchange @Viss@mastodon.social Honestly, there was nothing "obvious" about this one being a "big one" compared to all of the bugs we get, and fix, on a daily/weekly basis in the kernel. The ONLY thing different here from those bugfixes, was that someone made a web site, a simple reproducer, and announced it to the world. For 99.9% of the bugs we fix, that are reproducible like this, no one ever does that. That we know of... In other words, this was just another Tuesday for us.

@joshbressers @wdormann @deftpunk @Viss What do you mean, they told us, we fixed it, it got in some stable kernels, and so our work on the security team was done. The CVE team assigned a CVE after a while, and even gave it a CVSS score. The fact that no distro popped up that used older kernel versions to do the real work to backport to older kernels seems to be everyone's major problem here. That is outside of the kernel security team's work entirely. So take it up with the distros that people are paying support for to do this for them? And yes, Debian was vulnerable, that is not good, and once it was noticed people worked hard and quickly to fix that. Not bad for a community-based distro that no one pays for in my opinion.

@wdormann@infosec.exchange @joshbressers@infosec.exchange @deftpunk@fosstodon.org @Viss@mastodon.social Not ALL of the distros are on linux-distros. So that is one thing. The other being that I don't care what happens on linux-distros, for many public reasons I refuse to deal with them anymore, and strongly encourage no one else to do so either.

@deftpunk @joshbressers @wdormann @Viss no one did contact the kernel security team before they announced this. It was nice enough that they sent us a bug report and we got it fixed and pushed out to the latest stable kernel releases. That's all I can ever hope for.

@warthog9@social.afront.org @argv_minus_one@mastodon.sdf.org There should be fixes out for "everyone", that is what I got working yesterday morning. And yes, this is on the reporter, there's nothing the kernel security team, or kernel CNA can do differently here, sorry.

@wdormann @joshbressers @Viss I love it how people think that "coordination of vulnerabilities" is actually something that can be done these days. Think of just who uses the software in question, and who should, and should not, be on such a list to get a "early disclosure notification". As I have said for quite some time now, all early-disclosure lists are leaks, otherwise why would your government allow them to be in existence? Software, and specifically open source software, runs the world. So should the whole world be on that notification list? :)

@chinmay@social.treehouse.systems @wilfredh@mastodon.social So, "if a new option was added to this tool" kind of doesn't solve the issue today, or even tomorrow, if no one is adding such an option. Are you suggesting I do that? That's fine if you are, maybe that's the best end result, don't know, just want to make sure I'm not missing something already out there today.

@richlv @hanno Given that they have not been "found" yet by other tools that we know of, I would guess no. It's just "fuzzy pattern matching" which to be fair, is what LLMs are actually good at doing.

@samuel@social.meenzen.net "CVEmaxxing", if you don't mind, I'm going to steal that for my next talk! And the Google/Microsoft codebases are for _different_ products from those vendors, not just a single codebase, so you can't really compare them that way at all. Look at the product if you wish to compare for products. For products, our numbers are way way higher because most commercial vendors do not report all CVEs, only the "high" ones.

I can't resist keyboards in "odd" formats...

Here is a hopefully-useful notice about Linux kernel security issues, as it seems like this knowledge isn't distributed very widely based on the number of emails I get on a weekly basis:

- The kernel security team does not have any "early notice"
announcement list for security fixes for anyone, as that would only
make things more insecure for everyone.

- The kernel community does not assign CVEs, nor do we deal with them
at all. This is documented in the kernel's security policy, yet we
still have a number of people asking for CVE numbers even after
reading that policy. See my longer "CVEs are dead..." talk for full
details about how the CVE process is broken for projects like Linux:
https://kernel-recipes.org/en/2019/talks/cves-are-dead-long-live-the-cve/

- You HAVE to take all of the stable/LTS releases in order to have a
secure and stable system. If you attempt to cherry-pick random
patches you will NOT fix all of the known, and unknown, problems,
but rather you will end up with a potentially more insecure system,
and one that contains known bugs. Reliance on an "enterprise"
distribution to provide this for your systems is up to you, discuss
it with them as to how they achieve this result as this is what you
are paying for. If you aren't paying for it, just use Debian, they
know what they are doing and track the stable kernels and have a
larger installed base than any other Linux distro. For embedded,
use Yocto, they track the stable releases, or keep your own
buildroot-based system up to date with the new releases.

- Test all stable/LTS releases on your workload and hardware before
putting the kernel into "production" as everyone runs a different %
of the kernel source code from everyone else (servers run about
1.5mil lines of code, embedded runs about 3.5mil lines of code, your
mileage will vary). If you can't test releases before moving them
into production, you might want to solve that problem first.

- A fix for a known bug is better than the potential of a fix causing a
future problem as future problems, when found, will be fixed then.

I think I need to give another talk about this issue to go into the above in more detail. So much for me giving a technical talk at Kernel Recipes this year...

Dear semi-lazyweb,

Given a git diff of a C/Rust codebase, how to best determine which functions/defines have been modified between the two versions? Yes, the diff itself sometimes gives hints as to what has changed, but it's not always correct. Think about when it modifies the start of a function, but the diffstat "name" shows the previous function, a correct marking, but not what is needed.

Is the correct answer really going to be "compile the two versions and compare the AST" or something like that? No "diff library" somewhere that "knows" how to parse C (and Rust) that can do this in a faster way? Surely I'm missing something obvious here...

Posting this link here, as I always have to dig every few years when I need it: https://cdecl.org/ a C -> English translator for those "fun" const pointer to const array issues that you have to work out every so often...

@chinmay@social.treehouse.systems @wilfredh@mastodon.social difftastic is great, but a visual tool, not something that can be hooked up into a script to generate the list of modified symbols as far as I can tell. But again, I might be missing an option there, am I?

In a few minutes I get interviewed by Shuah Khan and might answer questions from the audience if we have time: https://www.linuxfoundation.org/webinars/lf-live-maintainer-series-my-life-as-a-linux-kernel-developer-and-maintainer-with-greg-kh-and-shuah-khan

It will be recorded for playback later as well. It's part of the great Mentorship video series that Shuah has been putting on for years, the back catalog is deep: https://events.linuxfoundation.org/lf-live-mentorship-series/

@warthog9@social.afront.org @argv_minus_one@mastodon.sdf.org Heck, this was even one of the very few CVE entries that we actually scored, giving it a 7.8 on the CVSS scale. We rarely score these things, but when we do, perhaps people should actually pay attention? Makes me wonder why we even do this sometimes... {sigh}

@uecker@mastodon.social @icing@chaos.social There are many reasons why this would not work. Again, step through the logic to prove it yourself.

My seat name tag for the EU CRA meeting today...

We've gotten five different "security reports" about the decades old USBIP protocol https://docs.kernel.org/usb/usbip_protocol.html and how it is "insecure" in the past few days.

Yes, it's only to be run between "trusted" devices, and we will gladly take patches so see the ones recently posted to the linux-usb mailing list to mitigate these issues, but this is very strange as to why all of a sudden this is being reported all at the same time by random different semi-anonymous accounts.

Is there some big usb-over-ip installation somewhere that people suddenly started caring about out there, or did some internal hacking tool that uses usbip just get leaked?

No one who we asked "why?" when they submitting these issues would give a very clear answer to that simple question so something is going on...

@joshbressers@infosec.exchange I will quote this in many presentations in the future because it is so true:

"The Kernel assigns lots of CVEs. They say it’s because they don’t really know how the Kernel is being used, so they err on the side of caution. Companies hate this because they have to deal with a lot of CVEs. Does the Kernel do this because it’s easier or do they have some sort of secret nefarious reason? Probably because it’s just easier and they have zero downside to disclosing and moving on. "

RE: @joshbressers@infosec.exchange

Starting to write up a series of articles about the Linux kernel CVE work that has happened in the past 2 years, starting with some "back to basics" information about how Linux kernels are numbered as many people/companies really don't know how we do this, and it matters a lot in tracking bugfixes and how to determine "vulnerable" and "fixed" kernel releases:
http://www.kroah.com/log/blog/2025/12/08/linux-cves-more-than-you-ever-wanted-to-know/
and
http://www.kroah.com/log/blog/2025/12/09/linux-kernel-version-numbers/

@warthog9@social.afront.org @argv_minus_one@mastodon.sdf.org I’d argue this not having a broader security push before the public release happened, is a pretty serious failure on someone’s part. And who is that “someone”? We fix bugs like this in the kernel on a daily basis. If people have not learned to constantly upgrade to stay ahead of this, then why even assign these 10 CVEs a day in the first place? :)

@uecker@mastodon.social @icing@chaos.social As is pointed, out, this is just a troll, but seriously, "worthy" isn't the issue. Again, you can not have one group "in" and one "out" without real reasons why anyone is "out". And again, my point remains, "All early release lists leak like a sieve, otherwise why does your government allow it to exist."

@joshbressers@infosec.exchange @deftpunk@fosstodon.org @wdormann@infosec.exchange @Viss@mastodon.social Loads of them.

@mei Not all that much, "best effort" is fine.

@lesto@hachyderm.io Yes to the first, and for the second "you can always change the code to adapt over time". Make it work now, don't worry too much about the future as you don't know what it will hold.

After talking with a bunch of different companies / groups, we've now bumped the length of a few of the longterm kernels we are supporting:
https://git.kernel.org/pub/scm/docs/kernel/website.git/commit/?id=d04587da86a3464881e0c97aabddd2c271105698

As always, the dates can be found at:https://www.kernel.org/category/releases.html

@jlhertel@fosstodon.org That doesn't seem to actually provide the "what function/symbol is modified" logic, nor would I expect it to, unless I am missing something obvious?

@uecker@mastodon.social @icing@chaos.social @joshbressers@infosec.exchange @wdormann@infosec.exchange @Viss@mastodon.social Why is it unconvincing? Who decides what group is on,or is not on, such a list? Your government? My governments? Their government? No government? Me? You? Someone else? And what is the criteria exactly for how? See how it breaks down when it hits the real world? As I have said many times, "All early-announce lists are a leak, otherwise why would your government allow it to exist?"

@penguin42 @deftpunk @joshbressers @wdormann @Viss I honestly don't remember, and if I did, we don't publish who asked for CVE ids from us as that's generally not a good idea to do so (and is not a requirement for being a CNA).

For “products” (which makes the vendor issue where a CNA issues for multiple software products go away), the numbers are a bit different: 2309 "product": "Linux", 1584 "product": "Chrome", 888 "product": "n/a", 497 "product": "OpenClaw", 284 "product": "Windows 10 Version 1607", 255 "product": "Firefox", 153 "product": "Android", 141 "product": "AVideo", 136 "product": "Red Hat Enterprise Linux 10", 124 "product": "iOS and iPadOS", Again, remember, vendors like Apple, Microsoft, and others only report the ones they determine to be “high” to CVE, while open source, as we can not dictate use of our code, have to report everything as we don’t know how it is used by others (i.e. severity is hard, if not impossible, to properly judge.) Again, gotta give props to OpenClaw for properly documenting all of their issues, I wish more vendors would learn from them…

After 25+ years of kernel development, I was finally forced to touch mm/ and it was due to a nommu "issue": https://lore.kernel.org/lkml/2026042334-acutely-unadorned-e05c@gregkh/

As @axboe@fosstodon.org said the other day, we aren't expecting a box of chocolates:

https://lore.kernel.org/r/2f2c91cb-f20e-44eb-8ba3-2d5b3d649642@kernel.dk

but these past weeks have made me feel like someone owes a few of us kernel developers a bunch of whisky at the very least...

In which I explain how I will treat LLM generated or assisted patches for the Linux kernel drivers/staging/ subsystem going forward: https://lore.kernel.org/all/2026080354-skater-urgent-31b2@gregkh/T/#u (hint, not allowed, except for security bugs that you can prove actually fix something by testing the issue on the actual hardware the driver controls.)

My build system right now, as it's one of "those" mornings....

Another one of those mornings....

As people keep guessing what/who gkh_clanker_t1000 is: https://lore.kernel.org/r/20260424054143.087847e1617a84df8b501313@linux-foundation.org

here it is after I cleaned up some of the horrid cable mess that had grown up around it.

Prediction for the potential future: When the AI coding agent companies are just about to run out of money, down to their last few % raised as none of their customers are actually paying the real cost required to run these services, they pivot and take all of the uploaded code that was willingly sent to them, turn it into thousands of products / services to sell / rent, disconnect the public api endpoints leaving their old customers helpless as they no longer remember how to program "in the raw" and can not understand their own codebases, and compete directly against them putting their own customers all out of business which finally results in a positive income stream and "validation" of the coding agent companies previously over-hyped business valuations. "But copyright law will prevent this!" you say...

In the same topic of "use frameworks to make bugs very hard to create", Alice Ryhl's patches for using a "range" api to access data from userspace: https://lore.kernel.org/r/20240913210031.20802-1-aliceryhl@google.com along with examples of how recent binder bugs were affected by this issue in C, and also were present in the Rust implementation, along with a proposal for how to prevent that are another good example of how the language can help us in kernel land by creating apis to help us do the right thing.