mastodon 4.8.0-alpha.3+glitchWorking at the intersection of security & code @circl@social.circl.lu / @0xrawsec@infosec.exchange. Aiming for a better, more open tech world. Developing in #rustlang, #golang and #python. Author of @kunai_project@infosec.exchange and several others.
🪲 Bug Fix in pure-magic!
I just fixed a tricky offset computation bug in pure-magic, your pure Rust port of libmagic for safe file identification. So, make sure you are using the latest version available for your toolings.
🔎 Check it out
- Rust Crate: https://crates.io/crates/pure-magic
- CLI Tool (replaces file): cargo install wiza
- Python Bindings: https://pypi.org/project/pure-magic-rs/
- Repository: https://github.com/qjerome/magic-rs
🚀 From C to Python to Rust: How We Built a Fast URL Parser in 3 Days ⚡
When our @ail_project@infosec.exchange (https://github.com/ail-project) started hitting performance walls with URL processing, we knew we needed a change. We were using Faup (https://github.com/stricaud/faup), a capable C library, but faced two growing pains: 🔄 architectural portability issues and cumbersome Python binding installations. Our first attempt was rewriting it in Python - which solved the installation problems but created new performance bottlenecks when processing millions of URLs. 🐢
That's when we decided to try Rust. Three days later, we had faup-rs (https://github.com/ail-project/faup-rs) - a zero-allocation URL parser with full Python bindings (https://pypi.org/project/pyfaup-rs/). 🎉
What made this possible? Two incredible Rust ecosystem tools:
- Pest (https://github.com/pest-parser/pest) - it might look a bit hostile at first, but this parser generator has consistently saved us weeks of development time across multiple projects. You define the grammar, and Pest handles a lot of the parsing magic. ✨
- PyO3 (https://github.com/PyO3) - which made creating Python bindings almost effortless. 🤝
The result speaks for itself:
- ⚡ Fast URL parsing callable from Python
- 🌍 True cross-platform compatibility
- 📦 Simple pip installation
- 🗑️ Zero allocations during parsing
This experience reinforced an important lesson: when you hit fundamental performance limits, sometimes the fastest solution is rewriting your bottleneck in Rust. The language's combination of speed, safety, and growing ecosystem makes these kinds of transformations not just possible, but practical even on tight timelines. 💡
You can try it today:
- Rust crate: https://crates.io/crates/faup-rs
- Python package: https://pypi.org/project/pyfaup-rs
As always, it is open-source and you can check it out: https://github.com/ail-project/faup-rs 🏗️
A new patched release is out for both the Rust crate and Python package of magic-rs—the memory-safe implementation of libmagic!
This fixes incorrect file identification (e.g., MS Office docs). If you use magic-rs, update now to avoid false positives:
🔗 Rust: https://crates.io/crates/pure-magic
🔗 Python: https://pypi.org/project/pure-magic-rs/
⚠️ Also, don't forget to update your command line file detection companion wiza (What is Zat?): https://crates.io/crates/wiza
Thanks to the @Rafiot@discourse.ossbase.org for reporting this!
🐍 Meet the new file identification package for Python!
Say hello to pure-magic-rs, the brand-new Python package that’s changing the game for file type detection!
✨ Safe – A reliable, pure Rust implementation of libmagic, ensuring high compatibility with existing magic rules.
🌍 Cross-Platform – No more headaches! Works seamlessly across all platforms without needing external dependencies or compiling C code.
📦 Self-Contained – Forget about managing external magic database files—it embeds its own, making setup a breeze.
Whether you're parsing unknown files, validating uploads, or building security tools, pure-magic-rs simplifies the process with safety, speed and accuracy.
Check it out and supercharge your file handling today: https://pypi.org/project/pure-magic-rs/
Just before the holidays, I ran a recorded Rust training at Luxembourg House of Cybersecurity — designed to help developers with experience in other languages get up to speed faster.
🔗 Watch the full session (broken English included!):
https://youtube.com/playlist?list=PLhSWiKucshm5vWvFCqtJePVVYgIXH6_6y&si=ffdPXpY52GPnd8DW
All training materials are open-source and available here: https://github.com/ngsoti/rust-training
Not about mastery, but about cutting through the noise and getting hands-on faster. Feedback welcome—what worked (or didn’t) for you?
Interested in an internal training? You can get in touch—this can be delivered in French or English.
@robo9k@mastodon.social https://crates.io/crates/pure-magic
Thanks for the name suggestion 😉