Remote

Roman

@guesser@sigmoid.social

mastodon 4.7.3

> How can we make use of AI without giving up ourselves?

I am a software engineer peeking into the next chapter of digital evolution. Wondering what technology brings us this time and how to make best use of artificial intelligence. Caring about our personal data, transparency and freedom.

0 Followers

0 Following

3 Posts

Joined December 18, 2025

Blog:

https://venkado.org/about

Self hosted personal AI assistant:

https://gitlab.com/RmMsr/relagent

Roman @guesser@sigmoid.social

· 7mo ago

Hello fellow beings,

my name is Roman and it time for an #Introduction . I earn my living as an IT engineer. Caring much about #opensource, #privacy and independence.

I am still very surprised by the raise and risks of #genAI . Instead of being affected by it, I'd rather understand it. So I put my curiosity and time into creating a completely local AI assistant for #selfhosting .

https://gitlab.com/RmMsr/relagent

Happy tooting!

sigmoid.social

Roman @guesser@sigmoid.social

· 5mo ago

Replying to

Nice article. Thanks for sharing your experience. @alan@lighthouse.co.im I really like reading real world reports from people using AI independent. Too many success stories leave out which API (or external token) usage was needed to get to the goal. Also many claim that running a solution all your own works because ollama serves the same API. I think there is a lot possible already with even moderate consumer hardware. It requires some engineering effort to tune such systems right. But its worth it.

Roman @guesser@sigmoid.social

· 5mo ago

Replying to

I agree @alan@lighthouse.co.im . Just getting any output is far from a good success metric. I haven't gotten to systematic testing yet. But want to do some reproducible evaluation soon. For my project I shift between a 12 GB nVidia (CUDA), a 24 GB AMD (ROCm) graphics card and a aged 16 GB laptop for CPU only inference. I get ok results running heavily quantized tunes of gpt-oss, olmo3, nemotron3 or gemma4 without tweaking inference parameters.