It has been in the works for months, but the recent increase of downtime has accelerated work. We've moved from Caddy to HAProxy for codeberg.org traffic.

Starting over with a new reverse proxy has enabled us to make choices that enable us to set up HAProxy to scale with our increasing traffic. For example, traffic is split (when possible, decided via heuristics) over several instances to load balance expensive traffic.

We hope to share more details at a later time about this change.

152

Codeberg Status @codebergstatus@social.anoxinon.de

· 2mo ago

We're excited to see @forgejo@floss.social releasing v16. We already deployed it to our internal staging environment and have not found any regressions. We plan to deploy it later today in the evening at 21:00 CEST (19:00 UTC) which requires downtime of around 10 minutes. We'll keep you updated in this thread.

Codeberg Status @codebergstatus@social.anoxinon.de

· 3mo ago

Boosted by @trending@homestead.social

We are struggling to keep Codebreg.org available for unauthenticated users due to massive abuse of expensive endpoints. Our current priority is keeping Codeberg.org responsive for authenticated users.

Codeberg Status @codebergstatus@social.anoxinon.de

· 7mo ago

Replying to

We were able to restore IPv4 and IPv6 connectivity. Performance is still not optimal, and connections are occasionally dropping. We've taken measures to try and maintain uptime overnight (with a certain probability). Routes protected by iocaine (crawler protection) may return 503 errors. Currently, it appears that these packets are also occasionally being lost. It's 2:30am, and we require a good night's sleep before we can look into this further.

129

Codeberg Status @codebergstatus@social.anoxinon.de

· 6mo ago

Hi all, we're aware that Codeberg is down right now and are investigating.

Codeberg Status @codebergstatus@social.anoxinon.de

· 5mo ago

Codeberg.org will be upgraded to Forgejo v15 in a few minutes. We expect downtime to be at most a few minutes, according to migration tests in the staging environment.

Codeberg Status @codebergstatus@social.anoxinon.de

· 5mo ago

Codeberg.org appears to be very slow right now, we are trying to resolve this as we speak.

Codeberg Status @codebergstatus@social.anoxinon.de

· 5mo ago

We're investigating a downtime of our primary instance.

Codeberg Status @codebergstatus@social.anoxinon.de

· 3mo ago

Boosted by @fedicat@pc.cafe

Replying to

We have recovered services after power returned, leading to a total downtime of about 7 hours. We apologize for the impact and we are in touch with our datacenter operator to discuss the incident.

Codeberg Status @codebergstatus@social.anoxinon.de

· 3mo ago

Boosted by @fedicat@pc.cafe

Replying to

We do not expect the issue to be resolved soon. We are waiting for a fix from the datacenter operator and will monitor the situation in order to restore service as soon as possible after power returns. Currently, one server is with power again, but the others remain unreachable. Our investigation shows that network equipment is also affected.

Codeberg Status @codebergstatus@social.anoxinon.de

· 6mo ago

We are aware of reports of Codeberg being down and are investigating. Thank you for your patience.

Codeberg Status @codebergstatus@social.anoxinon.de

· 2mo ago

https://codeberg.org and legacy Codeberg Pages server has seen a long outage since yesterday, with blips of uptime. We are still investigating the problem, but it is unlikely to be resolved any time soon, as there is no clear cause to explain the disruption. We apologize for the inconvenience and for not providing an update on this disruption since it started happening.

codeberg.org

Codeberg Status @codebergstatus@social.anoxinon.de

· 8mo ago

Replying to

We are entering a next phase. Codeberg Pages and Weblate and Hosted Woodpecker CI/CD will become unavailable. Forgejo will remain online.

Codeberg Status @codebergstatus@social.anoxinon.de

· 8mo ago

Maintenance is starting. Hosted Forgejo Actions logs will not be running. Performance degradation is possible