The Console is back to normal. The server was restarting repeatedly and holding on to database connections, so the pool ran out. We cleared the stale connections and restarted the server. We are adding a cap on connections and an alert on connection usage to prevent a repeat.

The Console (organizer and admin) is currently unavailable. Its server could not start because the database had no free connections left. The error was “max clients reached in session mode (pool size 15)”.

Impact

Organizers and Endurace staff can’t sign in or use the Console. That includes the dashboard, event editor, registrations, payouts and admin tools. Admin “Preview public” links also fail.
The athlete website is not affected. Browsing events, registering and paying work as usual, and no registration or payment data is at risk.

What we’re doing
We are freeing the database connections that are in use and restarting the Console server. We are also lowering how many connections it holds, so one restart loop can’t use up the pool again.

Next update: in 30 minutes, or sooner when service is restored.