Investigating - Farm's slurmdbd is having intermittent issues. If you see an error like below, it means the problem has occurred again, and we will restart slurmdbd to bring it back into service.

"""sacctmgr: error: _open_persist_conn: failed to open persistent connection to host:monitoring-ib:6819: Connection timed out
sacctmgr: error: Sending PersistInit msg: Connection timed out"""

We have a support case open with SchedMD and will update this issue as we learn more.

Apr 23, 2025 - 17:22 PDT
Login ? Operational
90 days ago
98.75 % uptime
Today
Storage ? Operational
90 days ago
98.75 % uptime
Today
File transfer node ? Operational
90 days ago
100.0 % uptime
Today
high2,med2,low2 ? Operational
90 days ago
100.0 % uptime
Today
high,med,low ? Operational
90 days ago
100.0 % uptime
Today
bmh,bmm ? Operational
90 days ago
100.0 % uptime
Today
bigmemh,bigmemm ? Operational
90 days ago
100.0 % uptime
Today
bgpu ? Operational
90 days ago
100.0 % uptime
Today
gpuh,gpum ? Operational
90 days ago
100.0 % uptime
Today
Email ? Operational
90 days ago
100.0 % uptime
Today
Virtualization Operational
90 days ago
100.0 % uptime
Today
Proxmox Virtualization Nodes Operational
90 days ago
100.0 % uptime
Today
Ganetti cluster ? Operational
90 days ago
100.0 % uptime
Today
Slurm ? Operational
90 days ago
82.11 % uptime
Today
Software Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Jul 18, 2025

No incidents reported today.

Jul 17, 2025

No incidents reported.

Jul 16, 2025

No incidents reported.

Jul 15, 2025

No incidents reported.

Jul 14, 2025

No incidents reported.

Jul 13, 2025

No incidents reported.

Jul 12, 2025

No incidents reported.

Jul 11, 2025

No incidents reported.

Jul 10, 2025

No incidents reported.

Jul 9, 2025

No incidents reported.

Jul 8, 2025

No incidents reported.

Jul 7, 2025

No incidents reported.

Jul 6, 2025

No incidents reported.

Jul 5, 2025

No incidents reported.

Jul 4, 2025

No incidents reported.