Monitoring - Quobyte storage is available again. We are continuing to monitor.
Jul 25, 2025 - 10:31 PDT
Jul 25, 2025 - 10:31 PDT
Identified - Quobyte storage is unavailable. We have identified the problem and are working to return service.
Jul 25, 2025 - 10:18 PDT
Jul 25, 2025 - 10:18 PDT
Investigating - Farm's slurmdbd is having intermittent issues. If you see an error like below, it means the problem has occurred again, and we will restart slurmdbd to bring it back into service.
"""sacctmgr: error: _open_persist_conn: failed to open persistent connection to host:monitoring-ib:6819: Connection timed out
sacctmgr: error: Sending PersistInit msg: Connection timed out"""
We have a support case open with SchedMD and will update this issue as we learn more.
Apr 23, 2025 - 17:22 PDT
"""sacctmgr: error: _open_persist_conn: failed to open persistent connection to host:monitoring-ib:6819: Connection timed out
sacctmgr: error: Sending PersistInit msg: Connection timed out"""
We have a support case open with SchedMD and will update this issue as we learn more.
Apr 23, 2025 - 17:22 PDT
Login
?
Operational
Storage
?
Operational
File transfer node
?
Operational
high2,med2,low2
?
Operational
high,med,low
?
Operational
bmh,bmm
?
Operational
bigmemh,bigmemm
?
Operational
bgpu
?
Operational
gpuh,gpum
?
Operational
Email
?
Operational
Virtualization
Operational
Proxmox Virtualization Nodes
Operational
Ganetti cluster
?
Operational
Slurm
?
Operational
Software
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.