Major outage on our European data center caused by a software detect
Last archived state: resolved
View archived record details
monitoring
Service has been fully restored for all clients.
We are aware of the cause of the incident (a known bug on CEPH) and are working to patch the system to prevent this from happening again.
In the coming hours, we will be working to stabilize the infrastructure, which may cause temporary down time due to reboots. While this might create inconvenience for a short duration, this will help prevent further issues that may arise caused by the defective component.
identified
Service is restarting on all clients, we expect all platforms to be fully available within minutes.
investigating
A large number of client platforms hosted on our Europe data center are currently unavailable, but are in the process of being restored progressively.
This incident was determined to be caused by a software bug in the the storage system component of our infrastructure (CEPH).
We are actively working on restoring service. We expect service to resume shortly for all impacted clients *without data loss*.
Our internal management platform (CCC) is already restored and is currently accessible.
...