EY02 mysql50-8-master resolved

We were able to restart the database almost immediately.  At this point all slices should be back to normal operation.  If you are continuing to experience any issue that may be related, please contact us immediately.  Thank you.

EY02 mysql50-8-master down

We have just received alerts from our monitoring system that the database mysql50-8-master on cluster EY02 is down.  We are responding immediately and hope to have operation fully restored shortly.  We will update in 15 minutes.

EY02: mysql50-8-master resolved

All slices have been checked and should now be back online.  If you are continuing to experience any issues whatsoever in relation to database connectivity with mysql50-8-master, please contact us via phone, irc or open a ticket immediately.  Thank you.

EY02 mysql50-8-master up

The database is restored to normal operation.  Now verifying that all application are properly connecting.  We will complete this shortly and report back.

EY02: mysql50-8-master down

Mysql50-8-master on cluster EY02 appears to be down.  We are responding immediately.

mysql50-8 problem on ey02

Customers using shared MySQL database server mysql50-8-master on our ey02 cluster experienced a brief outage at around 1500 PDT when the mysqld instance on that server segfaulted.  We were notified and brought the database server back online, however we do not have a root cause determined as yet.

ey04 GFS Issue — Resolved

Customers may have experienced a brief unavailability of their GFS partition while we performed a necessary and emergency reboot of a SAN device on cluster ey04. The affected slices were rebooted after the SANs device was restarted. All slices have been verified as up and working. This is now being marked as resolved. Total outage time lasted around 18 minutes.

EY01: MySQL issue on mysql50-3-master resolved

The MySQL database mysql50-3-master on EY01 is up and running as normal. If you are on cluster EY01 and are continuing to experience any issues with mysql50-3-master database, please alert us immediately by IRC, ticket or phone call.

EY01: MySQL issue on mysql50-3-master update

The MySQL process on mysql50-3-master has come back up and DBAs are checking databases for corruption.

EY01: MySQL issue on mysql50-3-master update

mysql50-3-master has come back up and the MySQL process has started and is recovering.