Google Cloud Datastore has Monday meltdown, tips other services over

Google Cloud Datastore has Monday meltdown, tips other  services over

Google Cloud is having a manic Monday, with a succession of the vendor’s services having problems getting up and running after problems with its core Datastore service.

The search to compute giant’s status page showed, at time of writing, that Cloud Datastore merited a big red outage stamp, while its Cloud Storage service was still experiencing problems. Its App Engine, Computer Engine, and BigQuery services had only just returned to full health – though none were subject to a complete outage.

The incident apparently began at 2.09 Pacific time (10.09GMT). The core problem seems to have been at Datastore, which, unsurprisingly, started tipping other services over.

At 4.32PST, an update for Datastore said the issue had been resolved, though other services were still listed as having problems. 

However, an overall update at 4.44PST put the dreaded red cross onto the datastore service, and said “We are investigating an issue with an infrastructure component impacting multiple products. We believe we have identified the cause and are currently rolling out mitigation.”

Other services experienced disruption earlier today, including Cloud Networking, Cloud Memory Store, Cloud Functions, CLoudSQL, Stackdriver, and BigTable..

And this being the new world, where services are in the cloud, and the cloud in question might be Google’s, other cloud services also experienced problems.

https://twitter.com/communityboxco/status/1193854684282474496
https://twitter.com/communityboxco/status/1193854684282474496

It may, or may not, be significant, that Google Kubernetes Engine has been enjoying a week long amber streak. This was due to “kernel panics following an upgrade in GKE version”. The vendor said while mitigating the issue was dragging on, and on, it was “downgraded to an orange category Service Disruption as the number of projects actually affected is very low.” Mitigation for this problem should finally be wrapped up this Wednesday.

Update:

Google’s cloud status dashboard declared at 5.36PST that, “The issue with multiple products is resolved for most of projects and our Engineering Team is working on full resolution for Stackdriver Monitoring and Google Cloud Storage for US multi-region.”