Not pointing fingers, but there was a project (maybe another) that uploaded many reports to the same commit. The results were massive reports that crushed our web services. Looking into better techniques to resolve this. Meanwhile, we are limiting uploads to a single commit.
Posted over 2 years ago. Feb 22, 2017 - 00:28 UTC
Working through the remaining build queue. Thank you all for your patience!
We are patching the server to prevent this issue again in the future.
Posted over 2 years ago. Feb 22, 2017 - 00:18 UTC
Slowly starting processing agents. Reports are coming through. Large delay.
Posted over 2 years ago. Feb 22, 2017 - 00:13 UTC
Uploads are still being accepted. Processing is paused.
Posted over 2 years ago. Feb 21, 2017 - 23:47 UTC
One or more reports uploaded huge content. We have paused our upload jobs to identify the culprit.
Posted over 2 years ago. Feb 21, 2017 - 23:46 UTC
We are seeing high error rates on web requests associated with database loads.