Bitbucket Cloud intermittently responding to requests
Incident
Report for Atlassian Bitbucket
Resolved
It's been about nine and a half hours since we physically moved all remaining network devices to new hardware, and there have not been any new indications of trouble. All new hardware is functioning as expected, and all servers are communicating as normal. We're marking this as resolved; please contact our support team (support.atlassian.com) if you experience any new issues.
Monitoring
We've moved devices off the malfunctioning network hardware and are monitoring site performance with cautious optimism.
Update
We have a technician on-site to begin the process of physically routing around the failed network hardware causing the availability problems experienced over the past 48 hours. We will have some more intermittent instability as this work is being done.
Update
The misbehaving memcache server has been removed for the pool of availible servers until we have completed network changes. Removing the memcache server has returned bitbucket.org to an operational status.
Update
A repeat of the previously mentioned memcache issue has produced another disruption in the bitbucket cloud service. Website may be unavailable for some customers.
Identified
An issue with one of the memcache servers was identified and the server has been removed from the cluster to return stability to bitbucket cloud.
Update
Ongoing network issues are continuing to significantly degrade the performance and stability of bitbucket.org. Our operations, networking and onsite engineers are working tirelessly to resolve the incident.
Investigating
Currently the Bitbucket API is experiencing latency resulting in connections being delayed up to 100ms. Our Operations team is investigating the issue.
This incident affected: Website and API.