Docs SSL errors
Incident Report for Help Scout

What happened

On Jan 21 from roughly 6:25 - 7:36pm EST, Docs SSL certificate validation began to fail for . The incident was caused by an issue with our upstream hosting provider, which then caused an OCSP check that we use to talk with the CA (Certificate Authority) to fail. By temporarily disabling the OCSP check, we were able to restore service back to normal.

The issue drew out longer than we’d like because we tried a number of things before finally tracking down the root cause. We always look at our own infrastructure first, and since this incident was caused by an external event that was not mentioned publicly, it took more time to resolve the problem.

What can we do better?

Our upstream provider seems confident the problem is permanently resolved, as we’ve been told they “would be very surprised if this ever reoccurs.” There isn’t much we could have done on our end to prevent this one, but we have updated our internal procedures so that we file a ticket with the upstream provider immediately, instead of waiting to diagnose a problem for ourselves.

Sorry again for the trouble on Monday, but the good news is that we don’t expect to ever see the same issue again. If so, we’ll better know how to resolve it.

Posted Jan 23, 2019 - 10:40 EST

All SSL errors have been resolved. Very sorry for the interruption today! We'll publish a postmortem with much more detail in the next 48 hours.
Posted Jan 21, 2019 - 19:41 EST
We're in the process of patching the SSL issues up now, and is securely serving up content again. Should be all set soon.
Posted Jan 21, 2019 - 19:36 EST
Really sorry for the trouble, folks. We're still working on this.
Posted Jan 21, 2019 - 19:18 EST
Still investigating, we're close to understanding the root cause and getting everything back to normal.
Posted Jan 21, 2019 - 18:59 EST
We're investigating SSL errors on Docs sites with custom domains. This includes our own site at More as we figure out what's up ...
Posted Jan 21, 2019 - 18:25 EST
This incident affected: Docs Sites.