Decrease in ingested Sessions

Incident Report for Hotjar

Resolved

On 07/12/2021 at 13:46 UTC our engineers identified an issue with WebSocket Data Ingestion.

What happened?

We detected a decrease in Sessions ingested through our WebSocket servers, however, within 1 minute of this issue being logged, Sessions were back to being ingested as normal.
It's possible that 20-40% of Recordings during this time window might have failed to be processed correctly.

Why did this issue occur?

A batch deployment at the infrastructure level recreated some DNS entries, which caused a temporary disconnection for some services. Within 1 minute, these services were reconnected, back up and running.

What will we do to prevent this from happening in the future?

If this is clear at time of resolution. It is fine to say “We will hold an internal postmortem to establish any relevant follow-up actions.”

Posted Dec 07, 2021 - 03:30 UTC