On 2020-02-17 at 12:40 UTC our engineers identified an issue with one of our primary databases. Due to this issue all Hotjar features were unavailable, including data collection from your websites.
What happened?
At approximately 12:30 UTC a fault developed on one of our primary databases. This caused an outage to our Recordings feature. In order to protect customer data we then placed the Hotjar application in maintenance mode at 13:00 UTC. Whilst in maintenance mode all Hotjar features were unavailable.
After repairing and restoring database services we left maintenance mode at 14:45 UTC. All Hotjar features were then made available again including data collection from websites.
A second fault developed on the database service at 15:45 UTC. We again placed the Hotjar application in maintenance mode shortly thereafter. We then worked to repair the database service and restore access. Once the database was repaired and restored we exited maintenance mode and restored access to services at 17:54 UTC.
What effect did this have?
Between 13:00 UTC and 14:45 UTC, and between 15:45 UTC and 17:40 UTC, data was not collected from your websites. We disabled access to the Hotjar application and thus all Hotjar features were unavailable.
During this time we lost access to two previous days of Recordings data. We have subsequently recovered this data and it is now available within the Hotjar application. The dates affected were the 5th of December 2019 and the 27th of January 2020.
Why did this issue occur?
The database which provides search and listing functionality for Recordings became unavailable. Unfortunately we have yet to identify the root cause of the database degradation. We will continue to work to identify the cause.
What will we do to prevent this from happening in the future?
We have made changes to the affected primary database to improve its stability and availability. We are continuing to investigate the root cause of the database degradation. We also intend to take further steps to improve the reliability of the affected database service.