On 2020-04-10 at 21:07 UTC our engineers identified an issue with the database which stores User Attributes sent through the Identify API. This resulted in an outage of the Identify API which was resolved on 2020-04-13 at 12:34 UTC.
On 2020-04-10 at 21:07 UTC, a fault developed in the service which manages User Attributes, which caused a high latency in requests to the Identify API. The situation continued to regress and we were unable to resolve the issue immediately. To mitigate any impact, we temporarily disabled the Identify API.
We deployed a fix and re-enabled the Identify API on 2020-04-13 at 12:34 UTC.
This only impacted sites using the Identify API, and during the period of this incident, User Attributes sent through the Identify API were not saved.
Why did this issue occur?
This outage was caused by a bug in the way we were constructing database queries.
Posted Apr 13, 2020 - 16:17 UTC
The issue has been resolved. Thank you for your patience!
Posted Apr 13, 2020 - 12:21 UTC
Our engineers have deployed a fix and are monitoring the situation to make sure everything is working smoothly again.
Posted Apr 13, 2020 - 11:52 UTC
The situation continued to degrade and we've put a temporary measure into place which stops saving User Attributes (sent through the Identify API) to our database until we understand what is causing the increased latency.
Posted Apr 11, 2020 - 09:25 UTC
We put measures in place to mitigate the issue and will be investigating further over the coming days.
Posted Apr 11, 2020 - 00:44 UTC
We are currently investigating an issue with an increased latency when writing User Attributes through the Identify API. Our engineers are working to fix the issue ASAP and will post updates to status.hotjar.com. Thanks for bearing with us!