Login Temporarily Disabled
Incident Report for Localize
Postmortem

We are pleased to notify you that this incident has been resolved. Your Localize Dashboard is now accessible.

During a routine system security audit, an error occurred that led us to disable customer access to the Localize Dashboard on September 23rd. We were able to resolve the error later that day, but decided to keep the Localize Dashboard disabled through the 24th as a precautionary measure as our engineers worked overtime to ensure that no mistakes were made in the course of resolving this incident.

Degraded service availability was isolated to the Localize Dashboard and did not impact on the deliverability of translations to Localize customers translating their websites or web applications using our service. The Localize API and CLI also remained operational during this incident.

We apologize for the inconvenience this incident may have caused. Localize values reliability, performance, and security to the highest degree, and these qualities are what has attracted more than 500 companies to rely on Localize as their primary localization platform. In this case, the internal security auditing and testing that we regularly perfrom to ensure that Localize is a highly secure platform resulted in the temporary degradation of the reliability of our platform.

As a result of this incident, 1% of user accounts will observe a small degree of data inconsistency. We will personally notify affected customers within 24 hours. 99% of users accounts were not affected by this incident, and we will be working with those who were affected over the coming days. In the unlikely case that your account was affected, the data inconsistency is relatively minor and we have full data backups that we are using to fully resolve the data inconsistencies on a case-by-case basis.

In response to this incident, we will be modifying our internal security auditing procedures and testing procedures so that the measures we deploy to ensure the ongoing security of the Localize platform does not compromise the reliability of the platform moving forward.

If you have any questions, please email support@localizejs.com and we will be happy to assist. For urgent inquires, you may email urgent@localizejs.com. You may also reach us by telephone Monday through Friday between the hours of 9am - 5pm PT at +1 (415) 651-7030.

Again, we apologize for the inconvenience caused by this incident. As our first major service degradation over the past 12 months, we are taking this incident very seriously. You can rest assured that we are taking every precaution to ensure our that we deliver on our promise to provide a consistently secure, reliable, and performant localization platform that all of our customers have come to depend on.

Posted Sep 25, 2016 - 00:18 EDT

Resolved
Access to the Localize Dashboard has now been restored.
Posted Sep 24, 2016 - 22:26 EDT
Update
Although we have restored all services, we are voluntarily continuing to limit access to the Localize Dashboard for the next few hours as a precautionary measure as we ensure that each customer account is fully operational. This partial degradation of service only impacts the Localize dashboard, and translations deployed to your website and through our backend API service have remained unaffected through the course of this incident. We will post a final update in the coming hours when the Localize Dashboard is fully restored for all customers. Please email "urgent@localizejs.com" if there are urgent changes that you must make to your account prior to full service restoration.
Posted Sep 24, 2016 - 18:37 EDT
Update
All approved and active translations are still running on your applications, but are currently not accessible from your Localize Dashboard. We're still working to restore functionality by end of day. We'll continue to update our progress.
Posted Sep 24, 2016 - 12:11 EDT
Update
We're still working to restore functionality. All approved and active translations are still running on your applications, but are currently not accessible from your Localize Dashboard. We'll continue to update our progress.
Posted Sep 23, 2016 - 21:38 EDT
Update
We're still working to restore functionality. We'll continue to update our progress.
Posted Sep 23, 2016 - 19:57 EDT
Identified
We’ve identified the issue and are working to restore full functionality. We’ll continue to update our progress.
Posted Sep 23, 2016 - 17:27 EDT
Investigating
We are aware that users may be experiencing problems with phrases. We've temporarily disabled login and our engineers are currently addressing the issue.
Posted Sep 23, 2016 - 13:43 EDT