We messed up. You'll see in the following post that we dropped the ball and it hurt you, and we're sorry.
At it's core, our mission is to help you grow. Events like this not only get in the way of that mission, they downright detract from it. So we're sharing the whole story so you can know what happened, and what we're going to do about it.
I woke up Saturday morning to log into Centori and review a blog post draft by one of our interns.
I tried logging in however I kept getting a message that my log in credentials were invalid.
Hm, that's weird.
When I tried resetting my password I got a message that my email address did not match any existing users.
That's really weird.
My heart began pounding as I dug through access logs and pieced together a series of events that unfolded around 2AM EST that morning. Between 2:07 AM - 2:25AM EST someone had created an account and found a vulnerability in one of our APIs that enabled them to delete users and data.
This included the entire Centori team, but also includes a large portion of our current user base. Users, teams, websites, tasks, and drafts were all deleted as a result.
It was around 7AM EST that we tried to get online and discovered the issue.
Before I continue, rest assured that no private information was accessible beyond the email address for your user. Passwords are hashed so they are not human readable in our system, nor are they accessible via our APIs. We also do not store billing and credit card information in Centori either (we let Stripe handle that), so your billing information is secure on our platform.
We immediately took Centori offline to ensure this bad actor could not re-gain access to their account and do any more damage. Then we spent the entire weekend securing all of our APIs so we could take the platform back online.
While that is good news, I am still sincerely sorry for what happened. Any time a company informs you of a security issue it is stressful and confidence-shaking. This is no exception and I'm sorry.
I am sorry to the folks who lost their accounts. I'm sorry to those who were unable to get work done this weekend because we were offline. Most of all, I am sorry because this represents a failure at meeting our mission to service you and help your business grow.
This was a very painful weekend for our customers and our team. We're not going to just put this behind us and continue on as we were. There are valuable lessons for us to glean here, and we are already shifting focus to put them to practice so we can serve you better.
First and foremost we are investing in security.
From securing our systems and APIs to ensure an event like this is impossible to double-checking our platform for any hint of vulnerability we are going to ensure all your data remains secure and all potential security issues are known and can be addressed. It's never too early to make security a priority, and for us it's our number one.
We're also going to ensure that our systems are fully backed up regularly in a secure location. We were not regularly backing up our systems which, in this case, means that the data we lost is permanently lost.
Centori is now back online and available for you to use. We are going to actively monitor usage for the coming weeks to ensure nothing else goes awry, and over the coming months will continue to make security a priority. I hope that you can give us another chance and continue to grow with us.
If you have any further questions/concerns please do not hesitate to reach out to me directly, you can reach me at email@example.com.