The anatomy of a security breach, and how to do good in a bad situation

On Tuesday, iThemes posted an announcement that they had suffered from a security breach of their website and servers. The attackers had reached the servers which stored customer information, including email addresses, IP addresses, full names, and yes, passwords.

iThemes was quick to notify customers via their blog, social media, and their full customer email list about the breach. Approximately 60,000 users were affected. They warned that passwords were vulnerable. In the second update, posted today, they gave more information about passwords, in response to many questions from users.

It turns out that passwords were stored in plaintext on iThemes’ server. That is, obviously, very bad practice.

Why Would You Store Passwords in Plain Text?

This is how the membership software we started using in 2009 did it. There are a number of factors for this, none that will make much of a difference at this point or make anyone feel any better about it, myself included.

Know that it’s not because we did not value your data. As an organization, we have been working on a very large migration process that has required us to interlink legacy systems with the latest technologies. Anyone that has ever gone through that process understands the complexities and challenges.

Frankly put, it’s been something we identified as a potential risk and are working rapidly now to rectify this issue as fast as humanly possible.

It’s also worth noting that their customer database and iThemes.com users were affected, but customers that use their Sync product to manage their own websites were not. So if you use iThemes Sync, and utilized your site passwords to connect, those accounts and passwords were not part of this breach.

aMember and legacy membership platforms

The membership platform that Cory highlights in the update is aMember, a membership management system that’s been around for many years. aMember only introduced encrypted passwords in version 4, which was released in November of 2011.

I discussed aMember and plaintext passwords with some other folks that have a significant history with the membership platform, and there are some significant problems that anyone using aMember have experienced.

First, most folks heavily using aMember aren’t using it out of the box. At the time, most membership sites were doing significant customizations to aMember to achieve desired functionality. So when the v.4 update came out, it was a very difficult update procedure for people to take advantage of the features.

iThemes would even tell you that their current version of membership software doesn’t look much like aMember at all.

iThemes is also not the first to be hacked and their aMember passwords leaked. Tuts+ Premium had the same issue in 2012.

I discussed aMember at length with Pippin Williamson. He has done a lot of work on his brother’s membership site, CGCookie, which also used aMember until 2012, when he did a huge migration of tens of thousands of members to a new platform.

At the time, Pippin notes that aMember did not disclose passwords were stored in plaintext, so CGCookie had no idea that their users were vulnerable until they learned of the Tuts+ hack, wherein they put a planned migration “into hyperdrive.”

The problem with iThemes’ situation is that they knew of the plaintext passwords and didn’t address the obvious security vulnerability.

All in all, the migration for CGCookie took months to perfect and significant juggling of priorities by their team.

Ticking time bomb

Speaking with Pippin, migrating from aMember was not an easy task. Paypal’s IPN handlers (a payment notification system) were tightly linked to aMember and preventing customer accounts from being disconnected from the membership site took weeks of engineering. Additionally, simply upgrading to the newer versions was also terrible.

Many other WordPress companies have used aMember in the past as well, storing plaintext passwords just like iThemes today.

So, aMember has definitely been a problem before now, but iThemes has absolutely slacked in their prioritization of the issue. Simply put, it’s inexcusable to put users into long term risk if you know their passwords are stored in plaintext.

That said, we should consider the potential consequences — though it doesn’t make it excusable. Because iThemes doesn’t store credit card details, emails and names and passwords are about the worst things at risk. Still, a tiny percentage of the general population uses responsible passwords, so getting a list of names and emails and passwords is a treasure trove of information for third party (as in other websites like banks and email providers) to steal identities and break directly into user accounts.

This should also be a significant lesson for any user to always use different passwords for every website where you keep an account. Tools like LastPass and 1Password make account password management easy. Additionally, learn about 2-factor authentication and use it whenever the service enables it.

Owning mistakes

This is clearly a horrible situation for iThemes to be in. Users have a right to be mad. In 2014 we should be able to expect that our passwords are encrypted, and that even if a server where our information is stored is hacked, that certain valuable details like credit cards or passwords are not exposed. Thankfully, iThemes doesn’t store credit card details, but those passwords should now be assumed stolen, and users have to pay that price.

But where I have enormous respect for iThemes is how they have owned it. Especially in the update post, Cory Miller — iThemes’ founder and CEO — took ownership of the issue and told the full story of the breach.

I realize this will generate a lot of concern. Again, I am deeply sorry for this mistake and how it has affected you.

Let me say: we have made mistakes in the past at iThemes … and as humans will make mistakes in the future. To make a promise otherwise would be absurd and misleading.

But my promise to you, our customers, is this … and it’s the same promise that I’ve held to since January 2008 when I started iThemes in my home:

1. We will identify mistakes as best we can. You have helped us with this and appreciate that accountability.
2. We will own up to our mistakes. Again, we’ve done this in the past, we did this yesterday and this post is another example of us living this value.
3. We will fix the mistake as fast as humanly possible. A number of priority issues have been unearthed, shone a hard light on, but we are working to resolve them.
4. We will learn and grow from it and be better for it and for you.

Additionally, as the founder and CEO, the leader of this company, I want you to know: the buck stops with me and me alone.

At the end of the day, I am responsible for our company, iThemes, and the work we do. I’ve often tried to defer credit for the great work we’ve done to our team, but as for the mistakes we make, that credit belongs solely to me.

Not every company that’s been breached can say the same. Even huge organizations (I’m thinking of Home Depot as the latest) have done a horrible job at responsible and honest disclosure to their customers.

iThemes could have made this much more quiet and kept it from being a priority to their users or a big deal in WordPress news-land. But they didn’t. They owned it, they apologized, and they’ve been rewarded for that.

In the comments of the security update post, users were appalled by the lack of responsibility on the security priorities, but the honesty paid off in the sense that users were verbally thankful for it so that they could accept the problem and deal with it, when the alternative was to be in the dark.

While this breach has cost iThemes some credibility, and some trust, I believe those things are recoverable. Had they obfuscated the hack itself, my post and users reactions (had they found out) would be very different.

What iThemes is doing now

iThemes has learned a hard lesson this week, as many other companies have before them. Now they must react, and react strongly. In a situation like this, nothing should take priority over getting this issue fixed.

Until they get their password storage issue fixed, they still are storing passwords in plaintext, even the new ones. Thankfully they’ve already updated password restrictions — which previously limited passwords to 20 characters — to now accept up to 255 characters. But storing them in plaintext means that they are still as vulnerable today as they were last week if someone gets into the server again. So they must be absolutely vigilant to protect their servers through this migration.

What we can all learn

Security is not a sexy industry. Too often we don’t consider it until it’s a problem. “Going on offense” should be our default when we consider security actions, but more often — even with some of the best companies and people in the tech space — we react to issues and aren’t appropriately proactive.

That said, our job is not done either. All of us should consider our own security situation and how we can improve it. Especially if you have users of your own (and many of my readers do), consider that state of your own security, and make sure it’s a priority in your business.

When we think about what pays the bills, it’s not security awareness and proactive security investments. But we should consider security breaches and vulnerabilities as risks that must be managed; meaning investment into security priorities should be a significant part of any business.

iThemes, Envato, and WooThemes are all “big” businesses from a WordPress business perspective. Each of them has fought a website security battle. Envato and WooThemes before iThemes have recovered and maintained their users and the community’s trust. I think iThemes will too. But before you, or me, decide “it won’t happen to us”, consider that it can happen to anyone, and our investments into security measures and best practices are not only worthwhile, but the only responsible thing to do.

And for users, use different passwords on every site. Use LastPass or 1Password. Enable 2-factor authentication. Be vigilant. Do not use the excuse that it’s too hard or a burden. What’s much more of a burden is dealing with a hacked email or bank account. Protect yourself and learn how to protect your accounts. It’s easy to adjust to and modern tools make it a much simpler task to manage.

Let’s all — web professionals and users — learn with iThemes here, and do better.