Filed under: Business, Internet, News
DreamHost's $7.5M billing accident: "Um, Whoops."
If you were a large company that subsists on customers paying monthly or yearly subscription fees, what's the worst possible thing you could do? While a number of things come to mind, automatically billing customers for a year in advance by mistake has to come in at the top. Now, how about when you're a hosting company that has a reputation for being a little too casual, a little bit unprofessional? Yikes.
Today is not a good day for DreamHost.
Last night a mistake was made that caused virtually every single DreamHost customer to be billed for a full year of hosting up until December 2008. This caused automated charges to be processed against these customer's credit card or other accounts from a few hundred dollars, to literally thousands of dollars. By their own admission, this was a $7.5M mistake. Seven and a half million dollars.
Calling that a mistake is an understatement. The right term starts with the word "cluster" and ends with something that's not printable here.
Given that DreamHost has a bit of a PR issue in that their cool and casual reputation is starting to look a little juvenile and irresponsible, how would you expect them to respond? Probably not with "Um, whoops," but that's exactly what they responded with.
Double Yikes.
DreamHost, it's time to grow up. The cool kid or class clown loses his charm when it becomes clear he's just a screwup.
With Halloween fast approaching, it's a great time to get in some practice defending your territory against zombies. In Graveyard Shift, you take aim at zombies and other creepy-crawlies, blasting them into splatters of cartoony green guts. It's a casual first-person shooter, and it's very easy to get the hang of - use the mouse to aim, click to fire. Graveyard Shift has at least 15 levels, and it might even have some secret stages I haven't unlocked yet.
They key to getting good at Graveyard Shift is learning to use ...
Reader Comments (Page 1 of 2)
rahook said 9:04PM on 1-15-2008
I would suggest a counter way of looking at it, based on many years in IT, far too many of them dealing with the aftermath of disasters. They have done several things to settle customers down, given that they have thousands of angry and confused customers. Including me.
1) issued frequent updates of what is happening, and what they are doing;
2) immediately took full responsibility for sorting things out with financial institutions;
3) have been completely transparent and open regarding the problem.
Overall I would say that their tone and responses will have helped reduce the stress and worry for a lot of customers.
Reply
Former Dreamhoster said 10:33PM on 1-15-2008
I agree, it is time for them to grow up. Should take a look at the coupon code blog.lunarpages.com is offering to current DreamHost customers that want to switch.
Reply
Lee said 1:17PM on 1-22-2008
Personally I like DH just the way they are, and to be honest after reading the blog post on lunarpages it seems they are just as "childish", trying to get customers from their competitions mistakes.
Rid said 9:36PM on 1-15-2008
As I read this article, I agreed, thinking that Dreamhost would be way out of line with a laid back or unprofessional response. But as I read their actual blog post, I changed my mind. Mistakes are bad, and the fact this happened is bad. But would a 'professional' response be a better follow up?
A professional response - that is, one we would expect from any other company - is a generic mass email saying there was an error, that they are looking into it, that they will address all concerns and value our loyal patronage. They would then proceed to block all support tickets containing the words 'billing error' for the next week.
Dreamhost gave the 'unprofessional' response. They owned up to their mistake without using generic corporate phrases, and explained their very human error so anyone could understand it was solely their fault.
After sitting on it and thinking a bit, I respect Dreamhost more for this. I wish more companies were more upfront about what went wrong and how it was fixed. It makes me think they actually have a vested interested in making a broken situation right.
Reply
Phil J Leitch said 9:52PM on 1-15-2008
I agree. I think Dreamhost did the right thing. They could have went and spent $100,000 on a top-notch PR firm to write them up some generic, bland, impersonal apology that every single Dreamhost customer would have known was phony.
I was a bit curious when I got an email saying I had been charged after just paying a few months ago. Later when I got the apology letter I shrugged my shoulders, smiled, deleted the message and got back to my day.
Nice to have 7.5mil in your bank account for a few hours though!
Reply
Paul said 10:24PM on 1-15-2008
Jason, I think you're being a little hard on the Dreamhost guys.
I was just discussing this with my bookkeeper/wife, and we both agreed that we preferred to have a company handle their mistakes with a human face than with more traditional, corporate drone-speak.
They made a mistake and they actually owned up to it as opposed to hiding behind lawyers or other typical CYA behaviour. That seems pretty grown up to me.
Considering that the majority of your recent posts have been "Time Wasters", I would think that you'd be the last guy to be telling people to grow up.
Reply
Jason Clarke said 10:34PM on 1-15-2008
Wow, I'm bit surprised at the responses here - did any of you that are cutting DreamHost actually get affected by their billing?
I did.
It gave me a good 10 minutes of stress and exasperation, because the way I found was that I got an email from DreamHost stating that they had been unable to bill me for the last two months. What, I thought? Is there something wrong with my credit card? A quick call to my wife to confirm that everything was okay with the credit card, but now she's agitated.
Then I try to place a support ticket with DreamHost, and that's where I discover a note admitting the problem. It was fairly straightforward, without any obvious attempts at humor. Fine, I wasn't impressed that I'd been billed way more than I would ever have expected, but for me it wasn't too bad since I had a cap on what I would allow DreamHost to bill me at any one time.
Others that were affected weren't so lucky. So if it was that stressful for me, how must it have been for them?
When I read the blog post, I had mixed feelings. I know they try to keep things lighthearted, but this instance in my opinion is far too serious to be joking about. Everything else about the post was fine, but seriously, don't make light about screwing up with other people's money. It's not cool.
Anyway, y'all already knew my thoughts... thanks for the comments!
Reply
Rid said 1:30AM on 1-16-2008
Nope, Jason, I was not affected by this, but I consider this just another error by just another company.
Some of my own recent dealings, though, to offer my empathy:
Microsoft's 64-bit Zune software automatically decided that the update it should download is the 32-bit version. This breaks the Zune software so thoroughly that only going back to a restore point and uninstalling and reinstalling the software allows me to access my Zune again. This takes three hours of painful phone support to do. I write an email to Microsoft, not angry, but informing them of a serious and theoretically easy to fix bug. I get a form response, and nothing more. As far as I know, the bug still exists and I need to make sure to never update through the Zune software. And that is exactly what I thought would happen, I knew the email would be an almost certain waste of time.
Walmart decides that a promotion it had took off too quickly (the video game preorder mess, for others who tried). It decides that a 29-game price cut was instead a price mistake. As of now, days later, there is still a large pending charge on my debit card, despite the fact that it should never have charged and was also supposedly refunded. Even now, they do not own up to the true motivations behind canceling the orders. I don't begrudge it, this is what I come to expect.
Sears has come to completely stop answering emails, not even giving form responses. Three emails in the last month have gone unanswered in a customer service request. Phone representatives can do nothing to help. There is literally a corporation giving me the silent treatment. This is an exceptionally bad example of normal corporate behavior, of course, but a true story even so.
But this is what I'm getting at. Whether it is Apple deleting posts and locking threads on their forums related to buggy displays, or corporations simply ignoring the little problems (how many readers did George Vaccaro [verizonmath.blogspot.com] get before Verizon gave him his $80 back, with no admission of guilt), this stuff does happen. All companies have huge mistakes and screwups under their belts, and all mess with people's money. If Dreamhost had simply given the standard PR 'Sorry, but something mysterious happened' this probably wouldn't have warranted a blog post. I won't hold what I consider to be surprising honesty against them.
Alok said 3:10PM on 1-16-2008
Jason, I've been at the recieving end of cockups worse than this. Dreamhost's way of handling the entire affair has been far better than the any of the 'professionals' I've seen so far. All the 'pro' responses would merely have engaged you in a nice game of hide and seek without actually letting you know what was being done except 'we are looking into it'.
If I was in their place, I'd be happy I got rid of the customers who get pissed off and close accounts despite having admitted my mistake and apologised as well as having made good their loss. Good riddance. In my experience, people who got pissed off the most at someone else's lapses were more likely to be incompetent at what they themselves do.
Finally, I am not a Dreamhost customer. But I'd subscribed to their blog and the Dreamhost status page via RSS (as I do Download Squad) to check them out before I finally commit to hosting with them. And now I think I will finally move my domains here.
tranglos said 10:45PM on 1-15-2008
What Rid, and Phil and Paul said. GLad to see this isn't turning into a Dreamhost bashing session (at least, not yet :)
People are understandably upset, but anyone who's been with DH for more than a month knows their style, and anyone who's ever had a technical problem, knows they are very friendly, very responsive and very attentive. Sure this is a huge screw-up (and the billing software should never have allowed it), but DH handling of the situation is much, much better - more reassuring - than the typicall faceless, canned drone.
And Jason, not *all* DH customers were affected. Automated billing is an option, better left unchecked. That's how I avoided the mishap. Those who worry about overdraft charges should not have alowed DH to bill them automatically in the first place. This doesn't excuse DH, but really, if you know your account is vulnerable to overdraft, don't leave your CC for strangers to charge any time they choose to. (And even if you do choose automatic billing, on the same page you can limit the max charge to any amount, like $10).
Reply
Paul said 10:55PM on 1-15-2008
Yep, I was affected by this. They double-billed me for the next 2 years. When I went looking for a reason why, I found their blog. No big deal, it's being fixed.
Reply
Nicholas said 12:28AM on 1-16-2008
I was affected ... Like many of the others, I thought DreamHost acted quickly and appropriately. Billing accidents happen. No big deal.
Reply
IsNoGood said 12:32AM on 1-16-2008
I say the blog gave a good Ida what was wrong and that looks to be done in good time, surely some of the other IT companies might learn from this (apple) regardless of the tone its some of the best I seen from a US based IT company in the last years
Reply
Grant Robertson said 12:50AM on 1-16-2008
"Calling that a mistake is an understatement. The right term starts with the word "cluster" and ends with something that's not printable here."
Oh Jason, you rock. Srsly.
Reply
Eachan Galewski said 1:48AM on 1-16-2008
I have been with Dreamhost for 3 years, and I was affected by this. Their friendly, personal, "unprofessional" attitude is exactly why I recommend Dreamhost to everyone I know. Mistakes like this happen ALL the time. If it weren't handled the way it was, it would have been filed away as a mystery billing mistake just like thousands of others made by other companies every year. I applaud Dreamhost, and their ability to not be influenced by the "professional" norm of other companies. Even after a mistake like this, I don't think I will ever be able to use a different hosting provider. They are simply the best.
Reply
Steve said 8:21AM on 1-16-2008
Deric: damn how did they make that kind of mistake
7.5 million
Steve: I don't know.... but somebody getting fired
Deric: @#$^% a whole department
"i wonder what whill happen if i do this
Steve: lol
"dam... nothing happened"... 12 days later....
KaBooooooooooooom
Deric: "sql=update payment set paid = NULL".... lmao
Steve: rotflmao
Reply
Tim said 8:32PM on 1-16-2008
I have a DreamHost account and yes, I received rather a large bill for something I didn't order. I did check the email out however and it said x amount is payable by 12/1/2009 - if not then we'll cut you off.
Luckily I paid by Google Checkout (for no particular reason, I was just interested in using it for the first time!) and thus my details were spared and I haven't been automatically billed.
They're a fairly decent company, cheap and cheerful webhosting and what-not. Shame they screwed up so bad.
Reply
joe said 11:44AM on 1-16-2008
they are owning up and taking care of it - much better than many corporations would do - they should get an award, not a post about being "a little juvenile and irresponsible".
you should be the one to grow up jason - make some real posts with real information, not your silly spin on the world
Reply
Amber Rhea said 9:36AM on 1-16-2008
Yeah, I was among the Dreamhost double-billed.
This is not the first major problem we've had w/ Dreamhost. Several months ago, they had a major security breach and a bunch of users' accounts were hacked... including my sites. Fortunately I had back-ups.
I originally moved all my hosting to Dreamhost several years ago because I noticed that's where a lot of the tech bloggers I read had their hosting. Now? Honestly, the only reason I'm still with them is laziness. It would be a huge PITA to move all my sites, and I'm too busy to deal with it at the moment.
Reply
Rusty Tanton said 9:49AM on 1-16-2008
FWIW, I think Dreamhost's response was pretty good, all things considered. I was amazed at how much detail the post provided.
Also FWIW, I still plan to move my sites off Dreamhost when I get time and find the right service at the right price. You get what you pay for, performance-wise.
Reply