DOS attack shuts down EVE Online for a day
June 3, 2013 3:53 AM   Subscribe

A DOS attack on Sunday, 3 June caused the moderators of the MMORPGs EVE Online and Dust 514 to shut down the server cluster that hosted both games. The games were offline for most of the day and into the following morning, having just recently been restored. The COO of EVE's parent company, CCP, described the situation this way:
What we can now confirm is that a person was able to utilize a vulnerability in one of the back-end services that support the operation of the Tranquility server. This vulnerability has now been secured and thoroughly tested. We would like to stress that at no time was customer data compromised or accessible in any way. The effort of returning the complex server structure of the EVE Universe and associated websites to service in a methodical and highly-scrutinized fashion began hours ago and Tranquility has now been brought online (at 10:13 UTC). Our teams will monitor the situation carefully in the coming hours to ensure that our services are accessible and that all customer data remains secure.
CCP also took the precaution of shutting down the games' websites, and so communicated with players via Twitter ("Your patience has been legendary and appreciated.") and its Facebook page.

The downtime had various effects on the in-game world. For example, players running player-versus-environment missions receive a bonus for quick completion, which will have expired by this time. In addition, mission targets appear to have fully respawned. Other objects in the game also are set to disappear after a given elapsed time and so may no longer be present when gamers log on.

After a previous outage, the game moderators compensated players with extra skill points. Moderators are considering a similar compensation after this event but have not yet announced details.
posted by Gelatin (39 comments total) 5 users marked this as a favorite
 
And of course there are all sorts of conspiracy theories about this being the work of one of the big player alliances, to gain some sort of well-timed advantage. (To be fair most people are just kidding about this, but you can never quite be sure...)
posted by Catseye at 4:01 AM on June 3, 2013 [4 favorites]


Oof. Yeah, when it comes to EVE, the "This might all be a trick... by whom?" factor is much stronger.
posted by Etrigan at 4:10 AM on June 3, 2013 [2 favorites]


And the awesome thing about EVE is that such paranoia is 100% justified and possibly smarter than assuming otherwise.
posted by Pope Guilty at 4:14 AM on June 3, 2013 [9 favorites]


One of my characters' names on Dust 514 is "President Romney." I made him before the elections.

That...that is all I have to offer to this conversation.
posted by GoingToShopping at 4:35 AM on June 3, 2013 [5 favorites]


It couldn't happen to a nicer bunch of people.
posted by justsomebodythatyouusedtoknow at 5:26 AM on June 3, 2013


What will this do the ISK trade? Could we see a short-term rise in the rates as the folks affected need to payoff loans in less time? I'm seeing some definite in-game loan sharking opportunities as a result of this.
posted by Nanukthedog at 5:30 AM on June 3, 2013


Their patience was legendary? There will be legends written about having to wait 24h to go to a website?
posted by anthill at 6:20 AM on June 3, 2013 [3 favorites]


There will be legends written about having to wait 24h to go to a website?

Hopefully. For our children's sake.
posted by Hicksu at 6:32 AM on June 3, 2013 [5 favorites]


Their patience was legendary? There will be legends written about having to wait 24h to go to a website?

lol. 24 hours is a loooong time in EVE world. Heck I've seen people lose it over 2 hours of being down. A day of being down could easily be worth billions of lost time, productivity and money lost.
It's was even more of big deal because a major expansion is dropping tomorrow which itself will likely have significant effects on the market for a number of reasons and having a whole day lost in prep for it could make a difference for many people.
posted by Jalliah at 6:34 AM on June 3, 2013 [1 favorite]


Your patience has been Legen... .wait for it.... wait for it... wait .... wait.... wait... wait for it... keep on waiting... just a little bit longer... hold on... wait.... dary.
posted by symbioid at 6:35 AM on June 3, 2013 [4 favorites]


Sort of on topic. This expansion video is pretty hilarious.
posted by Jalliah at 6:51 AM on June 3, 2013


Heck I've seen people lose it over 2 hours of being down.

When do these people go to work or take showers?

Wait...I probably don't want to know.
posted by DU at 6:53 AM on June 3, 2013 [2 favorites]


Like I said in the PINTO thread on MFC: Someone is trying way to hard to win the meta-game.
posted by RolandOfEld at 6:56 AM on June 3, 2013 [1 favorite]


Their patience was legendary? There will be legends written about having to wait 24h to go to a website?

Well, there are myths. One time, Posidon had a really slow connection, and he got so bored he flood a couple of islands just for the hell of it. At least the EVE users showed greater restraint than that.
posted by GenjiandProust at 6:58 AM on June 3, 2013 [1 favorite]


Jalliah: " lol. 24 hours is a loooong time in EVE world. Heck I've seen people lose it over 2 hours of being down. A day of being down could easily be worth billions of lost time, productivity and money lost. "

I'm so very glad I never got into EVE.
posted by zarq at 7:08 AM on June 3, 2013


Is it a DOS attack if they take their own servers offline?
posted by Brocktoon at 7:11 AM on June 3, 2013 [1 favorite]


A DOS attack is where someone breaks into the server, fires up the shitty command line interface in windows and runs a script that just prints the contents of directory a billion times a second.
posted by kaibutsu at 7:28 AM on June 3, 2013 [4 favorites]


But that didn't happen...
posted by Brocktoon at 7:30 AM on June 3, 2013


EVE isn't a game. It's an online economics course with its own stock exchange. Sheesh. All those brokers probably needed a break anyway.
posted by elendil71 at 7:40 AM on June 3, 2013 [4 favorites]


And of course there are all sorts of conspiracy theories about this being the work of one of the big player alliances, to gain some sort of well-timed advantage. (To be fair most people are just kidding about this, but you can never quite be sure...)

Not possible. I was assured last week nobody would use IRL as a battlefield in EVE.
posted by Ironmouth at 7:44 AM on June 3, 2013 [2 favorites]


Heck I've seen people lose it over 2 hours of being down.

When do these people go to work or take showers?


Oh, they're not necessarily playing for those 2 hours - it's about the timers. Lots of those people would only have been planning to log in for 30 seconds to change their market orders, repair/refuel their starbase, update their skill queue, or get supplies in place to rally and defend an outpost that's under attack. But if the servers are down, and they can't log on until later, and as a result they lose a huge amount of money on the market/lose their starbase/lose X hours of training on their skill queue/lose control of the outpost and therefore the solar system and therefore the region, changing the power structures of whole alliances containing thousands of other players, people will get angry to the point of outright screaming tantrums.

And this time we had a whole day of unexpected downtime, with a big game-changing expansion coming up, the sort of thing that's already going to affect the markets and people's trained skills and trigger more of those big thousand-player battles for territory. And the playerbase's response, pretty universally, was... to thank CCP for all their hard work, and make jokes about the surprise Real-World Interaction features of this new expansion on Twitter. It's not always a cold universe of Machiavellian capitalist sociopaths, honestly.
posted by Catseye at 7:48 AM on June 3, 2013 [5 favorites]


Catseye: " Oh, they're not necessarily playing for those 2 hours - it's about the timers.

AH! Okay, that makes more sense. I had the impression that EVE was a huge time suck.

Lots of those people would only have been planning to log in for 30 seconds to change their market orders, repair/refuel their starbase, update their skill queue, or get supplies in place to rally and defend an outpost that's under attack. But if the servers are down, and they can't log on until later, and as a result they lose a huge amount of money on the market/lose their starbase/lose X hours of training on their skill queue/lose control of the outpost and therefore the solar system and therefore the region, changing the power structures of whole alliances containing thousands of other players, people will get angry to the point of outright screaming tantrums. "

Forgive my ignorance, but in this instance, if the whole game is down, couldn't CCP simply stop everything and freeze gameplay so that nothing happens? Then "unpause" at a given time? The inconvenience to users would be annoying, but far less problematic.
posted by zarq at 7:57 AM on June 3, 2013 [1 favorite]


When do these people go to work or take showers?

Quoth the MetaFilter regular. ;)
posted by Celsius1414 at 7:58 AM on June 3, 2013 [6 favorites]


The attack happened at 2AM Reykjavik time. Middle of the night for the poor ops folks who run Eve's computers. I always feel so bad for the gnomes toiling down in the mines, unseen, sleepless.

I was assured last week nobody would use IRL as a battlefield in EVE.

That was total nonsense. People in Eve attack enemy alliances outside the game all the time. Mostly via hacking into forums, IRC channels, third party voice chat, that kind of thing. Espionage. But also more directed attacks, knocking crucial individuals offline at specific moments. People argue about where the line is, but out-of-game stuff happens a lot.
posted by Nelson at 8:30 AM on June 3, 2013


I always feel so bad for the gnomes toiling down in the mines, unseen, sleepless.

From what I've seen based upon my skimming of /r/eve there's mostly been supportive, if antsy, responses from the userbase. Just my opinion though, thought you might appreciate knowing.
posted by RolandOfEld at 8:35 AM on June 3, 2013


I'm seeing the same thing in the official forums btw. The exchange below, jokes about 'back end vulnerabilities' and simple curiosity about tech details aside, seems pretty common for player response. It's one of the reasons Eve really is a decent, cool place to be... and MFC's own PINTO corp that is of course. Looking forward to our corp-wide duel tourney btw even though I expect to die in a fire rather quickly.

Anyway:

From a Dev/CCP employee:
We apologize for any inconvenience that this extended downtime has caused.

A reply:
You... you what?

You stayed up all night for us. You gave up your weekend for us. You updated twitter every 3 hours for us. Your coffee pot valiantly died in the line of duty for us.

You don't need to apologize to us; You need a nap, a raise, and a new coffee pot.

Thanks for the hard work.

posted by RolandOfEld at 8:44 AM on June 3, 2013 [3 favorites]


symbioid: "Your patience has been Legen... .wait for it"

I see you shiver in antici...
posted by boo_radley at 8:45 AM on June 3, 2013 [4 favorites]


I see you shiver in antici...

"Say it! SAY IT!"
posted by Celsius1414 at 8:52 AM on June 3, 2013


People argue about where the line is, but out-of-game stuff happens a lot.

I experienced this first hand back when I played the game...it was during the BoB/ASCN war. Never in my life have I played a game that felt so much like a microcosm of real world espionage, where I actually felt at times concerned about my own real life privacy and safety given the level of clearance I had. Being on the ASCN side, we saw some impressive psyops from BoB, clever voice chat and forum spying, and all sorts of sabotage both in and out of game that ultimately broke apart our chances of fighting them (a much smaller but well organized alliance at the time) head on. We ultimately lost our Titan (largest ship in the game...back then would be the equivalent of $10-13k in real world dollars prior to conversion)

These things in any other MMO would've had players banned on site...but were actually encouraged by the Eve Devs back then. Being that this is a game that you can invest and profit off of (through game time cards), it ramps up the competition brutality as real money is often at stake....you can easily see years of hard work completely disappear within a few days time. (or in BoB's case, overnight when GoonSwarm was able to compromise one of their leaders to the point where he disbanded the alliance and sold off all if its assets...suddenly all if their members could no longer dock at their outposts and retrieve their personal belongings...just goes to show there's more ways to set back your enemy than using the combat mechanics...which gives all the more reason to be insanely paranoid in the game and not trust anyone).

Not everyone plays on that level of course....many are content roaming around in small bands doing smaller scale operations. But when it comes to large alliances, territory control, paranoia, and nulsec wars...timing is everything, and meta-game sabotage is very common.
posted by samsara at 8:58 AM on June 3, 2013 [4 favorites]


Oh hey, I was on the same side as you in the BoB/ASCN war. With Prime Orbital Systems down in Esoteria 16P-PX. I still have a Raven trapped in whatever the ASCN hub was in Feythabolis... AZN-D2? That was my first ever battelship. I haven't had an active account in that game in years, and I still remember all these historical details.
posted by Nelson at 10:41 AM on June 3, 2013 [1 favorite]


You don't need to apologize to us; You need a nap, a raise, and a new coffee pot.

Thanks for the hard work.


Man, if there's a more succinct and accurate summation of the words every game dev and IT worker want to hear, I've yet to encounter it.
posted by Ryvar at 10:41 AM on June 3, 2013


Forgive my ignorance, but in this instance, if the whole game is down, couldn't CCP simply stop everything and freeze gameplay so that nothing happens? Then "unpause" at a given time? The inconvenience to users would be annoying, but far less problematic.

They could, but it would cause a whole lot of other issues. Skill training is probably the biggest - in EVE skills train in real time (whether you're logged in or not, so no grinding gameplay to build up experience points), but you have to manually add skills to your ongoing skill queue in to be able to do so. The skill queue is 24-hours long, so you can add a 2-hour skill and a 6-hour skill and a 7-day skill to train after those, but you can't add anything after the 7-day skill until you've trained over 6 days of it and have space to add stuff again. The idea is to always have your skill queue full, so you're always training something. So unexpected downtime means you might miss an opportunity to log in and update your skill queue, and have it end up running empty as a result - but totally pausing gameplay would mean everybody's skill queue stopped training, and oh that would not go down well.

Even other parts of the game where pausing might seem to make more sense on the surface of it end up more complex underneath, because people know how the timers work and take full advantage of them. Say you have a POS - a Player-Owned Starbase, basically a little space station where you can keep ships and make things - and I want to attack it. The game has a mechanic where I can't just fly in there and blow it up immediately, no matter how many spaceships with guns I bring along. Instead, I'd shoot at it repeatedly until its shields took enough damage, at which time it would enter 'reinforced mode' and be totally invulnerable for a certain number of hours (which allows you to refuel the guns, get your ships to safety, etc).

That number of hours is determined by you, when you set up your POS. So what you'd usually do is to set the timer up to be, say, 12 hours or 36 hours from when the POS entered reinforced mode, in the hopes that this would end up being the middle of the night or the workday my time and I wouldn't be online playing. Your POS comes out of reinforced mode, I'm fast asleep, and the shields start repairing themselves and you can get the rest of your stuff out and to safety. But if CCP froze the game for four hours in between, maybe I'm awake and logged in and waiting to blow up your POS as soon as it's out of reinforced.

And that's just one timer mechanic...
posted by Catseye at 10:48 AM on June 3, 2013 [2 favorites]


Oh hey, I was on the same side as you in the BoB/ASCN war. With Prime Orbital Systems down in Esoteria 16P-PX. I still have a Raven trapped in whatever the ASCN hub was in Feythabolis... AZN-D2? That was my first ever battelship.

There's something pretty fucking fantastic about these few sentences which I can't quite put my finger on. The future is a hell of a thing.
posted by xbonesgt at 10:57 AM on June 3, 2013 [8 favorites]


Catseye: " but totally pausing gameplay would mean everybody's skill queue stopped training, and oh that would not go down well."

I only recently started playing, and yes I was very glad that I had queued up Drones V (over four days training time) hours before Tranquility went down. If I had lost a day of training because of the outage I would have been disappointed.
posted by vanar sena at 12:17 PM on June 3, 2013


Nelson: Oh hey, I was on the same side as you in the BoB/ASCN war. With Prime Orbital Systems down in Esoteria 16P-PX. I still have a Raven trapped in whatever the ASCN hub was in Feythabolis... AZN-D2? That was my first ever battelship. I haven't had an active account in that game in years, and I still remember all these historical details.

Hah same here. Remember "Keeping the Feyth?" I mainly played the role of a carrier scout/sentry over in the RIT triangle for eXceed. (Whew, 2006. Recap of the war for those following along) We had a pretty sweet carrier/POS logistics chain and were luckily far enough away from AZN to make it out relatively unscathed. We eventually re-established a much smaller presence up in drone space, I swiched corps and followed a lot of the Euro-ASCN guys to Invictus. We tried a few times to get RIT back though as it was a very defensible location (and profitable resources)....even loosely leaned on the Russians for a bit as the dust began to settle as they were established on the other side of the ASCN empire's older border. It was bizarre. The politics of it all began to blur immediately as ASCN fell...once enemies became allies, and vice versa, and all sorts of random corps started to flood in. Meanwhile Goonswarm was keeping BoB busy with relentless onslaught. The Goons and the quality of their propaganda was absolutely amazing and it was a pleasure making good on their promise in taking down our sworn enemy (we were necessarily allies, they had their own reasons). Interestingly the style of Goonswarm propaganda videos closely mirrors the videos put out by the hacker collective Anonymous. It's very likely that Anonymous members were either inspired or directly involved in some of the meta-gaming in Eve before Anonymous was even well known. (how's that for Eve-induced paranoia?)

All in all, it was a really gratifying and eye opening experience on the hardcore side of Eve-Online...the stuff you don't normally see when looking at the game on its surface. I'm glad I got to play a part in it...but could never be convinced to go back. I will say it's made me much more critical of people's motives in propaganda outside of the game (international politics in particular). For that, I owe Eve for putting me through the grinder and spitting out with a mindset similar to what I'd expect ex CIA officials feel when they retire. (I'm not kidding, I'm even pretty sure we had ex or existing MI5 on our side using Eve as some kind of psyop sandbox experiment to study player psychology and actions...one in particular we always suspected was playing both sides and knew WAY more than he led on...but to this day I'd still never mention his on-line handle on an open forum in fear of being targeted somehow).
posted by samsara at 1:03 PM on June 3, 2013 [6 favorites]


Maybe it's not about EVE at all. CCP now owns White Wolf, which owns Exalted, and Exalted's kickstarter has just posted some controversial rape-themed content which has pissed off lots of people. Including Goons. And Goons, from what I understand, basically run EVE in-game.
posted by Charlemagne In Sweatpants at 4:28 PM on June 3, 2013 [1 favorite]


Goonwaffe are, uh, not exactly a social justice-obsessed group. Don't make the mistake a lot of people make, where they see a particular group of SA users (most commonly Goonfleet or WoW's Goon Squad) and assume that that group is representative of the larger site culture.
posted by Pope Guilty at 6:31 PM on June 3, 2013 [1 favorite]


Charlemagne In Sweatpants: "Maybe it's not about EVE at all. CCP now owns White Wolf, which owns Exalted, and Exalted's kickstarter has just posted some controversial rape-themed content which has pissed off lots of people. Including Goons. And Goons, from what I understand, basically run EVE in-game."

Holy moly: "Evil elf seeks blond twink for vicious anal on wishing well. Must like collars and blood. Bring your own ghost hatchet. NO HEADGAMES."
posted by boo_radley at 7:18 PM on June 3, 2013 [3 favorites]


A really good write-up on one theory behind the recent attack on CCP servers/

So the point of the DDoS was to cover the attempted hack of a zero-day vulnerability in the CCP back-end. That's confirmation in this business that it never was about just lulz, though I'm certain a few were had by someone. But I have a more important question burning in the back of my brain. Why the attempted hack in the first place? That's a helluva escalation. Escalation of what you ask? The war on bots is what.

My experience with black-hats tells me it always boils down to money. When it comes to online gaming, the illicit money is in RMT. The people who write the bots that gather the ISK know enough to wear black-hats if they so wished. Did they wish to at this time? I can't help but think this might be an indication of some desperation in that camp. If true, it's a brazen escalation of the current war on bots. Still, it was inevitable I believe.


Some background information might be helpful, namely that this expansion involves changes that will, supposedly, make botting/afk mining for ISK harder. Various things like moving ore about, changing the contents of given ores, and making ice mining changes are going to have some sort of impact on people that mine across X number of accounts with Y number of characters for serious income, be it real world via Real Money Trading or in game.

The new login system can only have one purpose to my professional eye. It's yet another weapon in the anti-bot arsenal.

Very nice, I didn't think of that part of the new laucher, though I'm sure botters will find a way around it if they try hard enough.

Anyway, fun times.
posted by RolandOfEld at 9:17 AM on June 4, 2013 [1 favorite]


« Older The sky doesn't matter, it's the issues   |   But can you find it on a souvenir bicycle license... Newer »


This thread has been archived and is closed to new comments