Cyberattacks are increasingly common. The power grid is vulnerable.
December 9, 2015 1:54 PM   Subscribe

Cybersecurity is an increasingly important concern. The Washington Post recently ran a great special series on the issue. The rate of major hacks is growing. The power grid is especially vulnerable, and a hack on it will be especially damaging. It's not a question of if, but when.

The recent book Lights Out by Ted Koppel explores the power grids' vulnerabilities in detail. The odds of a major attack on the power grid are very high--"80-90%" according to Department of Homeland Security Secretary Janet Napolitano. There are 3 main grids in the U.S.: The Eastern Interconnect, the Texas, and the West Coast grids. Koppel says China and Russia are both known to have digitally penetrated at least one U.S. power grid.

A few other notes:

There was a great PBS NOVA documentary recently, "Rise Of The Hackers." It's not available for free viewing anymore, but it's on Netflix.

Wall Street is also pretty vulnerable.

Wikipedia has a couple categories containing decent lists of major hacking incidents.
posted by Sleeper (42 comments total) 22 users marked this as a favorite
koppel's book has had less than stellar reception by the infosec community
posted by p3on at 2:04 PM on December 9, 2015 [13 favorites]

You know this wouldn't be a problem if American had values of self-reliance and personal responsibility. Everyone would have solar power and a battery backup and it wouldn't matter one bit if the grid went down.
posted by Talez at 2:07 PM on December 9, 2015 [3 favorites]

I mean you don't even need to pwn something to take it down. The northeast blackout of 2003 was taken down by a race condition and foliage in the right place hitting high voltage wires. All you need to do is know where to poke the grid and I'm pretty sure people are starting to test some pressure points.
posted by Talez at 2:10 PM on December 9, 2015 [3 favorites]

The Northeast Blackout of 2003 was caused by "unpruned foliage" (a falling branch) in Ohio. It affected 45 million people, and completely blacked out the states of New York, Pennsylvania, New Jersey, Ohio, and Ontario, Canada. Fortunately, if happened in summer. If had happened in the middle of last winter (severe) or the winter before (very severe) there would have been many deaths. I would go so far as to say that a similar power outage taking place during a 10-day -period of below 20 degree F weather in the Northeast remains the most plausible accidental mass death scenario for that part of the country. As "Lights Out" notes, a blackout shuts down everything: gas stations, heat, water, heat, food delivery, heat, internet, heat ... Would you know where to go for warmth to save your life in the event of a multi-day, multi-state power failure in your city? I mean, somewhere where the several other million people in the same boat aren't trying to squeeze into already?
posted by Modest House at 2:15 PM on December 9, 2015 [1 favorite]

I work in cybersecurity. I mean. yeah, "the grid" could be taken down by a lot of things, but any breach or intrusion would be sponsored at a nation-state level and attack our SCADA infrastructure. Which is doable. But I don't know how fruitful that would be because we'd all notice PDQ and start working on it. I'm more concerned about people silently intruding on other things that maybe we aren't really paying attention to and diverting funds from the U.S or, say, compromising our certificate infrastructure. I dunno, I guess what I'm saying is that I'm afraid of the NSA?
posted by Annika Cicada at 2:41 PM on December 9, 2015 [15 favorites]

As "Lights Out" notes, a blackout shuts down everything: gas stations, heat, water, heat, food delivery

Mass cannibalism, within hours! And you wouldn't even need to put the leftovers in the fridge!
posted by XMLicious at 2:48 PM on December 9, 2015 [5 favorites]

The power grid is especially vulnerable

That video is just scaremongering word salad.
posted by peeedro at 3:20 PM on December 9, 2015 [3 favorites]

Darn terrorist squirrels!
posted by nofundy at 3:22 PM on December 9, 2015 [1 favorite]

My hope is that thanks to decades of privatization and the resulting fragmentation and cost cutting measures the systems controlling the grids are such a complete mess by now that it would be as impossible to cause failure on purpose as it is to cause efficient and reliable operation on purpose and failures would only be caused by stuff like random foliage-related incidents.
posted by Hairy Lobster at 3:53 PM on December 9, 2015 [3 favorites]

I mean, somewhere where the several other million people in the same boat aren't trying to squeeze into already?

In grad school, an ice storm hit the university during finals week. That evening was a series of transformer related fireworks, and it knocked out power for a week in our house. It turns out that as long as people have adequate caloric intake, in a well insulated house there comes a point where adding more people to the winter party requires you start opening windows. You would be doing this during flu season, however, with all the pandemic implications. At least in university, it doesn't represent a huge shift in social contact. At metro area levels though, that could turn out poorly.

You can choose to keep warm at night in your own home without a party. It's not impossible if you own a decent sleeping bag per person. And you realize you own one. I had to remind my roommate that his comforter was actually a sleeping bag, and that was why it had a zipper on it. His mood was considerably improved after I pointed this out to him.

Cooked food is harder to deal with; if you have a gas stove, you might be able to get by. We didn't. Equally important, our hot water heater was gas powered but still required electricity for certain things (like the pilot light? I forget). But the logistics of grocery delivery in a snowstorm without power is still a problem at scale.
posted by pwnguin at 4:13 PM on December 9, 2015 [3 favorites]

2013 felt pretty alarming to me.

The internet scan found two small hydroelectric plants in New York on the internet that were not secured. The plants supply electricity for the state of New York. WIRED attempted to reach the plants but was unable to connect with anyone to disclose the vulnerability, so the name of the plant in the screenshot has been blurred.

If there was anything nationally significant in there it should be fixed now. I think researchers are still working with these mass scanners, to develop/review/process the results. But this was one researcher in 15 minutes, finding 30000 remote-access computers with no password. We probably should keep working on it.
posted by sourcejedi at 4:17 PM on December 9, 2015

If there was anything nationally significant in there it should be fixed now.

It's unfortunately still fairly easy to find unsecured, or flimsily secured, internet-facing industrial control systems using Shodan. This Vice article from November cites a number of attacks that have happened since the Wired article was published. This is a problem that metastasizing with the spread of the IoT in ways we can barely comprehend, much less comprehensively address.
posted by ryanshepard at 5:01 PM on December 9, 2015 [1 favorite]

I operate a website that's been online for over a decade, and after my server was hacked a few times I erected some pretty aggressive extra defenses (e.g., anything that sends GET or POST data must satisfy a very surly automated gatekeeper, I capture and quarantine any file uploads from IP addresses I don't own, there is a blacklist* of IPs updated daily, etc).

I log all of the suspicious requests so I can learn the wrongdoers' ways and use them to further improve my defenses. This generates thousands of log files from thousands of (often sequential) IP addresses per day. Many trace back to Ukraine and China, but some could be proxies. Each one systematically pokes at a long list of known vulnerable scripts, hoping we have one installed (code libraries, common utilities, plugins/themes for popular CMSs, etc). These lists can have thousands of potential entry points to try, and as I look through the logs I don't know whether I should be more impressed by the resourcefulness of the hackers or depressed by the sad state of web software security.

* On my blacklist readers can still read, the server just no longer trusts them to send data, and returns a 403 Forbidden error. That way it's not too harsh on proxy users.
posted by Hot Pastrami! at 5:49 PM on December 9, 2015 [8 favorites]

Ukraine and China...nodding in in agreement. And the Netherlands in my experience.
posted by Annika Cicada at 7:02 PM on December 9, 2015

And this French server farm OVH is full of crap like that too.
posted by Annika Cicada at 7:03 PM on December 9, 2015

This is an area where I have some very specific expertise & experience. My last job was Vulnerability Analyst for the company that runs the power grid for the East Coast. These days I hunt nation state malware for a Fortune 50 firm. I don't have much to say about kinetic or EMP attacks but as far as cyber attacks on the grid, ain't gonna happen.

Outlying segments and those under local control of a power generation or distribution company may be more susceptible to attack but the grid itself is locked down tight. It'd take something on the scale of Stuxnet to breach the networks that control the grid. China & Russia might be able to pull it off but even that's dicey.

You'd need to acquire all the hardware for the testbed that exactly matches the production systems, no easy matter. You'd need a good idea of how it's configured, some solid generic 0-days & some SCADA 0-days on top of that. You'd need an entry point onto the target network, not just the employee user network or even the network that handles pricing & energy availability but the ICS network that houses the systems that control the actual grid. Good luck with that. You'd need seriously world class malware that can autonomously navigate the network until it finds the system it's looking for because connectivity to an external C&C is not gonna happen. Lacking connectivity it'd have to be a fire-&-forget weapon; once it's launched it's armed & counting down to ignition with no way to shut it off from home. All in all it's a very risky project.

Ted Koppel really should have spoken to an actual expert. I have no idea why he decided not to.
posted by scalefree at 7:24 PM on December 9, 2015 [17 favorites]

Would you know where to go for warmth to save your life in the event of a multi-day, multi-state power failure in your city?

I guess a lot of people live in apartments, but do most houses in cold areas have fireplaces? Seems like a backup at least outside super-dense areas where everyone is basically in a condo/apartment.

I mean, my house in LA has a fireplace. I don't keep any emergency wood supply or anything, though, because its Los Angeles and thus its never going to get dangerously cold unless there's a sudden Ice Age or something.
posted by thefoxgod at 7:40 PM on December 9, 2015

Guys, guys, guys. I'm sure if we just called Bill Gates that he'd fix this right up for us.
posted by dave*p at 8:04 PM on December 9, 2015 [1 favorite]

We recently passed legislation that would among other things give us a strategic reserve of transmission equipment. Power companies are already doing this on their own, and they think really hard about how to get them from A to B when needed. So there's a lot of good news here, sorry Ted.
posted by RobotVoodooPower at 8:48 PM on December 9, 2015 [2 favorites]

Modest House: " Fortunately, if happened in summer. If had happened in the middle of last winter (severe) or the winter before (very severe) there would have been many deaths."

35 people died during the 1998 Ice Storm that affected much of Ontario and Quebec knocking out power for over a week. Three years earlier a heat wave in Chicago killed over 700 in five days. Thousands died across Europe during the 2003 heat wave in part because the usually temperate climate means few have air conditioning. Temperatures weren't even extremely hot compared to many places in the US. EG: 1500 additional deaths were attributed to the heat in the Netherlands where the highest temperatures didn't even crack 38C.

Dealing with cold is mostly just a matter of layers. Dealing with 100F temperatures without air conditioning, especially for the elderly, is very difficult. A hypothetical attack that knocked out the Texas grid at the height of summer would kill thousands, maybe even tens of thousands.

And like pwnguin said, if you get enough people together in reasonably well insulated space you don't need external heat sources.

thefoxgod: "do most houses in cold areas have fireplaces?"

I don't know what it is like in the US but here in BC it's unusual at best. Many cities and regional districts have banned wood burning appliances in new construction. And many/most gas fireplaces require power to operate (and their use assumes the gas grid would stay up if power fails; not something I'd bet on). I've got the advantage of living only a few blocks from my mother who has wood heat available and my gas fireplaces don't require power to operate for as long as we have gas.

Broadly speaking though people have to prepared to shelter in place on their own for at least 72 hours (officially in my province) and more realistically a week. Most people are very ill prepared for a disaster. And it doesn't have to be all that expensive; especially if spread out over the course of a year. Unfortunately for people in tight living spaces the bulk of the items you need can be problematic; pooling resources can sometimes mitigate that somewhat.

Plastic pop/water bottles can be cleaned and used to store water if you can't afford to buy cases of water. 4L/per/day (IE: 112L for a family of four for a week. That's 12 24-500ml cases which will store under a bed). Have a way of melting/boiling it. Simplest is a camp stove of your choice; I prefer white gas but propane is good too.

Food can mostly be handled by not letting your pantry get down to scraps. Canned soup; ramen, rice, granola, tuna and processed stuff like Pop-Tarts, and granola bars are our big always have a bunch on hand items. I'm not saying we're going to eat particularly healthy for the duration but most people can handle that as long as they have calories.

A garbage bag, 5 gallon bucket, pool noodle kitty litter and some toilet paper allow you to keep a sanitary camp if the municipal water goes out. Put garbage bags, noodle and TP in the bucket for storage. There are commercial camping products available that integrate an actual toilet seat if you have some money to throw at this problem. This is something that is actually easier when the temperature drops.

Finally practice living un plugged. The funnest way for me anyways is to go camping.
posted by Mitheral at 8:54 PM on December 9, 2015 [3 favorites]

Speaking of cybersecurity efforts on the Hill, the Director of the FBI today asked tech companies to nerf their security features to satisfy law enforcement, comparing the use of encryption to pollution.
posted by RobotVoodooPower at 9:12 PM on December 9, 2015 [1 favorite]

Climate change and lack of regulation are the biggest threats to the power infrastructure by far.
posted by benzenedream at 9:21 PM on December 9, 2015 [2 favorites]

Annika Cicada's post above is the first thing I've ever seen, anywhere that combines the word "cybersecurity" with evidence of clue.

Still seems a very good rule of thumb that when someone talks about "cybersecurity", they're peddling fear and snake oil to the uninformed masses and goverment respectively.
posted by joeyh at 9:33 PM on December 9, 2015 [3 favorites]

We recently passed legislation that would among other things give us a strategic reserve of transmission equipment.

This has been the largest gap in our defenses for years if not decades so I'm quite relieved to hear it. Basically it takes a special kind of transformer to handle pushing power long haul. If you took a bunch of them out at once the RTOs would be crippled until they were replaced, since it would take a year or more to ramp up manufacturing capabilities to fill that need because as it stands we basically have no spares or ability to make them quickly. So yeah, great news that we're actually doing something about it now before the need arises instead of afterwards when we're all sitting around in the dark. But I bet that when that does happen it'll be because of a Carrington Event (solar flare damage) & not some Chinese hackers.
posted by scalefree at 9:38 PM on December 9, 2015 [6 favorites]

thefoxgod: do most houses in cold areas have fireplaces?

To expand on what Mitheral said, I grew up in a 300-year-old farmhouse in New England with fireplaces up the wazoo...and this fact wouldn't have helped us without ready (& dry) wood to burn in them* and the forethought to ensure the chimneys were clean. Neither of which the house really had until my parents installed a wood-burning stove about 5 years ago.

In addition: I currently live in an apartment in the outer San Francisco Bay Area (where it does, in fact, get down to vaguely uncomfortably cold temperatures at night 1/4 of the year) which has a fireplace...and as Mitheral noted, it's gas-driven and wouldn't work without power.

In this modern age, having a fireplace is definitely not proof against freezing your tush off during a prolonged power outage :(

* OK, technically, yes - you could burn some of your furniture in a pinch. Though goodness knows what's in the paint, polish etc!
posted by cyrusdogstar at 9:46 PM on December 9, 2015

Looking forward to getting my solar setup hooked into the Tesla Powerwall batteries. Might as well be prepared to run off the grid as much as possible...
posted by Windopaene at 10:11 PM on December 9, 2015

Are there any devices on the market that offer a way to retrofit a battery backup and maybe a small generator for gas furnace systems? Seems like a sensible idea, especially given that furnaces are really important in the winter, have ducting for exhaust, and have access to a different fuel than electricity.

Also, this is probably a good reason for the government to incentivize winterizing.
posted by mccarty.tim at 10:38 PM on December 9, 2015

Are there any devices on the market that offer a way to retrofit a battery backup and maybe a small generator for gas furnace systems?

This is not something I'm an expert in so caveat emptor but there seem to be plenty of videos & howtos on the subject available.
posted by scalefree at 10:53 PM on December 9, 2015

We've been hearing these "the grid is vulnerable!!!!" stories since the invention of the computer. The are only three types people worried about this: people selling books, security vendors selling infosec "solutions", and politicians selling fear.

Stuxnet is a good example of what an actual cyber (man I hate that word) attack looks like. It worked, but it took the resources of (probably) the US and Israeli to pull off.

The 1971 Sylmar Earthquake destroyed the Sylmar Converter Station , and it took something like 6 weeks to get all the transformers and whatnot to rebuild it. That's the kind of power grid problems we need to worry about; Amazon doesn't sell equipment rated 3.1 MW.
posted by sideshow at 11:06 PM on December 9, 2015 [2 favorites]

mccarty.tim: "Are there any devices on the market that offer a way to retrofit a battery backup and maybe a small generator for gas furnace systems?"

Simple back up generator. For the serious prepper you would have a propane generator and a big pig to fuel it. Usually though they are either gas (requires constant fuel maintenance) or natural gas (requires natural gas service). My father required power for an oxygen concentrator so I've got a couple 5000W gas generators in case of grid failure. They are pretty cheap if you are in the sort of situation where you need constant power to live. You can also get pad mounted units to power some or all of your house. They can be set up to require manual switch over or be fully automatic.

Depending on your electrical savvyness you might want to have an electrician come out and set you up for plug and play connection of your furnace.

Do not construct a suicide cord to mate your house to your generator.
posted by Mitheral at 2:19 AM on December 10, 2015

If you've got a privileged, motivated, skilled and hostile person on the inside of your system, then you're going to be vulnerable to attack, at least as far as I understand the current state of the art. And history. At the scale of national infrastructure, that should need a state-sized actor with the resources and experience to insert and maintain sleepers for many years, but with umpty-million cleared contractors circulating you've always got the risk of a Snowden.

You can and should design your systems to be robust against anything less than that, though, and on the whole I think that's sinking in. If you look at how the Def Cons et al have evolved into a communications channel between communities that Get It and those that Need It, it's quite heartening. The many and continuing facepalms in IoT should help drive the point home.

But classic espionage/subversion, rather than OMG CYBERHOLEZ!, is what would keep me awake at night, if I suffered from fear insomnia.

Meanwhile, if you want to learn about living off the grid, you could do worse than look at the UK's canal boating community. Most full-timers live plugged into the mains and using local sanitary disposal arrangements, and they all go to the supermarket from time to time, but there;s a fair sprinkling of die-hard, wood-foraging-seasoning-and-burning, solar-panelled, compost toileting nomads. With big water and diesel tanks and enough PV feeding big batteries, combined with a parsimonious diet and maintenance skills, a very useful degree of autonomy is available. (Floods and droughts are still very bad news, although I do have a vision of NB Fancy Free finding itself perched alone on the muddy plain that once was Hampstead Heath as the waters recede from the dead city...)
posted by Devonian at 3:18 AM on December 10, 2015 [1 favorite]

At this point I'm just gonna start assuming that anything with the prefix "cyber" is now about Cybermen.
cyberattacks = attacks by cybermen
cybersecurity = defences against cybermen attacks
cyberwarfare = the ongoing war with the cybermen
cyberpunk = kids dressing up in cybermen costumes to annoy their parents
posted by EndsOfInvention at 4:09 AM on December 10, 2015 [3 favorites]

It's really too bad. Cybernetics is a fascinating field, far predating sci-fi bs about cyborgs etc. Control systems and self regulation are kind of important. But I guess the prefix is useless now because someone decided it means "has something to do with computers".
posted by idiopath at 5:03 AM on December 10, 2015 [1 favorite]

I still snicker sometimes because I remember when "cyber" was generally used as an abbreviation for "cybersex" in chatrooms.

There are valid concerns involving information security and the power grid, but (in my not-so-humble opinion) raving panic about "OH GOD THE HACKERZ ARE GONNA GET US" is really not going to be as useful as a more general strategy for resilience and recovery. Squirrels and bad weather are still way ahead of hackers when it comes to taking down our power infrastructure. And I'd be willing to bet cash money that there's some critical (or unexpectedly critical) systems in use that are running operating systems that are no longer supported by the vendor, but cleaning up boring security issues like that isn't nearly as profitable or mediagenic as worrying about superhackers.

It's like people telling you to spend lots of money on a high-end Medeco lock on your door when there's a screen window right next to the front door.
posted by rmd1023 at 6:52 AM on December 10, 2015 [1 favorite]

Annika Cicada's post above is the first thing I've ever seen, anywhere that combines the word "cybersecurity" with evidence of clue.

I did hold my nose while typing that word. It resonates though. If I say "I'm a network security engineer" people look at me like "WTF is that?". I usually answer "I work for AMWAY making sure no one is allowed to ever leave the program".
posted by Annika Cicada at 7:24 AM on December 10, 2015 [4 favorites]

And any way I'm too busy tracing threat actors to get frustrated by the "cyber" prefix but goddamn it all the absolutely unnecessary military terminology in the security world feels like the heteropatriarchy on steroids some days. I can't wait until I don't have to do this anymore, honestly. Mostly because men.
posted by Annika Cicada at 7:31 AM on December 10, 2015 [2 favorites]

I always wonder if the reason these scare stories circulate periodically is that huge government defense contractors are trying to stir up business for themselves. I'd think they'd be pleased to accept a few multi billion dollar contracts to "protect" the electric grid.
posted by Bee'sWing at 8:24 AM on December 10, 2015

OK, now that I finished reading the comments, what Annika Cicada said.
posted by Bee'sWing at 8:33 AM on December 10, 2015

Thanks, but honestly scalefree said it better. Shower them with favorites too.
posted by Annika Cicada at 8:47 AM on December 10, 2015

I still snicker sometimes because I remember when "cyber" was generally used as an abbreviation for "cybersex" in chatrooms.

You could write two different genres if you based your work on this confusion. One would be a rolicking sex farce where some chump looking to get his wank on encounters an FBI agent online looking for terrorists. Ever escalating confusion and hijinks as they talk past each other and the civilian chump does weirder and weirder things trying to get odd and the FBI chump misinterprets the civilian's actions.

Keep writing a little longer and you can have a tragedy/law drama when the FBI charges and imprisons the clueless chump anyway.
posted by phearlez at 8:50 AM on December 10, 2015 [1 favorite]

Whenever I read about these concerns about hacking the grid I think that if we were serious about power security we'd hold electric meters in with more than a 12ga clip of metal. Pull that sucker out and the power's gone to the whole property.
posted by phearlez at 8:57 AM on December 10, 2015

« Older Devil's Hole Pupfish, the tiny, tenacious desert...   |   KOUIGN-AMANN Newer »

This thread has been archived and is closed to new comments