At the intersection of machine learning & journalism: Spy planes
August 7, 2017 2:05 PM Subscribe
BuzzFeed News Trained A Computer To Search For Hidden Spy Planes. This Is What We Found (Buzzfeed News) The algorithm was not infallible: Among other candidates, it flagged several skydiving operations that circled in a relatively small area, much like a typical surveillance aircraft. But as an initial screen for candidate spy planes, it proved very effective. In addition to aircraft operated by the US Marshals and the military contractor Acorn Growth Companies, covered in our previous stories, it highlighted a variety of planes flown by law enforcement, and by the military and its contractors.
Wow, this is some great reporting.
posted by sixpack at 2:20 PM on August 7, 2017 [1 favorite]
posted by sixpack at 2:20 PM on August 7, 2017 [1 favorite]
Fascinating. This feels a lot like spam detection ten years ago. You can obfuscate the metadata (the plane registration, by analogy) but you can't obfuscate the advertising message (the flight path).
posted by Leon at 2:28 PM on August 7, 2017 [6 favorites]
posted by Leon at 2:28 PM on August 7, 2017 [6 favorites]
Flying for the military, some were still transmitting the identifying code linked to their old civilian registration numbers, which had since been recycled.Oops! You'd think that would would be the first item on the check list when converting civilian craft to secret military use.
posted by Mitheral at 2:29 PM on August 7, 2017 [2 favorites]
We were initially baffled by some of the aircraft the algorithm flagged: They included one plane that identified itself as a powered parachute, and another as a small drone owned by a photographer.
Sounds like cross-checking the registration database against the flight data and flagging anything that's flying too fast/slow/high/far might throw up some weird craft this algorithm might have missed.
posted by Leon at 2:33 PM on August 7, 2017 [2 favorites]
Sounds like cross-checking the registration database against the flight data and flagging anything that's flying too fast/slow/high/far might throw up some weird craft this algorithm might have missed.
posted by Leon at 2:33 PM on August 7, 2017 [2 favorites]
Wasn't this (or something very similar) reported last year? I'm having some serious deja vu.
Edit: Yep.
posted by rp at 2:33 PM on August 7, 2017 [2 favorites]
Edit: Yep.
posted by rp at 2:33 PM on August 7, 2017 [2 favorites]
Oh I can't wait until 2020 when basically every aircraft will have ADS-B by law. A lot of military flights already do have Mode S transponders and they even leave them on for many training/repositioning flights, so applying big data techniques to crowdsourced data will make a lot of organizations like the company in Davie get their knickers in a twist because they find it harder to keep secrets about where their assets are operating.
Even without ADS-B, aircraft are easily trackable using multilateration off their Mode A/C transponder, and now that the software and hardware to do that is getting ever more widely available and ever cheaper, respectively, it's getting to the point where if ATC can see you, so can everyone else, even if you do have your tail number blocked.
posted by wierdo at 2:48 PM on August 7, 2017 [1 favorite]
Even without ADS-B, aircraft are easily trackable using multilateration off their Mode A/C transponder, and now that the software and hardware to do that is getting ever more widely available and ever cheaper, respectively, it's getting to the point where if ATC can see you, so can everyone else, even if you do have your tail number blocked.
posted by wierdo at 2:48 PM on August 7, 2017 [1 favorite]
Couldn't the military/police/contractors/deep-pocketed private agencies/whoever theoretically buy up/acquire a large pool of aircraft IDs (possibly with assistance from the FAA) and swap them regularly, perhaps not using one for more than one covert operation? Or even get dispensation to generate new ones, much in the way that phones generate new MAC addresses to avoid tracking.
Of course, this would just transform the problem from spam detection to Twitter-bot detection (i.e., flag newly registered accounts/ones with fake-looking histories). So they'd need to “pre-age” these IDs; perhaps fly them on routine flights for a while, or even (if electronic databases suffice) have fake histories inserted into the databases.
posted by acb at 2:56 PM on August 7, 2017 [3 favorites]
Of course, this would just transform the problem from spam detection to Twitter-bot detection (i.e., flag newly registered accounts/ones with fake-looking histories). So they'd need to “pre-age” these IDs; perhaps fly them on routine flights for a while, or even (if electronic databases suffice) have fake histories inserted into the databases.
posted by acb at 2:56 PM on August 7, 2017 [3 favorites]
Wasn't there a mefite doing this?
Yes, jjwiseman tracked FBI aircraft. Discussed in this thread. It was a fun thing to read about then as is this post. This stuff is fascinating to me. It's simpler than I would guess to spot these flights.
posted by Clinging to the Wreckage at 4:54 PM on August 7, 2017 [2 favorites]
Yes, jjwiseman tracked FBI aircraft. Discussed in this thread. It was a fun thing to read about then as is this post. This stuff is fascinating to me. It's simpler than I would guess to spot these flights.
posted by Clinging to the Wreckage at 4:54 PM on August 7, 2017 [2 favorites]
A lot of military flights already do have Mode S transponders and they even leave them on for many training/repositioning flights...
Have you seen anyone spoofing data? I'm admittedly not very knowledgeable about this stuff, but I've been running PiAware for a little over a year. I often pull up my local page when I hear low-flying planes overhead, and there was one day in which I had an entry that came up as a military-registered DC-9 or DC-10 (can't remember), but when I sighted it visually, it was definitely 4 small planes in formation, not a single jetliner.
posted by god hates math at 5:05 PM on August 7, 2017 [1 favorite]
Have you seen anyone spoofing data? I'm admittedly not very knowledgeable about this stuff, but I've been running PiAware for a little over a year. I often pull up my local page when I hear low-flying planes overhead, and there was one day in which I had an entry that came up as a military-registered DC-9 or DC-10 (can't remember), but when I sighted it visually, it was definitely 4 small planes in formation, not a single jetliner.
posted by god hates math at 5:05 PM on August 7, 2017 [1 favorite]
Since I'm in flyover country, I recently decided to run a piaware. I haven't had time to watch it much lately, but for my first few days I followed it obsessively. Here's a few interesting ones I've noticed:
Callsign "Rider69", labeled (at the time) as a C-17. Here's some more tracking of this C-17, including some alternate callsigns.
US Transportation Command 562
United States Department Of Justice Prisoner And Alien Transportation Service 431
It's also interesting the people who block tracking. Anytime a tail number comes up as blocked, I look up the registration. And sometimes sites less scrupulous than Flightaware will have partial tracks of the blocked aircraft. Lots of private planes overhead which wish I didn't know they were there. One afternoon in our endless Kansas sky, the hidden flights I was tracking included the founder of Foxconn heading east, while the Jordanian Royal Family was heading west, and the Assemblies of God Annuities Department were on their way to being comfortably godly in Wichita.
posted by honestcoyote at 9:25 PM on August 7, 2017 [9 favorites]
Callsign "Rider69", labeled (at the time) as a C-17. Here's some more tracking of this C-17, including some alternate callsigns.
US Transportation Command 562
United States Department Of Justice Prisoner And Alien Transportation Service 431
It's also interesting the people who block tracking. Anytime a tail number comes up as blocked, I look up the registration. And sometimes sites less scrupulous than Flightaware will have partial tracks of the blocked aircraft. Lots of private planes overhead which wish I didn't know they were there. One afternoon in our endless Kansas sky, the hidden flights I was tracking included the founder of Foxconn heading east, while the Jordanian Royal Family was heading west, and the Assemblies of God Annuities Department were on their way to being comfortably godly in Wichita.
posted by honestcoyote at 9:25 PM on August 7, 2017 [9 favorites]
I did start down the road of automatically detecting surveillance aircraft, but didn't go very far for a few reasons. One was that I didn't have access to flights beyond what I could pick up with my radio (I love that Buzzfeed got access to seemingly everything Flightradar24 has, and I'd be interested in hearing more about that arrangement), and it was very easy to spot the surveillance flights I did pick up. Another was that it was so easy: Literally every* surveillance flight that I know about flies in circles for hours. Most at around 5,000 feet altitude. Squawking 4414 or 4415 on the transponder means they're almost certainly a federal agency doing surveillance, but lots of them don't use that squawk. Their classifier code found the same thing, with the most important variables being steern, squawk_1, duration4 and altitude3.
Besides those behavioral markers, I found additional metadata that was significant.
* Once exception might be a clover-shaped flight track. The aircraft in question ticks a number of other boxes (it's a Cessna T206H owned by a defense contractor flying for long periods at about the right altitude) but I could never confirm it was doing surveillance.
Have you seen anyone spoofing data?
Reports are that tactical (fighters etc.) aircraft often spoof their Mode S code. For example.
posted by jjwiseman at 1:47 PM on August 8, 2017 [5 favorites]
Besides those behavioral markers, I found additional metadata that was significant.
* Once exception might be a clover-shaped flight track. The aircraft in question ticks a number of other boxes (it's a Cessna T206H owned by a defense contractor flying for long periods at about the right altitude) but I could never confirm it was doing surveillance.
Have you seen anyone spoofing data?
Reports are that tactical (fighters etc.) aircraft often spoof their Mode S code. For example.
posted by jjwiseman at 1:47 PM on August 8, 2017 [5 favorites]
« Older seducing them with imagined realities | Everything's Coming Up Rosies Newer »
This thread has been archived and is closed to new comments
posted by CrystalDave at 2:05 PM on August 7, 2017 [1 favorite]