Eavesdropping on the Hidden World
November 28, 2015 1:54 PM Subscribe
In March of 2010, programmer Eric Fry discovered a cheap digital tuner from Realtek could be modified to receive more than mere TV and FM Radio. Much more.
For years, Amateur Radio has mired by expensive hardware and a difficult learning curve. However thanks in part to the FCC eliminating the requirement to learn Morse Code as a part of license examination, as well as a new cadre of hardware hackers and tinkerers, Scanning and Ham Radio is enjoying a Renaissance. With the purchase of an inexpensive USB dongle and the addition of a small driver patch, anyone can surf the radio waves using a computer or tablet. Radio operators around the world have added web-controlled receivers to their arsenals of tools, allowing anyone to Listen in. (Previously)
Support for hobbyists new to the field can get up to speed with a variety of guides, user groups, and knowledge repositories. But what can you listen to besides Commercial Broadcasting? The range of modern SDR devices covers broad frequencies including Aviation and Marine tracking, decoding Trunked radio systems, downloading Weather Satellite images, reversing wireless protocols, hunting for Numbers Stations, or even watch the International Space Station broadcast Slow-scan TV.
Side projects include Signal Identification and Frequency Mapping, and many guides exist for help in receiving an Amateur Radio license.
- - -
(Missed First Post month by, well, a lot. I was still inspired to make this small contribution. HI MOM!)
For years, Amateur Radio has mired by expensive hardware and a difficult learning curve. However thanks in part to the FCC eliminating the requirement to learn Morse Code as a part of license examination, as well as a new cadre of hardware hackers and tinkerers, Scanning and Ham Radio is enjoying a Renaissance. With the purchase of an inexpensive USB dongle and the addition of a small driver patch, anyone can surf the radio waves using a computer or tablet. Radio operators around the world have added web-controlled receivers to their arsenals of tools, allowing anyone to Listen in. (Previously)
Support for hobbyists new to the field can get up to speed with a variety of guides, user groups, and knowledge repositories. But what can you listen to besides Commercial Broadcasting? The range of modern SDR devices covers broad frequencies including Aviation and Marine tracking, decoding Trunked radio systems, downloading Weather Satellite images, reversing wireless protocols, hunting for Numbers Stations, or even watch the International Space Station broadcast Slow-scan TV.
Side projects include Signal Identification and Frequency Mapping, and many guides exist for help in receiving an Amateur Radio license.
- - -
(Missed First Post month by, well, a lot. I was still inspired to make this small contribution. HI MOM!)
Favoriting this to read later, because getting involved with HAM Radio and getting my license has been on my bucket list for like, forever. Is 2016 THE YEAR I DO IT?
posted by Major Matt Mason Dixon at 3:47 PM on November 28, 2015 [7 favorites]
posted by Major Matt Mason Dixon at 3:47 PM on November 28, 2015 [7 favorites]
Wanna stream Video over the air from your webcam? Be the sequel to Pump Up the Volume!
posted by The Power Nap at 4:41 PM on November 28, 2015 [2 favorites]
posted by The Power Nap at 4:41 PM on November 28, 2015 [2 favorites]
Huh. Do you need a license to listen, it just to talk?
posted by rebent at 4:43 PM on November 28, 2015
posted by rebent at 4:43 PM on November 28, 2015
If you want to have a taste of what it's like without investing in anything, web-based Software Defined Radio is a great way to check out what's floating around out there. My favorite is the one from the University of Twente, which will let you listen across a huge range of wavelengths. The time change is especially nice-- shortwave tends to get better reception at night and since it's across the Atlantic, I can listen during the day here in the States. No dongle required.
posted by WidgetAlley at 4:47 PM on November 28, 2015 [11 favorites]
posted by WidgetAlley at 4:47 PM on November 28, 2015 [11 favorites]
I've experimented with a few of these dongles, and they are a terrific and extremely cheap way to do stuff that would cost orders of magnitudes more not so very long ago. Even the upconverters are just a few dollars, so with an RTL dongle, one of those and some free software you can build a reasonably exciting communications receiver that in some respects has the capabilities of a thousand-dollar box from ten years ago. I find it particularly joyous to bung one on the end of a USB To Go adaptor on a cheap Android phone, download one of the apps, and have something in my pocket that does stuff a deskful of pre-2000 radio kit couldn't manage.
A few caveats, learned the hard way. You do need to be a bit sensible about the antenna. Many dongles come with a little wire thing on a few inches of crapulous coax, and this will pick up nothing except the noise from your computing gear/networking/plasma telly and some strong broadcast FM. Get something - anything - outside in the open air, use some half-decent coax, and things will come alive.
There are still driver and chipset issues, which can be as fun to diagnose as you may imagine. Be attentive to the exact sequence of numbers that describes your dongle's radio chip.
Beware GNURadio. It is a fabulously powerful dev environment that rolls up stupendous amounts of radio smarts, and it most certainly works with RTL dongles, but it is not packaged or designed as a tinker toy. I found my best results came when using a distro that had GNURadio preinstalled; I found (more than once) that trying to install it from cold on a vanilla Ubuntu distro led to so much unsolvable dependency-related pain that I had to trash the whole thing. (When it took out VLC, I knew I was beaten.) I'm sure this was me being a Gnubie - I recognised the experience from early Linux days - but I haven't lost that badly for a while.
This was about a year ago, things may have improved as I haven't had the time to get as seriously stuck into this as I'dve hoped. I do have one major project in abeyance that needs GNURadio; I shall return!
The radio side of the RTL dongles isn't very good, in terms of frequency accuracy, sensitivity, strong signal performance and noise. It's not terrible-terrible, and for the price it's astonishing, but if you get into it you'll find that you'll need more and more help from good antennas, filters and that sort of thing.
But - if you're prepared to invest a bit of time, this is the golden age for radio fun. If you do get interested, the really good news is that if you spend a bit more, you can get SDR radios for between $100-$200 that perform spectacularly well, and the free software to control them is a wet dream.
Have at it!
posted by Devonian at 5:24 PM on November 28, 2015 [32 favorites]
A few caveats, learned the hard way. You do need to be a bit sensible about the antenna. Many dongles come with a little wire thing on a few inches of crapulous coax, and this will pick up nothing except the noise from your computing gear/networking/plasma telly and some strong broadcast FM. Get something - anything - outside in the open air, use some half-decent coax, and things will come alive.
There are still driver and chipset issues, which can be as fun to diagnose as you may imagine. Be attentive to the exact sequence of numbers that describes your dongle's radio chip.
Beware GNURadio. It is a fabulously powerful dev environment that rolls up stupendous amounts of radio smarts, and it most certainly works with RTL dongles, but it is not packaged or designed as a tinker toy. I found my best results came when using a distro that had GNURadio preinstalled; I found (more than once) that trying to install it from cold on a vanilla Ubuntu distro led to so much unsolvable dependency-related pain that I had to trash the whole thing. (When it took out VLC, I knew I was beaten.) I'm sure this was me being a Gnubie - I recognised the experience from early Linux days - but I haven't lost that badly for a while.
This was about a year ago, things may have improved as I haven't had the time to get as seriously stuck into this as I'dve hoped. I do have one major project in abeyance that needs GNURadio; I shall return!
The radio side of the RTL dongles isn't very good, in terms of frequency accuracy, sensitivity, strong signal performance and noise. It's not terrible-terrible, and for the price it's astonishing, but if you get into it you'll find that you'll need more and more help from good antennas, filters and that sort of thing.
But - if you're prepared to invest a bit of time, this is the golden age for radio fun. If you do get interested, the really good news is that if you spend a bit more, you can get SDR radios for between $100-$200 that perform spectacularly well, and the free software to control them is a wet dream.
Have at it!
posted by Devonian at 5:24 PM on November 28, 2015 [32 favorites]
if you spend a bit more, you can get SDR radios for between $100-$200 that perform spectacularly well
Can you recommend a good place to start?
posted by Songdog at 6:00 PM on November 28, 2015 [1 favorite]
Can you recommend a good place to start?
posted by Songdog at 6:00 PM on November 28, 2015 [1 favorite]
Can you recommend a good place to start?
There's a very good overview at the RTL-SDR site. From everything I've heard, the SDRPlay RSP looks like the best value for money option at its price point, it's getting regular updates and a lot of hams are incorporating it into their setups with good results.
It's not the best - you can spend a lot more and get some remarkable abilities, like being able to watch an AM broadcast channel overnight receiving multiple stations and measure each frequency drift to sub-Hertz accuracies. Other radios cover higher frequencies, or have transmit options, or can deliver a bigger chunk of the spectrum for instantaneous monitoring, but if any of this matters to you you're already qualified to sort those decisions out for yourself...
Should have said - thanks to Endotoxin for the FPP! Radio rules!
posted by Devonian at 8:08 PM on November 28, 2015 [3 favorites]
There's a very good overview at the RTL-SDR site. From everything I've heard, the SDRPlay RSP looks like the best value for money option at its price point, it's getting regular updates and a lot of hams are incorporating it into their setups with good results.
It's not the best - you can spend a lot more and get some remarkable abilities, like being able to watch an AM broadcast channel overnight receiving multiple stations and measure each frequency drift to sub-Hertz accuracies. Other radios cover higher frequencies, or have transmit options, or can deliver a bigger chunk of the spectrum for instantaneous monitoring, but if any of this matters to you you're already qualified to sort those decisions out for yourself...
Should have said - thanks to Endotoxin for the FPP! Radio rules!
posted by Devonian at 8:08 PM on November 28, 2015 [3 favorites]
Also, MeFi's Own™ jjwiseman used this kit to track secret surveillance aircraft over LA.
They're also very handy for decoding the little data chirps and burps made by transmitters like remotes and low-tech radio switches in the ISM bands. I had fretted for ages using arduinos and receivers to try and decode a power monitor transmission. In a couple of weeks, two folks with RTL-SDR dongles had worked out the protocol and written a robust decoder. Impressive stuff.
posted by scruss at 8:56 PM on November 28, 2015 [5 favorites]
They're also very handy for decoding the little data chirps and burps made by transmitters like remotes and low-tech radio switches in the ISM bands. I had fretted for ages using arduinos and receivers to try and decode a power monitor transmission. In a couple of weeks, two folks with RTL-SDR dongles had worked out the protocol and written a robust decoder. Impressive stuff.
posted by scruss at 8:56 PM on November 28, 2015 [5 favorites]
I still remember when one of my friends pointed a sat dish at the sky and got Internet traffic.
And then another started up SASA, the Southern Appalachian Space Agency...
...which is to say, there's a lot of weird radio out there, assuming it's hard to pick up. False assumption, getting falser.
posted by effugas at 9:57 PM on November 28, 2015 [2 favorites]
And then another started up SASA, the Southern Appalachian Space Agency...
...which is to say, there's a lot of weird radio out there, assuming it's hard to pick up. False assumption, getting falser.
posted by effugas at 9:57 PM on November 28, 2015 [2 favorites]
If any readers are wondering what's broadcasting in shortwave, a great place to start is with the websites for the major international broadcasters; one can listen to their programs online (or even as podcasts), and it might just be the inspiration to try to listen to them over the air. Here are my top five: Radio France International, Radio Netherlands, Deutsche Welle, BBC, and Voice of America.
Oh, and thanks for the post; now I have a new project for the winter break!
posted by math at 10:26 PM on November 28, 2015 [4 favorites]
Oh, and thanks for the post; now I have a new project for the winter break!
posted by math at 10:26 PM on November 28, 2015 [4 favorites]
I have two friends who have recently gotten into ham radio, one of whom was able to get their granddad's old call sign and the other intends to. So ham has maybe skipped a generation?
posted by tavella at 11:24 PM on November 28, 2015
posted by tavella at 11:24 PM on November 28, 2015
IIRC, back in the day, the emergence of SDR really freaked out TPTB. The idea of no gatekeepers enforcing regulation? Bureaucratic Minds Blown. Is transmitting still a PITA?
posted by mikelieman at 1:47 AM on November 29, 2015
posted by mikelieman at 1:47 AM on November 29, 2015
The HackRF One can transmit, and is sanely priced though not cheap. Also has a wide range response. It would need an amp to get any distance, though. Licensing is an exercise for the user.
posted by Bovine Love at 5:50 AM on November 29, 2015 [1 favorite]
posted by Bovine Love at 5:50 AM on November 29, 2015 [1 favorite]
I used to listen to short wave alot when I was a kid (in the 70s!). It was a real window to the world, in the same way that the internet is now. Recently I tried to demonstrate SW to my son using my trusty Longines Symphonette. Maybe because it was daytime, but all I could seem to pick up were religious stations. I'm sure the internet has caused many SW stations to shut down. It felt really empty.
posted by jabah at 7:09 AM on November 29, 2015
posted by jabah at 7:09 AM on November 29, 2015
The 70s and 80s were the best time to do shortwave broadcast listening. There was no Web, satellite TV was still being born, so there was no other way for countries to get their voice heard abroad in the way they wanted it. During the Cold War, that was important - and shortwave broadcasting was well funded and highly competitive.
Most countries have shut down their shortwave BC transmitters, either moving online or just not bothering. There's still a lot of stuff out there, but it's much more fragmented, usually not in English, and harder to hear (China is the exception). On the plus side, there are some good online resources, and as mentioned it's never been easier to go to a WebSDR site and listen from your browser. Some of us still like scanning the bands old-school, though.
I can highly recommend Jonathan Marks' Media Network Vault for those who used to listen to broadcast shortwave. It's a huge collection of Radio Nederlands' Media Network programmes from the 80s through to the 2000s This was a weekly news/reviews/features show covering international broadcasting and matters pertaining. It documents the whole process of shortwave (and other) broadcasting changing as the Cold War ended and the Internet turned up, as everything happened. It's a unique window on a very interesting corner of media culture, and listening to the programmes in high quality (when you remember hearing them on crackly shortwave) is quite something.
posted by Devonian at 7:32 AM on November 29, 2015 [1 favorite]
Most countries have shut down their shortwave BC transmitters, either moving online or just not bothering. There's still a lot of stuff out there, but it's much more fragmented, usually not in English, and harder to hear (China is the exception). On the plus side, there are some good online resources, and as mentioned it's never been easier to go to a WebSDR site and listen from your browser. Some of us still like scanning the bands old-school, though.
I can highly recommend Jonathan Marks' Media Network Vault for those who used to listen to broadcast shortwave. It's a huge collection of Radio Nederlands' Media Network programmes from the 80s through to the 2000s This was a weekly news/reviews/features show covering international broadcasting and matters pertaining. It documents the whole process of shortwave (and other) broadcasting changing as the Cold War ended and the Internet turned up, as everything happened. It's a unique window on a very interesting corner of media culture, and listening to the programmes in high quality (when you remember hearing them on crackly shortwave) is quite something.
posted by Devonian at 7:32 AM on November 29, 2015 [1 favorite]
Cool. I'm definitely going to check this out; I recently bought one of the ARRL books on getting a Technician license, my goal for 2016; my dad is a ham, and I was joking with him the other day on the phone that I think the morse code for CQ and QSL is permanently imprinted onto my brain somewhere.
It kills me how many broadcasters have stopped their shortwave; some of my early and fonder childhood memories are from sitting at my dad's radio station in the laundry room and scanning through the channels on his different antennas and hearing the BBC, Radio Cuba, and a million other international stations that gave me a tiny glimpse into an untapped world and put its hooks in me. Twenty years later, a lot of those childhood memories are permanently off the air, which feels like such a loss [empty, as a prior poster said].
Ham radio was the original DIY for me, way before computers, watching my dad build and fix (and sell) antennas and switches and every part you might possibly need; he impressed on us the conviction that you are absolutely capable of figuring out a solution for your problems. So I'm happy to see that it's not going anywhere, especially if it gets more people under the age of 50.
posted by circle_b at 10:17 AM on November 29, 2015 [1 favorite]
It kills me how many broadcasters have stopped their shortwave; some of my early and fonder childhood memories are from sitting at my dad's radio station in the laundry room and scanning through the channels on his different antennas and hearing the BBC, Radio Cuba, and a million other international stations that gave me a tiny glimpse into an untapped world and put its hooks in me. Twenty years later, a lot of those childhood memories are permanently off the air, which feels like such a loss [empty, as a prior poster said].
Ham radio was the original DIY for me, way before computers, watching my dad build and fix (and sell) antennas and switches and every part you might possibly need; he impressed on us the conviction that you are absolutely capable of figuring out a solution for your problems. So I'm happy to see that it's not going anywhere, especially if it gets more people under the age of 50.
posted by circle_b at 10:17 AM on November 29, 2015 [1 favorite]
I just mentioned my planetracking in the Raspberry Pi thread, but I'll mention a few more details here relevant to SDR and its applications.
If you want to try planetracking/planespotting/radarspotting, a relatively easy and very cheap way to start is with pretty much any RTL-SDR dongle and dump1090, which is a free decoding program. That's what I used--with the dumb little stock antenna that comes with the dongle--to figure out all the FBI surveillance stuff. I don't recommend PlanePlotter (linked in the post): It's Windows-only, costs money, is a usability nightmare, and has a relatively low quality community around it.
There are multiple versions of dump1090, but you can't really choose a bad one. By default I would recommend the fork by mutability/Oliver Jowett because it's under active development and has the most features. Oliver Jowett also seems to be working for/closely with FlightAware, which brings me to the easiest way to take yourself to the next level in planetracking:
FlightAware (and FlightRadar24, though I know less about them) offer a shopping list and instructions for building a tracking station that includes a Raspberry Pi and purpose-built antenna for under $100, which will be capable of picking up planes up to 300 miles away. If you run their software piaware, which is basically dump1090 plus a program to communicate with their servers, you will get a premium account for free, and be able to participate in their multilateration network. Multilateration lets you calculate the position of aircraft that aren't broadcasting their position via ADS-B. I find that useful because many of the aircraft I'm interested in (like FBI surveillance planes) don't use ADS-B.
You can see details and statistics for my piaware station here: http://flightaware.com/adsb/stats/user/jjwiseman
Here's a screenshot of the little virtual radar web app that's built into dump1090: https://www.flickr.com/photos/wallofhair/22773687884. I took that just now, so it's showing all the aircraft I'm tracking at this moment. Gray icons are aircraft that are broadcasting position with ADS-B, and blue icons are aircraft whose positions have been multilaterated.
I also run a fancier tracking web app called Virtual Radar Server that takes the output of dump1090 as input. Here's a screenshot: https://www.flickr.com/photos/wallofhair/23402055675. The main reason I use it is because it can show track trails for all aircraft at once, which makes it easier for me to find new surveillance aircraft, which typically fly in circles, at a glance.
(BTW, Shodan makes it very easy to find other people's dump1090 servers that are open to the internet. I would be very nervous about doing that with a web server that's written in C.)
If you're really into aviation, SDR will also let you decode ACARS messages and listen to air communications.
I don't have as much experience with other SDR applications, but since I don't think it's been mentioned in this thread yet I'll link to gqrx, which I found to be a good receiver app.
Finally, I hope more people get involved in SDR. I find some of the current areas of the community to be throwbacks to the bad parts of "amateur radio/software guy" culture, in terms of unfriendliness and the desire to keep information proprietary but also just in old fashioned development culture. Some small examples of this are the fact that there are probably a half dozen forks of dump1090 that all have the same name, the popularity of some awful proprietary software, and the drama that pops up around sharing (e.g.) databases of aircraft information. I'd love to see more fun and ideas of radical openness come into the community.
posted by jjwiseman at 11:05 AM on November 29, 2015 [31 favorites]
If you want to try planetracking/planespotting/radarspotting, a relatively easy and very cheap way to start is with pretty much any RTL-SDR dongle and dump1090, which is a free decoding program. That's what I used--with the dumb little stock antenna that comes with the dongle--to figure out all the FBI surveillance stuff. I don't recommend PlanePlotter (linked in the post): It's Windows-only, costs money, is a usability nightmare, and has a relatively low quality community around it.
There are multiple versions of dump1090, but you can't really choose a bad one. By default I would recommend the fork by mutability/Oliver Jowett because it's under active development and has the most features. Oliver Jowett also seems to be working for/closely with FlightAware, which brings me to the easiest way to take yourself to the next level in planetracking:
FlightAware (and FlightRadar24, though I know less about them) offer a shopping list and instructions for building a tracking station that includes a Raspberry Pi and purpose-built antenna for under $100, which will be capable of picking up planes up to 300 miles away. If you run their software piaware, which is basically dump1090 plus a program to communicate with their servers, you will get a premium account for free, and be able to participate in their multilateration network. Multilateration lets you calculate the position of aircraft that aren't broadcasting their position via ADS-B. I find that useful because many of the aircraft I'm interested in (like FBI surveillance planes) don't use ADS-B.
You can see details and statistics for my piaware station here: http://flightaware.com/adsb/stats/user/jjwiseman
Here's a screenshot of the little virtual radar web app that's built into dump1090: https://www.flickr.com/photos/wallofhair/22773687884. I took that just now, so it's showing all the aircraft I'm tracking at this moment. Gray icons are aircraft that are broadcasting position with ADS-B, and blue icons are aircraft whose positions have been multilaterated.
I also run a fancier tracking web app called Virtual Radar Server that takes the output of dump1090 as input. Here's a screenshot: https://www.flickr.com/photos/wallofhair/23402055675. The main reason I use it is because it can show track trails for all aircraft at once, which makes it easier for me to find new surveillance aircraft, which typically fly in circles, at a glance.
(BTW, Shodan makes it very easy to find other people's dump1090 servers that are open to the internet. I would be very nervous about doing that with a web server that's written in C.)
If you're really into aviation, SDR will also let you decode ACARS messages and listen to air communications.
I don't have as much experience with other SDR applications, but since I don't think it's been mentioned in this thread yet I'll link to gqrx, which I found to be a good receiver app.
Finally, I hope more people get involved in SDR. I find some of the current areas of the community to be throwbacks to the bad parts of "amateur radio/software guy" culture, in terms of unfriendliness and the desire to keep information proprietary but also just in old fashioned development culture. Some small examples of this are the fact that there are probably a half dozen forks of dump1090 that all have the same name, the popularity of some awful proprietary software, and the drama that pops up around sharing (e.g.) databases of aircraft information. I'd love to see more fun and ideas of radical openness come into the community.
posted by jjwiseman at 11:05 AM on November 29, 2015 [31 favorites]
BTW, endotoxin: 135 favorites (currently) for a first post is doing pretty well! :)
posted by jjwiseman at 11:09 AM on November 29, 2015 [2 favorites]
posted by jjwiseman at 11:09 AM on November 29, 2015 [2 favorites]
I find some of the current areas of the community to be throwbacks to the bad parts of "amateur radio/software guy" culture, in terms of unfriendliness and the desire to keep information proprietary but also just in old fashioned development culture.
This. There is so much closed source, windows only, outsiders-unwelcome software around SDR it's crazy. This is a field that could benefit like mad from being able to share code and make incremental improvements, but the culture just doesn't do that. dump1090 is a great counterexample (written by the guy who made Redis), but it's a rarity. ADS-B is thus basically a solved problem. There are so many other protocols that should be solved problems, but they're not because they only half-work in closed source programs, so nobody can build off them and they have to decide between starting from scratch or living with broken software. P25, the digital voice format that is becoming very common, is one of these. This could be a killer feature for SDR since radios that support it are otherwise expensive, but the software is just not there (a group actually took an open source program that was 80% there, applied their bugfixes, and took it closed source). I've implemented myself the protocols that are most useful for me, but it feels like what's the use in writing the one half I do know about (against a stale codebase) when the other half I need is in a proprietary fork.
Similarly, "scanning" a list of known frequencies for voice traffic, like one normally uses a regular radio for, is just not up to par using the existing software (unless you want a proprietary plug in to a proprietary program running on .NET). SDRs should totally kill at signal discovery and logging, but there's no software.
I guess this pains me just because it's all so close, yet so far. So many folks are excited by the few protocols that work well (because they are small enough that one person can implement them in their spare time e.g. ADS-B ) that it seems we could go a lot farther if it was done cooperatively instead of in silos.
posted by kiltedtaco at 11:47 AM on November 29, 2015 [3 favorites]
This. There is so much closed source, windows only, outsiders-unwelcome software around SDR it's crazy. This is a field that could benefit like mad from being able to share code and make incremental improvements, but the culture just doesn't do that. dump1090 is a great counterexample (written by the guy who made Redis), but it's a rarity. ADS-B is thus basically a solved problem. There are so many other protocols that should be solved problems, but they're not because they only half-work in closed source programs, so nobody can build off them and they have to decide between starting from scratch or living with broken software. P25, the digital voice format that is becoming very common, is one of these. This could be a killer feature for SDR since radios that support it are otherwise expensive, but the software is just not there (a group actually took an open source program that was 80% there, applied their bugfixes, and took it closed source). I've implemented myself the protocols that are most useful for me, but it feels like what's the use in writing the one half I do know about (against a stale codebase) when the other half I need is in a proprietary fork.
Similarly, "scanning" a list of known frequencies for voice traffic, like one normally uses a regular radio for, is just not up to par using the existing software (unless you want a proprietary plug in to a proprietary program running on .NET). SDRs should totally kill at signal discovery and logging, but there's no software.
I guess this pains me just because it's all so close, yet so far. So many folks are excited by the few protocols that work well (because they are small enough that one person can implement them in their spare time e.g. ADS-B ) that it seems we could go a lot farther if it was done cooperatively instead of in silos.
posted by kiltedtaco at 11:47 AM on November 29, 2015 [3 favorites]
Couldn't agree more, kiltedface. There are some open-source endeavours that are bearing fruit, but they're mostly ham radio focused. I've long felt the lack of a decent open framework for not just the raw radio side (which, despite my gripes about usability, GNURadio provides) but for automation, UX experimentation, and modular development. I have been wondering if I'm just missing stuff that's out there, but your post makes me think that perhaps that's not it.
I hope, in the New Year, to have a go at doing a proper review of who's doing what where in SDR open source developments, what the available frameworks/APIs are, and what might be the best way to contribute what I can to move things along. I've been using open source for long enough and wondering what I could give back that wouldn't founder on the rocks of frustration or lack of interest, that I think it's about time. We need the digital equivalent of the old-school analogue radio exploration experience, and we don't have it. And it could do so much more.
Basically, I want more cool stuff, and I want more people to have cool stuff, and all the bits are just lying around in the Internet's equivalent of the junk box. Time to warm up the virtual soldering iron and start making.
posted by Devonian at 1:11 PM on November 29, 2015 [3 favorites]
I hope, in the New Year, to have a go at doing a proper review of who's doing what where in SDR open source developments, what the available frameworks/APIs are, and what might be the best way to contribute what I can to move things along. I've been using open source for long enough and wondering what I could give back that wouldn't founder on the rocks of frustration or lack of interest, that I think it's about time. We need the digital equivalent of the old-school analogue radio exploration experience, and we don't have it. And it could do so much more.
Basically, I want more cool stuff, and I want more people to have cool stuff, and all the bits are just lying around in the Internet's equivalent of the junk box. Time to warm up the virtual soldering iron and start making.
posted by Devonian at 1:11 PM on November 29, 2015 [3 favorites]
Ach, Kiltedtaco. Not kiltedface. Optical pattern recognition not my strong point these days! Apologies.
posted by Devonian at 1:18 PM on November 29, 2015 [1 favorite]
posted by Devonian at 1:18 PM on November 29, 2015 [1 favorite]
great post endotoxin, and many thanks to jjwiseman for additional infos. pretty sure I'd been using realtek dongles for at least a year or two before I was finally inspired to get my Amateur license in 2012 (hey I was too lazy for Technician - and any primate can pass Amateur exam) - and it seemed to die out after a while.
so it's great that diy-places are finally selling/encouraging this.
major NB: the ones you can get insanely-cheap e.g. on ebay, are often missing important voltage-protection diodes. I've bought more than a few of those over the years, but recently to feel safer I pay a few dollars more for ones like NooElec products that I know are not skimping on the electronics-parts or taking shortcuts.
I've always preferred Malcolm Robb's branch, but this newer branch you guys mentioned has renewed my interest.
nice that people are shipping packages/repos now! in the past I have had zero problem pulling from git and compiling directly on:
- BBB/debian
- BBB/archlinux
- BBB/ubuntu
- PogoPlug/archlinux-armv5
- Intel Edision / debian-i386
- OUYA/debian
- Amazon fireTV/LUCI -gone-wrong-thing
- various LG and Samsung Android phones + OTG (also using LUCI - it was just so much easier!)
- generic ubuntu-i386
- generic ubuntu-x86_64
even on super-low-power machines, this hardly taxes resources at all. (but! in my experience, it is important to note, however, that you most likely need a powered USB hub in between dongle and server-machine)
I love the built-in web interface of dump1090, and the fact you can also remotely use any UI like rtl1090 and adsbscope to point at the dump1090 server. (adsbscope in particular will let you add in map-things like runway markers and other cool stuff)
odinsdream, as Devonian and others have mentioned, the antenna that comes with most of these, is shite for this purpose. so the best you will likely get is indeed as you say, mainly AM/FM. which can still be fun. you need to determine what it is you mainly want to listen to, to figure out what sort/length of antenna you want. this takes a bit of work, but it sounds like now that places like adafruit are into it, they will probably be more helpful unlike in the past where they had no clue. ('cos yeah you really also need some filters to clean up otherwise FM is you will likely ever get.)
I really only listen to ADS-B i.e. aircraft stuff; I picked up some J-pole/slimjim types (highly recommend callsign N9TAX to buy jpoles from), in the correct-ish wavelength - nice b/c they take up almost no space, work pretty well from indoors, and are so light and portable if you want to e.g. go hiking with one. I'm on an island in the Atlantic and I easily get planes from NYC around 400 miles away.
also you will need to figure out what pigtail/converter (the shorter the better!) you need to connect to your antenna. and what sort of other cabling to use (hint: 75ohm cable may be cheaper and easier to find, and for RX-only applications, the impedance-mismatch is vaguely negligible, if not technically-correct.) - I spent so much time back in the day, on ebay researching MCX stuff (which actually has great response specs and MTBF/duty), but apparently now people are shipping SMA which may be easier. (also, whatever the connection on the dongle-side is, it may not be a bad idea to consider FME on the other end. it is so generic and you can find just about any converter from it to anything, plus has great frequency response, plus gas-proof, plus TINY so if you want to drill/pull cables thru walls, no need to worry about ginormous N stuff, plus excellent duty cycles etc. etc. etc.)
posted by dorian at 5:40 PM on November 29, 2015 [4 favorites]
so it's great that diy-places are finally selling/encouraging this.
major NB: the ones you can get insanely-cheap e.g. on ebay, are often missing important voltage-protection diodes. I've bought more than a few of those over the years, but recently to feel safer I pay a few dollars more for ones like NooElec products that I know are not skimping on the electronics-parts or taking shortcuts.
I've always preferred Malcolm Robb's branch, but this newer branch you guys mentioned has renewed my interest.
nice that people are shipping packages/repos now! in the past I have had zero problem pulling from git and compiling directly on:
- BBB/debian
- BBB/archlinux
- BBB/ubuntu
- PogoPlug/archlinux-armv5
- Intel Edision / debian-i386
- OUYA/debian
- Amazon fireTV/LUCI -gone-wrong-thing
- various LG and Samsung Android phones + OTG (also using LUCI - it was just so much easier!)
- generic ubuntu-i386
- generic ubuntu-x86_64
even on super-low-power machines, this hardly taxes resources at all. (but! in my experience, it is important to note, however, that you most likely need a powered USB hub in between dongle and server-machine)
I love the built-in web interface of dump1090, and the fact you can also remotely use any UI like rtl1090 and adsbscope to point at the dump1090 server. (adsbscope in particular will let you add in map-things like runway markers and other cool stuff)
odinsdream, as Devonian and others have mentioned, the antenna that comes with most of these, is shite for this purpose. so the best you will likely get is indeed as you say, mainly AM/FM. which can still be fun. you need to determine what it is you mainly want to listen to, to figure out what sort/length of antenna you want. this takes a bit of work, but it sounds like now that places like adafruit are into it, they will probably be more helpful unlike in the past where they had no clue. ('cos yeah you really also need some filters to clean up otherwise FM is you will likely ever get.)
I really only listen to ADS-B i.e. aircraft stuff; I picked up some J-pole/slimjim types (highly recommend callsign N9TAX to buy jpoles from), in the correct-ish wavelength - nice b/c they take up almost no space, work pretty well from indoors, and are so light and portable if you want to e.g. go hiking with one. I'm on an island in the Atlantic and I easily get planes from NYC around 400 miles away.
also you will need to figure out what pigtail/converter (the shorter the better!) you need to connect to your antenna. and what sort of other cabling to use (hint: 75ohm cable may be cheaper and easier to find, and for RX-only applications, the impedance-mismatch is vaguely negligible, if not technically-correct.) - I spent so much time back in the day, on ebay researching MCX stuff (which actually has great response specs and MTBF/duty), but apparently now people are shipping SMA which may be easier. (also, whatever the connection on the dongle-side is, it may not be a bad idea to consider FME on the other end. it is so generic and you can find just about any converter from it to anything, plus has great frequency response, plus gas-proof, plus TINY so if you want to drill/pull cables thru walls, no need to worry about ginormous N stuff, plus excellent duty cycles etc. etc. etc.)
posted by dorian at 5:40 PM on November 29, 2015 [4 favorites]
Aside from the pure SDR tools, there's also a bunch of work happening around the digital comms modes that many cheap consumer products use. From repurposing a toy as a generic development platform, to purpose built (open hardware!) platforms, and the various amusing scenarios they enable.
posted by russm at 7:20 PM on November 29, 2015 [2 favorites]
posted by russm at 7:20 PM on November 29, 2015 [2 favorites]
I was playing with the web app and found a signal at around 26968.00 kHz that pulses every few seconds with what sounds like a short burst of morse. Does anyone have any idea what that is?
I was also watching the spectrogram, and at certain points a line goes across the entire thing... is that like a solar flare or something?
posted by codacorolla at 6:17 AM on November 30, 2015
I was also watching the spectrogram, and at certain points a line goes across the entire thing... is that like a solar flare or something?
posted by codacorolla at 6:17 AM on November 30, 2015
codacorolla it could be likely only site-related - things like energized-CAT5/6 in close proximity can easily cause such effect. try powering-off all your intarwebs and see if you still get it?
posted by dorian at 10:45 AM on November 30, 2015
posted by dorian at 10:45 AM on November 30, 2015
I was playing with the web app and found a signal at around 26968.00 kHz that pulses every few seconds with what sounds like a short burst of morse. Does anyone have any idea what that is?
I assume we're talking about the Twente online SDR, in which case it's a pager transmitter. It's so strong that you'll also find harmonics of it scattered all around.
posted by kiltedtaco at 11:02 AM on November 30, 2015
I assume we're talking about the Twente online SDR, in which case it's a pager transmitter. It's so strong that you'll also find harmonics of it scattered all around.
posted by kiltedtaco at 11:02 AM on November 30, 2015
Ah, got it. Both seem like they're local to the Netherlands, then.
Playing with the web app (the Twente one) was very fun. I actually found a ham radio at a thrift store for about 15 dollars a few weeks ago, but passed it up... I'm sort of regretting that now. I might have to look into the USB antenna after the holidays.
posted by codacorolla at 11:42 AM on November 30, 2015
Playing with the web app (the Twente one) was very fun. I actually found a ham radio at a thrift store for about 15 dollars a few weeks ago, but passed it up... I'm sort of regretting that now. I might have to look into the USB antenna after the holidays.
posted by codacorolla at 11:42 AM on November 30, 2015
If you see a horizontal line on the waterfall of an SDR, it's either a wide-band noiseburst, which puts energy into lots of frequencies at once, or (if you're on shortwave) an ionosonde. These transmit a swept signal across all of shoftwave in about a minute, looking for reflections from the ionisophere to tell what condition it's in and what frequencies are useful for long-distance links. As the transmission sweeps across the tiny bit of the bands your waterfall covers, it'll look thin and horizontal.
You can use SDRs to track these ionosonde transmissions and pick up your own scan - which is actually really quite something, if you're deeply into this stuff.
There are lots and lots (and lots) of other things out there, too...
posted by Devonian at 2:40 PM on November 30, 2015 [1 favorite]
You can use SDRs to track these ionosonde transmissions and pick up your own scan - which is actually really quite something, if you're deeply into this stuff.
There are lots and lots (and lots) of other things out there, too...
posted by Devonian at 2:40 PM on November 30, 2015 [1 favorite]
This is an excellent thread.
posted by Songdog at 9:48 PM on November 30, 2015 [1 favorite]
posted by Songdog at 9:48 PM on November 30, 2015 [1 favorite]
I have a cheap radar-detector in my car. The difference between a cheap one and an expensive one is that an expensive one identifies the signals of police radar devices before giving and alarm, so there are not false positives, whereas a cheap one triggers when there is a device beaming on the band that the police equipment uses, so the radar in a nearby store's automatic door can give you a false positive, and the radar from driving past an airport can give you a faLSE POSITIVE OMG IT'S SO STRONG THEY MUST BE RIGHT ON TOP OF US!! MAXIMUM ALERT LEVEL: PaNiC!!! :)
I live in an area where police basically don't use radar or lidar, so there is no vehicle-related purpose in having the detector, but I like it because I like the tiny little window it gives into this invisible world of signals all around us (which is why I mentioned this seemingly random thing in this thread). The signals in the bands it watches are very fixed in location, so you learn the "terrain" of signals as you drive. When you find a signal that breaks that pattern, well, that's interesting too; in rare cases there has clearly been a vehicle on the freeway with me with some kind of equipment on board that is emitting in that band, but not police and not marked. So as traffic patterns shift and dips and hills alter visibility, I try to figure out which vehicle from the crowd is the anonymous one operating the mysterious equipment...
I might do something similar with putting a geiger counter in the car. That responds to geology rather than signals, but it's still a window into an invisible world that we don't often think about.
posted by anonymisc at 4:03 PM on December 1, 2015 [5 favorites]
I live in an area where police basically don't use radar or lidar, so there is no vehicle-related purpose in having the detector, but I like it because I like the tiny little window it gives into this invisible world of signals all around us (which is why I mentioned this seemingly random thing in this thread). The signals in the bands it watches are very fixed in location, so you learn the "terrain" of signals as you drive. When you find a signal that breaks that pattern, well, that's interesting too; in rare cases there has clearly been a vehicle on the freeway with me with some kind of equipment on board that is emitting in that band, but not police and not marked. So as traffic patterns shift and dips and hills alter visibility, I try to figure out which vehicle from the crowd is the anonymous one operating the mysterious equipment...
I might do something similar with putting a geiger counter in the car. That responds to geology rather than signals, but it's still a window into an invisible world that we don't often think about.
posted by anonymisc at 4:03 PM on December 1, 2015 [5 favorites]
When you find a signal that breaks that pattern, well, that's interesting too
YES.
posted by dorian at 6:03 PM on December 1, 2015
YES.
posted by dorian at 6:03 PM on December 1, 2015
I've been meaning to dust off my Technician license and play around with SDR for years now. Happy to see it's become this accessible.
posted by snuffleupagus at 6:38 PM on December 1, 2015
posted by snuffleupagus at 6:38 PM on December 1, 2015
One last little bit of radio fun - a network of amateurs keep a look-out for signals that appear in the shortwave ham bands but shouldn't be there. The daily update is a fascinating snapshot of the sort of thing that's out there - although there's much more on shortwave alone. When you get into satellites, natural radio, your local VHF/UHF and up environment... well, beats cable.
posted by Devonian at 3:41 AM on December 2, 2015 [1 favorite]
posted by Devonian at 3:41 AM on December 2, 2015 [1 favorite]
My father had his ham radio license for more than 60 years. His last call sign was retired when he died - W6 DMO. This may just inspire me to finally do something. Maybe I can get his call sign.
posted by Altomentis at 5:08 PM on December 5, 2015 [1 favorite]
posted by Altomentis at 5:08 PM on December 5, 2015 [1 favorite]
I should just mention that UK law forbids reception of anything except legal broadcasters, radio hams, CBers, and weather/navigation information.
This is, as you may imagine, impossible to enforce, full of silliness, and everyone ignores it. But it is the reason you won't find any live ATC feeds on the Web from the UK (unless someone knows better?); even though it is perfectly acceptable in practice to have an airband radio at airshows and in the spotters' hangouts at airports it's still strictly speaking illegal.
Myself, I consider the duty imposed on me by the terms of my amateur radio licence to avoid causing interference to other services to be the more pressing legal responsibility, so I do monitor other services where appropriate. If it came to court, it would be an interesting case.
Prosecutions under the law are very rare, and happen when someone's doing something very silly with what they hear. Criminals who actually use sigint to further their evil ways tend to have much more serious legal problems when they're caught, so nobody bothers with those charges in those cases.
posted by Devonian at 7:07 AM on December 6, 2015
This is, as you may imagine, impossible to enforce, full of silliness, and everyone ignores it. But it is the reason you won't find any live ATC feeds on the Web from the UK (unless someone knows better?); even though it is perfectly acceptable in practice to have an airband radio at airshows and in the spotters' hangouts at airports it's still strictly speaking illegal.
Myself, I consider the duty imposed on me by the terms of my amateur radio licence to avoid causing interference to other services to be the more pressing legal responsibility, so I do monitor other services where appropriate. If it came to court, it would be an interesting case.
Prosecutions under the law are very rare, and happen when someone's doing something very silly with what they hear. Criminals who actually use sigint to further their evil ways tend to have much more serious legal problems when they're caught, so nobody bothers with those charges in those cases.
posted by Devonian at 7:07 AM on December 6, 2015
« Older Also, Play Stations, kitty litter, and dinners at... | This Is Not Real Newer »
This thread has been archived and is closed to new comments
posted by tommasz at 2:04 PM on November 28, 2015 [1 favorite]