High Frequency Trading on Shortwave Radio
January 14, 2021 4:52 PM   Subscribe

 
If notoriety public doesn't come along to explain, we were talking about this same thing recently. I don't know the difference between microwave towers and shortwave radio, but he indicated this has been going on for awhile now and has its own disadvantages, mainly the bandwidth you can achieve and the impact on weather vs fiber. There's also a limited number of firms that can really take advantage of this particular HFT. I don't think it is a growing field, I believe he indicated that it has been done for years now.

The article made a point about how fiber isn't in a straight line usually but there was a firm that dug fiber in as a straight line as possible from NYC to Chicago.

More interesting to me is the FPGA and specialized network cards. There's a lot of strange tech in the world of HFT that you don't see anywhere else.

Maintaining a network of microwave antennas is expensive.

The article mentioned the firms were secretive and I don't doubt that but I don't think "making things as fast as possible" isn't the secret per se as you can always find engineers to make things go faster. It is nature of the order execution itself that is secretive. Not so much straight arbitrage in that APPL is trading at 1 cent cheaper in Chicago than NYC but things like order batching and more complex trading strategies that need to be executed quickly.

Maybe np comes along and tells me everything I said was wrong, but haven't seen him so thought I'd summarize poorly on what he told me.
posted by geoff. at 5:51 PM on January 14, 2021 [5 favorites]


It certainly gives new meaning to "shortwave numbers station".
posted by hippybear at 6:04 PM on January 14, 2021 [10 favorites]


Microwave is very high frequency and best suited for point-to-point communications. Microwave towers were used extensively by the phone system many years ago, and a number of those towers exist, though I think they're pretty much all defunct now.

The HF (shortwave) stuff has been interesting to watch. I guess any sort of edge is worth pursuing if you're trying to get a millisecond or so advantage over someone else in trading, but the unpredictability of HF propagation...well, it's what makes amateur radio simultaneously fun and frustrating. These bands rely on ionospheric propagation - you're literally bouncing your signal off the ionosphere and back down again, then back up and back down again, etc. If there's another station where the signal comes down, congratulations, you can establish communications. If not, you won't hear each other.

And as a bonus, these spots shift around quite a bit. You can make some educated guesses by looking at the space weather forecasts, because sunspot activity has a direct effect on whether or not the ionosphere will be amenable to propagation. For some bands, you can use what's called the "grey-line," which is where the day/night terminator is, to get reliably heard in, say, South America from North America. It's a bit of science and luck, but hey, that's the draw. We're in the low-ish part of the current sunspot cycle, though things seem to be picking up again a little.

Trying to do something critical, though, seems like a head-scratcher to me. It's all we can do most days to trade enough small bits of information to qualify as a loggable contact. For everything else, there's the Internet. In any case, I've seen signals like that one before on 20m and it's popped up in some ham IRC channels too. There are web-based SDRs (software defined radios) out there that will let you tune around and see what's up.

I can go on at length about this. 73 DE AA4JQ.
posted by jquinby at 6:10 PM on January 14, 2021 [14 favorites]


I don't know the difference between microwave towers and shortwave radio

Microwave towers: signals are in the GHz range. Antennas are measured in centimetres. Signals travel in straight lines, and only work when the towers have line of sight to each other - so only over relatively short distances. But once you have a link set up, it will generally work reliably.

Shortwave radio: signals are in the MHz range. Antennas are measured in metres. Signals can - but don't always - bounce off layers in the ionosphere, allowing them to travel around the world... sometimes. That long-distance propogation depends on the weather, the time of day, and solar activity - sunspots, coronal mass ejection, etc. It's wacky as fuck and hard to predict. It's only good for opportunistic communication, really. But when it works, it will get to the other side of the world faster than anything else short of a neutrino beam.

Both travel faster than optical signals on fibre optic cables - even if the fibre is in a straight line - because the speed of light in glass is slower than in air.
posted by automatronic at 6:19 PM on January 14, 2021 [9 favorites]


I'd love to see someone jam those signals. HFT is an evil practice that needs to be stopped. Financial Transaction Tax Now!
posted by SansPoint at 6:35 PM on January 14, 2021 [20 favorites]


There are no flat earthers among microwave tower technicians, because accounting for the curvature of the earth is vital to properly lining up two antennas on towers that are barely visible to each other over the horizon.
posted by Mr.Encyclopedia at 6:35 PM on January 14, 2021 [11 favorites]


I find articles like this fascinating. One of these days I wanna learn about radio, maybe do a little fooling around with an SDR kit.
posted by signsofrain at 6:56 PM on January 14, 2021 [1 favorite]


There are no flat earthers among microwave tower technicians, because accounting for the curvature of the earth is vital to properly lining up two antennas on towers that are barely visible to each other over the horizon.

Curvature of the earth my ass. Everyone knows that radio signals are attenuated by angels and you need to include an angel absortivity correction factor when calculating transmission power over large distances on the flat earth.
posted by mr_roboto at 7:10 PM on January 14, 2021 [10 favorites]


When I'm dictator, my thunderdome battles-to-the-death will consist of everyone involved in high frequency trading vs. everyone involved in cryptocurrency. The twist will be that no one actually gets to leave the dome, I'm gonna pour in some sort of acid or something.
posted by maxwelton at 7:29 PM on January 14, 2021 [13 favorites]


HFT is up there with cryptocurrency mining as completely worthless wastes of resources on an enormous scale. They're both technically fascinating, but they're in dire need of regulation. The SEC already has a financial transaction tax on some trades, but it's 0.002%: current proposals to bump it up to 0.1% should curb the worst excesses of the HFT crowd. I think it should be closer to sales tax (and a wealth tax closer to property tax), but it's a start.
posted by netowl at 7:32 PM on January 14, 2021 [19 favorites]


> Delmarva Broadband is authorized 14.35-14.99 MHz at 186 kW.

No wonder they're spamming everyone else; they're potentially putting out almost 200x the power of a high-powered amateur station. An elegant solution this is decidedly not.
posted by Kadin2048 at 8:14 PM on January 14, 2021 [2 favorites]


more complex trading strategies that need to be executed quickly

i.e., frontrunning
posted by hypnogogue at 11:10 PM on January 14, 2021 [1 favorite]


His map shows a source very near Chicago, so perhaps this is commodities-related trading.
posted by jamjam at 12:11 AM on January 15, 2021


His map shows a source very near Chicago, so perhaps this is commodities-related trading.

Chicago is the THE hub for HFT firms across all assets.
posted by JPD at 4:55 AM on January 15, 2021 [2 favorites]


HFT is up there with cryptocurrency mining as completely worthless wastes of resources on an enormous scale. They're both technically fascinating, but they're in dire need of regulation. The SEC already has a financial transaction tax on some trades, but it's 0.002%: current proposals to bump it up to 0.1% should curb the worst excesses of the HFT crowd. I think it should be closer to sales tax (and a wealth tax closer to property tax), but it's a start.

to repeat ad nauseum: HFT trading replaced a business which was actually more rapacious and extracted more value from investors. if you're gonna complain about who makes money from market making (which is a necessary task) - which is more odious to you - 20 bro-y dudes from Breezy Point who love a good fleece vest or 2 washed out academic mathematicians and comp sci folks?

Also, financial transaction taxes usually exclude market making - which is ultimately what HFT is. Not to mention HFT actually makes the business that you probably want to target, short-term prop trading, mostly economically unviable.

The argument most often put forth re:HFT is that as they have no obligation to make markets (unlike the historic market makers - the business they basically destroyed) so in times of stress markets will be even less efficient. The evidence for this isn't great, but it does have a certain logic to it.

Regardless, my understanding from talking to some of the earliest players in the industry is that the Volatility the marginal HFT firm needs to generate a reasonable return on investment on these crazy projects is much much higher than we've seen absent very short periods of incredible stress. We're past peak HFT craziness.
posted by JPD at 5:08 AM on January 15, 2021 [1 favorite]


I imagine this is an opportunistic thing too - on the days it works, they print money with their latency advantage? Other times, they do nothing?

To be honest I hadn't thought about the power they're using. It's probably enough to push data even when propagation's not terrific, especially if they're using high-gain antennas. 186kW? That's...a lot.
posted by jquinby at 6:48 AM on January 15, 2021


Didn't expect to be name-dropped in the first comment (h/t geoff.), but yes, I am in the electronic finance industry (for 21 years now), and have pontificated on these topics before. I say "electronic finance" because when I started, HFT wasn't really a thing in the same way it is now. It has evolved TREMENDOUSLY over time. And even now, I am not at a firm that plays the high speed game. You never want to be SLOW, because you will miss opportunities and get picked off, but you don't need to be the very fastest to still trade well.

When I started, machines running trading algos weren't colocated, and often weren't even in the same city! The software latency was still dramatically larger than the transmission latency- nobody was caring about whether it took a whole 20 milliseconds for the order to get to the exchange in New York from Chicago.

Eventually, the machines got fast enough that the order transit time started to be a big enough slice of the total time that colocating the machines in the same datacenter was an easier way of making your order faster than trying to speed up the code or the hardware. They didn't STOP speeding up the code or the hardware, but paying attention to transmission latency came into the mix as something to work on.

The shorter fiber optic connections between colos started to happen then, because it was a way to shave off a couple of milliseconds over Chicago-NYC distance (and a smaller but still important amount between the important datacenters in the NY/NJ area), at a time when there was no longer milliseconds left to shave anywhere else. The staggering capital expense ensured it was only done once all cheaper options were exhausted.

And then, after the massive investment made by the fiber optic people, they themselves got sniped within a few years by the even more massive investment of doing the same short-path thing via microwave, because light propagation speed in fiber is maybe .6-.7c, while in air it's only a tiny fraction less than 1. Over the distances involved this again is enough to be a competitive edge in terms of total latency to market.

Now, when you start getting into microwave, you have many issues. It is NOISY, so the effective bandwidth is constrained a lot by the noise floor. Additionally, the noise is very weather sensitive, so rain, fog, etc can knock you out intermittently for significant periods of time, and you need to handle that. The limited bandwidth is actually a pretty critical thing, because your latency advantage is merely a head start, and your (many orders of magnitude slower) data transmission rate eats into that head start very quickly. So you can't send the entire NYC marketdata book to Chicago to trade off of- you have to ruthlessly curate each trading signal sent. Because it doesn't matter how fast the start of your signal arrives, only how fast it fully transmits. And you have to limit the number of signals you send, because you can't send the second signal while the first is still transmitting, so that second signal is going to wait in a queue before it even goes out the door. To get a sense of scale, these signal messages, including full header information for network routing, are on the order of 64 bytes. You don't get very much to work with.

A final note is that the super-fast trades can only be kind of stupid. They tend to be "if a new order arrives for instrument XYZ, and the price is above X, sell it. If it's below Y, buy it." But how do you decide what the X and Y values are? the more complicated that decision is, the longer your X value can be sitting there at a price you might not want, because you were still figuring out you wanted to back off. So the sophistication of those buy/sell levels is very limited for the super-fast trading algos. That's where firms like mine come in, where we spend a lot more effort on longer-term price prediction, which can't be done in the microsecond-to-submicrosecond time frames of the bleeding edge HFT firms.

Still with me? Let's get back to the actual topic!

Whoever is doing this, it was noticed a while ago somewhere in the various chatter groups we HFT folks gossip in, and what we expect these guys are doing, is the "very tiny bandwidth, very noisy, very short signal messages" thing, but from Chicago to Europe, using the 20 meter band. The reason is because of the Kennelly-Heaviside Layer, which can channel radio of certain wavelength ranges around the curvature of the earth, and get an acceptable delivered to someplace far out of line of sight.

This has the potential of delivering a signal around the world with lower latency than any other technique available. However, the signal to noise ratio of doing this SUCKS. Which is why these sociopaths are blasting it at enormous power, in order to get some kind of reasonable data rate arriving at the other end. Whether it works at all to trade with, I dunno, but that is the generally accepted rationale for what they're doing.

Sorry about the long winding road to relevance here, but I thought it was worthwhile to describe the background of WHY somebody would be doing something as crazy as this. The tl;dr answer is "money", as it usually is, but the interesting part is why and how it might be worth money.
posted by notoriety public at 6:58 AM on January 15, 2021 [34 favorites]


His map shows a source very near Chicago, so perhaps this is commodities-related trading.

The landing point is literally next door to the Chicago Mercantile Exchange's data center in Aurora, IL. The CME is a virtual trading floor run out of this building now - the open-outcry trading you saw in Trading Places ended a long time ago.

This location is heavily fought over.
posted by JoeZydeco at 8:03 AM on January 15, 2021 [2 favorites]


notoriety public's post gives a lot of great background on the practice and motivations for HFT.

This bit in particular is crucial to understanding why HFT traders would want to use shortwave radio versus fiber optic (emphasis mine):
> ... because light propagation speed in fiber is maybe .6-.7c, while in air it's only a tiny fraction less than 1. Over the distances involved this again is enough to be a competitive edge in terms of total latency to market."
posted by muchomas at 10:57 AM on January 15, 2021


There is really SO MUCH to unpack in the current state of the industry that I struggled to make my post as short as possible while still providing the waypoints by which we got to the subject matter of this particular post.

To step back a bit to address the commenters above who vilify the HFT industry: You're not wrong. It requires a pretty staggering amount of cognitive dissonance on my part to reconcile my professional heartless capitalism with my personal socialist worldview. People are good at that, which is kind of how we all got to the place we are today. I'd say i'm under no illusions about what I do, but no, I am merely pretty far up the food chain in a sea of predators.

I do think there's social value in explaining the hows and whys that it works. Because there's a LOT of misinformation out there, and I want the negative opinions about finance to be formed from knowledge rather than ignorance. Not least because nobody's yelling louder about the evils of HFT firms, than the ones who used to have that niche and want to go back to exploiting it themselves. When you come to kill all the sharks, be sure to cast a wide net.
posted by notoriety public at 11:17 AM on January 15, 2021 [2 favorites]


I shared with a friend who is into radio and said they tracked down where they think the house is (not him, but he found online a discussion). And he checked out they have an antenna and he thought maybe CHI exchange, but it wasn't pointing (I guess it's a very directional antenna on the site). So he said it almost looked like it was pointed to WI, and thought that didn't make sense but followed the line and...

BINGO! London, as if they're working with London exchange. At least that's his conjecture.

He said apparently there's direct point-to-point fiber registered at the house, too. weird shit.
posted by symbioid at 11:18 AM on January 15, 2021 [1 favorite]


(The above is all assuming that the location someone found (which - interestingly - is blurred out in google maps all the way around, so whoever lives there/works there is clearly intent on privacy). Everything else in the neighborhood is just regular suburban neighborhood...

So yeah, I bet that's it. Especially if it is over to London, that difference in distance would add more value I bet to the time difference.
posted by symbioid at 11:21 AM on January 15, 2021 [1 favorite]


If this stuff interests you, the Sniper in Mahwah blog goes DEEP into all of these mystery HFT towers and cables to figure out who is using them and why. It's pretty fascinating reading. HFT in my Backyard is a good starting point.
posted by JoeZydeco at 11:24 AM on January 15, 2021 [2 favorites]


> BINGO! London, as if they're working with London exchange. At least that's his conjecture.

Electromagnetic waves propagate along great circles, which can be hard to visualize if you're looking at a garden variety "Web Mercator" map.

For funsies you could generate an azimuthal map with, say, Chicago as its center point, and then compare the alignment of a given dish to see if its great circle path leads to somewhere else like, say, London.
posted by muchomas at 1:50 PM on January 15, 2021 [2 favorites]


This little snippet from an article about billionaires Club for Growth, funding Hawley and Cruz, and Boebert and the rest of the asses trying to overturn the election, this snippet, offered an alternative use for these short wave towers. Just a snippet took me straight to this FP, "Jeffrey Yass, the co-founder of Susquehanna International Group, an options trading group based in Philadelphia that also owns a sports betting company in Dublin."
posted by Oyéah at 6:31 PM on January 15, 2021


I would assume modern Machine Learning techniques have revolutionized modeling of the ionosphere for most likely bounce paths from/to any two points on Earth. Anyone in this particular scene who can confirm/deny?
posted by effugas at 6:51 PM on January 15, 2021


I thought ML might be at play here as well but I’m looked into it and saw no research that would indicate ML can overcome problems with shortwave transmission. It doesn’t mean it isn’t happening, but if there’s not basic research there I doubt there’s anything beyond just the obvious.
posted by geoff. at 9:47 PM on January 15, 2021


I would assume modern Machine Learning techniques have revolutionized modeling of the ionosphere for most likely bounce paths from/to any two points on Earth.

Not that I've ever heard of. There are networks of beacons that let you determine the status of the ionosphere in real time, but as far as I know there's no way to predict it in advance or anything. It's a chaotic system driven largely by solar flares and the Sun's "weather" patterns, and the models are just as prone to failure as traditional terrestrial weather modeling. More, even, because there are more variables involved.

Getting "good DX" (long distance communications) over HF involves a lot of luck, or a lot of power, or sometimes both.
posted by Kadin2048 at 7:46 PM on January 16, 2021 [3 favorites]


« Older The knights alone can account for as much as 50%...   |   ignorance of history serves many ends Newer »


This thread has been archived and is closed to new comments