High frequency networking
October 18, 2013 3:47 PM Subscribe
In New York, there are at least six data centers you need to collocate in to be competitive in equities. An in-depth look at the insanity behind modern high-frequency trading, where the speed of light is the only limit.
10 Milliseconds of trading in Merck on May 16.
posted by absalom at 3:55 PM on October 18, 2013 [1 favorite]
posted by absalom at 3:55 PM on October 18, 2013 [1 favorite]
It is just so fucking nuts this isn't illegal.
posted by lattiboy at 3:59 PM on October 18, 2013 [21 favorites]
posted by lattiboy at 3:59 PM on October 18, 2013 [21 favorites]
This is great.
posted by Samuel Farrow at 4:20 PM on October 18, 2013
posted by Samuel Farrow at 4:20 PM on October 18, 2013
There are certainly a lot of issues with HFT, but this is just fucking cool:
In many markets, the length of the cable within the same building is a competitive advantage. Some facilities such as the Mahwah, New Jersey, NYSE (New York Stock Exchange) data center have rolls of fiber so that every cage has exactly the same length of fiber running to the exchange cages.3
posted by graphnerd at 4:21 PM on October 18, 2013 [2 favorites]
In many markets, the length of the cable within the same building is a competitive advantage. Some facilities such as the Mahwah, New Jersey, NYSE (New York Stock Exchange) data center have rolls of fiber so that every cage has exactly the same length of fiber running to the exchange cages.3
posted by graphnerd at 4:21 PM on October 18, 2013 [2 favorites]
So ripe for hacking...
posted by Annika Cicada at 4:24 PM on October 18, 2013 [1 favorite]
posted by Annika Cicada at 4:24 PM on October 18, 2013 [1 favorite]
It is just so fucking nuts that people get worked up about this. Unless you're either day trading (it's bad for you) or you're an institution trying to make very large transactions (it's good for you), this does not affect you.
posted by indubitable at 4:29 PM on October 18, 2013 [9 favorites]
posted by indubitable at 4:29 PM on October 18, 2013 [9 favorites]
I used to work for the company that ran the backend systems for NYSE. When they decided to relocate their data centers it was a massive secret where it was going. Working in HR, I learned early on, and it was very explicit that it was not to be leaked to anyone, and this was the exact reason why. There was talk about how much real estate would need to be obtained, and how the knowledge of where the servers would be located would drive up prices almost instantly.
posted by nevercalm at 4:38 PM on October 18, 2013 [1 favorite]
posted by nevercalm at 4:38 PM on October 18, 2013 [1 favorite]
I'm not sure I agree, indubitable. It might indirectly affect me. How?
1) How rapidly will the exchanges notice and stop faulty trades? If humans make a trading error, the size of the problem is limited by the speed with which they can place orders. These computers run many more trades, so it could be a much bigger problem by the time it is recognized and stopped.
2) Human effort, and material resources, are at best being wasted. (Unless the increase in liquidity for very large institutions results in sufficient increase in other productive economic activity). I'd rather the smart and capable people doing this were doing something more useful.
posted by nat at 4:41 PM on October 18, 2013 [9 favorites]
1) How rapidly will the exchanges notice and stop faulty trades? If humans make a trading error, the size of the problem is limited by the speed with which they can place orders. These computers run many more trades, so it could be a much bigger problem by the time it is recognized and stopped.
2) Human effort, and material resources, are at best being wasted. (Unless the increase in liquidity for very large institutions results in sufficient increase in other productive economic activity). I'd rather the smart and capable people doing this were doing something more useful.
posted by nat at 4:41 PM on October 18, 2013 [9 favorites]
Related: there was a really interesting Radiolab segment about HFT a little while ago.
posted by archagon at 4:44 PM on October 18, 2013 [2 favorites]
posted by archagon at 4:44 PM on October 18, 2013 [2 favorites]
I heard something a while ago about the Australian Stock Exchange - where new fibre was laid across Sydney Harbour explicitly to improve response times for traders on the North Shore...
Weird.
posted by Jimbob at 4:44 PM on October 18, 2013
Weird.
posted by Jimbob at 4:44 PM on October 18, 2013
But making money on these sorts of trades is zero-sum, is it not? Where does the money come from?
It's skimmed off the productive part. This adds no social value and is at best it's a scavenger, at worst it's an outright parasite.
Post-econopocalypse, I vote parasite
posted by Pirate-Bartender-Zombie-Monkey at 4:47 PM on October 18, 2013 [24 favorites]
It's skimmed off the productive part. This adds no social value and is at best it's a scavenger, at worst it's an outright parasite.
Post-econopocalypse, I vote parasite
posted by Pirate-Bartender-Zombie-Monkey at 4:47 PM on October 18, 2013 [24 favorites]
This site has never really been a great place to discuss the financial industry, but a few of my thoughts:
"Today a big Wall Street trader is more likely to have a Ph.D from Caltech or MIT than an MBA from Harvard or Yale. "
Disagree. Some of the very quant-y traders have PhD's, but most sell side traders do not have more than a bachelor's or maybe an MBA; same goes for hedge funds that are not quant shops. When I think of "big" traders, I think of traders that take big bets and move markets, and that is not what HFT's do. HFT's make lots of little tiny bets and generally do not move markets.
The reality is that automated trading is the new marketplace
Sure, HFTs make the markets (liquidity provider), but they generally do not move the markets (liquidity taker). In the end, it's still geopolitical events, macroeconomic data, and company earnings that drive the stock market.
I have traded both at a sell-side institution, and for my own account as a retail trader. You can think of HFTs as frenemies. Some people blame them for everything and wail that they're going to implode the market, but they're really not as big a deal as people make them out to be. All the high-tech stuff and hand waving over the volume they trade is really a red herring I think.
From the institutional standpoint, if you are executing a large amount of stock, the HFTs essentially attempt to detect your large order and front-run you. Let's say I need to buy 100,000 shares of IBM stock quickly; if I just lift every offer I'm going to get a horrible price. So what my algo tries to do is piece out the order in smaller lots so I don't move the market too much. Now what the HFT might do is dangle a tantalizingly low offer in front of me, but as soon as I go to lift some shares he pulls the rest of his offer, buys it in front of me, then offers it higher. So while he may have "increased" liquidity by narrowing the bid/ask initially, it's a sort of phantom liquidity that disappears when you go to take it.
From the retail standpoint, if I only need to buy a couple hundred shares for my P.A., HFT's give me a tighter market - but it's not like it really matters since the few cents of savings per share don't equate to much for retail size.
posted by pravit at 4:51 PM on October 18, 2013 [29 favorites]
"Today a big Wall Street trader is more likely to have a Ph.D from Caltech or MIT than an MBA from Harvard or Yale. "
Disagree. Some of the very quant-y traders have PhD's, but most sell side traders do not have more than a bachelor's or maybe an MBA; same goes for hedge funds that are not quant shops. When I think of "big" traders, I think of traders that take big bets and move markets, and that is not what HFT's do. HFT's make lots of little tiny bets and generally do not move markets.
The reality is that automated trading is the new marketplace
Sure, HFTs make the markets (liquidity provider), but they generally do not move the markets (liquidity taker). In the end, it's still geopolitical events, macroeconomic data, and company earnings that drive the stock market.
I have traded both at a sell-side institution, and for my own account as a retail trader. You can think of HFTs as frenemies. Some people blame them for everything and wail that they're going to implode the market, but they're really not as big a deal as people make them out to be. All the high-tech stuff and hand waving over the volume they trade is really a red herring I think.
From the institutional standpoint, if you are executing a large amount of stock, the HFTs essentially attempt to detect your large order and front-run you. Let's say I need to buy 100,000 shares of IBM stock quickly; if I just lift every offer I'm going to get a horrible price. So what my algo tries to do is piece out the order in smaller lots so I don't move the market too much. Now what the HFT might do is dangle a tantalizingly low offer in front of me, but as soon as I go to lift some shares he pulls the rest of his offer, buys it in front of me, then offers it higher. So while he may have "increased" liquidity by narrowing the bid/ask initially, it's a sort of phantom liquidity that disappears when you go to take it.
From the retail standpoint, if I only need to buy a couple hundred shares for my P.A., HFT's give me a tighter market - but it's not like it really matters since the few cents of savings per share don't equate to much for retail size.
posted by pravit at 4:51 PM on October 18, 2013 [29 favorites]
indubitable
Detractors are quick to point out that the market is no longer a level playing field for smaller investors, that it's easier to hide illegal activity (e.g., Trillium) and that it was a big factor in the 2010 flash crash.
I'm not personally convinced that the pros outweigh the cons.
posted by Brian Puccio at 4:54 PM on October 18, 2013 [2 favorites]
It is just so fucking nuts that people get worked up about this. Unless you're either day trading (it's bad for you) or you're an institution trying to make very large transactions (it's good for you), this does not affect you.I think there's more to it than that. Proponents argue that the market needs the liquidity that HFT provides and that it also contributes to market efficiency.
Detractors are quick to point out that the market is no longer a level playing field for smaller investors, that it's easier to hide illegal activity (e.g., Trillium) and that it was a big factor in the 2010 flash crash.
I'm not personally convinced that the pros outweigh the cons.
posted by Brian Puccio at 4:54 PM on October 18, 2013 [2 favorites]
this is nuts. it has nothing to do with the market finding the real price or introducing efficiencies. It's merely skimming off some technical flaws in the market machinery. illegalize +1
posted by j_curiouser at 4:55 PM on October 18, 2013 [1 favorite]
posted by j_curiouser at 4:55 PM on October 18, 2013 [1 favorite]
If you're going to make HFT illegal, how are you going to ensure fair access to the digital stock exchange?
posted by archagon at 4:58 PM on October 18, 2013
posted by archagon at 4:58 PM on October 18, 2013
I don't think it did well with critics or audiences, but I thought Cronenberg's Cosmopolis (never read the Delillo book) was a fascinating portrait of this world.
posted by Saxon Kane at 4:58 PM on October 18, 2013 [1 favorite]
posted by Saxon Kane at 4:58 PM on October 18, 2013 [1 favorite]
Unless you're either day trading (it's bad for you) or you're an institution trying to make very large transactions (it's good for you), this does not affect you.
It affects everyone.
It's bizarre and fascinating to me that we have evolved into a space where so much human talent and material expenditure is spent on an activity (front running) that is nothing more than an attempt to skim the tiniest piece of liquidity from every market order.
posted by MillMan at 5:02 PM on October 18, 2013 [6 favorites]
It affects everyone.
It's bizarre and fascinating to me that we have evolved into a space where so much human talent and material expenditure is spent on an activity (front running) that is nothing more than an attempt to skim the tiniest piece of liquidity from every market order.
posted by MillMan at 5:02 PM on October 18, 2013 [6 favorites]
If you're going to make HFT illegal, how are you going to ensure fair access to the digital stock exchange?
With a transaction tax. That's the traditional way of mitigating societal losses to deadweight exchanges like these.
posted by mhoye at 5:07 PM on October 18, 2013 [37 favorites]
With a transaction tax. That's the traditional way of mitigating societal losses to deadweight exchanges like these.
posted by mhoye at 5:07 PM on October 18, 2013 [37 favorites]
Unless you're either day trading (it's bad for you) or you're an institution trying to make very large transactions (it's good for you), this does not affect you.
I think that's mostly right. Except it does affect everyone. In a very real way. But not everyone—in fact a very tiny minority—of people are actually qualified to comment on the merits and dangers here.
Which is really a fascinating problem. I wish I knew enough to say more about it. But I don't.
posted by graphnerd at 5:08 PM on October 18, 2013 [4 favorites]
I think that's mostly right. Except it does affect everyone. In a very real way. But not everyone—in fact a very tiny minority—of people are actually qualified to comment on the merits and dangers here.
Which is really a fascinating problem. I wish I knew enough to say more about it. But I don't.
posted by graphnerd at 5:08 PM on October 18, 2013 [4 favorites]
Proposing a "trade clock":
8:00:00 AM: OUTGOING: You can send information to the exchange
8:00:05 AM: TRADE: Exchange acts on information sent to it in the last 5 seconds
8:00:10 AM: INCOMING: You receive information
8:00:15 AM: (repeat at OUTGOING)
You can still do thousands of trade cycles per day. You can still programatically attempt to skim pennies off the bid/ask. And you don't have to spend millions pushing your server three inches closer to the router.
posted by 0xFCAF at 5:10 PM on October 18, 2013 [4 favorites]
8:00:00 AM: OUTGOING: You can send information to the exchange
8:00:05 AM: TRADE: Exchange acts on information sent to it in the last 5 seconds
8:00:10 AM: INCOMING: You receive information
8:00:15 AM: (repeat at OUTGOING)
You can still do thousands of trade cycles per day. You can still programatically attempt to skim pennies off the bid/ask. And you don't have to spend millions pushing your server three inches closer to the router.
posted by 0xFCAF at 5:10 PM on October 18, 2013 [4 favorites]
My question is: if robots can play the markets (HFT) and robots can go to war (drones), how long before they demand the right to get married? And then who can stop them ADOPTING our children?
posted by the quidnunc kid at 5:19 PM on October 18, 2013 [11 favorites]
posted by the quidnunc kid at 5:19 PM on October 18, 2013 [11 favorites]
A paper with a potential solution (it's batch trades). I remember hearing about a new private exchange on the radio; the founder (like many institutional investors) felt like they were getting ripped off by HFT running ahead of their trades.
posted by a robot made out of meat at 6:00 PM on October 18, 2013 [2 favorites]
posted by a robot made out of meat at 6:00 PM on October 18, 2013 [2 favorites]
But making money on these sorts of trades is zero-sum, is it not? Where does the money come from?
its like superman 3: they take fractions of a penny from everyone.
posted by jpe at 6:12 PM on October 18, 2013
its like superman 3: they take fractions of a penny from everyone.
From the crippled children?
posted by Banky_Edwards at 6:21 PM on October 18, 2013
From the crippled children?
posted by Banky_Edwards at 6:21 PM on October 18, 2013
There's a quip going around the net "The best minds of my generation are thinking about how to make people click ads." One of my friends adds "or how to move stock orders faster." He ought to know, as he's high up in a company whose product helps customers do both.
It is a great waste, but if it should come as a consolation that at least the algorithms used by the HFT people are also extremely important for smart grid applications, which in turn are vital for integrating renewable energy sources into the electric grid. There is some benefit to the common good coming from the HFT arms race. Can't say the same for the click-on-ads people/.
posted by ocschwar at 6:24 PM on October 18, 2013 [1 favorite]
It is a great waste, but if it should come as a consolation that at least the algorithms used by the HFT people are also extremely important for smart grid applications, which in turn are vital for integrating renewable energy sources into the electric grid. There is some benefit to the common good coming from the HFT arms race. Can't say the same for the click-on-ads people/.
posted by ocschwar at 6:24 PM on October 18, 2013 [1 favorite]
But making money on these sorts of trades is zero-sum, is it not? Where does the money come from?
One of the definitions of "alpha" is the pot of money that the market at large pays the stock traders for the service of establishing the proper prices of the securities they trade in. It is zero sum. Every cent an HFT trader makes off these trades is a cent that is either taken from a rival HFT trader (and one who by definition is less competent at this) or a cent paid by a lower frequency trader for this service.
posted by ocschwar at 6:34 PM on October 18, 2013 [1 favorite]
One of the definitions of "alpha" is the pot of money that the market at large pays the stock traders for the service of establishing the proper prices of the securities they trade in. It is zero sum. Every cent an HFT trader makes off these trades is a cent that is either taken from a rival HFT trader (and one who by definition is less competent at this) or a cent paid by a lower frequency trader for this service.
posted by ocschwar at 6:34 PM on October 18, 2013 [1 favorite]
'BUT THE MARKET IS SO EFFICIENT', everybody yells.
posted by thsmchnekllsfascists at 7:13 PM on October 18, 2013
posted by thsmchnekllsfascists at 7:13 PM on October 18, 2013
There is some benefit to the common good coming from the HFT arms race. Can't say the same for the click-on-ads people.
The high-frequency traders have given us LMAX Disruptor and and some interesting insight into high-speed messaging systems and pushing the limits of Java. The folks working on serving up targeted ads have done interesting work in natural language processing, big data tools, and graph algorithms (which, really, are a big gnarly intersecting mess).
...I use a fair number of tools and techniques listed above in my work in very non-ad-serving, non-stock-trading domains. I'm glad someone's done this work, perhaps it would have been better if it had come from different sources...
posted by combinatorial explosion at 8:07 PM on October 18, 2013 [5 favorites]
The high-frequency traders have given us LMAX Disruptor and and some interesting insight into high-speed messaging systems and pushing the limits of Java. The folks working on serving up targeted ads have done interesting work in natural language processing, big data tools, and graph algorithms (which, really, are a big gnarly intersecting mess).
...I use a fair number of tools and techniques listed above in my work in very non-ad-serving, non-stock-trading domains. I'm glad someone's done this work, perhaps it would have been better if it had come from different sources...
posted by combinatorial explosion at 8:07 PM on October 18, 2013 [5 favorites]
We often say you get to the front of the queue via two methods: promotion, which is a second-level queue becoming a first-level queue, and joining, which is, exactly as it sounds, joining the newly created queue.
It seems odd to me that at the heart of the market making algorithm is a series of soviet style bread lines.
posted by pwnguin at 8:12 PM on October 18, 2013 [2 favorites]
It seems odd to me that at the heart of the market making algorithm is a series of soviet style bread lines.
posted by pwnguin at 8:12 PM on October 18, 2013 [2 favorites]
PHD from MIT or MBA from Harvard, one thing still holds true: A small number of very smart people continue to do their best to get ahead and burn everything behind.
posted by Halogenhat at 8:14 PM on October 18, 2013 [3 favorites]
posted by Halogenhat at 8:14 PM on October 18, 2013 [3 favorites]
Yesterday I literally literally saw an entire party of businesspeople wearing fedoras partying outside the New York Stock Exchange. I was like, I'm a take a picture cause there's people on the Internet who would love to hate on this. Enjoy.
posted by save alive nothing that breatheth at 8:15 PM on October 18, 2013 [3 favorites]
posted by save alive nothing that breatheth at 8:15 PM on October 18, 2013 [3 favorites]
One funny thing about trading speed being limited "only by the speed of light": When someone makes illegal trades based on leaked info, they need to be careful they don't appear to violate causality.
posted by mbrubeck at 8:58 PM on October 18, 2013 [13 favorites]
posted by mbrubeck at 8:58 PM on October 18, 2013 [13 favorites]
If you're going to make HFT illegal, how are you going to ensure fair access to the digital stock exchange?
Put all requests in a queue with a random(100) millisecond delay?
posted by CheeseDigestsAll at 9:22 PM on October 18, 2013 [4 favorites]
Put all requests in a queue with a random(100) millisecond delay?
posted by CheeseDigestsAll at 9:22 PM on October 18, 2013 [4 favorites]
By the way, I think "HFT" is kind of a nebulous term and they really do more than just front-run institutional orders; I just related that example because that's the extent of my personal experience with HFTs. I have heard HFT blamed for a lot of things, e.g. the knee-jerk reaction that comes after economic data or news is released. I honestly don't care that much; the instantaneous reaction is often different from where the market ends up trending over the next few hours. And in the case of Fed statements, they're sometimes so ambiguous that trading on them is no better than flipping a coin; you don't gain any advantage by having light-speed connections or computerized text parsers.
But making money on these sorts of trades is zero-sum, is it not? Where does the money come from?
A high-frequency market maker is basically engaging in the same business that dealers at stock exchanges have been doing for hundreds of years. A dealer has an inventory of stock that he is willing to buy and sell. Ideally, people will buy about as much stock from the dealer as they sell, and the dealer collects the bid/ask spread. The dealer gets compensated because he takes risk. In between those buys and sells, the dealer is long/short stock and exposed to price risk. Imagine if somebody knows something about the stock, and sells to the dealer @ $100. Dealer lowers his bid to $99. They sell @ $99. Dealer lowers his bid to $98. They keep selling until the dealer is sitting on a bunch of stock that he has bought for an average price of $95 and nobody else is willing to pay more than $90 for it. In trading parlance, this is called "getting run over."
Now, if the dealer was smart, he might realize people were selling the stock, cease bidding for it, and start selling the stock to a different dealer. HFT's are really no different; it's just that all of this happens in microseconds and they do many many trades for a very small margin that add up to a large profit.
The fact that many many trades are happening quickly, or that they're at such small margin doesn't change the basic story that the HFT is just trying to collect the bid/ask spread and make small directional bets based on the flow that it is seeing. It's bad news for the old-time stock dealers on the exchanges, but their time was over long ago.
Personally I'm not sure why people get worked up about it, other than that it sounds scary (robots! millions of trades in microseconds! PhDs!). I don't see any reason why market making shouldn't be taken over by robots, unless you want to ensure the employment of human market makers - who are just as prone to exacerbating irrational selloffs (remember 1987?). It's interesting that a lot of people are proposing solutions to HFT or declaring its criminality without explaining why they think it's a problem in the first place.
posted by pravit at 9:33 PM on October 18, 2013 [6 favorites]
But making money on these sorts of trades is zero-sum, is it not? Where does the money come from?
A high-frequency market maker is basically engaging in the same business that dealers at stock exchanges have been doing for hundreds of years. A dealer has an inventory of stock that he is willing to buy and sell. Ideally, people will buy about as much stock from the dealer as they sell, and the dealer collects the bid/ask spread. The dealer gets compensated because he takes risk. In between those buys and sells, the dealer is long/short stock and exposed to price risk. Imagine if somebody knows something about the stock, and sells to the dealer @ $100. Dealer lowers his bid to $99. They sell @ $99. Dealer lowers his bid to $98. They keep selling until the dealer is sitting on a bunch of stock that he has bought for an average price of $95 and nobody else is willing to pay more than $90 for it. In trading parlance, this is called "getting run over."
Now, if the dealer was smart, he might realize people were selling the stock, cease bidding for it, and start selling the stock to a different dealer. HFT's are really no different; it's just that all of this happens in microseconds and they do many many trades for a very small margin that add up to a large profit.
The fact that many many trades are happening quickly, or that they're at such small margin doesn't change the basic story that the HFT is just trying to collect the bid/ask spread and make small directional bets based on the flow that it is seeing. It's bad news for the old-time stock dealers on the exchanges, but their time was over long ago.
Personally I'm not sure why people get worked up about it, other than that it sounds scary (robots! millions of trades in microseconds! PhDs!). I don't see any reason why market making shouldn't be taken over by robots, unless you want to ensure the employment of human market makers - who are just as prone to exacerbating irrational selloffs (remember 1987?). It's interesting that a lot of people are proposing solutions to HFT or declaring its criminality without explaining why they think it's a problem in the first place.
posted by pravit at 9:33 PM on October 18, 2013 [6 favorites]
HFT absolutely causes harm. First, the amount of volume they generate is punishingly enormous, there's a cost in the market's IT just to manage it. Second, sometimes HFT goes wrong and you have market swings that make no rational sense because someone screwed up the robot. I'm not sure the HFT form of market making is any worse than alternatives though, but smarter people than me think to. Europe is still working on their transaction tax to stop it, but I tend to agree with the experts that a simple tax will distort the market more than it will fix it.
Why are bid/ask spreads still a whole penny on the US market? I should be grateful it's not still 1/8 I guess, but why not $0.0001?
posted by Nelson at 9:54 PM on October 18, 2013 [1 favorite]
Why are bid/ask spreads still a whole penny on the US market? I should be grateful it's not still 1/8 I guess, but why not $0.0001?
posted by Nelson at 9:54 PM on October 18, 2013 [1 favorite]
I sometimes work with HFT developers looking for a change of pace.
these guys are knowledgeable in much different aspects if development than I am and its always interesting to work with them.
A few years back I decided to take one particularly good dev into interviews I was doing.
I was hiring a junior guy to do ASP.NET for a basic intranet app. I got through few softball questions until my coworker started firing off questions. We got through C++ calling conventions and he started on esoteric x86 operators for pushing registers to the stack. I could maybe answer on my best day. The kid we were interviewing looked like someone clubbed him on the back of the head with a rock.
We spent the rest of the day debating shit just so I could leave feeling I deserved my job.
posted by Ad hominem at 9:56 PM on October 18, 2013 [2 favorites]
these guys are knowledgeable in much different aspects if development than I am and its always interesting to work with them.
A few years back I decided to take one particularly good dev into interviews I was doing.
I was hiring a junior guy to do ASP.NET for a basic intranet app. I got through few softball questions until my coworker started firing off questions. We got through C++ calling conventions and he started on esoteric x86 operators for pushing registers to the stack. I could maybe answer on my best day. The kid we were interviewing looked like someone clubbed him on the back of the head with a rock.
We spent the rest of the day debating shit just so I could leave feeling I deserved my job.
posted by Ad hominem at 9:56 PM on October 18, 2013 [2 favorites]
As a BSEE/MBA w/a 30-year employment history in (in no particular order) 1) semiconductor devices and fabrication technologies, 2) telecommunications networks and architectures (including PSTN --> packet migration and network time and sync distribution) and 3) trading systems (really!) ... I know how I'll be spending tomorrow morning ...
Thanks!
posted by ZenMasterThis at 9:59 PM on October 18, 2013 [1 favorite]
Thanks!
posted by ZenMasterThis at 9:59 PM on October 18, 2013 [1 favorite]
What I can't not wonder, as a person who once had to write an article on HFT but is generally not super well informed about it... what would the world be like if this simply didn't exist? Because to me, it seems like, I don't know, useless. Like a bitcoin mining operation only with real money.
posted by showbiz_liz at 10:57 PM on October 18, 2013 [3 favorites]
posted by showbiz_liz at 10:57 PM on October 18, 2013 [3 favorites]
I was looking into the logistics of the Chicago/New York optical link a while back and came to the startling realization that it takes light over 4 milliseconds to make the trip. My whole life I've been fed this crap about how fast light speed is supposed to be and it turns out to be pretty damn slow all things considered.
posted by Tell Me No Lies at 11:24 PM on October 18, 2013 [4 favorites]
posted by Tell Me No Lies at 11:24 PM on October 18, 2013 [4 favorites]
> Unless you're either day trading (it's bad for you) or you're an institution trying to make very large transactions (it's good for you), this does not affect you.
Oh, please. (Long ago ex-Wall Street person speaking here...)
First, there's zero reason to believe that HFT makes things any better for traditional institutions running traditional transactions - quite the contrary, since they make up the great majority of the market, the institutions are losing the majority of those pennies that the HFT traders are winning.
What traders do affects every American to some degree, and so does HFT. At the very least, HFT traders are extracting money out of the markets, money for which they exchange no value whatsoever - because for the two group that use securities, investors and corporations, have no use whatsoever for liquidity below one second. This is money out of the economy of people who make things and invest in things, into the HFT trader's pocket, and everyone's paying a little bit for it.
In the worst case, they are running unattended, automated programs with the power to trade, and running them at an incredible rate. It's perfectly reasonable to believe that this has the possibility to cause increased volatility, and outright market crashes.
Mind you, it probably won't come to that, but it's really not clear why We The People should allow this sort of thing at all, since it has zero value to the economy, and poses some risk. Financing corporations and managing risk - this is the whole use of Wall Street to the economy, and they really should be strongly discouraged from doing other things, particularly technical strategies which only manipulate markets, particularly when these strategies come with an entirely unknown risk.
I was a little disappointed by the tech, frankly. I didn't expect them to use a real-time kernel - I didn't expect them to use a kernel at all. Why have the overhead of a general purpose kernel when you're only doing one thing? 250µs seems like a tiny period of time to react, but in a 3GHz processor, that's 750,000 processor cycles, at least 75,000 full instructions (and probably a lot more).
posted by lupus_yonderboy at 11:59 PM on October 18, 2013 [13 favorites]
Oh, please. (Long ago ex-Wall Street person speaking here...)
First, there's zero reason to believe that HFT makes things any better for traditional institutions running traditional transactions - quite the contrary, since they make up the great majority of the market, the institutions are losing the majority of those pennies that the HFT traders are winning.
What traders do affects every American to some degree, and so does HFT. At the very least, HFT traders are extracting money out of the markets, money for which they exchange no value whatsoever - because for the two group that use securities, investors and corporations, have no use whatsoever for liquidity below one second. This is money out of the economy of people who make things and invest in things, into the HFT trader's pocket, and everyone's paying a little bit for it.
In the worst case, they are running unattended, automated programs with the power to trade, and running them at an incredible rate. It's perfectly reasonable to believe that this has the possibility to cause increased volatility, and outright market crashes.
Mind you, it probably won't come to that, but it's really not clear why We The People should allow this sort of thing at all, since it has zero value to the economy, and poses some risk. Financing corporations and managing risk - this is the whole use of Wall Street to the economy, and they really should be strongly discouraged from doing other things, particularly technical strategies which only manipulate markets, particularly when these strategies come with an entirely unknown risk.
I was a little disappointed by the tech, frankly. I didn't expect them to use a real-time kernel - I didn't expect them to use a kernel at all. Why have the overhead of a general purpose kernel when you're only doing one thing? 250µs seems like a tiny period of time to react, but in a 3GHz processor, that's 750,000 processor cycles, at least 75,000 full instructions (and probably a lot more).
posted by lupus_yonderboy at 11:59 PM on October 18, 2013 [13 favorites]
There's a beautiful contrast between HFT's, trading by the microsecond, and the advice generally given to the individual investor, which is to invest for the long term.
There's a chasm there no amount of cynicism can bridge.
Just remember, it's always your own fault if you get ripped off, though, because we are all adults and should know what we're doing.
posted by one weird trick at 4:52 AM on October 19, 2013 [1 favorite]
There's a chasm there no amount of cynicism can bridge.
Just remember, it's always your own fault if you get ripped off, though, because we are all adults and should know what we're doing.
posted by one weird trick at 4:52 AM on October 19, 2013 [1 favorite]
I also want to clarify one thing. When that HFT "front-runs" an institution, it isn't front-running in the traditional, illegal sense - which is trading in front of a known client order. The HFT doesn't actually know there is institutional size behind the order - it is making a bet and taking risk. They aren't omniscient - and they certainly aren't run "unattended." People spend their entire working days monitoring HFT's and tweaking their algos.
OK, you might say, but isn't it bad that the HFT is driving up the price in front of the institution? Well, yes and no. It is bad for the institution, but can be good for the rest of the marketplace (and I say this as someone who has had my own orders front-run by HFT's). Let's say a hedge fund (informed investor) does a bunch of analysis on a $100 stock and concludes that it should be worth $90. Now let's say there are a bunch of retail traders (uninformed investors) looking to buy the stock. In the old days, the uninformed investors would buy the stock for $100, and the HF would gradually drive the stock down to $90 and collect the profit at everyone else's expense.
But when HFT's detect that HF's selling flow, they get in front of the HF and sell the stock, and it gets to $90 much more quickly. The result is that the HF's knowledge about the stock - that it should be worth only $90 - is disseminated to the rest of the market much more rapidly. The retail traders will buy it for a better price - or think twice about buying it at all. If you don't know anything about a stock, at the very least the price history itself gives you information.
That's why I call HFT's frenemies. Nobody likes it when they yank a dollar on a string in front of you, but by doing that they disseminate information about stocks much more quickly, and help to even out the information asymmetry in the markets. If you've ever traded and noticed a stock suddenly gap on no news, only for it to come out a few minutes later - well, that HFT might have just saved you a few dollars, not cents.
I think HFT's absolutely provide value to the economy - it's the same value that dealers have always provided; it's just done more rapidly and by robots instead of by people.
posted by pravit at 7:12 AM on October 19, 2013 [8 favorites]
OK, you might say, but isn't it bad that the HFT is driving up the price in front of the institution? Well, yes and no. It is bad for the institution, but can be good for the rest of the marketplace (and I say this as someone who has had my own orders front-run by HFT's). Let's say a hedge fund (informed investor) does a bunch of analysis on a $100 stock and concludes that it should be worth $90. Now let's say there are a bunch of retail traders (uninformed investors) looking to buy the stock. In the old days, the uninformed investors would buy the stock for $100, and the HF would gradually drive the stock down to $90 and collect the profit at everyone else's expense.
But when HFT's detect that HF's selling flow, they get in front of the HF and sell the stock, and it gets to $90 much more quickly. The result is that the HF's knowledge about the stock - that it should be worth only $90 - is disseminated to the rest of the market much more rapidly. The retail traders will buy it for a better price - or think twice about buying it at all. If you don't know anything about a stock, at the very least the price history itself gives you information.
That's why I call HFT's frenemies. Nobody likes it when they yank a dollar on a string in front of you, but by doing that they disseminate information about stocks much more quickly, and help to even out the information asymmetry in the markets. If you've ever traded and noticed a stock suddenly gap on no news, only for it to come out a few minutes later - well, that HFT might have just saved you a few dollars, not cents.
I think HFT's absolutely provide value to the economy - it's the same value that dealers have always provided; it's just done more rapidly and by robots instead of by people.
posted by pravit at 7:12 AM on October 19, 2013 [8 favorites]
> it's the same value that dealers have always provided
Sorry, that just isn't so. As I pointed out above, dealers provide liquidity - for execution times above one second. HFTs provide liquidity only for times smaller than one second - which has NO use whatsoever to anyone. HFTs aren't buying and holding the securities, so if there is no actual market for the securities, the HFT people aren't going to provide one.
posted by lupus_yonderboy at 7:43 AM on October 19, 2013
Sorry, that just isn't so. As I pointed out above, dealers provide liquidity - for execution times above one second. HFTs provide liquidity only for times smaller than one second - which has NO use whatsoever to anyone. HFTs aren't buying and holding the securities, so if there is no actual market for the securities, the HFT people aren't going to provide one.
posted by lupus_yonderboy at 7:43 AM on October 19, 2013
It sounds like a rather invasive process that ensures that whoever has the most money and resources gets to "share information" to the market and all they get in return is first dibs on everything. Like Jim Cramer style stock advice we can rest assured that the most powerful have long acted on this information using their HFT algorithms way before any regular schmuck has a chance. It sounds horribly manipulative... If the service they provide is so great let the govt be the one to handle it because it just doesn't make sense that any for profit institution is providing a net benefit to other traders that isn't cancelled out by their sheer massiveness and influence on the market even if they are only probing into the real value and slurping some up like a mosquito in the process which I suspect we would all do better without.
posted by lordaych at 7:46 AM on October 19, 2013
posted by lordaych at 7:46 AM on October 19, 2013
Here's an article describing a related sort of plan for the EBS trading platform:If you're going to make HFT illegal, how are you going to ensure fair access to the digital stock exchange?Put all requests in a queue with a random(100) millisecond delay?
In an interview with EBS CEO Gil Mandelzis, FT reported that EBS may be planning to change its existing ‘first in fist out’ trading methodology with one that aggregates orders and randomizes the executions. The goal of such a move would be to place limits on high frequency traders as their orders will no longer be executed solely on the speed.posted by Anything at 8:23 AM on October 19, 2013 [1 favorite]
According to EBS’s CEO, Gil Mandelzis, the primary participants of the venue aren’t interested in HFT, and their existence is disruptive to platform. Mandelzis told FT “It is a technology arms race to the bottom, and a huge tax on the industry, since people are having to make significant investments in speed without any connection to their trading strategy,” said Gil Mandelzis, EBS chief executive. “Speed has little to do with why many participants come to our markets. These are serious players who come to the market to exchange risk; they do not come to race.”
And regarding the above, on a quick think something like that seems like the only sane way to do things, as opposed to the status quo, given the present realities. I wonder if there are any good counterarguments (apart from the costs already sunk into the arms race, of course).
posted by Anything at 8:27 AM on October 19, 2013
posted by Anything at 8:27 AM on October 19, 2013
Without taking an opinion on their merits, I will point there are often unintended side effects to changes such as batch auctions.
For example, in any kind of batch auction you're likely to get a rush of orders right before the actual auction time. The closer a trader can get their order in to the auction the less risk of price movement they face and the less they 'show their hand' to the market. You also have to deal with people canceling or changing their orders right before the auction and potentially significantly moving the price. This is often addressed today with techniques such as a random open - Search for 'Random ends to auction call periods' or a period before the end of the auction where orders can't be modified or cancelled, but those add their own complexities to the auctions.
People also tend to speak of high frequency trading (buying and selling quickly) as if it were the same as high speed trading (sending an order quickly). While HFTs absolutely need be high speed traders, generally speaking in any market structure - electronic or not - there will always be an advantage to acting on public information as soon as possible.
posted by true at 9:05 AM on October 19, 2013 [2 favorites]
For example, in any kind of batch auction you're likely to get a rush of orders right before the actual auction time. The closer a trader can get their order in to the auction the less risk of price movement they face and the less they 'show their hand' to the market. You also have to deal with people canceling or changing their orders right before the auction and potentially significantly moving the price. This is often addressed today with techniques such as a random open - Search for 'Random ends to auction call periods' or a period before the end of the auction where orders can't be modified or cancelled, but those add their own complexities to the auctions.
People also tend to speak of high frequency trading (buying and selling quickly) as if it were the same as high speed trading (sending an order quickly). While HFTs absolutely need be high speed traders, generally speaking in any market structure - electronic or not - there will always be an advantage to acting on public information as soon as possible.
posted by true at 9:05 AM on October 19, 2013 [2 favorites]
As others have said above, there are good reasons to be worried about HFT, in particular the potential to cause "flash crashes" and unforeseeable "black swan" events.
But another problem is that they blunt incentives in the market.
Now, markets do have an underlying benefit to society, by identifying quality. For instance, suppose an investor spots that the company "Better Mousetrap Inc." is a great company: well-managed, with an excellent lineup of products in growing markets. He also realises that the company "Chocolate Teapot Ltd" is a terrible company: badly managed, with products nobody wants bought by a shrinking demographic. So, he sells his shares in "Chocolate Teapot Ltd" and buys shares in "Better Mousetrap Inc." By doing that, he makes money. But he also identifies a good company, and by buying shares, he makes it possible for "Better Mousetrap" to potentially issue more shares if they need capital to expand.
Now, there are other ways than markets to identify good firms. The Soviet Union had central planning committees trying to identify good factories and allocate resources correctly. But without the incentive of potentially getting stinking rich, they proved to be not nearly as good as markets at doing that.
Now with HFT, what happens is that when the shrewd value investor who's done all the research tries to buy shares in "Better Mousetrap Inc.", they almost instantly bid up the price, so he makes less money out of it. When he tries to sell shares in "Chocolate Teapot Ltd", they instantly bid down the price, so he loses money anyway.
So, HFT reduces the incentive to identify quality in the market. It thus makes the markets less effective at doing that.
Now, HFT has benefits as well. All these automated buyers and sellers increase liquidity in the market: with all these extra trades going on, it's easier to find someone willing to buy or sell what you're interested in.
But it's definitely debatable whether the benefits of HFT outweigh the costs.
What's interesting is that some people seem much more concerned about some kinds of incentive-blunting than others. For instance, some people seem very concerned about increases to tax or decreases in tax thresholds, since that potentially blunts the incentive to work hard or do high-value work. But if you think that it's true that small reductions in incentives can cause big problems for the economy, you ought to be very concerned about things like HFT too.
posted by TheophileEscargot at 9:17 AM on October 19, 2013 [9 favorites]
But another problem is that they blunt incentives in the market.
Now, markets do have an underlying benefit to society, by identifying quality. For instance, suppose an investor spots that the company "Better Mousetrap Inc." is a great company: well-managed, with an excellent lineup of products in growing markets. He also realises that the company "Chocolate Teapot Ltd" is a terrible company: badly managed, with products nobody wants bought by a shrinking demographic. So, he sells his shares in "Chocolate Teapot Ltd" and buys shares in "Better Mousetrap Inc." By doing that, he makes money. But he also identifies a good company, and by buying shares, he makes it possible for "Better Mousetrap" to potentially issue more shares if they need capital to expand.
Now, there are other ways than markets to identify good firms. The Soviet Union had central planning committees trying to identify good factories and allocate resources correctly. But without the incentive of potentially getting stinking rich, they proved to be not nearly as good as markets at doing that.
Now with HFT, what happens is that when the shrewd value investor who's done all the research tries to buy shares in "Better Mousetrap Inc.", they almost instantly bid up the price, so he makes less money out of it. When he tries to sell shares in "Chocolate Teapot Ltd", they instantly bid down the price, so he loses money anyway.
So, HFT reduces the incentive to identify quality in the market. It thus makes the markets less effective at doing that.
Now, HFT has benefits as well. All these automated buyers and sellers increase liquidity in the market: with all these extra trades going on, it's easier to find someone willing to buy or sell what you're interested in.
But it's definitely debatable whether the benefits of HFT outweigh the costs.
What's interesting is that some people seem much more concerned about some kinds of incentive-blunting than others. For instance, some people seem very concerned about increases to tax or decreases in tax thresholds, since that potentially blunts the incentive to work hard or do high-value work. But if you think that it's true that small reductions in incentives can cause big problems for the economy, you ought to be very concerned about things like HFT too.
posted by TheophileEscargot at 9:17 AM on October 19, 2013 [9 favorites]
Sorry I keep coming back, but it's an interesting topic - and it would be awesome if someone who has actually worked for an HFT in a trading or programming capacity could chime in.
Despite my example above about HFT's front-running large orders, I think HFTs actually do provide a ton of useful liquidity to the markets. It is inaccurate to say that the liquidity HFT's provide lasts for less than a second and is thus not of value. When you trade stock electronically, most of the time you are trading with an HFT market maker on the other side. The whole reason why bid/ask spreads on most big-cap names are only a penny is because of HFT's. Names like CSCO, MSFT, BAC trade like water - they're so liquid you can easily lift the screens for a couple million dollars worth of stock without moving the market. In this situation HFT's are your friend, and they actually do save you money in bid/ask. The presence of multiple market makers (HFT or otherwise) in heavily traded stocks actually strengthens what the market can bear in terms of liquidity. What would be an ugly spike in an illiquid stock gets smoothed out to a little blip in a liquid stock. If the HFT sells you a couple million shares of stock @ $20 but he knows a millisecond later he can buy a couple milion shares for $19.99, he has no need to front run you. He makes the 1 cent in bid/ask, you are saved a couple cents in bid/ask - only person who loses is the old-time stock dealer who used to quote markets in 1/8ths.
But if you are trying to take more liquidity from the market than it can bear - in other words, you are trying to trade a large volume of stock in a short time - then the mere fact that you want to do that is valuable information that should influence the price. If somebody told you that a big hedge fund wanted to buy $100 million worth of some random biotech stock right now, would you go ahead and sell them $100 million of stock at 1 cent above the bid? Of course not! The hedge fund has information the rest of the market does not have, and wants to get the "best price" - the price that does not reflect the information they hold. If the HFT's detect a big order hitting the market, it makes sense for them to trade in front of it, and the new price is "discovered" more rapidly, which benefits other market participants who do not have the information.
This is part of the reason why non-HFT dealers are still in business, because institutions who don't want to compete with HFT's can quote the dealers for large blocks of stock or options. The institution pays the dealer a commission and gets a guaranteed price. Then it's up to the dealer to execute the order. If the market moves too much, then the dealer crosses the rest of the stock to the institution at the guaranteed price from their own books (e.g. I guarantee you 100,000 shares with $101 limit. I buy 90,000 shares on your behalf up to $101. I then sell you 10,000 shares @ $101 out of my own book, so I'm left short 10,000 shares. My problem, not yours). This leads to "algo arms races" as dealers compete with HFT's for best execution.
If you are investing for the longer term, and you believe one stock is overvalued relative to another, then in terms of execution you shouldn't be that worried about HFT's. No long-term investor is going to blast hundreds of millions worth of stock instantaneously. You're spreading your buying and selling volume over entire days and trying not to be too much of a % of the total volume.
posted by pravit at 12:53 PM on October 19, 2013 [2 favorites]
Despite my example above about HFT's front-running large orders, I think HFTs actually do provide a ton of useful liquidity to the markets. It is inaccurate to say that the liquidity HFT's provide lasts for less than a second and is thus not of value. When you trade stock electronically, most of the time you are trading with an HFT market maker on the other side. The whole reason why bid/ask spreads on most big-cap names are only a penny is because of HFT's. Names like CSCO, MSFT, BAC trade like water - they're so liquid you can easily lift the screens for a couple million dollars worth of stock without moving the market. In this situation HFT's are your friend, and they actually do save you money in bid/ask. The presence of multiple market makers (HFT or otherwise) in heavily traded stocks actually strengthens what the market can bear in terms of liquidity. What would be an ugly spike in an illiquid stock gets smoothed out to a little blip in a liquid stock. If the HFT sells you a couple million shares of stock @ $20 but he knows a millisecond later he can buy a couple milion shares for $19.99, he has no need to front run you. He makes the 1 cent in bid/ask, you are saved a couple cents in bid/ask - only person who loses is the old-time stock dealer who used to quote markets in 1/8ths.
But if you are trying to take more liquidity from the market than it can bear - in other words, you are trying to trade a large volume of stock in a short time - then the mere fact that you want to do that is valuable information that should influence the price. If somebody told you that a big hedge fund wanted to buy $100 million worth of some random biotech stock right now, would you go ahead and sell them $100 million of stock at 1 cent above the bid? Of course not! The hedge fund has information the rest of the market does not have, and wants to get the "best price" - the price that does not reflect the information they hold. If the HFT's detect a big order hitting the market, it makes sense for them to trade in front of it, and the new price is "discovered" more rapidly, which benefits other market participants who do not have the information.
This is part of the reason why non-HFT dealers are still in business, because institutions who don't want to compete with HFT's can quote the dealers for large blocks of stock or options. The institution pays the dealer a commission and gets a guaranteed price. Then it's up to the dealer to execute the order. If the market moves too much, then the dealer crosses the rest of the stock to the institution at the guaranteed price from their own books (e.g. I guarantee you 100,000 shares with $101 limit. I buy 90,000 shares on your behalf up to $101. I then sell you 10,000 shares @ $101 out of my own book, so I'm left short 10,000 shares. My problem, not yours). This leads to "algo arms races" as dealers compete with HFT's for best execution.
If you are investing for the longer term, and you believe one stock is overvalued relative to another, then in terms of execution you shouldn't be that worried about HFT's. No long-term investor is going to blast hundreds of millions worth of stock instantaneously. You're spreading your buying and selling volume over entire days and trying not to be too much of a % of the total volume.
posted by pravit at 12:53 PM on October 19, 2013 [2 favorites]
it would be awesome if someone who has actually worked for an HFT in a trading or programming capacity could chime in.
You might get some former people chiming in, but as someone who currently works for an HFT, I don't think you'll find that many currently employed HFT employees that interested in joining the discussion. Even if my user name was more anonymous (it's my actual name) I don't think it would be wise. I know that everything I commit to paper/disk/whatever in the course of my job is legally discoverable by regulators and courts. I actually don't know if my personal speech is or not, but basically I couldn't risk saying anything that might be misunderstood or misconstrued.
Beyond that, it's a controversial issue, and I don't think my employer would like their employees in the middle of it.
posted by RustyBrooks at 2:51 PM on October 19, 2013 [1 favorite]
You might get some former people chiming in, but as someone who currently works for an HFT, I don't think you'll find that many currently employed HFT employees that interested in joining the discussion. Even if my user name was more anonymous (it's my actual name) I don't think it would be wise. I know that everything I commit to paper/disk/whatever in the course of my job is legally discoverable by regulators and courts. I actually don't know if my personal speech is or not, but basically I couldn't risk saying anything that might be misunderstood or misconstrued.
Beyond that, it's a controversial issue, and I don't think my employer would like their employees in the middle of it.
posted by RustyBrooks at 2:51 PM on October 19, 2013 [1 favorite]
Fascinating stuff in a kind of tangential and horrifying way. What would really float my boat is a similarly concerted effort to rebuild the broken-down infrastructure of the country. I'm a high-frequency traveler over bridges and would like to feel more confident about getting to the other side. Pardon my plebeian outlook.
posted by CincyBlues at 3:06 PM on October 19, 2013
posted by CincyBlues at 3:06 PM on October 19, 2013
Serious question: What about the person who sold shares at $19.99 that they could have sold at $20? Seems to me they lost out.
If the market is $19.99/$20, then the person who wants to sell shares has two choices:
1) Offer the shares at $20 and wait for someone to lift them
2) Hit the $19.99 bid.
By choosing 2), you ensure a $19.99 fill, but you cross bid/offer and sell shares at $0.005 worse than the mid-market price.
By choosing 1), you can potentially sell the shares @ $20, but the risk is that the market moves lower and that you never get filled.
Nobody is forcing the person to sell shares @ $19.99.
It's really difficult to engage in "what-if" discussions. What if there were no robots allowed in market making? Well, stocks would probably be more illiquid, and the money that HFT's grind out cent by cent, would instead go dime by dime into the pockets of old-school stock dealers making wide markets. I don't know, I can think of lots of industries that serve no obvious economic purpose, and lots of money that could be better used for lots of different things. But I don't know if you can unplug one thing and plug it into another socket so easily. It's like my parents used to say, "if only you spent as much energy on school work as you do on video games..."
posted by pravit at 3:14 PM on October 19, 2013 [2 favorites]
If the market is $19.99/$20, then the person who wants to sell shares has two choices:
1) Offer the shares at $20 and wait for someone to lift them
2) Hit the $19.99 bid.
By choosing 2), you ensure a $19.99 fill, but you cross bid/offer and sell shares at $0.005 worse than the mid-market price.
By choosing 1), you can potentially sell the shares @ $20, but the risk is that the market moves lower and that you never get filled.
Nobody is forcing the person to sell shares @ $19.99.
It's really difficult to engage in "what-if" discussions. What if there were no robots allowed in market making? Well, stocks would probably be more illiquid, and the money that HFT's grind out cent by cent, would instead go dime by dime into the pockets of old-school stock dealers making wide markets. I don't know, I can think of lots of industries that serve no obvious economic purpose, and lots of money that could be better used for lots of different things. But I don't know if you can unplug one thing and plug it into another socket so easily. It's like my parents used to say, "if only you spent as much energy on school work as you do on video games..."
posted by pravit at 3:14 PM on October 19, 2013 [2 favorites]
There are inventions and medical treatments and research that won't even exist, because the people who could have made them were busy dicking around on Wall Street. And the reason we can't have those things is "liquidity".
I don't know if i buy this argument. There's people out there who could be curing a specific type of cancer or inventing the perfect artificial heart or whatever, but are living their dream working part time at a coffee shop or some microsoft sub contractor cranking out bong hits and shredding guitar in a basement practice space too. No, but seriously, some of the smartest people i've ever met were just content being artists/musicians in this sort of lifestyle. Right up there with the guys who got full ride scholarships to MIT in 10th grade.
Are they somehow detracting from the future of the human race too? Maybe these guys think that designing these ASICs and writing this software to do battle with the other geniuses is fucking awesome.
I mean, i realized that years ago while fairly smart this sort of programming/software engineering stuff Wasn't For Me, but i absolutely get the appeal of this being some badass William Gibson novel future shit. You're basically playing a game of chess with some of the other smartest guys in the world to see whose robot is the best.
How is that not cool as hell? If someone told me that was even something i could potentially do when i was 5 or 8 i would have dreamed about it regularly. It's like, actual, honest to god cyberpunk shit right there.
So yea, i don't think the smartest minds are being "wasted" on this without any agency just because that's where the money is or anything. I would be very, very surprised if they weren't in this just for the game of it.
When this all dies someone is going to write a great memoir about the golden age of weak-AI cyber warfare.
Maybe it'll be written and edited in 5 seconds by the first strong AI 2 milliseconds after trading is shut down for the last time.
I just question that by default all these people would be in other fields advancing the human race if this didn't exist. They may very well have been drawn in to this type of dev/programming/hardware design/etc just to compete in this battle royale. It's not like they'd be at spacex developing warp engines otherwise in some alternate universe.
posted by emptythought at 5:15 PM on October 19, 2013 [2 favorites]
I don't know if i buy this argument. There's people out there who could be curing a specific type of cancer or inventing the perfect artificial heart or whatever, but are living their dream working part time at a coffee shop or some microsoft sub contractor cranking out bong hits and shredding guitar in a basement practice space too. No, but seriously, some of the smartest people i've ever met were just content being artists/musicians in this sort of lifestyle. Right up there with the guys who got full ride scholarships to MIT in 10th grade.
Are they somehow detracting from the future of the human race too? Maybe these guys think that designing these ASICs and writing this software to do battle with the other geniuses is fucking awesome.
I mean, i realized that years ago while fairly smart this sort of programming/software engineering stuff Wasn't For Me, but i absolutely get the appeal of this being some badass William Gibson novel future shit. You're basically playing a game of chess with some of the other smartest guys in the world to see whose robot is the best.
How is that not cool as hell? If someone told me that was even something i could potentially do when i was 5 or 8 i would have dreamed about it regularly. It's like, actual, honest to god cyberpunk shit right there.
So yea, i don't think the smartest minds are being "wasted" on this without any agency just because that's where the money is or anything. I would be very, very surprised if they weren't in this just for the game of it.
When this all dies someone is going to write a great memoir about the golden age of weak-AI cyber warfare.
Maybe it'll be written and edited in 5 seconds by the first strong AI 2 milliseconds after trading is shut down for the last time.
I just question that by default all these people would be in other fields advancing the human race if this didn't exist. They may very well have been drawn in to this type of dev/programming/hardware design/etc just to compete in this battle royale. It's not like they'd be at spacex developing warp engines otherwise in some alternate universe.
posted by emptythought at 5:15 PM on October 19, 2013 [2 favorites]
There are inventions and medical treatments and research that won't even exist, because the people who could have made them were busy dicking around on Wall Street. And the reason we can't have those things is "liquidity".
Well, what is so goddamn great about liquidity anyway? And is it really the case that the only way to get it is to engage in a crazy technological arms race?
You had me at "arms race".
The Cold War arms race produced many wonders -- this internet thingy we're communicating over for starters. Valuable engineers were wasted dicking around:
Now I personally prefer an arms race between trading houses because nobody gets all killed and stuff, and twenty years from now when you are:
posted by Tell Me No Lies at 7:48 PM on October 19, 2013 [1 favorite]
Well, what is so goddamn great about liquidity anyway? And is it really the case that the only way to get it is to engage in a crazy technological arms race?
You had me at "arms race".
The Cold War arms race produced many wonders -- this internet thingy we're communicating over for starters. Valuable engineers were wasted dicking around:
- Connecting computers together, a pointless task at best considering that sending magnetic tapes was about 10,000 times faster.
- In a nuke proof network supporting tens of thousands of nodes. Even sillier because there weren't that many computers to begin with.
Now I personally prefer an arms race between trading houses because nobody gets all killed and stuff, and twenty years from now when you are:
- Being operated on by the finest cardiovascular surgeon in the world from his beach hut in the Barbados,
- Using VR assisted software that that requires as much network bandwidth as the entire country of Madagascar consumes today,
- With a 3 millisecond latency on the laser scalpel control
posted by Tell Me No Lies at 7:48 PM on October 19, 2013 [1 favorite]
The protocols used in trading platforms have very little to do with those we use to move actual traffic around the Internet these days -- reliability relying on dropped acknowledgments imparts too much delay to recovery to be useful in this environment -- but we will certainly reap the benefits in terms of available technology for latency reduction down the road. Bandwidth not so much, or rather, there are far more generally interesting application areas and economic incentives driving increases in bandwidth.
That said, you're never, ever going to hit 3ms latency on a laser scalpel with an operator in Barbados and a patient anywhere other than Barbados: c is still in the way, optical fiber propagates at a fraction of c (on the order of .66, I believe), you've got (sometimes significant) delay in the optical-physical boundary as well, and I hope to hell that you're encrypting the stream of commands to that laser scalpel -- block ciphers induce their own delay and jitter. Some problems you can throw way too much money at, and some will remain as tradeoffs inherent in the system.
posted by Vetinari at 3:07 AM on October 20, 2013
That said, you're never, ever going to hit 3ms latency on a laser scalpel with an operator in Barbados and a patient anywhere other than Barbados: c is still in the way, optical fiber propagates at a fraction of c (on the order of .66, I believe), you've got (sometimes significant) delay in the optical-physical boundary as well, and I hope to hell that you're encrypting the stream of commands to that laser scalpel -- block ciphers induce their own delay and jitter. Some problems you can throw way too much money at, and some will remain as tradeoffs inherent in the system.
posted by Vetinari at 3:07 AM on October 20, 2013
High Frequency Trading – An Asset Manager’s Perspective
posted by BobbyVan at 11:02 AM on October 21, 2013 [1 favorite]
posted by BobbyVan at 11:02 AM on October 21, 2013 [1 favorite]
Bandwidth not so much, or rather, there are far more generally interesting application areas and economic incentives driving increases in bandwidth.
I'm not sure I agree with that. There are economic incentives driving lots of little streams multiplexed into a single stream, but not so much huge individual streams. There are a limited number of low latency/big data problems running around and even fewer that have Big Money behind them. In the long run the ability to constantly share terabyte data sets between New York and Chicago could drive things more than you think.
That said, you're never, ever going to hit 3ms latency on a laser scalpel with an operator in Barbados and a patient anywhere other than Barbados: c is still in the way.
Well okay, 3 ms is pushing it a bit. In fact the worst case for light speed delay is Bali, which is 43 ms as the neutrino flies.
you've got (sometimes significant) delay in theoptical neutrino-physical boundary as well
This is true, but the lag caused at media boundaries is exactly the area where network vendors are focusing in order to sell to the trading houses. I think we'll see it continue to drop, and of course it's not that big compared to 43 ms to start with...
I hope to hell that you're encrypting the stream of commands to that laser scalpel
To quote a friend from my Cisco days "People talk about the Internet like it's run through public restrooms or something". So no, don't really care about encryption.
In any case there's no reason to run this through the Internet in the first place, and the logistics of pulling a man-in-the-middle attack on a beam of neutrinos traveling directly through the earth are significant.
posted by Tell Me No Lies at 2:09 PM on October 21, 2013
I'm not sure I agree with that. There are economic incentives driving lots of little streams multiplexed into a single stream, but not so much huge individual streams. There are a limited number of low latency/big data problems running around and even fewer that have Big Money behind them. In the long run the ability to constantly share terabyte data sets between New York and Chicago could drive things more than you think.
That said, you're never, ever going to hit 3ms latency on a laser scalpel with an operator in Barbados and a patient anywhere other than Barbados: c is still in the way.
Well okay, 3 ms is pushing it a bit. In fact the worst case for light speed delay is Bali, which is 43 ms as the neutrino flies.
you've got (sometimes significant) delay in the
This is true, but the lag caused at media boundaries is exactly the area where network vendors are focusing in order to sell to the trading houses. I think we'll see it continue to drop, and of course it's not that big compared to 43 ms to start with...
I hope to hell that you're encrypting the stream of commands to that laser scalpel
To quote a friend from my Cisco days "People talk about the Internet like it's run through public restrooms or something". So no, don't really care about encryption.
In any case there's no reason to run this through the Internet in the first place, and the logistics of pulling a man-in-the-middle attack on a beam of neutrinos traveling directly through the earth are significant.
posted by Tell Me No Lies at 2:09 PM on October 21, 2013
Two fo my favorite docus on HFT are by the Dutch TV show "Tegenlicht"(Backlight).
Money & Speed: Inside the Black Box looks at the role HFT and black box trading algorithms in the 2010 Flash Crash, when the Dow Jones dropped suddenly by 9%, only to recover minutes later. They raise some good questions like what happens when the programmers let the algorithm program itself? What decisions will it come up with then, based on what criteria? What happens when the smarter algorithms realize its time to get out of the market, but the dumber ones keeps trading at a million miles an hour?
Quants: The Alchemists of Wall Street is about "the math wizards and computer programmers in the engine room of our global financial system who designed the financial products that almost crashed Wall st."
My money is still on "This is parasite activity skimming resources off of the system while waving the Liquidity flag in peoples faces".
posted by Pirate-Bartender-Zombie-Monkey at 7:21 PM on October 21, 2013
Money & Speed: Inside the Black Box looks at the role HFT and black box trading algorithms in the 2010 Flash Crash, when the Dow Jones dropped suddenly by 9%, only to recover minutes later. They raise some good questions like what happens when the programmers let the algorithm program itself? What decisions will it come up with then, based on what criteria? What happens when the smarter algorithms realize its time to get out of the market, but the dumber ones keeps trading at a million miles an hour?
Quants: The Alchemists of Wall Street is about "the math wizards and computer programmers in the engine room of our global financial system who designed the financial products that almost crashed Wall st."
My money is still on "This is parasite activity skimming resources off of the system while waving the Liquidity flag in peoples faces".
posted by Pirate-Bartender-Zombie-Monkey at 7:21 PM on October 21, 2013
« Older The Women and the Thrones | New News is Good News Newer »
This thread has been archived and is closed to new comments
It takes six minutes to display.
posted by absalom at 3:54 PM on October 18, 2013 [7 favorites]