What am I, an idiot? I mean, kind of?

November 7, 2012 9:26 AM

Ask A Banker: What's The Deal With High Frequency Trading? From the planet money NPR team.
posted by garlic (35 comments total)

If you want more reading, these guys have strong opinions on HFT - The Themis Trading blog:

http://blog.themistrading.com/
posted by de void at 9:39 AM on November 7, 2012

A more concrete example of how HFT "improves price" for market participants:

http://market-ticker.org/akcs-www?post=163801

... A couple of years ago if you entered a limit order for $26.40 with the market at $26.10 odds are excellent that most of your order would have filled down near where the market was when you entered the order - $26.10.

Today, odds are excellent that most of your order will fill at $26.39, and the HFT firms will claim this is an "efficient market." The truth is that you got screwed for 29 cents per share which was quite literally stolen by the HFT firms that probed your book before you could detect the activity, determined your maximum price, and then sold to you as close to your maximum price as was possible. ...

And a proposal to stop HFT market manipulations:

http://market-ticker.org/akcs-www?post=209947

...

- All market participants must have margin available to clear all orders they have open at any instant in time on an unaggregated basis. Since you may only enter an order you intend to execute under black-letter law you must be required to have the margin capacity through either cash or secured and proved-available credit, to clear the trade. Period.


- All orders must be exposed to actual execution risk by all market participants. Since you may only enter an order that you intend to execute the market must be able to act on each and every order you place into the market. This requires that each order, once placed, by valid for a reasonable minimum period of time so that it is exposed to a a reasonably-large percentage (for all purposes all) of the market. This means that the minimum human reaction time plus the round-trip time for all reasonable technologies in use must be the minimum order validity time; an order must either be valid for that time or it must execute. A reasonable definition of this time is 2 seconds.

...

posted by de void at 9:56 AM on November 7, 2012 [8 favorites]

Where's #4? You know, this takes Big Iron and Big Math and is completely out of the reach of ordinary humans, even individual clever humans, thereby allowing only powerful megacorps to wring every last drop of available profit from transactions before pitiful, solo fleshbags can dare to have a lick at the now bone-dry fabric of our economy?
posted by adipocere at 9:57 AM on November 7, 2012 [3 favorites]

But all trading is, in some sense, a stupid thing for people to devote their energies to. Just moving around stocks and money between people is a mostly-pointless activity in itself; if you buy my shares of Facebook stock no new investment happens, no new factories are built, no new diseases are cured. Each individual trade looks pointless from a social perspective. But the ability to buy and sell stocks efficiently makes it easier for new companies to raise money. The trades are mostly pointless, but the system of trading is valuable.

Incidentally, most people think that we do spend too many resources on this whole trading business. There's a wonderful recent paper by Thomas Philippon of NYU, who points out that, while the income of the financial industry has grown from ~5% to ~8% of the American economy since 1980, the value it provides, in terms of liquidity services and financing of businesses, has not grown nearly as fast. As he puts it, the "cost of financial intermediation" has actually gone up.

Here's the paper by Phillipon. The "banker" here really needs to delve into what he means by efficient, since at least by the simplest definition, the paper he cites by Phillipon says that the getting money for your new business has gone up over the last 30 years, i.e. raising money on Wall Street has become less efficient. But, I suppose that would lead to a very short essay and an end to his writing career: the biggest profit centers of Wall Street firms amount to parasitic rent seeking on regular business.

Selling corporate bonds and stock issuance for mature companies is just not that profitable if you aren't allowed to pump 'n dump, front run, or other wise swindle the little old ladies like what the Goulds, Morgans, and Rockefellers of yore built their legends on.
posted by ennui.bz at 10:01 AM on November 7, 2012 [3 favorites]

A while back I read something about how long stocks were held. What I found quickly today was 70% of trade positions are held for an average of 11 seconds.
This hardly seems to be in line with the idea that the stock market is for raising capital for companies to grow.
But wait, I learned that in high school... I should have known that I was being conned.
posted by MtDewd at 10:05 AM on November 7, 2012 [2 favorites]

It doesn't exactly require "Big Iron" to do this sort of thing, even low-end Xeon processors are quite capable of performing this sort of task; that's not where the bottleneck is.

The huge advantage that the trading firms have is that their hardware is colocated at the stock exchange's datacenter, giving them the minuscule network latencies that they need.
posted by 1970s Antihero at 10:06 AM on November 7, 2012 [7 favorites]

This is a good article.

For a while, it seemed like anti-HFT people were concentrating on how "the little guy" was supposedly being ripped off, and it never fit with empirical reality--bid-ask spreads remain very low and markets are quite deep well over 99 percent of the time. As Felix Salmon, a general critic of modern Wall Street pointed out in an article against HFT:

high-frequency trading has been genuinely wonderful for small investors like you and me. We might not be particularly clever, but when we put in an order to buy this or sell that, the order gets filled immediately. We pay almost nothing in trading costs — just a few pounds, normally. And we get the very best price in the market: something called NBBO, for “national best bid/offer”. If you look at all the prices being quoted on all of the stock exchanges in the country, we get the lowest price of all if we’re buying, and the highest price of all if we’re selling.

That wasn’t true ten years ago.

The real problems he points out are a waste of resources in an "arms race," and that HFT firms may leave the market in times of distress. If we're moving away as a society from the naive, and often wrong, simplistic view of "the vampire squids are ripping off the little main street guy" criticisms of Wall Street to messages like this, I think that's a very good sign.
posted by dsfan at 10:07 AM on November 7, 2012 [1 favorite]

Agreed, good article

1)Arms race also sows the seeds of the end of HFT as a profit generator. Everyone is now collocated, most of the easy money is gone.
2)Agree leaving the market at times of distress is the only real concern
3)Themis is in the category of people inside finance who are anti-HFT because it is destroying their own business which is essentially outsourced trading. They are protecting themselves, not other market particiapants.

HFT takes most of its profit from market makers and very very frequent traders. If you aren't in those camps its pretty easy to avoid it as a meaningful cost.

A couple of years ago if you entered a limit order for $26.40 with the market at $26.10 odds are excellent that most of your order would have filled down near where the market was when you entered the order - $26.10.

Today, odds are excellent that most of your order will fill at $26.39, and the HFT firms will claim this is an "efficient market." The truth is that you got screwed for 29 cents per share which was quite literally stolen by the HFT firms that probed your book before you could detect the activity, determined your maximum price, and then sold to you as close to your maximum price as was possible. ...


Uhm - if you didn't want to buy the shares at 26.40 then don't put in a limit at 26.40. If those 29 cents changes the calculus of the trade for you then you are a speculator, and well - you just got out speculated.
posted by JPD at 10:29 AM on November 7, 2012 [13 favorites]

If we're moving away as a society from the naive, and often wrong, simplistic view of "the vampire squids are ripping off the little main street guy" criticisms of Wall Street to messages ...

Well, who exactly they are ripping off with the increase in "price of financial intermediation" might get complicated, but Phillipon's article is basically all vampire squid:

The role of the finance industry is to produce, trade and settle financial contracts that can be used to pool funds, share risks, transfer resources, produce information and provide incentives. Financial intermediaries are compensated for providing these services. The income received by these intermediaries measures the cost of financial intermediation. This income is the sum of all spreads and fees paid by non-financial agents to financial intermediaries, and it is also the sum of all profits and wages in the finance industry. The first contribution of the paper is empirical.
I show that the income of financial intermediaries as a share of GDP varies a lot over time. The income share grows from 2% to 6% from 1870 to 1930. It shrinks to less than 4% in 1950, grows slowly to 5% in 1980, and then increases rapidly to more than 8% in 2010. This finding is robust to alternative measures, e.g., excluding net exports of financial services, or scaling by services instead of GDP.

After observing these large historical changes in the finance income share, it is natural to ask the following questions: Is finance a normal good? Should we expect finance to grow with income per capita? How do productivity growth in the non financial sector or technological progress in intermediation affect the size of the finance industry?

To answer these questions, I introduce financial services for firms and households in the neoclassical growth model. This is the second contribution of the paper. Under the assumption of homogenous monitoring (a natural assumptions for monitoring and screening technologies), the model predicts no income effect (i.e., no mechanical tendency for the finance income share to grow with per-capita GDP). The intuition for this result is simple. As borrowers become more productive, the value of monitoring increases even though the monitoring technology itself does not change. Since the opportunity cost of being a banker is the wage in the non-financial sector, and since this wage is proportional to aggregate productivity, the income share of finance remains constant on the balanced growth path. I test this hypothesis and find that it holds well.

The growth in income for finanicial firms to 8% of GDP is a big fat blood bolus of rent-seeking meta-capitalism i.e. not efficient.
posted by ennui.bz at 10:33 AM on November 7, 2012 [2 favorites]

Here's the paper by Phillipon. The "banker" here really needs to delve into what he means by efficient, since at least by the simplest definition, the paper he cites by Phillipon says that the getting money for your new business has gone up over the last 30 years, i.e. raising money on Wall Street has become less efficient. But, I suppose that would lead to a very short essay and an end to his writing career: the biggest profit centers of Wall Street firms amount to parasitic rent seeking on regular business. But that's not really germane to the topic of HFT and I'm pretty sure I've seen Levine write on those topics elsewhere and that he basically agrees with you that the capital raising process is inefficient, overpriced, and anti-competitive.


Selling corporate bonds and stock issuance for mature companies is just not that profitable if you aren't allowed to pump 'n dump, front run, or other wise swindle the little old ladies like what the Goulds, Morgans, and Rockefellers of yore built their legends on.

Underwriting equity offerings is up there with M&A advisory in terms of profitability relative to capital requirements. Debt offerings aren't nearly so exciting because they require a big balance sheet, but its still not bad business. Basically you are wrong on this.
posted by JPD at 10:35 AM on November 7, 2012 [1 favorite]

But that's not really germane to the topic of HFT and I'm pretty sure I've seen Levine write on those topics elsewhere and that he basically agrees with you that the capital raising process is inefficient, overpriced, and anti-competitive.

It's sort of a simplistic essay, but it's frustrating that he's basically making a simple general argument that all trading beyond market-making is economically useless (and thus there is no good argument for not heavily regulating it):

There's a wonderful recent paper by Thomas Philippon of NYU, who points out that, while the income of the financial industry has grown from ~5% to ~8% of the American economy since 1980, the value it provides, in terms of liquidity services and financing of businesses, has not grown nearly as fast. As he puts it, the "cost of financial intermediation" has actually gone up since the 1970s, unlike most industries where increasing use of technology has lowered costs. High frequency trading may be a part of this, but it's unlikely to be the main part; it is a symptom of the excessive dedication of resources to trading, rather than a cause.

without going into the detail of just how useless HFT may be i.e. much of it is high-tech front running on the market makers themselves... parasites on parasites yet leaving th ereader feeling like somehow his argument is about HFT.

Politically, the case for regulation of HFT is tied up in the regulation of trading i.e. any "financial transaction tax," say, will be "Death Taxed" to Death in Congress, with a line of bankers testifying how it will tax the american dream of stock trading your way to a comfortable retirement to death. So, in order to regulate HFT you actually have to make a case for regulating T, which he is but saying he isn't.
posted by ennui.bz at 10:44 AM on November 7, 2012

"Based on our analysis, we believe that High Frequency Traders exhibit trading patterns inconsistent with the traditional definition of market making. Specifically, High Frequency Traders aggressively trade in the direction of price changes. This activity comprises a large percentage of total trading volume, but does not result in a significant accumulation of inventory. As a result, whether under normal market conditions or during periods of high volatility, High Frequency Traders are not willing to accumulate large positions or absorb large losses. Moreover, their contribution to higher trading volumes may be mistaken for liquidity by Fundamental Traders. Finally, when rebalancing their positions, High Frequency Traders may compete for liquidity and amplify price volatility.
Consequently, we believe, that irrespective of technology, markets can become fragile when imbalances arise as a result of large traders seeking to buy or sell quantities larger than intermediaries are willing to temporarily hold, and simultaneously long-term suppliers of liquidity are not forthcoming even if significant price concessions are offered."

From here.
posted by Dmenet at 10:50 AM on November 7, 2012

Because he's not making a case to regulate HFT, never mind T. The HFT algorithms are well well well on their way to commoditization. A few years from now they'll just be used by execution traders to fill orders for third partiers.

Secondly in the case of Equity trading - which is really what he is talking about - the value intermediated out (just in this space) and captured by WS firms has absolutely collapsed over the last twenty years. This is not where Wall Street has found a way to make more profits. The Cash Equity trading business is quite literally near death.

Instead they've discovered ways to intermediate other markets.

Trading beyond market making - sure it is economically useless, but I'm not sure why that matters. That capital gets sucked up into other uses. Every dollar some HFT guy is reinvesting in his business today is going towards really smart comp sci sorts.

Of all the things to be pissed off about the lack of regulation in the financial sector this one seems pretty pointless.
posted by JPD at 10:52 AM on November 7, 2012 [1 favorite]

JPD,

Themis trading may be attempting to protect their own industry, but that doesn't invalidate their arguments in a broader sense.

I noticed you did not address the issue in one of the market ticker articles linked earlier - that placing orders that are not intended for execution, but instead made solely for price discovery or to manipulate the market higher or lower are against the law (securities act of 1934).

... Uhm - if you didn't want to buy the shares at 26.40 then don't put in a limit at 26.40. If those 29 cents changes the calculus of the trade for you then you are a speculator, and well - you just got out speculated.

Let's take that philosophy and apply it to an admittedly contrived example in another market:

Say you wanted to buy a house, were interested in those available in a certain area at $300K, but were willing to spend up to $350K for the "right" place.

Your agent's phone system has a flaw where every other agent at the brokerage can listen to their phone calls.

Suddenly every property that is available for sale in the area has its asking price changed to $349,999.00.

So if you didn't want to spend $350K for a house, you shouldn't have been in the marketplace!?
posted by de void at 10:55 AM on November 7, 2012 [4 favorites]

JPD, you're also choosing to overlook to the very real and tangible risk of HFT causing a major market meltdown. What's to prevent another March 2010 from happening again?
posted by 1970s Antihero at 11:03 AM on November 7, 2012

Well Themis was a leech. Another leech came along and is better at leeching money from you. When times are normal spreads are smaller and commissions are lower. You'll excuse me if that doesn't instantly make skeptical of the first leeches argument.

Its not clear to me how placing those orders and cancelling them actually impacts a real investor in any stock with real liquidity, so I don't really care at all. As long as I am mostly buying and selling on limits, then who cares? Its illegal, and sure laws should be enforced, but it seems like a vestige of another time when those rules were needed to protect traditional market makers.

The calculus is really really really simple. If you are an investor you shouldn't actively be trading and if you are a speculator who actively trades then I don't really care if you get out speculated.

The only argument that holds any water to me at all is the withdrawal of liquidity in times of crisis, but really that becomes an opportunity for you as long as you are set up to never be a forced seller, which is really how you should be set up.


Right that's a contrived example in another market that has no liquidity amongst assets that aren't fungible. Really the original example is the equivalent of me bidding 350k for a house without realizing the sellers would have accepted a 325k bid. Whether that goes to the homeowner or someone who owned it for a day and flipped it to me for a small profit doesn't matter to me, because I was happy to pay 350k for it.
posted by JPD at 11:06 AM on November 7, 2012 [2 favorites]

JPD, you're also choosing to overlook to the very real and tangible risk of HFT causing a major market meltdown. What's to prevent another March 2010 from happening again?

No. I'm not. I just don't care about a flash crash and neither should anyone who is an investor and not a speculator.
posted by JPD at 11:07 AM on November 7, 2012 [1 favorite]

The flash crash could have been a whole lot worse. From ennui.bz's link:

The more obvious problem with exchanges run by computers is that computers don’t have any common sense. We saw this on the 6th of May, 2010 — the day of the so-called “flash crash”, when in a matter of a couple of minutes the US stock market plunged hundreds of points for no particular reason, and some stocks traded at a price of just one cent. It was sheer luck that the crash happened just before 3pm, rather than just before 4pm, and that as a result there was time for the market to recover before the closing bell. If there hadn’t been, then Asian markets would have sold off as well, and then European markets, and hundreds of billions of pounds of value would have been destroyed, just because of a trading glitch which started on something called the e-mini contract in Chicago.

posted by 1970s Antihero at 11:19 AM on November 7, 2012

That wouldn't have been worse, that would have been better.
posted by JPD at 11:25 AM on November 7, 2012 [1 favorite]

computer's don't have any sense that we don't give them. Noticing a missing, necessary sense means people can put that sense in place.
posted by garlic at 12:28 PM on November 7, 2012 [1 favorite]

This link claims the definative cause of the flash crash is unknown, and circuit breakers should prevent it from happening again.
posted by garlic at 12:31 PM on November 7, 2012

> If those 29 cents changes the calculus of the trade for you then you are a speculator, and well - you just got out speculated.

But they get to take 29 cents from each and every investor, without providing any sort of positive service. It bothers me in the same way that someone blocking my way into the subway and asking for a nickel would.

The worst is that they actually provide a negative service to the market - by increasing volatility. It's entirely unclear that they are ever going to provide liquidity - there's no advantage to them staying in to make a market.

And it seems to me that they are breaking both the letter and the spirit of the law and of the agreements with the markets they are trading on - both of which are considering a bid or an offer being an attempt in good faith to make a transaction, which high frequency trading is not.
posted by lupus_yonderboy at 12:52 PM on November 7, 2012 [3 favorites]

> This link claims the definative cause of the flash crash is unknown, and circuit breakers should prevent it from happening again.

If you don't know why something happened, how can you be sure to prevent it from happening again?
posted by lupus_yonderboy at 12:53 PM on November 7, 2012

But they get to take 29 cents from each and every investor, without providing any sort of positive service. It bothers me in the same way that someone blocking my way into the subway and asking for a nickel would. If it bothers you then don't buy on limits. When you put a limit order out there you should expect it to get filled somewhere near where the limit is.

The worst is that they actually provide a negative service to the market - by increasing volatility. It's entirely unclear that they are ever going to provide liquidity - there's no advantage to them staying in to make a market.
I don't think the evidence is that during normal market operations they increase volatility, they do increase volatility during small periods of dislocation, but also get paid less for their market making compared to the traditional market makers, so the math is really not "what do HFT cost me when I can't find liquidity" but rather is that cost greater or less than the cost difference between what them and the traditional market makers earn during normal operations.

If you have yourself structured in such a way that you don't need liquidity at an moment from the market then the math is pretty clear. Being a provider of liquidity during periods of illiquidity is one of the classic ways for investors to make money.
posted by JPD at 1:04 PM on November 7, 2012 [1 favorite]

I see a lot of people deriding HFT, but I'd be willing to bet that if if humanity ever invents FTL travel or communication, it will be because of these guys.
posted by blue_beetle at 2:23 PM on November 7, 2012

I see a lot of people deriding HFT, but I'd be willing to bet that if if humanity ever invents FTL travel or communication, it will be because of these guys.

I'd be willing to bet that if the thousands of brilliant minds involved in HFT had been focused on problems other than shaving 0.00001 seconds off of a transaction, or making 0.001 cents on a trade, we'd be better off as a species. We have a company spending 300 million dollars laying a submarine cable to shave the travel time of signals between London and NYC from 65 milliseconds down to 60 milliseconds. But the same traders are going to pay to use this new cable, and I don't think the independent investor is particularly worried about the 5 microseconds extra it's taking for their trade to execute.

It's like someone manufacturing boxes for people to stand on to watch a parade -- at the end of the day, a bunch of trees got cut down, the box maker made a few bucks, but everybody has the same view they had before.
posted by Homeboy Trouble at 3:02 PM on November 7, 2012 [3 favorites]

BBC Radio 4's More or Less (hosted by Tim Harford, the Financial Times' Undercover Economist) had a an good episode on HFT. Much of it, including a brief introduction to algo-sniffers, is also in Tim's BBC magazine article.
posted by NailsTheCat at 3:09 PM on November 7, 2012

I interpreted blue_beetle's comment as a joke. Reminds me of how in Primer, the first thing the protagonists do with their time machine is use their knowledge of the future to make money in the financial markets. Necessity is the mother of invention and all.
posted by [expletive deleted] at 3:28 PM on November 7, 2012 [1 favorite]

There's been a huge brain drain out of physics and math --- the quants --- into finance. Instead of working on quantum entanglement or better cryptography, this is (part of) what they're doing.
posted by bonehead at 4:43 PM on November 7, 2012

The more obvious problem with exchanges run by computers is that computers don’t have any common sense. We saw this on the 6th of May, 2010 — the day of the so-called “flash crash”, when in a matter of a couple of minutes the US stock market plunged hundreds of points for no particular reason, and some stocks traded at a price of just one cent. It was sheer luck that the crash happened just before 3pm, rather than just before 4pm, and that as a result there was time for the market to recover before the closing bell. If there hadn’t been, then Asian markets would have sold off as well, and then European markets, and hundreds of billions of pounds of value would have been destroyed, just because of a trading glitch which started on something called the e-mini contract in Chicago.

Let them go down. The idiots who sell stocks they bought for $21 for $0.03 deserve what they get.
posted by gjc at 6:01 PM on November 7, 2012 [1 favorite]

But they get to take 29 cents from each and every investor, without providing any sort of positive service. It bothers me in the same way that someone blocking my way into the subway and asking for a nickel would.

But that's how the market works- people offer what they are willing to pay, and only sell what they are willing to sell. If there is 29 cents to be made, someone will do it. It doesn't matter who it is.
posted by gjc at 6:04 PM on November 7, 2012

The idiots who sell stocks they bought for $21 for $0.03 deserve what they get.

I don't have any particular moral objection to this, but a major problem is that exchanges have, rightly or wrongly, canceled trades in situations like this. This disincentivizes people from stepping in to act as a market maker in times of distress (as JPD notes, this is ordinarily a good way to make money).

Rather than trying to stamp out HFT for this issue, the more straightforward solution, it seems to me (and some others), is simply to ban market orders. Most people who enter in a 'sell at any price' order don't really it to be filled literally at any price.
posted by dsfan at 6:15 PM on November 7, 2012 [1 favorite]

I've worked with a few HFT guys. The guys I worked with were always the bare metal guys who worried about micro optimization of method calls and inner loops.

I did take a javascript "master class" from an HFT guy who developed his own algorithms. He did some of his data modeling and tested algorithms in javascript, it allowed him to iterate very quickly.

He didn't say a whole hell of a lot about his trading algorithms. But some of the things he did say made it seem like there are a whole shitload of HFT and algorithmic trading guys just playing games with each other. Ferreting out triggers and conditions of how competing algorithms work and leveraging those triggers to trick other "bots" to make tiny bits of money.

Every once in a while articles likethis pop up and everyone piles on to say how real HFT guys would eat him for lunch cuz his stuff is just too slow.

As always.I can't help but think some of the hate is sour grapes, nerds who are trying to game the system instead of selling or even knowing all that much about business.
posted by Ad hominem at 6:28 PM on November 7, 2012 [1 favorite]

What a well written article. While it's technically about HFT I'll likely use it in educating people about basic finance.

What I found quickly today was 70% of trade positions are held for an average of 11 seconds.

That stat alone doesn't tell you much. It could be (and likely is) the same 500 stocks being traded over and over again. Or it could all be trading in penny stocks. Or both.

How much total value (for some value of value) trades every 11 seconds would tell you a lot more about what the stock market gets used for in the big picture.
posted by Tell Me No Lies at 6:46 PM on November 7, 2012

Let's keep it legal but have, oh, I dunno, an 8% sales tax for high-frequency trading, trading instruments faster than, oh, I dunno, let's just say 365 days (you know, to encourage personal savings and fiscal responsibility).

The 8% would pay the national debt off in record time. After that, it should just go into the pockets of the people, sort of like reverse socialism. Everyone wins!

Or, we can wait and see. My friends in Wall Street all know this is all going to come crashing to a halt one day, and they all have plans to flee the scene. Check it out - look for the rich ones, and ask if they have ranches in Wyoming as a hedge on the coming Wall Street apocalypse.

You know who you are.

Just sayin.
posted by Monkey0nCrack at 7:44 PM on November 7, 2012