High Frequency, Fat Target
Michael Lewis misses the competitive benefits of computerized Wall Street trading.

Flash Boys: A Wall Street Revolt, by Michael Lewis, W.W. Norton and Company, $27.95
For the last five years, the press has been sounding alarms about high-frequency trading (HFT), a practice in which investors use fast computers driven by secret algorithms to rapidly trade securities. Time wondered in a 2012 headline whether the practice is "Wall Street's Doomsday Machine." Mother Jones in 2013 worried it could "set off a financial meltdown." In March of this year, 60 Minutes aired an infomercial-toned segment promoting the new Investor's Exchange (IEX) trading venue, which, according to IEX's website, is "dedicated to institutionalizing fairness in the markets" by slowing down trades.
Now we have Flash Boys, Michael Lewis' highly lauded attempt to explain the dark ways of Wall Street to the masses. The book seems aimed at convincing the public that high-frequency trading is bad and that firms should be trading on IEX, because the other "markets are rigged." This emotion-provoking tagline has prompted cries for regulation, launched legislative committee hearings, and forced trading firms to defend the integrity of the financial markets.
Lewis' book is primarily about the individuals who developed the IEX. These characters are compelling, and the book provides interesting insights into the types of people lurking around Wall Street's back rooms. The hero is Brad Katsuyama, a former Royal Bank of Canada trader who became frustrated by his inability to beat high-frequency traders to a particular stock price.
Katsuyama found that when he wanted to purchase 100 shares of a stock at $50 each, he might only be able to purchase 10 shares at that price even when it appeared that 100 were available when he executed his order. The remaining 90 shares might have to be purchased for a penny or two more. Through trial and error, Katsuyama discovered that HFTs were beating him to the stocks, because getting 100 shares at $50 required him to trade across multiple trading venues. The computers used by HFTs were so fast at processing data, they were able to detect the demand for the stock at $50 and rush to the other venues to buy it before he could. The HFTs then sold it back to Katsuyama and others like him whose orders were executed more slowly.
Trading venues exist at specific physical places. When Katsuyama, or any trader, placed an order on his computer, it would arrive at the physically nearest venue first. At that point HFTs picked up the signal and raced to the more distant trading venues to beat traders to the $50 price. To resolve this problem, Katsuyama developed an algorithm that slowed down the speed of his trades but changed the order in which they arrived at different exchanges. By sending his orders to the most distant exchanges first, he was able to beat HFT algorithms to the stock.
It's a story well told, but it doesn't support Lewis' thesis about markets being dangerously rigged. The fact that this new strategy was indeed able to neutralize the HFT strategy suggests that markets are excellent at producing competitive corrections to any distortion. If the financial markets were truly rigged, Lewis' band of merry men and women wouldn't be able to take from high-frequency traders and give to IEX.
Logical gaps like this appear throughout the book. Lewis raises several important issues, but time and again, he either misses the solution or fails to see the problems in a broader context, preferring a simplistic if entertaining heroes-and-villains narrative.
The book does provide some value to the lay reader. Lewis is particularly on point in describing the unintended consequences of the Securities and Exchange Commission's Regulation National Market System (Reg NMS), implemented in 2007. Reg NMS was designed to encourage competition in the marketplace and to ensure that investors receive the best price execution for their orders. To this end, traders are required to buy or sell equities for their clients on whichever exchange will give them the best price at any given moment.
This has created at least two important unintended consequences. The first is what Lewis describes as market "fragmentation." In the past, most trades were handled by either the New York Stock Exchange (NYSE) or the NASDAQ. Now, trading is divided between 13 public exchanges and about 50 alternative trading systems within the United States alone. Reg NMS was successful in making markets more price-competitive by breaking the domination of the NYSE and NASDAQ. This very fragmented environment, Lewis claims, laid the groundwork for high-frequency trading.
Fragmentation did indeed create the need for computerized trading systems to quickly process price and market information across multiple venues for the best price. But this alone needn't have been a problem. Faster and more efficient markets with increased price competition have driven trading costs down for everyone, and that's a good thing.
The problem arises with the Securities Information Processor (SIP), which consolidates and disseminates prices across trading venues. The SIP is managed by a 15-member committee of exchanges, market participants, and regulators, and it is operated, via contract, by the company that owns the NASDAQ stock market. To facilitate discovery of the best available price across so many different venues, Reg NMS required that the nationwide best bid and offer price be compiled in one place. The venue that this information was presumed, but not required, to be compiled, was in the SIP.
The problem is that in order to provide the national best bid and offer, the information has to be collected, calculated, and disseminated by the SIP. The technology used to do so is old and much slower than that used by high-speed traders.
To overcome this gap, HFTs began creating their own internal versions of the SIP, but ones that process the information much faster. Because these homegrown clearinghouses process information so much faster than firms still relying on SIP-distributed data, they can detect demand for equities well before the competition. Thus HFTs are able to buy at better prices and sell for a penny or so more to those using price data direct from the SIP.
Lewis suggests that the SIP's sluggishness means we should eliminate high-frequency trading technology. But high-speed traders jumping ahead of slower competitors is just a symptom. Since the slowness of the SIP caused the perceived problem, why not fix the SIP, or require centralization of national best-bid and-offer information in a more modern system? Or maybe we should repeal Reg NMS instead of creating a Rube Goldbergesque tangle of regulations designed to regulate the regulations. Blaming high-speed trading technology for informational inequities is like blaming the train for a derailment caused by a stalled vehicle on the rail.
Lewis also fails to mention just who is most interested in banning HFT: the exchanges and legacy trading firms that have not competed as well in this new environment. The first firm to trade on the new IEX venue was Goldman Sachs-and after Flash Boys was published, Goldman's chief operating officer declared HFT bad for U.S. financial markets. Now the New York Stock Exchange is lobbying against HFT.
Why? Fragmentation creates competition, and both Goldman Sachs and the NYSE are losing market share in this new environment. They want their customers back.
Lewis' markets-are-rigged mantra is gaining purchase. After the book came out, the FBI launched a probe of HFT and New York Attorney General Eric Schneiderman made his year-old investigation of the practice public. Some brokerages have received requests from their clients to have their securities traded on the new IEX venue, and IEX's trading volume has risen dramatically.
Lewis and his compatriots have rallied the villagers to take up torches and pitchforks to fight market monsters that they know little about. They are arriving at their brokers' doors demanding they trade with IEX, and they are calling their legislators to insist on new investigations and regulations.
Yet the jury is still out on whether the high-frequency traders' overall market impact has been negative or positive. HFT is not taking over markets; both its share of trading volume and its profits have been declining steadily since 2009, according to The New York Times. There is also evidence that HFT has dramatically decreased the costs of trading for all market participants, increasing liquidity and decreasing spreads.
It is attractive to believe that markets are rigged, since someone always seems to be better at investing than we are. If Lewis' book and the emotions it provokes manage to have more influence on public policy than reasoned analysis does, then the crusader against rigged markets may have pulled off one of the greatest market manipulations of all time.
Editor's Note: As of February 29, 2024, commenting privileges on reason.com posts are limited to Reason Plus subscribers. Past commenters are grandfathered in for a temporary period. Subscribe here to preserve your ability to comment. Your Reason Plus subscription also gives you an ad-free version of reason.com, along with full access to the digital edition and archives of Reason magazine. We request that comments be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of reason.com or Reason Foundation. We reserve the right to delete any comment and ban commenters for any reason at any time. Comments may only be edited within 5 minutes of posting. Report abuses.
Please
to post comments
my roomate's sister makes $88 /hour on the laptop . She has been out of work for 8 months but last month her paycheck was $12338 just working on the laptop for a few hours. Look At This...
????? http://www.netjob70.com
This type of trading has very little to no effect on personal investors.
First - editors, reconsider your use of the term "investor" in ' high-frequency trading (HFT), a practice in which investors use fast computers driven by secret algorithms to rapidly trade securities.' THey are not investors.
As to the comment that this has no effect, that is false.
There is no deep liquidity in stocks or bonds anymore as a result of HFT. Order book? gone. Market makers? gone (and please don't refer to HFT as 'market makers'). Liquidity crash? Here in spades.
They're better at arbitrage than you are. Too bad.
My only beef is if they get special access, i.e. co-locate servers at the exchanges to reduce latency to the exchanges. My solution would be to lock everything to a reasonable clock, say every second, and execute all trades accumulated in that second at once. If there's a settlement issue then I'd suggest that the transaction favors the seller, i.e. they get the highest price offered.
There are a several companies starting to offer algorithmic trading to the masses (e.g., Quantopian).
How close to the exchange are their servers? Not as close as BOA's and Goldman's, that I guarantee.
Merrill Hess|10.26.14 @ 12:39AM|#
"How close to the exchange are their servers? Not as close as BOA's and Goldman's, that I guarantee."
So your concern here is the physical proximity of the equipment and "C"?
Well, you're sort of stuck with the same constraints that made Holiday Inn a good investment; location, location, location.
Problem is they're not recognizing arbitrage, they're creating arbitrage. Big difference.
My only beef is if they get special access, i.e. co-locate servers at the exchanges to reduce latency to the exchanges.
Your only beef? That's the entire steer.
"This type of trading has very little to no effect on personal investors."
I've said this from the first day, but a larger question is whether, for example, I should be able to extract one penny from each person in the USA...even though it really does not put any dent in anyone's pocket.
This is called "skimming" in the real world....if I worked in a gold foundry and brought home a little dust each day, the world nor my employer would likely not suffer any measurable loss....(maybe a couple cents per week per employee). But is it right?
This is really the same thing. It falls into that category of "if they do it. shouldn't we be able to do it?" - or are only those who find ways to hack systems allowed to profit from them?
Casino's bar you and sometimes jail you if you count cards or find a little bug in their system.
craiginmass|10.25.14 @ 8:00PM|#
"I've said this from the first day, but a larger question is whether, for example, I should be able to extract one penny from each person in the USA...even though it really does not put any dent in anyone's pocket."
Hi, ASSHOLE!
No. Fuck you.
What makes you think you aren't able to do it? Since the beginning, if you know your stuff, you either can get hired or you can get investors to support you. These days, algorithmic trading is increasingly offered as a commodity: you can rent computers and access the way you would any other online service.
Your ignorance is not a barrier to entry.
Mark, I think you're missing asshole's mendacity.
Through either his stupidity or sophistry, the claim is that those using the algorithms are 'hacking'; IOWs, those using any sort of competitive advantage are cheating according to asshole, and he therefore presumes that random begging is the equivalent.
Asshole is a mendacious lefty imbecile; presume dishonesty and you'll be correct almost always.
Right, ASSHOLE?
I don't understand why you are getting rude with me. Nor does your name calling really advance the libertarian argument.
I think you misunderstand. I have no gripe with you, and yes, I am constantly rude to the asshole known as craig, simply because he is a lying asshole.
Oh, and presuming you could 'advance a libertarian position' with the asshole known as craig means you have yet to see his constant lies and mis-representations.
Are you really sure you want to deal honestly with an asshole who claims that libertarianism is somehow coercion in that it allows the Koch Bros(tm) to provide funds to Reason?
Help yourself; I'll continue to refer to asshole as asshole.
And if you really know card counting tricks, you can beat the house at the blackjack table in every casino in Las Vegas.
That doesn't mean playing blackjack, or hiring someone to play blackjack with your money for a cut of the winnings, is a reasonable suggestion for a normal person.
You want to play your rigged Fed-propped-up game, go ahead. I'll take my money elsewhere, thanks.
Trading is unsafe at any speed.
"There is also evidence that HFT has dramatically decreased the costs of trading for all market participants, increasing liquidity and decreasing spreads."
Could we see the data and analysis supporting this assertion?
Anything they show you will conveniently leave out the vacuum that has resulted from HFT when "news" or other events take place. What were small gaps before are now gaping holes. Have a nice day when your s/l order is filled points away.
That's my expectation, but I thought I'd give them an opportunity to show something.
Wouldn't the brokerage firm that fills your S/L order also have access to HFT?
If your broker is slow, switch to a faster one. It's no different from any other service.
Awesome, so now I get to pay for proximity of the servers to the exchange, in addition to paying for the service I actually wanted to buy. That must be how you're "decreasing the costs of trading", eh?
Merrill Hess|10.26.14 @ 12:36AM|#
"Awesome, so now I get to pay for proximity of the servers to the exchange, in addition to paying for the service I actually wanted to buy."
Did you want to buy slow trades?
Actually I wanted to buy the ability to trade, period. Not to get extorted by BOA and Goldman for extra cash because they've essentially put a toll booth in front of the exchange.
Merrill Hess|10.26.14 @ 12:51AM|#
"Actually I wanted to buy the ability to trade, period."
You've got that.
"Not to get extorted by BOA and Goldman for extra cash because they've essentially put a toll booth in front of the exchange."
So you don't want to pay what it costs to trade rapidly? Well, trade slowly.
I wouldn't mind trading slowly (ie in time increments greater than nanoseconds), except BOA and Goldman are skimming money off my trades.
Merrill Hess|10.26.14 @ 12:56AM|#
..."except BOA and Goldman are skimming money off my trades."
How so?
So you're commenting on something you have no knowledge of. Awesome.
Go read up on the subject first, when you come back you may have something to contribute.
Merrill Hess|10.26.14 @ 1:07AM|#
"Go read up on the subject first, when you come back you may have something to contribute."
Oh, I'm sure whiners are posting all over the net.
Go read up on how a market works and come back when you may have something to contribute.
So you're commenting on something you have no knowledge of. Awesome.
I was following this exchange when Sevo asked a straightforward question, "How so".
Then you descended into this and evaporated. That tells me your argument is largely based on emotion and thin on fact.
The developed world and complex markets run on technology. If your brokerage can boast faster trades, that's called a value-added service vs. the guy who gives a paper chit to a runner.
That tells me your argument is largely based on emotion and thin on fact.
Or that the machinations of what he was asking about are complex, and not conducive to explanation in a blog comment, but they are available in more scholarly sources. If you're going to attack Lewis' work, you really should familiarize yourself with it rather than just breaking in to a CAPITALISM RAH RAH RAH rant.
More accurately, you can't build an argument where anyone is actually "skimming" in any accurate sense of the term anything from your personal trades, so you'd rather not get into the complexities of it because they'd make you look like an even bigger whinging retard.
The developed world and complex markets run on technology. If your brokerage can boast faster trades, that's called a value-added service vs. the guy who gives a paper chit to a runner.
Nice strawman. I'm asking for milliseconds, dude, not the Pony Express. There is no advantage to a retail investor from a trade completed in nanoseconds vs. milliseconds, other than that privileged market players are going to rip you off in the latter case.
Michael Lewis wants us to believe the market is rigged, that we will never get the best deal because the intermediaries are in the way.
The argument goes something like this:
When High Frequency traders are involved, you can always get your order filled but you will never get a better price than your limit.
However, I know from personal experience, this is just not true. As an individual investor, it is not unusual for my trades to be executed at a better price than my limit.
And, programmatic trading (not just HFT) does provide more liquidity. Again, I know this from personal experience - I have had trades executed when the bid and ask prices indicated there wasn't much interest in my trade - but because someone's algorithm detected a good arbitrage opportunity, my trades were executed - to my advantage.
However, I personally think one of the problems we have in the market is that we have enacted so many safeguards to protect the 'retail investor' that we have created a moral hazard. People do not educate themselves enough about all of the risks inherent in the market, because they believe someone else is performing that due diligence for them or that the government has regulated these risks out of the market. Then, when someone like Lewis explains a natural phenomenon like front-running, and then declares that the market is rigged because this natural condition exists, these same people believe him.
Mountain of documented evidence in Lewis' book over here, random poster's claimed personal experience over there. Which one am I to believe?
Calling front-running "a natural phenomenon" isn't helping your side in the balance.
Merrill Hess|10.26.14 @ 12:32AM|#
"Mountain of documented evidence in Lewis' book over here, random poster's claimed personal experience over there. Which one am I to believe?"
Evidence of what?
True, Lewis gave us a very interesting book on the mechanics of HFT. HFT is out there, and it does impact pricing (to a very small extent). I just can't get worked up about it, and neither should most 'retail investors'. Especially now, as HF traders are losing market share and making less money, as the market reacts.
And yes, Front Running is a natural phenomenon. It's prevalent in many markets (concert tickets for instance) and occasionally appears in other markets (the latest iThingy from Apple). Someone has the resources to buy the product first at the 'face value' or 'list' price, and they resell it at the 'market' price.
So "front running" is nothing other than a pejorative for "arbitrage"?
Not in the strictest sense - many see 'front running' as a bad thing, as some of the folks here have referred to it as putting a toll booth in the system.
Wouldn't it be nice if you could buy your concert ticket directly from the box office, without having to pay some third party extra money just because they have a computer system that can buy up almost all the tickets as soon as they go on sale. Certainly, the scalper is not buying the ticket to enjoy the concert, he is buying the ticket to make a profit when he resells.
So, in a sense it's 'forced' arbitrage. But, how do you prevent it? Outlawing scalping penalizes people that purchase the tickets then can't go to the concert.
The ticket sellers could control it, until someone finds a way around it. The airline industry has solved it by issuing tickets in the traveler's name, and not allowing tickets to be transferred.
But what it really comes down to for me is convenience - if I want to go to a concert, what price am I willing to pay to get the tickets as conveniently as possible. If the ticket broker makes my life easier, it might be worth it to me to pay his mark-up
The problem in the scenario you describe is that the original seller is charging the scalpers a price that's too low. If the scalper immediately goes out and sells the tickets at market price, the end consumer hasn't really lost anything. It would only be a problem if the scalpers cornered the market and artificially lowered supply by releasing only a few tickets at a time for sale.
The analogy for HFT frontrunning would be the scalper noticing that you are in the middle of placing an order to buy the last 10 tickets to some obscure concert, jumping in and grabbing the tickets before you complete the transaction, then offering to sell them to you for 10% more. That's not speculation, that's outright cheating.
One of the points that is missing from this argument is why HFT works in the first place. It's because when brokers are trying to buy or sell large numbers of shares, they try to do it without letting the market know how big their actual order is. Why do they do this? to maximize their price. (I guarantee you that a senior trader at RBC was not making trades for 100 shares unless there was a much larger order behind it)
What the HFT guys figured out is that by watching what types of orders were entered, the volumes of these orders, and who was placing the orders that they could tell (or their algorithms could tell) that there was a much larger order behind it. They read this market signal, and take advantage of it.
Just like if a scalper saw that a band sold out in Chicago, and will probably sell out in St. Louis.
That's ordinary HFT, not the front-running kind we're talking about, which actually interrupts orders in progress. It's like you get to the supermarket checkout line with a cart full of groceries, the cashier tells you it's $100, and before you can swipe your card, the cart is gone; the cashier informs you that someone else bought it for $100. Then you go out to the parking lot and are offered the contents of your cart for $110 by a guy in a pinstripe suit and a toupe.
Just like if a scalper saw that a band sold out in Chicago, and will probably sell out in St. Louis.
Not "just like" that at all. That would be ordinary following of the market. "Hey, Apple just got a million shares bought, so I'll buy 10,000 because more people are probably going to want it in the future." Nothing sinister about that.
The examples you give are of ordinary speculation (both occurring because the "list price" is obviously too low). That's very different from X buying Y because he saw Z place an order for it, and wants to force Z to buy it at a higher price.
Merrill Hess|10.26.14 @ 1:05AM|#
..."That's very different from X buying Y because he saw Z place an order for it, and wants to force Z to buy it at a higher price."
Can you say "arbitrage"? Yeah, I know you can!
And you're not willing to pay what it costs to keep up.
Tough.
LOL. You guys need to get your story straight.
On the one hand you claim the market isn't rigged, on the other you tell me I have to pay off the big boys if I want to have a chance in the game.
You having to possibly pay more for a broker that can compete with HFT computers doesn't make the market "rigged". You're throwing a petulant tantrum because someone else has a competitive advantage over you and you don't like it.
The ability to place an electronic toll booth in front of the market is not "competitive advantage". The ability to buy at the price the seller is offering is fundamental to the operations of a market, and the HFT frontrunners are destroying that ability. The option to pay them off to get out of the way does not legitimize what they're doing.
How is it different? I 'know' that everyone wants to see the latest boy band when it comes to town and they want to buy their ticket at list price, but would be willing to pay more. If I have the capital and the technology then I can insert myself in between the retailer and the consumer, and make a profit as a result.
I don't know this for a fact, but my guess is that these ticket resellers have algorithms to help them figure out how popular an event will be, and how many tickets they should buy. Otherwise, if they just bought up every ticket for every event, they would probably not make as much money as they could.
So, this really is no different than programmatic trading.
It's different because in one case you're taking advantage of the fact the original seller underpriced their goods (and everybody involved knows they are underpriced), while in the other case, the good was not considered underpriced, you're reacting to a single buyer starting an order.
It's also a weird example to use because limited-attendance entertainment providers often collude with the scalpers to give them early access to the tix. The entertainment provider gets good PR for "keeping ticket prices low" and they get kickbacks from the scalpers.
The safeguards you mention don't just create a moral hazard, they also create inefficiencies and barriers to entry, so in tat sense, I think the market is "rigged" and intermediaries are getting in the way.
Of course, as usual, these problems and rent seeking are caused by people in the guise of helping others.
Not much being done here to counteract the perception that libertarianism = corporatism.
But we all know that BOA and Goldman got their position in the market by pure hard work and pleasing their customers, nothing else. LOL.
Merrill Hess|10.26.14 @ 12:55AM|#
"Not much being done here to counteract the perception that libertarianism = corporatism."
Surprise, surprise, surprise! Losers always revert to ad-homs when sympathy is hard to come by.
Buzz off.
Is there a suggestion that HFTs only exist because Goldman and BOA got bailouts?
Somewhere else maybe, not in my comment. The bailouts do cast a spectre over attempts to make this into a RAH RAH CAPITALISM issue.
Don't get me wrong, there is such a thing as good HFT, just like there is such a thing as good bureaucracy. In both cases, once it's around for long enough the bad kind is going to dominate.
Your total inability to differentiate between regulatory capture resulting from government interference in markets and competitive advantage gained from exploiting natural market opportunities doesn't make them the same issue.
I'm not sure you understand what corporatism is.
http://en.wikipedia.org/wiki/Corporatism
Libertarianism are for completely free markets.
So yeah, they are for things like this, but pretty much all the other stuff that the government does at the behest of corporations to keep out competition (or in some cases, flat out give them money).
Libertarians would be happy for there to be a financial meltdown. That's how markets correct themselves.
That's really the problem - there is no risk anymore - the government basically underwrites those from taxpayer money.
I understand that libertarianism is not corporatism -- indeed many classical liberals were very anti-corporatist, viewing corporations as creatures of the state.
My issue is that several people on this thread have not looked at Lewis' work (or many other people's work on the same subject), see that corporations and Wall Street are being slighted, and immediately jump in with the RAH RAH CAPITALISM nonsense.
Libertarians would be happy for there to be a financial meltdown. That's how markets correct themselves.
Maybe that's how it works in books, but not in the real world. The two big financial meltdowns we've seen in the last century have given us the New Deal and Obamanomics, respectively. The statists never let a crisis go to waste, so why libertarians seek crisis is beyond me.
Unfortunately, libertarians and arch-conservatives alike dramatically underestimate the profound damage caused by deflation and the inevitable moves toward a less free society that are demanded by the frightened public thereafter.
my co-worker's mother makes $71 /hr on the laptop . She has been unemployed for 9 months but last month her payment was $17334 just working on the laptop for a few hours. published here
----------------http://shorx.com/onlineatm
Just ask anyone who runs equity portfolios of any reasonable size for a living how HFT has changed things. They'll all tell you that in the post-HFT world, liquidity is illusory in that the moment they cross markets by hitting a bid or taking an offer, suddenly the market for whatever they're trading moves against them, often sharply. Overwhelmingly, HFT is viewed as front running and a direct wealth transfer from underlying investors to HFT principals.
Precisely. People need to take their ideological blinders off here.
I should specify that HFT was originally a good thing, taking existing smart market practices and speeding them up... but once HFT became more widespread, that kind of "good HFT" ran into diminishing returns. The only profitable type of HFT left is the frontrunning variety.
Hopefully, as the big exchanges like NYSE lose credibility and lose trades to "speed bumped" exchanges like IEX, the market-market will do its magic to solve the problem. But right now there are major penalties for being an early adopter of the "speed bumped" exchanges, so it's going to take a while.
my co-worker's mother makes $71 /hr on the laptop . She has been unemployed for 9 months but last month her payment was $17334 just working on the laptop for a few hours. published here
----------------http://shorx.com/onlineatm
Can you have some spare time to sit back in your chair having your laptop with you and making some money online for some interesting online work said Jenny Francis in the party last nightsee more what is for you there to increase your pocket money??.
http://shorx.com/clickforsurvey