This guys explanation of "quote stuffing" seems incorrect to me:
"What benefit could there be to whomever is generating these extremely high quote rates? After thoughtful analysis, we can only think of one. Competition between HFT systems today has reached the point where microseconds matter. Any edge one has to process information faster than a competitor makes all the difference in this game. If you could generate a large number of quotes that your competitors have to process, but you can ignore since you generated them, you gain valuable processing time."
I don't think this is correct, since it's not a trivial matter to ignore your own quotes. Say there are two orders about to come in, and you place a third. The market data feed then tells you [order 1, order 2, order 3]. It does not tell you which order is yours. So you need to separately run code which remembers when you placed orders, and then guess that "look, order 1 and order 2 match the order I placed, I guess order 1 is mine, lets ignore it when doing the rest of our calculations". That's not fast code.
Besides, if your broker thinks you are DOSing the markets (as well as the broker's system), they will shut you down FAST.
My "stupidity, not malice" explanation: novices didn't throttle the oscillations in their code and feedback loops developed. Some were self-loops - basically, you just don't have the self-order detection code I described above, and try to trade based on your own actions. Some were loops between traders - trader 1 places an order, trader 2 cancels in response, trader 1 cancels in response to 2, trader 2 places an order in response to the cancel, etc. The band-saw patterns look like bidding wars on wildly undervalued stocks (e.g., one system places an order for accenture at 0.05, another at 0.06, another at 0.07, etc), which is actually a very good thing since it pushes prices back up to their proper levels.
Technically speaking, it may not be a trivial matter to ignore your own quotes, but it's not very difficult, either - if it takes M operations to handle your typical fully processed quote, and N operations to hash the quote and compare it to a table of quotes that you've delivered, then as long as M > N you've got some room for manipulation as long as you're able to pump enough bullshit into the system that everyone else has to deal with; I assume that in these situations M is usually substantially greater than N, probably on the order of at least 10:1 unless the quote is discarded as obviously irrelevant. It's just awful hard to imagine that the amount of processing done per quote in a HFT system can be less than a few hash table lookups...
I'd imagine the real difficulty would be making sure that the quotes you're putting in are a) relevant enough not to be immediately and trivially screened out by your competitors, b) not in danger of actually getting acted upon, and c) seemingly legitimate even in the face of after-the-fact scrutiny so that you avoid getting butt raped by the SEC for manipulation. a) and b) would likely be pretty easy to satisfy if you didn't have to worry about c), so...I don't know, this doesn't seem terribly implausible to me.
Then again, I don't know much about HFT, there may be factors in play that would make it a lot trickier to manipulate things in this way. And I'm certainly not convinced that this is what was behind the crash - "stupidity, not malice" is a good rule of thumb, and I'd have to see more evidence to assume that someone deliberately manipulated the markets here.
If you are doing high frequency trading, processing a quote isn't that hard either. It can't be, or you'd never respond fast enough. I suppose the best way to do it would be to ignore quotes X distance from the book, and then do your DOS X+0.01 from the book, though this might not be compatible with your regular strategy. Whatever DOS you want to do has to be compatible with your already existing strategy. I'm not saying it's 100% impossible, only that it isn't easy.
Additionally, your broker starts getting annoyed at you when your fill rate drops below 0.5% or so. If you DOSed the market, you wouldn't even be in that ballpark.
FWIW, looking at those graphs, it does appear that a lot of the extraneous action was happening right on the edge of the inside bid/ask.
Additionally, your broker starts getting annoyed at you when your fill rate drops below 0.5% or so. If you DOSed the market, you wouldn't even be in that ballpark.
That's definitely true, and I'm sure it would be a serious problem with doing anything of this nature.
Is there always a middleman that cares about what quotes you're pushing through, though, or do some people have closer connections that wouldn't be watched very closely?
Some larger funds become a broker/dealer, so they can cut their brokers out of the loop. I really doubt that anyone big enough to do that would be willing to risk their business on a DOS attack, however.
I fully agree that it's possible to do. I just find other explanations, like badly written algorithms, much more plausible.
Do you have a technical background in trading/exchanges? (FYI, I have a background in trading, but not HFT or connecting to exchanges) I think we should defer somewhat to the people who provide exchange data professionally.
Don't you think the people who work with this data (and its consumers) professionally know what's fast enough to run and what's not?
Don't you think they know what behavior gets you shut down, how quickly you get shut down (and therefore how long you can game the system), and what doesn't?
I'm not saying you're explanation isn't possible/plausible, or that you're skepticism is unfounded, but I've seen you post several times about Finance/HFT and I haven't usually found your commments/assertions were very substantive.
Thanks, I didn't know about his background (I had checked his profile/web pages but didn't see anything about the HFT stuff). The hilarious (perhaps stubborn) thing is that I still don't weight his information very highly.
"What benefit could there be to whomever is generating these extremely high quote rates? After thoughtful analysis, we can only think of one. Competition between HFT systems today has reached the point where microseconds matter. Any edge one has to process information faster than a competitor makes all the difference in this game. If you could generate a large number of quotes that your competitors have to process, but you can ignore since you generated them, you gain valuable processing time."
I don't think this is correct, since it's not a trivial matter to ignore your own quotes. Say there are two orders about to come in, and you place a third. The market data feed then tells you [order 1, order 2, order 3]. It does not tell you which order is yours. So you need to separately run code which remembers when you placed orders, and then guess that "look, order 1 and order 2 match the order I placed, I guess order 1 is mine, lets ignore it when doing the rest of our calculations". That's not fast code.
Besides, if your broker thinks you are DOSing the markets (as well as the broker's system), they will shut you down FAST.
My "stupidity, not malice" explanation: novices didn't throttle the oscillations in their code and feedback loops developed. Some were self-loops - basically, you just don't have the self-order detection code I described above, and try to trade based on your own actions. Some were loops between traders - trader 1 places an order, trader 2 cancels in response, trader 1 cancels in response to 2, trader 2 places an order in response to the cancel, etc. The band-saw patterns look like bidding wars on wildly undervalued stocks (e.g., one system places an order for accenture at 0.05, another at 0.06, another at 0.07, etc), which is actually a very good thing since it pushes prices back up to their proper levels.