Computational complexity in quantitative finance Predicting the stock market is hard! Can TCS make this sentiment more formal?

Recently I have started thinking a little bit about finance, and was wondering how knowledge of TCS could help. Hedge funds and investment firms seems to use algorithmic trading, machine learning, and AI all the time, but TCS results seem to be few. In particular, I only know two papers:

Sanjeev Arora, Boaz Barak, Markus Brunnermeier, and Rong Ge, Computational Complexity and Information Asymmetry in Financial Products, 2009.

Philip Z. Maymin, Markets are efficient if and only if P = NP, 2011.


The first paper shows that derivatives can amplify the cost of informational asymmetry (instead of the desired goal of reducing it) for computationally-bounded agents. The second paper challenges the popular belief of efficient markets by showing that market efficiency can be used to solve NP-hard problems.
Are there any books/surveys or seminal papers on related ideas? Especially things related to the difficulty of predicting or approximating markets or trading optimally (or close to optimally) in such markets?
A slightly more meta question: why does there seem to be an abscence of papers on this? Is there no interest, or is that all the interested parties become quants that are hiden behind no-publication agreements?
Related question
Algorithmic lens in the social sciences
What is the Complexity Classification of Portfolio Theory in Financial Economics?
The question you begin with relates to predicting the stock market, but you seem to have broader concerns.  I'll attempt to tackle your meta-question; apologies in advance for my sweeping generalizations.

As far as I can tell, academic computer science is far removed from the actual concerns of hedge funds and people who try to model and predict markets.

The current focus areas in algorithmic game theory are not obviously relevant to finance practitioners.  In particular, worst case results are not seen as useful at all, and average case analysis based on artificial distributions seems largely irrelevant also.  Yet the only way to obtain information about real distributions seems to be to actually engage in the market, updating one's information using a variety of learning techniques.  This creates messy models that change dynamically and are not amenable to most types of analysis.

As an example, there has been a focus in finance on understanding the microstructure of trades.  Market microstructure is an emergent property of the specific low-level market mechanisms that are in place, such as how frequently pending trades are matched, what information traders believe exists in the order book, techniques used to obfuscate that information, the roll-back mechanisms in place, contractual arrangements relating to settling trades, network latency in receiving updates about the current state of the order book, and many other factors.  Market microstructure is a highly reflexive system, so the clean models typical of TCS seem beyond reach.

The market design community is trying to tackle questions like this (for instance see Huang and Stoll and the recent Kirilenko et al. paper on the flash crash), but they do not seem to have much interaction with TCS.

Finance has become increasingly complex as IT has pervaded markets.  This means that most markets now consist of multiple interlocking systems which it may not be possible to meaningfully model separately.  In addition, as markets move closer to continuous trading, I am not sure the TCS lens of computation is currently all that useful in finance; control theory, graphical models, fluid dynamics, and many other areas of applied mathematics seem more directly useful.

TCS methods could well be useful, but one needs to expend effort to understand what happens in finance, to find a place to apply the lever, and to acquire an appropriate mathematical toolkit.  Personally I would like to see more work along the lines of Arora/Barak/Brunnermeier/Ge, which engage with deep questions.  For instance, does adding more degrees of freedom to financial systems lead to good outcomes for the users of these systems?  Or does adding complexity mainly serve to help intermediaries set up asymmetric zero-sum games against the users?  There is probably a neat complexity-based argument waiting to be discovered...

So in a nutshell: you haven't seen much TCS/finance research because it is hard to apply TCS to finance.