Recent reports suggest that artificial intelligence will “crack” the financial market code using big data – machine learning. Given the vision of the success of machine learning in language domains, we should not be surprised by the strong demands or expectations in the capital markets.

After running two decades of computer-based investment software, I do not believe there is a code to break. What exists is a constant search for a systematic “edge” where the car knows when and how much risk to take.

Those who intend to spend their money on such programs should ask tough questions about what gives them an “advantage” – most importantly – it will be sustainable. In addition, the sobering law of the car dealership is that there is an inverse relationship between the “power” of the program. Systematic AI machines are subject to the same law.

To clarify the role of machine learning in forecasting, it is helpful to ask if practicing artificial intelligence is like learning to drive. The answer is no, but examining the differences is important to form realistic expectations of emergencies in capital markets.

In the case of a car, there really is a code that needs to be cracked. The problem mainly involves geometry, immutable laws of motion և known roads – all real estate.

The small change that will take place gradually is that if not all cars will become autonomous. But this should only make the task of machine learning easier, due to the unpredictability of road operators.

Second, the training data is huge, collected from many machines in real conditions. In five years’ time, autonomous vehicles will move even better than they do now, and they may eventually become error-free. Every progress in navigation is built in collaboration with the research community. Fixed target ավելի higher data density will break the code.

Financial markets are not volatile. They are constantly changing due to political, social, economic or natural events. Data is limited by how often: how much we want to predict the future. As eloquently described in the book “Flash Boys”, cars are able to learn predictable current patterns in the financial markets that arise from the actions of people և machines. Such data is very dense in the sense that the car has 480 one-minute samples during the eight-hour trading day, from which one can learn to make one-minute forecasts. It should get more than 10 thousand views in a month.

But if you want to learn how to make one-day forecasts, data is relatively scarce, so you need long enough stories of different things for different conditions to create reliable models. The density of such data grows much more slowly over time without drivers.

Equally important is the fact that markets are highly competitive in two respects. First, any new ideas or terms are quickly copied և competing away. Therefore, it can be argued that the role of intelligence in financial markets is not to find the Holy Grail, but to have a process that can recognize changing conditions and opportunities and adapt accordingly. This makes the prediction problem much more difficult.

The second source of unhappiness is that making larger deals does not give you a wholesale discount, on the contrary. It may be relatively easy to trade 100 shares of IBM at the current price most often, but it is impossible to buy and sell 1,000 shares at that price. The presence of size makes the market an adversary. This universal law applies to all car dealers.

The image below plots the performance և capacity և relationship, measured in millions of dollars invested using industry-standard risk-adjusted profitability, the Information Ratio (approximately 0.4 for the S&P 500 in the long run). The larger the storage, the longer it should be stored. Consequently, the data available for learning are scarce and the results more vague. Performance deteriorates rapidly with storage time, especially if you store overnight. There is no free lunch.

In the early 2000s, I ran a high-frequency program that rarely lost money, but could not exceed a few million dollars in capital. The regulatory change changed the market dynamics և eliminated its advantage, but it brought about other software operators who capitalized on the microstructural effects of the change. There are currently several high-frequency software operators that feed on what liquidity they can find to operate, but high-frequency trading is not a viable business model for a large asset manager or regular investor. It is another animal.

My forthcoming research quantitatively determines the uncertainty of decision-making behavior in machine learning systems in the face of various problems. It explains why a collection of self-driving predictive models taught on large data set variations would agree that the object in front is a pedestrian rather than a tree, while a set of models based on small variations in market history is unlikely to agree with tomorrow. about the direction.

This translates into more uncertain behavior of artificial intelligence systems in areas of low predictability, such as the stock market compared to sight.

If you are considering an artificial intelligence investment system, you need to do some serious homework, starting with the real story. Ask yourself if the program is based on sufficiently dense learning data given its average duration. Does the operator have a well-defined process that consistently follows the scientific method? What do you say about the typical model uncertainty և performance results you should expect? How much will the performance decrease if the operator increases the power? After all, will the edge of preservation be preserved in the future, or is it endangered out of competition?

Do not invest if you do not have clear answers to these questions. You want to invest, not play games.

Vasant Dhar is a professor at Stern Business School at New York University. program at the Data Science Center. He is the founder of SCT Capital Management, a system-based hedge fund based on machine learning in New York.