Finance, much like many fields, has become an increasingly siloed industry, especially in relation to technology. There are people who know finance and there are others who can code. This is the situation I faced when arriving as a summer analyst at JP Morgan in the summer of 2017.
As the technology department generally lacked a working knowledge of the financial industry, there were many opportunities to integrate the two fields. In an attempt to do this, I set out to develop an algorithm that would utilized the technical skills held by software teams to help fully meet the needs of the financial analysts. As our division focused primarily on Chinese equity I opted to develop a machine learning algorithm that could outperform the archaic research methods that were in use.
Building a hypothesis
I began this project asking several fundamental questions. What do the financial analysts want? What could they have right now that could make there lives easier and their decisions more robust? So I started to talk to people, as many as I could, and as I did, the message became abundantly clear: there was too much information.
When an analyst wants to determine what a market will do in the coming months, he/ or she can look at hundreds, if not thousands of variables. For an analyst, and their finite brainpower, this is an impossible task.
So to solve this problem, I began building a machine learning algorithm that could analyze a variety of factors to determine future market movements by comparing the current market environment to past ones.
After a little research, I realized that when the market environment–denoting the aggregated movement of different market factors like the three listed above–is similar to historic ones, the outcomes tend to be the same. This is shown above as the market environments–composed of the three factirs–in both March of 2014, and July of 2016 were very similar, and have resulting market movements seem to mimic each other.
Launching the idea
As I now had a hypothesis, I set out to build an algorithm that could do this on a large scale. After going through multiple iterations, and hearing feedback from both programmers and analysts I landed on an algorithmic design:
This model could pull in factor data, process it by breaking it down into first and second derivatives, aggregate the factor results, search for similar instances, output predictions and then continuously back-test the algorithm to fine tune the predictive strength.
Testing the results
After implementing this architecture through python code, the testing process began. I did this by running my algorithm through a k-fold assessment model, which back-tested my model over multiple time periods. The predictions proved strong, as shown in the example below for 2017:
Over a 20 month testing period this algorithm proved an 85% success rate in predicting Chinese market movements.
When used to decide whether to buy or sell Chinese stocks, this model created significantly more value than the standard buy-and-hold strategy that was used by JP Morgan. The stark contrast in the investment returns over a 5 year period can be shown below:
Due to it’s success, this algorithm is now being utilized by JP Morgan to create value for its clients on a day to day basis.
This example is just one of many in how technology, due to a disconnect between industry experts and software developers, is not being fully utilized. There is a strong need for people, in todays workplace, who have a firm understanding of both business goals and technology.