Backtesting is an incredibly powerful tool that can be used to understand how our trading algorithms might fare in the real world (might is the key word). However, if you’re a data scientist coming from a non-finance background it can be rather difficult to understand what it all means. Sortinos or Sharpes? Returns or profitability? This shouldn’t discourage you, as some of the best funds in the world (see RenTech) are comprised of non-finance folk. Instead, its time to learn.
In this article, we’re going to discuss some crucial metrics for evaluating the performance of your trading strategy. Truly understanding these basic metrics, will give you a strong base to be able to critically assess different strategies.
Creating Metrics to Measure Performance
These are some of the starting metrics or parameters that will help you gauge the performance of your trades. In general the metrics cover two important aspects of a strategy: how your portfolio value changed and the risk in achieving those gains/losses. By understanding these two areas, it will help you identify its weaknesses and strengths.
Metrics focused on P/L
The metrics in this section all tell you something about how much money you made (or lost) with a particular strategy. The starting point is obviously to look at the final amount of money, but there are other metrics that give us a bit more detail:
- Annualized Return: The yearly average % Profit(or Loss) from your trading strategy.
- Win/Loss, Average Profit/Loss: Sum(or Average) of Profits from trades that results in profits/Sum(or Average) of losses from trades that results in losses
- % Profitability = % of total trades that resulted in profits
We normally talk about return on capital as a percentage, with the idea that the strategy is a multiplier on your starting capital. This is useful in most situations but we should also remember that it’s only true up to a point (e.g. diminishing returns caused by limited asset liquidity or increasing slippage as you buy more).
If we want to understand our strategy in more detail, we next need to understand how we’re making money. For example, are we making lots of small losses and then big wins or are we making lots of small but consistent wins? By looking at different combinations of profitability, win/loss and profit/loss, we can start to understand how our strategy will work.
Metrics focused on risk
Equally important to seeing massive returns, is understanding the probability of the strategy losing money in the future. The famous adage ‘no risk no reward’ is only spoken by winners, not the much greater proportion of people who took a risky bet and it didn’t pay off. Our key metrics here are:
- Annualized Volatility: The standard deviation of daily returns of the model in a year. Volatility is used as a measure of risk, therefore higher vol implies riskier model.
- Max Drawdown: The largest drop in Pnl or maximum negative difference in total portfolio value. It is calculated as the maximum high to subsequent low difference before a new high is reached.
Drawdown is an important risk factor to consider as our backtest will run over the entire period regardless, but in reality we’re far less likely to sit on a losing trade. Imagine you brought Amazon stock in ’98, in hindsight it would have been a good idea to never sell a share, and to buy as much as you could in the dotcom bust. In reality, it’s highly unlikely that anyone would stay in a trade as the saw their money go -10%, -20%, -40%, -80% etc.
Combined risk and reward metrics
There are then a third type of metric which give you a combined view of the returns and risk in one metric. For both of these you want to get them as high as possible. The metrics are:
- Sharpe Ratio: The reward/risk ratio or risk adjusted returns of the strategy, calculated as Annualized Return/Annualized Volatility
- Sortino Ratio: Returns adjusted for downside risk, calculated as Annualized Return/Annualized Volatility of Negative Returns
In the previous section we noted that volatility was a measure of risk. The truth is that not all risk is bad and that is why we have two metrics here. They differ in that Sharpe looks at all volatility and Sortino just considers downside volatility. As a rule of thumb, you want to see a high Sharpe and a higher Sortino.
Proper Accounting of Costs
The other important considerations in evaluating trading strategies are the cost involved in making the required trades. As a beginner, it is common to overestimate the performance of your strategies and this is one of the major reasons why.
A lot of quants assume that only commissions to brokers are the only transaction costs of a trading strategy. However, there are a few more factors to be aware of. Two other important examples are:
As you know that it’s really difficult to trade without an intermediary known as a broker. Acting as an exchange, brokers offer transactional services and get paid commissions in return. What may come as a surprise are the added expenses and fees that brokers sometimes charge. This includes additional services, exchange mandated costs and any government tax that get imposed on the financial transaction.
A key aspect that often goes unnoticed as an evaluation factor is slippage. Any difference between the price that you wanted to trade at and the price you actually end up trading is called slippage.
Why is there a difference in these prices? There can be many factors. For example, you maybe wanted to buy a 100 shares of AAPL at 100$, but only 50 people were selling at 100$, and 50 more at 101$. The price you would have traded at is 100.50 and your slippage would have been 50 cents.
A component of transaction costs, slippage can quickly turn a theoretically profitable strategy into one that performs poorly. In the previous example, if you planned to sell your shares back at 102$, slippage would have reduced your profit by 25%! Slippage can be minimized by building an efficient execution system but it’s important to understand how it could impact your trades.
A quick note on personal biases
Everything starts from within and by everything we mean both profit and loss. Though the market and its volatility are crucial in determining how much profit or loss we’re going to hit, there’s always a voice within that guides us through in a trade. Sometimes, these voices can be helpful; but mostly, these are personal biases. Often, such personal biases drive us through a range of emotions and make us take decisions we weren’t supposed to in the first place. Such personal biases are emotions we need to discipline to know when to stop and when to proceed with a trade.
Our decision making abilities are stalled by a number of emotions and so are crucial parameters to evaluate a trading strategy. Some of the emotions that cloud our minds include excitement, thrill, optimism, fear, anxiety and panic. Since these are the driving emotions, a check on them at the right time will always help us increase the performance of a trade for the better.
For more on this, we’ve also have a resource on biases in backtesting and risk management. This will give you further insights on personal biases and how you can stay away from them for a better trade.
These are some of the most common parameters on evaluating a trading strategy. Understand them and try implementing them on your strategies, don’t be afraid to make notes. Once you’ve assessed the results, you can start making tweaks to further boost the performance of your trades.
If you’re interested in quant finance, our data science competition platform will let you learn on real hedge fund data and teach you vital skills.