I’m using data derived from a walk-forward of the White Bumblebee system and the DX future contract (US Dollar). The out-sample data is over 10 years in length, and the total number of walk-forward periods comes to 10, as I’ve optimized on 400 days and traded on 200 days. The different fitness functions include Net Profit, Net Profit/Drawdown, and Pessimistic Return on Margin (PROM). For Net Profit, the system returns the parameter set that had the highest Net Profit during the optimization period and then uses those settings for the trading window. Same logic for the other fitness functions.

I’ve broken down the analysis into two sections: Profit Metrics, and Drawdown Metrics. In each category, I’m interested in the total, range, max, min, mean and standard deviation. My backtesting software is TradersStudio and the statistical software that paints the graphs is R.

First let’s look at the Profit Metrics.

In the end, all three produced about the same total profit over 10 years. Net Profit registered the largest profit during a single period, but also the highest range. All three fitness functions had losing periods, but it is interesting to note that the Net Profit/Drawdown function had the least worst period, or it lost small when it lost.

Next, let’s look at the Drawdown Metrics.

Here again, the Net Profit/DD fitness function yields the most attractive results. It experienced the least drawdown in a period and also the least amount of total drawdown during the 10-year testing period.

Keep in mind that the sample size is fairly small here with only 10 walk-forward periods being analyzed. You could easily argue that these results are not statistically valid, but at least it points out that all fitness functions are not created equal. Finding the best one may require some time and brain cells on the system trader’s part, but hopefully well-spent.

It can be anything such as large cap stocks, penny stocks, major currency pairs, or just one stock or currency pair. Or just the ES futures. Or you can make the case of trading what is cool and in vogue, such as GOOG into earnings, or gold stocks when everyone is buzzing about gold. I’ve chosen to trade the universe of liquid futures. And I’ve wittled down the list of qualifying candidates to just fifty-two.

My list is crafted carefully around major market groups, which have a tendency to inter-relate and correlate. The basic groups include:

1. Currencies (7)
2. Interest Rates (10)
3. Equities (9)
4. Metals (5)
5. Energy (4)
6. Commodities (17)

Currencies:

This group includes the Dollar, Euro, Pound, Yen, Aussie, Swiss Franc and Canadian Dollar. Pretty much the major currencies with the Aussie and Canadian related to commodities.

Interest Rates:

This groups consists of US Treasuries (2-year, 5-year, 10-year, 30-year), the Eurodollar and foreign treasuries that include the Canadian 10-year, the Gilt Long Bond (British debt), and the German debt complex of Eurobund (10-year), Euro BOBL (5-year) and Euro Shatz (2-year). Potential inter-market relationships between these markets and both the currency and equity markets abound.

Equities:

Okay, the standard US equity markets need to be included. I’ve chosen to leave out the DOW futures and focus on the S&P 500, Nasdaq 100 and the Russell 2000. For international exposure I’ve included the Nikkei (Japan), CAC 40 (France), Euro STOXX 50 (Europe), FTSE 100 (Britain), DAX 30(Germany) and the Heng Seng (Hong Kong).

Metals:

You can’t leave out gold or silver. Their ratio is actually an important metric. I’ve also included other precious metals such as platinum and palladium (think catalytic converters and cell phones), and the base metal of copper.

Energy:

Of course we include crude oil and natural gas. Also included are refined products such as heating oil and gasoline.

Commodities:

This is the largest group, so let’s divide it up into Grains, Livestock, Forest and Softs (I’m following the CME Group convention here).

Grains include corn, wheat, soybean, soybean oil, soybean meal, oats and rough rice.

Livestock includes live cattle, lean hogs, feeder cattle and everyone’s favorite future – pork bellies.

Forest includes lumber.

Softs include cocoa, coffee, cotton, sugar and orange juice.

And that’s it.

Just 52 underlyings, many of which have interesting inter-market relationships.

This is my final roster of players, and they will all participate in future backtesting regimes. Will I trade them all? No. Money management may limit the total number of underlyings to a small percentage of the universe, but those 5-7 futures will represent a good diversification of product, and will also be the best representative for a particular trading strategy.

Dory got confused once in Finding Nemo and said: ‘I don’t know where I am… I don’t know what’s going on. I think I lost somebody but I, I can’t remember.’ Don’t be that fish.

Cut-and-paste versus read-in:

After you’ve generated vast amounts of data from an optimization run, you find yourself with the loathsome task of cutting and pasting the data into an Excel spreadsheet that you’ve probably already pre-formatted to look at the data. It requires you to select the data and not miss any rows, and it opens up the possibility that something gets left out. With R, you don’t touch it once you’ve converted it to a .csv file. You simply reference it in your script.

Manipulating columns of data:

In Excel, you know the drill. Select a cell to compute the mean of a row and then select the elements of a column that you’d like to apply the formula to. When you have over 1,000 rows, it takes a while to drag that stupid mouse. Just as you approach the end of your values, the cursor goes wild and runs down to infinity. Ugh. Well, with R that is no longer an issue since you define the column of data (called a vector) to a script that quickly returns the value you’re looking for.

Add-in packages:

Remember how excited you were to install the Data Analysis pack in Excel and how it gave summary statistics for a column of data that included mean, standard deviation and other stuff you had no idea what they were? Well, that’s a standard part of R. And as you graduate from Statistics 101, you’ll find that there are add-in packages in R that range from neural networks to Bayesian statistics, from genetic algorithms to copulas (side note: the geniuses who created tranched derivatives that came home to roost last year used gaussian copulas, amongst other things).

Cost:

I think I paid in the neighborhood of $250 for my copy of Excel. I paid nothing for R, since it is open-source and free.

The issue of crashing Windows:

I have crashed my computer by asking Excel to compute too much data. So far, R hasn’t produced the same results. And based on its architecture, I’m not expecting it.

Graphs and dynamic data:

If you want to create a chart or graph of something in Excel, you need to navigate through that stupid menu of crazy pictures to get something that’s clunky and difficult to maintain. In R, you use scripts to create charts and graphs, and the data file feeding the chart is dynamic so that each time data is added, R will incorporate the new data. No offsetting shenanigans.

There are video tutorials available at the usual places you’d find that sort of thing, and a very robust community of R users to help get you started. So put down your toys and get to work. There is much data that needs analyzing.

System trading distinguishes itself from other forms of trading in that it has a high threshold for capital allocations. Feelings, notions, guru tips and gut instinct are not sufficient for the system trader. There needs to be a formalized approach to the trade that can be back-tested on historical data. This first step is not all that involved, really. There are many off-the-shelf programs that can perform a back-test of a trading system. But once you get the results, what do you do with it?

You ponder, perhaps, and cast a skeptical eye upon promising results. You start wondering (and rightfully so) whether past performance has any correlation with future returns. Well, how would you know? At this point you can either sell the program to others or try it out yourself.

A way to test the back-test is to perform an optimization. This is running the back-test with different parameters (ie, moving averages with varying length for an MA crossover system). If it performs well under this process, you can gain some more confidence in your system. But how would one define ‘performs well’? There are many metrics to look at including net profit, drawdown and ratios of those two. How many permutations were profitable as a percentage of all the permutations that were run? What percentage of permutations had a net profit that exceeded maximum drawdown? Is that a good number? What is the best parameter set? What was the measuring stick used to determine the winning set? Now we see where the quagmire begins.

You are now the proud owner of stacks of statistics that you need to sort out. It’s like going through your coupon drawer. You know, the one that has mostly expired coupons, and some for junk you wouldn’t buy if it were free. The optimization results are not a self-evident validation or rejection of a trade system. It takes a thinking person to evaluate what they’ve got.

And after this is sorted out, there is still another step of testing the system on out-of-sample data. Why? Well, because your optimization process found good parameters, but it fit the data to do it. The process is called curve-fitting in statistics and involves making things fit. Not necessarily bad, but it can be overdone. That’s why we walk-forward, or test on out-of-sample data. It’s the same thing as trading the system in real-time. After this process is complete, you find yourself with another truckload of disorganized coupons.

Besides the programming skills needed to do the basics, you need a decent grasp of statistics, good organization skills (where did I put that file on the Coffee market between 1998 and 1990?), and the basic understanding of how markets work. Understanding markets is crucial, as many nerdy statisticians with excellent programming skills have proven by losing large sums of money. Don’t be afraid of having to learn a lot of stuff, though. Trading is not for the lazy.

The formula for correlation is actually quite simple. I could derive it for you, but this is not high-school statistics, so I’ll just show it below:

It’s the best depiction I found on the Internet. Look at the bottom part of the equation and just substitute that gobblygook with standard deviation of X and standard deviation of Y and you have a much cleaner looking equation.

Oh, yawn. Where’s the stock tip? I need to know what stock I need to buy tomorrow so please, stop with the algebra already.

These highly trending equity markets will not sustain themselves indefinitely, and you need to start thinking about the range-bound scenario that may happen in the next couple years. For that you need a system. And I offer you a metric now to compare your system’s equity curve with Perfect Profit’s equity curve.

What does a high correlation indicate? Well, that you are extracting profits (albeit at a slower pace than Perfect) in-line with what your competition is doing.

What does low correlation indicate? Your system is not achieving its destined returns based on what the market offers.

If your system has low correlation with your ultimate competitor, Perfect Profit, then you probably are just lucky right now. There is nothing wrong with being lucky and being rich, but consider your fortune as a potential predicament.

To get an accurate handle on the concept, we first ask what time frame we will be using. For end-of-day traders or swing traders, the daily bar is your competition. If you day-trade off the five-minute bar, well then use the five-minute bar to gauge your performance.

This concept is used in system trading and because vast amounts of data are typically used, we system traders need to resort to our super-duper calculators, also referred to as our programming language. This is an example of what code looks like for our daily perfect trader. The language is TradersStudio’s version of Visual Basic, but you can get the idea and use it with any program you wish. It’s the basic loop function.

For i = FirstBar to LastBar step 1

Next

PerfectProfit = PerfectProfit + Range [i]

You’ll need to dimension your PerfectProfit variable as an array if you want it to tally up daily ranges.

But you don’t need to be a systems trader to get some use from the concept. You can simply take a sheet of paper and tally the sum of all the bar ranges for your time frame. Take your net profit and divide it by perfect profit for a sobering assessment of how far from perfect you really are.

If you are 20% of perfect, then you have an impressive system. Don’t be discouraged to find out you’re more in the 2% range. What they say about flyfishing applies. First you try to catch a fish (profitable trade), then you try to catch a lot of fish (consistently profitable), then you try to catch big fish (participating in large moves) and then you seek the fish that has never yet been caught.

Good luck fishing.

My first impression is that it’s not an idea one would actually trade real money with. But is it an interesting factoid? Well, to pursue the idea, I quickly coded the idea (I use TradersStudio) with three parameters that include VIX Spike, N Days to Hold and Percent Stop Loss. Preliminary research showed that fine-tuning a stop loss percentage was not all that meaningful, so I focused on varying only the amount of VIX spike that would trigger the trade, and the number of days to hold the long Apple stock. Since I’m not developing this idea as a trade system, I did not protect any data from the perils of optimization’s curve-fitting process. I didn’t even really backtest the idea since I’m not sure what parameter sets would make sense but instead ran straight into optimization. The period is from October 1, 1999 to October 1, 2009 (10 years). I stepped the N days to hold from 2 to 30 days in chunks of 2 days, and the VIX spike from 2% to 20% in chunks of 1%. That’s 285 permutations that yields the following space with respect to profitability (above zero line is profitable):

Of all the permutations, 41% were profitable. Hmmm, maybe we can improve on the idea. What if no trades were taken when there was a positive correlation between AAPL and VIX? After all, it is a contrarian trade idea. Well, a little line in the code does the trick and the following optimization space is the result:

It yields less trades with the filter (obviously) but there is also a decrease in the percentage of profitable parameter sets with the total coming to 36%. Oh well, at least now we know. Anyone who made money using this system over the past ten years did so on the basis of pure luck. If , however, you have a time machine that can go back to 1999, I suggest you buy APPL whenever the VIX spikes at least 4% and hold the long stock for 2 days. You will have been a genius and could brag about an annualized return of 47% over ten years.

Instead of going through the whole process again like I did with Bumblebee, let me give you the psuedocode right quick and I’ll post the real code later. Assume the code is correct and now we are up to the backtest. I’ll do a quick backtest and post the data below. Later, I’ll run the optimization process to get a feel for how this insect performs under stress. Then I’ll follow up with walk-forward data later.

Ladybug takes a long position when the close is greater than the upper Keltner band. It takes a short position when the close is less than the lower Keltner band. Long positions are exited when the the close is below the simple moving average on which the ATR bands are based. Likewise logic for short exits. The simple moving average is 30 days for the initial backtest, with a ATR band of 1.5. These rules apply to White Ladybug, or the first version of the system. I’m happy to share this system, but later versions will require a non-disclosure agreement from you, in addition to copious amounts of money.

White Ladybug was backtested on 9 markets from 1987 to 1996. Later, we’ll run an optimization on the same period to see how robust this creature really is.

Here are the backtest results for nine markets on White Ladybug:

Once again, SP 500 performs abysmally. How does one trade that stupid market anyway? On the bright side, we have crazy great results from Dollar, Yen, Gold and Coffee, with cumulative returns in excess of 300% over ten years.

If you look closely, you’ll see that there is some pain that must be endured to experience these results, probably more pain than can be asked of any trader, systems or not.

But it’s a good start on White Ladybug. As someone once said:

There will be time yet for a hundred indecisions,
And for a hundred visions and revisions,
Before the taking of a toast and tea.

Your system’s equity curve is how much money it’s making. Any standard package will be able to draw the simple chart for you. It has its ups and downs, with hopefully more ups for your sake. The goal for system traders is to create a smooth equity curve, meaning there are no radical departures from a gradually increasing trendline.

If your system’s equity curve looks sloppy and you can’t live with it, you either stop trading the ugly beast or you start a process of making it look better, which usually involves adding filters (to get the bad trades out) and other overfitting exercises. If you start down the path of overfitting so that everything looks neat and pretty, you might as well quit trading right now and donate your trading capital to a local charity. You’ll save yourself frustration and feel good about contributing to your community.

A natural tendency amongst system traders upon viewing an ugly equity curve is to revert to their discretionary trading habits. Take this trade, but not that one. It’s no longer system trading at that point, but discretionary trading with system tools. There is another way to maintain the integrity of the vast amounts of statistical data you acquired about your system during its development. And that is to code an algorithm to throttle or feed your system based on its equity curve performance.

I’ll offer three equity curves but instead of the naked one you’re used to viewing, I’ve added a MACD on a lower panel and two simple moving averages, a red one (50 day) and a green one (20 day).

The first one is a no-brainer. With an equity curve like this one, you want to keep trading it. Feed it, nurture it and read it bedtime stories.

It’s steadily climbing with the 20-day average nicely above the 50-day, and the MACD not showing any warning signs. If this is your equity curve, nice work.

Next, let’s look at one that may be in a little bit of trouble.

It has had a steep run-up just lately, but now the spread between the 20-day and 50-day is tightening to the point where the 20-day is threatening a breach of the 50-day. The MACD shows stalling momentum and a general decline.

Finally, let’s look at a not-so-pretty equity curve. This one has had its 20-day crossover its 50-day, and the MACD is confirming a deteriorating situation of steady decline and sub-zero line occupation.

Do yourself a favor, and let this system tell you what it’s going to do next before allocating anymore funds to it.

For system traders, it’s just a matter of programming when your system takes a trade or leaves a signal alone. You can program a moving average crossover or a host of other technical indicators that you may use in any typical trading system. You’re not being discretionary if you program under what equity curve conditions a trade is taken. The programming aspects involve creating a shadow or virtual system that follows along your regular system. I haven’t programmed one yet myself, but I know I can do it in TradersStudio. I’ll post the system once I’ve gotten it correctly coded.

Equity curves are not unlike stocks, and stocks are not unlike equity curves. That’s why we sometimes refer to stocks as equities. Your system is basically a stock that you’ve invested time and energy into. You can employ the buy-and-hold philosophy with your system, or you can trade it.

To emphasize the fact that you can view your equity curve just like you would any stock, ask yourself what favorite stocks the three equity curves above look very similar to. If you guessed AAPL, IBM and F, then you’re right.