Walking forward the Bumblebee – We have issues

September 23, 2009

I was hoping it would be a non-event to blog about how successful the Bumblebee trading system performed under the rigors of walking forward. But alas, the little critter has been tripped up a bit. The initial results I’ve gotten are abysmal. As in I wouldn’t even commit Monopoly money to the enterprise. We started this journey with nine markets. Two of these markets (SP 500 and Coffee) showed early signs that they wanted nothing to do with the system. That’s okay. We’ll deal with them later. But of the seven remaining markets, three showed great promise in the optimization cycle of system development. Clearly, I’m disappointed.

Here is a spreadsheet of results for how White Bumblebee (the system’s first version) performed on fresh data. The basic idea with a walk-forward is to optimize on a set of data and use those optimal parameters in a walk-forward window. The window is forward data, or data that was not subjected to the fondling of the optimization process. It’s fresh, and unbiased. Historical, yes, but the system doesn’t know what year it is so we get away with a little slight-of-hand, to be sure.

The column that is labeled ‘# Periods’ is the number of windows that were traded. The relationship between optimization data (fishing for parameters) and the walk forward window (okay, let’s see what you got) is 200 to 50. Two-hundred days are used for optimization, and the system is traded on 50 days before the process begins anew. It doesn’t take too much mental yoga to see that your optimization data is moved forward after the window results have been recorded. That’s the walking part of walk-forward.

I’ve been giving some limited thought to getting a more clever metaphor for what we’re doing here, but so far walk-forward seems to capture the essence of the process best.

Okay, back to our issues with Bumblebee. I’ve made it easy to spot profitable versus non-profitable cells by painting them green or red. You can see that there is a dearth of green cells. The sigma levels show what happens if the system experiences standard deviations on the downside (where we lose money). Ideally, we’d like to get a little sigma cushion, but none of the walk forwards gave us anything.

So where does this leave us? Well, we can throw the system into the trash can. That would be clean and simple. But sometimes, you just need to play with the pain to win. So, we’re not going to give up just yet.

I’m not sure where things went so terribly wrong. I probably need to check my degrees of freedom and see if I’m doing something stupid there. Or maybe my window is too short and the system is getting re-optimized too much. Or maybe my fitness function is no good.

Uh-oh, I’ve introduced a new concept right when I was wrapping up.

What is a fitness function? Well, it is essentially the arbiter of what are the best parameters. Why give one arbiter dibs on making the decision of what parameters we’ll be using going forward? Essentially, someone’s got to do it. Who do we pick as our front man? We could choose the function that finds the best net profit on a historical basis. But Net Profit is sort of … well, dull and monolithic, unlike the markets we’re trying to squeeze money from. The concept of what is the best fitness function, or by what measure can we determine the parameters that have the best chance of producing profits in the future, is a whole separate topic, worth its own blog at some later time.

I used the pessimistic return on margin (PROM) as my fitness function in the walk forward of Bumblebee. It adds to the losers by its standard error and takes away from the winners by its standard error. I don’t think it’s the problem, quite frankly. But I’ll triple check my programming just to be sure it’s not eating my Bumblebee for no good reason.


Vladimir Visotsky redux

September 18, 2009

This is how the market bears feel about the market. Timeless.

Nothing is as it seems.

Upcoming Classes, Objects, Properties and Methods

September 15, 2009

Maybe I’m doing a little too much programming lately, but I’ve decided to lay out my plans for system development into classes and objects. This is more than just creating a super-duper trading system that uses data mining as its justification for massive, unheard-of returns. And it’s more than just creating a ‘system.’ It’s about a style of trading that uses systems, but more in the manner of a symphony and less in the manner of a 3rd grade recorder. I’ll be organizing systems under different classes. An instance of a class of systems will be an object, which itself contains properties that makes it unique from other objects within the class. Getting a little programmy for you? Not to worry. I’ve laid out some of what’s in store below.

So far I’ve shared the development of the Bumblebee trading system. It is a rules-based system, meaning it trades off signals derived from a single market that are clearly defined. The long entry for Bumblebee is when the fast moving average closes above the upper Bollinger band of the slow moving average. That’s an example of a rule-based trigger. Now let’s start organizing.

Bumblebee is an object of the class of systems known as rules-based systems. More specifically, it belongs to the Insect class of trading systems.

Insect Trading Systems (class)

These systems use price action and technical indicators to trigger entries and exits. The source of the price action and the inputs to technical indicators is the market that is traded.

Bird Trading Systems (class)

These systems use intermarket data to trigger entries and exits. Slightly more complicated than the Insect class, Bird class systems include data about what a separate market is doing to trigger trades.

Safari Animal Trading Systems (class)

These systems use neural networks with either basic back-propagation or radial net algorithms, or kernel regression modeling. The inputs into the network can include a wide-range of data. Because of greater jeopardy to overfitting, these systems deploy neural network outputs as enhancements to already viable trading strategies. Too many neural networks have blown up trading accounts, and we’re not going to let that happen here.

Dinosaur Trading Systems (class)

Reserved for more advanced strategies that may use multiple neural networks, for example.

So from that simple outline, one can easily see that the Bumblebee system uses rules to trigger trades, and that its triggering data is all derived from the market to be traded. You likely don’t know about my Pelican trading system, but you can tell from its name that it is an intermarket system that uses data from a separate market to trigger trades.

Now I’d like to designate some properties. We will be using a simple color system that will sound familiar to parents whose children participate in martial arts.

The color property of a system designates the version of the system. The first color is White. White Bumblebee is the first version of the Bumblebee trading system, and the one for which the code, backtesting, optimization profile, etc are publicly available here. Later versions will follow the following order:

Yellow: second version
Purple: third version
Orange: fourth version
Green: fifth version
Blue: sixth version
Brown: seventh version
Red: eighth version
Black: ninth version

There, that’s simple, no?

But we’re just coming to the place where it all comes together. Systems are not going to be traded as stand-alone toys. Instead, they will be grouped together and traded together. They will all play in the same sandbox. And that leads us to a new class known as box. The box is a trading system that incorporates more than one trading system. So the White Box includes the following systems, for example:

White Bumblebee
White Firefly
White Finch
White Zebra

Our White Box trading system incorporates two simple rules-based systems, one inter-market system and one neural network system. The Box’s ‘White’ property signifies that the Box includes only White trading systems and only trades single contracts.

More advanced boxes will follow the same color scheme mentioned above and will get progressively more complex to include more advanced money management schemes, and neural network-based constrictors and expanders, meaning it will have the ability to adapt capital allotments to different systems and markets.

You may be wondering what a particular instance of the White Box system would be. We have designated the Box as a class and White as a property, but there is no Object. Not to worry, each instance of the White Box will be given a name that will differentiate it from other White Boxes. A specific instance of the White Box could include any number of white systems and trade any number of markets, so each time a White Box is created, it will be given a hurricane name.

There, that completes the overview.

This should be fun.

Black Box Chronicles: a Conversation with Optimization

September 14, 2009

I’m having the most pleasant conversation with Optimization over a couple of espressos and biscotti when this hideous creature walks into the room. The she-devil says she has to talk with Optimization in private, so I muddle over to the counter to get two more espressos. But I’m listening carefully. There are threats being volleyed back and forth and some talk about children. The creature leaves and I saunter back to the table with fresh espressos in hand. Curious about the creature, I ask my new friend what that exchange was all about.

‘Who was that?’ I ask.

‘She likes to call herself Optimal, but I call her Overfit.’

Sounds like the perfect description of an ex. I come to find out that Overfit is a mistress of my new friend Optimization. And apparently, they have lots of children together. Optimization is caught in this relationship, and he can’t get out of it. It’s just one of those things.

‘Now, back to my main point,’ Optimization continues.

He gets back into his rant about how it’s not his primary job to find the optimal parameters of a trading system, but rather to test how sturdy a strategy really is.

‘There are really a very wide range of parameters any decent trader can pick from a very wide range of objectives. I offer much more than giving you a dream set. I can tell you if your system has the cajones to trade in future.’

I explain to Optimization that I get it. I also remind him that his job is to find optimal parameters, but also acknowledging that he needs good direction on what to look for from the trader.

‘I can find best parameters based on a variety of measurements,’ he says, ‘I can tell you what the best parameters are based on net profit, net profit/maximum drawdown or some other more clever measurements.’

‘How fond are you of finding best parameters based on net profit,’ I ask. It’s kind of a trick question on my part, really. I realize this isn’t the best way to measure the potential for future profits, but I’d like to hear his take on this.

His eyes start fluttering and he puts down his espresso, almost as a measure to ensure he has nothing to throw at the wall.

‘I don’t like that method, quite frankly.’

I’m about to leave it at that and then he starts in about Overfit.

‘You know that I have certain relationships that I cannot justify to you. Let me be clear, the best way to lead me into my own circle of hell is to ask me to give you the best parameters based on past net profit.’

Okay. Time to change the subject. Clearly not a favorite topic and a little awkward. Just as I try to switch the discussion over to clever fitness functions, in walks a very strange creature named Data Mining.

I wish I could properly describe the look on Optimization’s face when he saw that thing come into the cafe. It was sheer horror that looked comical at the same time.

Data Mining is the albatross around Optimization’s fragile neck. When traders deploy this creature, they are basically torturing Optimization to the core. Data Mining is most annoying even to the uninvolved observer. He mumbles nonsense and has a queer look of promise on his face, not unlike your favorite purveyor of useless kitchen gadgets. As the little troll came into the room, I could hear him repeating a phrase that was barely audible.

‘The chance that the S&P 500 will be up for the year is directly correlated to how many players in the World Series for that year have last names that start with the letter A through M. If most of the players have last names beginning with N through Z, expect a down year.’

‘Is that true?’ I ask Optimization.

‘Probably, but let’s not go there for now.’

I can see how tortured a soul Optimization really is. He is very much a nice fellow should you ever meet him. But he is plagued in the Box. He is an indentured servant to the trader and though he doesn’t always like or agree with what he is made to do, he performs regardless.

I’m starting to see that the Box is not only filled with interesting creatures, but ominous ones as well. I will tread lightly as a precaution.

Optimizing the Bumblebee

September 4, 2009

The purpose of optimization is not to find the optimal parameters. Yes, that’s not a typo. The main purpose of optimization is to validate backtest results. You’ll remember it was the backtest’s job to validate a trading idea. Well, now backtesting is on the hot seat and optimization is the judge and jury. Optimization will validate that backtest results are not simply the product of chance, and that the system stands a chance of making money in the future. It will do this by creating a profile sketch of the system. I know, sketches are not the first thing you’d associate with spreadsheets. But in our world, it’s what we have to work with.

We are going to go over the basic setup of the optimization run in just a moment, but let’s take a simple statistic from the optimization run and see if it helps illuminate our objective. The statistic will be ‘profitable parameter sets.’ These are permutations that made money on historical data. If we randomly picked a parameter set, what would be our chances that it would have made money? If 50% of our optimized parameter sets are profitable, that means you can basically throw a dart and your chances of making money is, well, 50/50. Of course we not going to throw darts, but you get the idea.

If only 10% of optimized parameter sets are profitable, then you’d better have the aim of a trained assassin. If 80% are profitable, call your half-blind neighbor over and have him fire away. If you absolutely have to be dead-nuts on in your choice of parameter sets, you may consider whether the enterprise is worth your efforts.

The space we’d like to paint (don’t worry, I’ve got some cool looking graphs below) should be basically square in two dimensions. A scan that is too fine will lead to over-fitting. That’s bad. We cannot avoid curve fitting as that’s basically what we’re doing, but we can avoid getting carried away.

Besides providing us with information about how lucky you need to be to make money with your system, optimization will provide an insight into other features of your system, such as what the winning percent is, what sort of drawdowns you should prepare for and how frequent are trades for winning parameters versus the frequency of trades for losers.

Bumblebee was initially backtested with a slow moving average of 30, a fast moving average of 10 and a Bolllinger band around the slow MA of 0.5 standard deviations. We are going to fiddle with the slow MA period and the fast MA period and leave the Bolllinger band width alone. The reason is simple. Moving average length will have the greater impact on the system than tweaking the standard deviation length, and we don’t have time to calculate too many permutations with the method of choice. That method is called brute force. Yes, you need to be brutish and forceful to do this, but it won’t be too terribly distressing, I promise.

Permutations are statistical combinations. If we ran a test with a slow MA of 30 and fast MA of 10, we’d like to ask what would happen if we used a slow MA of 30 and a fast MA of 11. How about 12, 13. 14 and so on. You get the idea. Let me derive a simple method for calculating this monster in the following statistical proof. Or, on second thought, why not you look it up in your statistic book and take my word for it. If we run a range of 20 periods for the fast and 20 periods for the slow, we will have run 400 permutations. Mathematically savvy readers will see that 20 x 20 is 400, and that is the secret shortcut you forgot you once knew.

In keeping with the square space ideal, we are running the fast MA from 3 to 24 in steps of one (that means, 3, 4, 5, etc.) and the slow MA in steps of 2 from 11 to 49 (11, 13, 15, etc.) This yields 410 permutations. After we take out pairs that don’t make sense (where the slow is faster than the fast), we end up with 371 parameter sets. And with no further ado, below are the results of this optimization sketch. First let’s observe the percent of parameter sets that are profitable. If it’s under 30%, I highlighted it in red to indicate potential problems. If it’s over 80%, I highlighted it in green, to indicate the blind squirrel principle applies (ie, a random dart thrower can make money using the system).

I also put a color code to the left of the market to indicate whether or not the system is useful in a particular market. The obvious losers with the Bumblebee include the S&P 500 and the Coffee markets. The markets that show the most promise include the Crude Oil, Gold and Ten-Year markets. The other markets show potential, but also show some warnings signs, so they’ve been designated as cautionary.

Let’s look at the optimization space for a obvious wanker and a potential best-friend. First the wanker. Below is the optimization space painted for the S&P 500 from 1991-92. Only six percent of potential parameter sets showed a net profit. Yes, that ugly.

You can see that overall, you needed to pick just the right parameter set to make money with the system during this two-year period.

Let’s now look at what a more lovely optimization looks like. Below is the optimization space for the Cotton during 1987-88.

You can clearly see that if you avoided a certain area of tiny slow and fast MAs, you’d at least make money. And if you shot for the area in the upper right corner, you’d make some nice returns.

Here is a sample of some other data gleaned from the optimization profile.

This is the spreadsheets results for the Ten-Year between 1995-96. You can see that 92% of parameter sets were profitable. And, 347 parameter sets yielded trades during the period out of a potential 371 sets. That’s impressive. But looking deeper, you can see that the winning parameter sets had a winning percentage of just 51%. The losing parameter sets had a winning percentage of around 35%.

Also, the winning parameter sets (the good guys on our team) traded an average of 10 trades during the two-year period while the losing parameter sets (the bad guys on the other team), traded much more heavily with an average of 35 trades during the two-year period. These are just a few of the revealing characteristics of the winning parameter sets versus the losing parameter sets that can alert you to when things are going wrong. In other words, when you have given the ball to the other team.

Black Box Chronicles – Getting Lost

August 28, 2009

I’m trapped inside a black box. I know, I was warned not to come here in the first place, told my best option was to keep it simple: trade the probabilities, let your winners run and limit your losses. It started as a simple glance, really. Then a gaze and then I took the first step into the black box. Now, I don’t know how to get out. Don’t be alarmed, I’m comfortable here. The people (creatures, really) are pretty nice so far. But I need to leave here at some point because from everyone I’ve spoken to, you can’t trade while you’re in the black box. That seems patently absurd to me since the black box is system trading, and ‘trading’ is part of system trading. Right now though, I’m not in the mood to argue. There’s some pretty cool stuff in here and interesting creatures roaming about that I’ve got to bring out.

For starters, there is enough gold in here that if I could just get a small fraction of it out, and the price of gold went to zero, I’d still be rich. I’ll worry about that later. Let me first relate the conditions surrounding my entrapment.

It started when I decided to leave the world of discretionary trading. That was a tough choice, but sometimes defining moments present themselves to you and you simply need to do what it takes to define yourself. Almost by default, system trading was the last method standing. So that’s the path I took. With system trading, I must warn you now, once you start poking around it doesn’t take long to follow so many winding paths that you can’t remember how to get back out. That’s what happened to me. My pursuit of system trading has led me into the trader’s black box. Trust me, it’s a lot more than a group of algorithms. I feel lost now, but at the same time I feel like I’m at home.

The first creature I came upon in the black box is a curious fellow. He’s a nice little man that reminds me of Yoda. I couldn’t figure out at first if he’s the janitor or actually runs the place. I’m still not sure.

So I asked him,’What is this place?’

He replied ‘It’s a black box.’

‘As in a trading black box?’

‘Yes,’ he says.

‘Where can I go to place a trade then?’ I’m feeling a little giddy at this point, like I’ve arrived by default in trader’s nirvana.

‘You can’t place a trade while you’re in the box,’ he says.

Not the answer I was expecting. It’s called system trading. And black box or no black box, there is trading in system trading.

‘Fine,’ I said, ‘then can you direct me how to get out of here?’

‘You don’t ever get out of the black box once you’ve entered it. You have to become the box.’

More bad and unexpected news. The song ‘Hotel California’ starts looping in my brain. But between you and me, I think he’s just the janitor.

I decided to keep exploring and leave this nice little man alone for awhile. So far, I’ve met many old friends whom I’ve never met in person before. Simple Moving Average is here, as is RSI, MACD and even Head&Shoulders (he’s a big fella, as one would expect). There are also these little furry balls that run around the place screaming “got to find the best parameter” in a soft, squeaky voices. They call them swarming bots. I’d like to kick one across the room, quite frankly.

I also ran into ADX. He was kind of bummed out telling me how it’s hard for him to get work lately. I promised him I’d send some work his way once I got out of the box.

Curiously, he told me the same thing as that nice little man. ‘You can’t leave the box.’

I’ll attribute that to the medication he’s obviously taking. You see, contrary to popular trading myth, there actually is emotion in the black box.

There is so much to explore in here, I’m cool with hanging out for a while. For instance, I’ve always wondered what the deal was with Fuzzy Logic. I just saw him pass by and he’s on my list of people to interview.

Yes, I realize that one can spend too much time exploring every possible trading method and form of technical analysis. After all, traders trade. We’re not perpetual students of methodologies. But since I can’t figure how to get out of this place at the moment anyway, why not spend some time enjoying myself?

On my list of places/people to see while I’m trapped in here.

1. Genetic Algorithms
2. Particle Swarm
3. Intermarket Correlation
4. Visual Basic and C# programming
5. Statistics
6. Neural Network
7. Kernel Regression
8. Fuzzy Logic

And that’s about it. After that, I’ll commit to getting out of here. I’m not too concerned about it, really. These walls are sort of fluffy. And the air is refreshingly crisp.

Closed SPX and RUT Iron Condors for August

August 19, 2009

Yes, I’ve filed for a divorce with high-probability trading, but there is the matter of leftover trades that must be dealt with. These are the last Iron Condors I’ll be doing for a while. The next one I do will be hatched under the roof of systems trading. You’ll see that I didn’t do what I said I was going to do with both these trades, and my assumptions about what the market was going to do for the next six weeks was completely wrong. I changed my mind as the market conditions changed, and in the mean time I endured the rather unpleasant emotions of dread, loathing, fear, resignation, hope, more dread and so on.

I put on the SPX Iron Condor trade for August at the strikes of 785/790 (puts) and 1010/1020 (calls). My credit was $1.15. You will notice that I skewed the trade towards the downside. My risk was greater on the upside ($10 on the call spread) than on the put side ($5 on the put side). The reason: I wasn’t too concerned about a rally, but more concerned about retesting the lows. Okay, you can laugh now.

I put on the RUT Iron Condor trade for August at the strikes of 450/460 (puts) and 560/570 (calls). Credit received was $3.15. My reasoning was that the market wasn’t going anywhere for a couple weeks, and I would take the trade off for a small profit with more than two weeks to expiration. Yes, you can continue to laugh.

If you trade with discretion, you really need to get past the fact that you will be wrong at times, sometimes dismally wrong. Discretionary traders who have been stung by wild market action that is in complete contradiction to their views tend to hedge their opinions going forward. I’m not going to make attempts to convince anyone that they should hedge their opinions in the market. Not because I’m concerned about you neophytes not listening to me, but only because it is completely worthless. Worthless because it takes personal experience to learn this lesson in trading, and good-intentioned words from others probably does more to hinder your development as a trader than help. Pay your own tuition. It’s the only way.

An Iron Condor is a hedged position by definition, because your further out strikes protect you from unlimited loss. The SPX trade this month had a total risk of $8.85 for a potential reward of $1.15. The RUT trade had a risk of $6.85 for a potential reward of $3.15. Not your typical positive expectancy scenario, but this is the strange world of options we’re talking about here.

Anyone following the market in the last few weeks would have observed that the SPX breached the short strike of 1010 and almost hit the hedge of 1020 intraday. In the RUT, the short strike of 560 was breached and the hedge of 570 was also breached as the index traded at 577 intraday. That’s enough to discourage just about anyone. It’s usually around the time when the price starts touching your short option that you begin devising ‘adjustment’ plans. You can move your short call spread up a strike by simply purchasing a like-option butterfly. For example, to move the RUT 560/570 call spread up to the 570/580 strikes (which gives you more breathing room), just buy the 560/570/580 call butterfly. Diagram it out on paper, it works.

Adjustments cost money though, and it doesn’t take long for you to become confused as to what you’re trying to accomplish. I chose not to make adjustments.

On Wednesday before expiration (tomorrow is the last day to trade AUG SPX and RUT options), I took the trades down. That is, I took off all the risk.

For RUT, I decided that buying the 500/560 AUG strangle was the best choice. It was marked around $4.20 before the open, but the futures were down so I put in a standing bid of $1.99. While I was buttering my toast, I heard the opening bell ring. Sauntering over to the monitor, I noticed that the order was not working, but was in fact filled. Holy smokes, that was fast! Okay, good. No risk is left and I have a small profit.

Next up the SPX Iron Condor. The strangle approach wasn’t working out so well. I had a bid for the 800/1010 strangle for $0.75 and when I saw it being marked at $0.62 after the open, I canceled and replaced it with a bid of $0.50. The spread got away from me and went to a $1.00. Now the sell-off from the close was getting challenged and that 1010 call was getting more expensive to buy back. The 800 put was going into no-bid land (zero bid, ask a nickel) where nobody wants to fill you. It was mucking it up. Another approach was to buy back my short 790 put for a nickel and work on the short 1010 call. And that’s what I did. Bought back the 790 put for $0.05.

I realize it’s hard to keep track of all this, but you will recall that though I neutralized my RUT Iron Condor (took off all risk), I still had long options, namely the 460 put (worth less than a nickel) and the 570 call (worth around $0.80). I was thinking of the long 570 call as a hedge against a melt-up on the markets and against my short 1010 call in SPX. Different markets, yes, but it helped me take a breath and consider an orderly exit strategy.

I bought back the SPX 1010 call for $0.90. That made the profit total $0.20. Big whipty do, I understand. But remember I was facing a loss of over $8.00 just last week. Now with my risk in SPX off, I sold the RUT 570 call for $0.65, and the long 1020 SPX call for $0.30.

After all was said and done, the profit from the SPX Iron Condor totaled $0.50, and the profit from the RUT Iron Condor totaled $1.81.

I was worried that when I announced my break-up with high-probability discretionary trading, she would sell my golf clubs to the punk kid next door for $20. But she hasn’t done that. Maybe she wants to get back together, who knows. You never can tell about these things.

Backtesting the Bumblebee (Part Two)

August 16, 2009

I only broke this up into two separate blogs because if you endured the first part, you may be interested in the second part. But you also may have had enough. I know polite readers will finish what they’ve started, so I wanted to give them a logical stopping point. In this part we will continue our exploration of how Bumblebee performs on historical data. So far, we got decent results from a 10-year test window in one market. Now, we continue down the rabbit hole.

Bumblebee’s performance in crude oil over separate 2-year windows is depicted in the following spreadsheet:

I’ve highlighted in blue the pleasant surprises, and used orange to point out the not-so-good results.

Our next step is to see how Bumblebee works on different markets. There is no expectation that it will do as well or as poorly in other markets. We’d like to see it do well in at least some other markets though. The eight other markets include corn, cotton, coffee, dollar, yen, gold, S&P 500 and Ten-year note. Once again, I’ve used blue to highlight pleasant surprises and orange to highlight dubious results.

So far, we can see that coffee and S&P 500 futures didn’t do so well. They lost money. But the good news is that about 78% of the markets actually made money, though cotton and gold pulled in a meager performance of around 3% each.

Nine markets over ten years yielded 641 trades, which is statistically valid. Taking the good with the bad, the consolidated performance gave us a 23.7% annualized return. Hmmm, I do believe this system bears further investigation. Let’s get granular again and divide up these overview reports into 2-year chunks to see if they yield any more information about Bumblebee.

Here is the S&P 500. Clearly not impressive. From 1989 to 1992, this system was dismal, to be kind. The only impressive result was long trades between 1995 and 1996.

Coffee is another market that eats this system alive. Six out of 10 years were losers, some with painful drawdowns.

Before we get completely depressed, let’s admire how the system performed with 10-year notes. The periods of 1991 to 1992 and from 1995 to 1996 yielded better than 100% returns, with long and short trades taking turns in the bounty.

The Dollar also shows some nice returns with the system, including some nice 300% returns from the short side in three separate test windows.

Gold is kinda lackluster, but at least the losing windows didn’t see too large a drawdown. Nice short trade in 1995 to 1996.

As for the Corn trade, not that impressive. The big long winner in 1987 to 1988 helps keep this one above water.

The Cotton market responds very similarly to the Corn market, with lackluster results, the decent returns from 1989 to 1992 notwithstanding.

I’ve saved the best for last, but what would you expect. The system performed well in the Yen market from 1987 to 1988. Overall annualized return was 132%, which was helped mightily by a 959% return on long trades during the period. This sort of result is almost too good to be true, so we have to give it guarded respect. It’s cool to look at though, and would be even cooler if it happened again in the future.

That is a lot of data to digest. It is just a start though. Next we optimize. As the backtest was a guardian against the trade idea as whimsical fancy, the optimization process will be a guardian against the backtest being the product of pure chance.

Backtesting the Bumblebee (Part One)

August 16, 2009

After one has formulated a trading idea, coded it and tested that the code does what it’s supposed to do, it’s time to put the notion to a test based on historical data. Don’t be scared and quit biting your nails. This is good for you. Backtesting is the arbiter of the trading idea. You can’t make the swim team unless you can first swim across the length of the pool. You cannot trade a system that has not shown it would have actually made money in the past. I’m assuming you are a trader whose objective it is to make money in the markets, and if you’re a different sort of trader, you may not want to keep reading. By the way, I pass no judgment on you if you choose to trade for reasons other than to make money. For the profit-minded sort, it will not require too much persuasion to convince you that we only want to trade systems that have shown potential for making money. Backtesting will show that a system either has the potential to be a traded with real money or is nothing more than whimsical fantasy. If you suspect your trading idea is whimsical fantasy, then you may choose to keep it somewhere safe, because this is going to get a little rough.

I’m taking 20 years of data to backtest Bumblebee. You will recall that the system is a trend-following system that uses a Bollinger Band around the slow moving average to create four lanes for our faster moving average to occupy. We enter long when the fast is in the upper lane, and close the trade after it moves into the third lane. We enter short when the fast MA is in the bottom lane, and cover when it breaches the second lane from the top. Entries and exits are on the open of the next bar.

My market for the initial test will be crude oil (continuous contract). I’m only backtesting the first 10 years of data and leaving the second block of 10 years in a super-secret location that the system doesn’t know about. I told you it was going to get rough. Below are the results of how Bumblebee performed from Jan 1, 1987 to Dec 31, 1996.

You will notice that the annualized profit of this system is 45.24% over this 10-year period. Not bad. But hold on skippy. We should be asking ourselves if all the profit came from one year while the other nine years were pain-suffering malaise. We don’t know from our first run of the data. So it’s time to get granular. We are next going to divide up that 10-year block of data into five 2-year chunks. That way, we can check if the system is consistent or if it gets lucky once a decade. Before we do that, let’s notice some other characteristics of the system.

Winning percentage is less than 50%. Not atypical for trend-following systems, but if you’re psychologically disposed to having the bulk of your trades being winners, you may decide it’s safer to eat glass than to trade this system.

Profit factor is 2.56. That means at the end of the day, our gross winnings outnumber our gross losses by that factor. If we were to break even for the period, we’d have a profit factor of 1.0. Any number below 1.0 is a net loser. Obviously, we’d like our profit factor (PF) to be large. It would mean money is flowing into our account, and we are charged little to be at the table. Our profit factor is not bad, but let’s keep an eye on it just the same.

Net profit as a function of maximum drawdown. I’ll admit, it’s a mouthful. Basically you’re trying to see how much pain you must endure to realize your profits. With the handy little calculator provided free-of-charge on your PC, you can see that Net Profit ($43,590) divided by Max DD ($9,640) is 4.52. Essentially, we put one dollar at risk to make $4.52. This number goes negative whenever we are overall losers. An interesting metric we shall also keep an eye on.

Long returns versus short returns. It’s not in that crummy little graphic above, but let me just tell you what it is. Longs had a 59% annualized return while short trades had a annualized return of 36%. Not totally skewed either way. That’s a good sign that the system is not overly disposed to either the long side or short side.

In Part Two we will see the results of crude oil divided up into five equal 2-year periods. And then to test the robustness even more, we will do the same thing on eight other markets so we end up with 45 reports. Five reports per market, nine markets total. This is why I purchased Excel.

Trading shibboleths are acid on the brain

August 6, 2009

It’s virtually impossible to have a meaningful discussion about trading with someone whose contribution is proudly limited to spewing trading shibboleths. Their account never suffers a loser because of their faithfulness to petty phrases such as ‘the trend is your friend’ or ‘trade what you see, not what you feel’ or ‘keep your losers small and let your winners run.’ Shibboleths are popular phrases that have become devoid of meaning. Those who repeat them are revealing themselves for who they are.

What does ‘the trend is your friend’ actually mean anyway? (donning my Andy Rooney hat) Jesse Livermore planted that phrase into the minds of traders as a means of gaining an edge over them. Get them babbling about cute phrases that rhyme and take their money in the interim. They’ll be content to watch their accounts dwindle as long as they have some shibboleth to grab hold of to explain the recent depreciation. Whenever they have a winner, program them to say ‘I let my winner run.’ And whenever they have a loser, program them to say ‘I should have kept my losers small.’ That way, instead of them thinking about what they’re actually doing, they’ll be thinking about applying the most appropriate shibboleth as the situation demands. Confusion in your adversary is key.

Do you ever notice how dumb you feel when you repeat some retarded trading shibboleth? That is by sinister design. It’s psychological warfare out there and the dirty little secret is that other traders constantly try to gain an edge over you. When you feel dumb, you’re likely to act dumb.

Next time you have the urge to spout meaningless dribble, resist. Don’t even try inverting a popular phase, such as cleverly stating ‘the trend is not always your friend’. You’re wasting your time and brain cells. Think about what you should be doing instead of what phrase you should be repeating.

Now go grab the bull by the horns and get trading. Or something like that.