Archive for the ‘Bumblebee’ Category

Goodbye Bumblebee, Hello Ladybug

September 25, 2009

I had grand schemes and lofty goals for the Bumblebee system. But barring an egregious mistake in my walk-forward of the system, I have to let the critter go. The White Bumblebee (first version) suffered massively in its synthetic real-world experience, like a patient etherised on the table only to suffer reality’s cruel but necessary knife. I had other versions queued up and ready to improve upon the simplest version, but they did no better in their walk-forwards. Not to worry, there are many more insects in the hopper. Next one sprawling on a pin is the Ladybug. It is based on Keltner band breakouts. Keltner bands are like Bollinger bands, but they’re based on ATR instead of standard deviation. The basic idea is that once the band is breached, take a trade in the direction of the excursion and enjoy great inflows of capital into your account. (more…)

Advertisements

Walking forward the Bumblebee – We have issues

September 23, 2009

I was hoping it would be a non-event to blog about how successful the Bumblebee trading system performed under the rigors of walking forward. But alas, the little critter has been tripped up a bit. The initial results I’ve gotten are abysmal. As in I wouldn’t even commit Monopoly money to the enterprise. We started this journey with nine markets. Two of these markets (SP 500 and Coffee) showed early signs that they wanted nothing to do with the system. That’s okay. We’ll deal with them later. But of the seven remaining markets, three showed great promise in the optimization cycle of system development. Clearly, I’m disappointed.

Here is a spreadsheet of results for how White Bumblebee (the system’s first version) performed on fresh data. The basic idea with a walk-forward is to optimize on a set of data and use those optimal parameters in a walk-forward window. The window is forward data, or data that was not subjected to the fondling of the optimization process. It’s fresh, and unbiased. Historical, yes, but the system doesn’t know what year it is so we get away with a little slight-of-hand, to be sure.

The column that is labeled ‘# Periods’ is the number of windows that were traded. The relationship between optimization data (fishing for parameters) and the walk forward window (okay, let’s see what you got) is 200 to 50. Two-hundred days are used for optimization, and the system is traded on 50 days before the process begins anew. It doesn’t take too much mental yoga to see that your optimization data is moved forward after the window results have been recorded. That’s the walking part of walk-forward.

I’ve been giving some limited thought to getting a more clever metaphor for what we’re doing here, but so far walk-forward seems to capture the essence of the process best.

Okay, back to our issues with Bumblebee. I’ve made it easy to spot profitable versus non-profitable cells by painting them green or red. You can see that there is a dearth of green cells. The sigma levels show what happens if the system experiences standard deviations on the downside (where we lose money). Ideally, we’d like to get a little sigma cushion, but none of the walk forwards gave us anything.

So where does this leave us? Well, we can throw the system into the trash can. That would be clean and simple. But sometimes, you just need to play with the pain to win. So, we’re not going to give up just yet.

I’m not sure where things went so terribly wrong. I probably need to check my degrees of freedom and see if I’m doing something stupid there. Or maybe my window is too short and the system is getting re-optimized too much. Or maybe my fitness function is no good.

Uh-oh, I’ve introduced a new concept right when I was wrapping up.

What is a fitness function? Well, it is essentially the arbiter of what are the best parameters. Why give one arbiter dibs on making the decision of what parameters we’ll be using going forward? Essentially, someone’s got to do it. Who do we pick as our front man? We could choose the function that finds the best net profit on a historical basis. But Net Profit is sort of … well, dull and monolithic, unlike the markets we’re trying to squeeze money from. The concept of what is the best fitness function, or by what measure can we determine the parameters that have the best chance of producing profits in the future, is a whole separate topic, worth its own blog at some later time.

I used the pessimistic return on margin (PROM) as my fitness function in the walk forward of Bumblebee. It adds to the losers by its standard error and takes away from the winners by its standard error. I don’t think it’s the problem, quite frankly. But I’ll triple check my programming just to be sure it’s not eating my Bumblebee for no good reason.

Optimizing the Bumblebee

September 4, 2009

The purpose of optimization is not to find the optimal parameters. Yes, that’s not a typo. The main purpose of optimization is to validate backtest results. You’ll remember it was the backtest’s job to validate a trading idea. Well, now backtesting is on the hot seat and optimization is the judge and jury. Optimization will validate that backtest results are not simply the product of chance, and that the system stands a chance of making money in the future. It will do this by creating a profile sketch of the system. I know, sketches are not the first thing you’d associate with spreadsheets. But in our world, it’s what we have to work with.

We are going to go over the basic setup of the optimization run in just a moment, but let’s take a simple statistic from the optimization run and see if it helps illuminate our objective. The statistic will be ‘profitable parameter sets.’ These are permutations that made money on historical data. If we randomly picked a parameter set, what would be our chances that it would have made money? If 50% of our optimized parameter sets are profitable, that means you can basically throw a dart and your chances of making money is, well, 50/50. Of course we not going to throw darts, but you get the idea.

If only 10% of optimized parameter sets are profitable, then you’d better have the aim of a trained assassin. If 80% are profitable, call your half-blind neighbor over and have him fire away. If you absolutely have to be dead-nuts on in your choice of parameter sets, you may consider whether the enterprise is worth your efforts.

The space we’d like to paint (don’t worry, I’ve got some cool looking graphs below) should be basically square in two dimensions. A scan that is too fine will lead to over-fitting. That’s bad. We cannot avoid curve fitting as that’s basically what we’re doing, but we can avoid getting carried away.

Besides providing us with information about how lucky you need to be to make money with your system, optimization will provide an insight into other features of your system, such as what the winning percent is, what sort of drawdowns you should prepare for and how frequent are trades for winning parameters versus the frequency of trades for losers.

Bumblebee was initially backtested with a slow moving average of 30, a fast moving average of 10 and a Bolllinger band around the slow MA of 0.5 standard deviations. We are going to fiddle with the slow MA period and the fast MA period and leave the Bolllinger band width alone. The reason is simple. Moving average length will have the greater impact on the system than tweaking the standard deviation length, and we don’t have time to calculate too many permutations with the method of choice. That method is called brute force. Yes, you need to be brutish and forceful to do this, but it won’t be too terribly distressing, I promise.

Permutations are statistical combinations. If we ran a test with a slow MA of 30 and fast MA of 10, we’d like to ask what would happen if we used a slow MA of 30 and a fast MA of 11. How about 12, 13. 14 and so on. You get the idea. Let me derive a simple method for calculating this monster in the following statistical proof. Or, on second thought, why not you look it up in your statistic book and take my word for it. If we run a range of 20 periods for the fast and 20 periods for the slow, we will have run 400 permutations. Mathematically savvy readers will see that 20 x 20 is 400, and that is the secret shortcut you forgot you once knew.

In keeping with the square space ideal, we are running the fast MA from 3 to 24 in steps of one (that means, 3, 4, 5, etc.) and the slow MA in steps of 2 from 11 to 49 (11, 13, 15, etc.) This yields 410 permutations. After we take out pairs that don’t make sense (where the slow is faster than the fast), we end up with 371 parameter sets. And with no further ado, below are the results of this optimization sketch. First let’s observe the percent of parameter sets that are profitable. If it’s under 30%, I highlighted it in red to indicate potential problems. If it’s over 80%, I highlighted it in green, to indicate the blind squirrel principle applies (ie, a random dart thrower can make money using the system).

I also put a color code to the left of the market to indicate whether or not the system is useful in a particular market. The obvious losers with the Bumblebee include the S&P 500 and the Coffee markets. The markets that show the most promise include the Crude Oil, Gold and Ten-Year markets. The other markets show potential, but also show some warnings signs, so they’ve been designated as cautionary.

Let’s look at the optimization space for a obvious wanker and a potential best-friend. First the wanker. Below is the optimization space painted for the S&P 500 from 1991-92. Only six percent of potential parameter sets showed a net profit. Yes, that ugly.

You can see that overall, you needed to pick just the right parameter set to make money with the system during this two-year period.

Let’s now look at what a more lovely optimization looks like. Below is the optimization space for the Cotton during 1987-88.

You can clearly see that if you avoided a certain area of tiny slow and fast MAs, you’d at least make money. And if you shot for the area in the upper right corner, you’d make some nice returns.

Here is a sample of some other data gleaned from the optimization profile.

This is the spreadsheets results for the Ten-Year between 1995-96. You can see that 92% of parameter sets were profitable. And, 347 parameter sets yielded trades during the period out of a potential 371 sets. That’s impressive. But looking deeper, you can see that the winning parameter sets had a winning percentage of just 51%. The losing parameter sets had a winning percentage of around 35%.

Also, the winning parameter sets (the good guys on our team) traded an average of 10 trades during the two-year period while the losing parameter sets (the bad guys on the other team), traded much more heavily with an average of 35 trades during the two-year period. These are just a few of the revealing characteristics of the winning parameter sets versus the losing parameter sets that can alert you to when things are going wrong. In other words, when you have given the ball to the other team.

Backtesting the Bumblebee (Part Two)

August 16, 2009

I only broke this up into two separate blogs because if you endured the first part, you may be interested in the second part. But you also may have had enough. I know polite readers will finish what they’ve started, so I wanted to give them a logical stopping point. In this part we will continue our exploration of how Bumblebee performs on historical data. So far, we got decent results from a 10-year test window in one market. Now, we continue down the rabbit hole.

Bumblebee’s performance in crude oil over separate 2-year windows is depicted in the following spreadsheet:


I’ve highlighted in blue the pleasant surprises, and used orange to point out the not-so-good results.

Our next step is to see how Bumblebee works on different markets. There is no expectation that it will do as well or as poorly in other markets. We’d like to see it do well in at least some other markets though. The eight other markets include corn, cotton, coffee, dollar, yen, gold, S&P 500 and Ten-year note. Once again, I’ve used blue to highlight pleasant surprises and orange to highlight dubious results.


So far, we can see that coffee and S&P 500 futures didn’t do so well. They lost money. But the good news is that about 78% of the markets actually made money, though cotton and gold pulled in a meager performance of around 3% each.

Nine markets over ten years yielded 641 trades, which is statistically valid. Taking the good with the bad, the consolidated performance gave us a 23.7% annualized return. Hmmm, I do believe this system bears further investigation. Let’s get granular again and divide up these overview reports into 2-year chunks to see if they yield any more information about Bumblebee.


Here is the S&P 500. Clearly not impressive. From 1989 to 1992, this system was dismal, to be kind. The only impressive result was long trades between 1995 and 1996.


Coffee is another market that eats this system alive. Six out of 10 years were losers, some with painful drawdowns.


Before we get completely depressed, let’s admire how the system performed with 10-year notes. The periods of 1991 to 1992 and from 1995 to 1996 yielded better than 100% returns, with long and short trades taking turns in the bounty.


The Dollar also shows some nice returns with the system, including some nice 300% returns from the short side in three separate test windows.


Gold is kinda lackluster, but at least the losing windows didn’t see too large a drawdown. Nice short trade in 1995 to 1996.


As for the Corn trade, not that impressive. The big long winner in 1987 to 1988 helps keep this one above water.


The Cotton market responds very similarly to the Corn market, with lackluster results, the decent returns from 1989 to 1992 notwithstanding.


I’ve saved the best for last, but what would you expect. The system performed well in the Yen market from 1987 to 1988. Overall annualized return was 132%, which was helped mightily by a 959% return on long trades during the period. This sort of result is almost too good to be true, so we have to give it guarded respect. It’s cool to look at though, and would be even cooler if it happened again in the future.

That is a lot of data to digest. It is just a start though. Next we optimize. As the backtest was a guardian against the trade idea as whimsical fancy, the optimization process will be a guardian against the backtest being the product of pure chance.

Backtesting the Bumblebee (Part One)

August 16, 2009

After one has formulated a trading idea, coded it and tested that the code does what it’s supposed to do, it’s time to put the notion to a test based on historical data. Don’t be scared and quit biting your nails. This is good for you. Backtesting is the arbiter of the trading idea. You can’t make the swim team unless you can first swim across the length of the pool. You cannot trade a system that has not shown it would have actually made money in the past. I’m assuming you are a trader whose objective it is to make money in the markets, and if you’re a different sort of trader, you may not want to keep reading. By the way, I pass no judgment on you if you choose to trade for reasons other than to make money. For the profit-minded sort, it will not require too much persuasion to convince you that we only want to trade systems that have shown potential for making money. Backtesting will show that a system either has the potential to be a traded with real money or is nothing more than whimsical fantasy. If you suspect your trading idea is whimsical fantasy, then you may choose to keep it somewhere safe, because this is going to get a little rough.

I’m taking 20 years of data to backtest Bumblebee. You will recall that the system is a trend-following system that uses a Bollinger Band around the slow moving average to create four lanes for our faster moving average to occupy. We enter long when the fast is in the upper lane, and close the trade after it moves into the third lane. We enter short when the fast MA is in the bottom lane, and cover when it breaches the second lane from the top. Entries and exits are on the open of the next bar.

My market for the initial test will be crude oil (continuous contract). I’m only backtesting the first 10 years of data and leaving the second block of 10 years in a super-secret location that the system doesn’t know about. I told you it was going to get rough. Below are the results of how Bumblebee performed from Jan 1, 1987 to Dec 31, 1996.

You will notice that the annualized profit of this system is 45.24% over this 10-year period. Not bad. But hold on skippy. We should be asking ourselves if all the profit came from one year while the other nine years were pain-suffering malaise. We don’t know from our first run of the data. So it’s time to get granular. We are next going to divide up that 10-year block of data into five 2-year chunks. That way, we can check if the system is consistent or if it gets lucky once a decade. Before we do that, let’s notice some other characteristics of the system.

Winning percentage is less than 50%. Not atypical for trend-following systems, but if you’re psychologically disposed to having the bulk of your trades being winners, you may decide it’s safer to eat glass than to trade this system.

Profit factor is 2.56. That means at the end of the day, our gross winnings outnumber our gross losses by that factor. If we were to break even for the period, we’d have a profit factor of 1.0. Any number below 1.0 is a net loser. Obviously, we’d like our profit factor (PF) to be large. It would mean money is flowing into our account, and we are charged little to be at the table. Our profit factor is not bad, but let’s keep an eye on it just the same.

Net profit as a function of maximum drawdown. I’ll admit, it’s a mouthful. Basically you’re trying to see how much pain you must endure to realize your profits. With the handy little calculator provided free-of-charge on your PC, you can see that Net Profit ($43,590) divided by Max DD ($9,640) is 4.52. Essentially, we put one dollar at risk to make $4.52. This number goes negative whenever we are overall losers. An interesting metric we shall also keep an eye on.

Long returns versus short returns. It’s not in that crummy little graphic above, but let me just tell you what it is. Longs had a 59% annualized return while short trades had a annualized return of 36%. Not totally skewed either way. That’s a good sign that the system is not overly disposed to either the long side or short side.

In Part Two we will see the results of crude oil divided up into five equal 2-year periods. And then to test the robustness even more, we will do the same thing on eight other markets so we end up with 45 reports. Five reports per market, nine markets total. This is why I purchased Excel.

Testing the code for the Bumblebee

June 23, 2009

Now that we’ve expended brain cells in the coding of a trade idea, we’d like to make sure the thing works as we think it should. Before we spend any more brain cells determining how well our system will perform going forward, we’d better complete this important step.

With TradersStudio, a list of trades is generated with a backtest run. These trades can be depicted on a chart as neat little arrows at the point where a trade was transacted. The Bumblebee system requires the programming of a proprietary indicator, which I’ve done behind the scenes. Now we overlay the chart (with the neat little arrows) with the proprietary indicator to see if the arrows are where they’re supposed to be.


The first time I ran this process through I was horrified to find a mistake in my programming, which generated all manner of errant trades. After removing the poltergeist, I was able to get a clean programmed code.

I ran the code test on data from 1973 to 1974 in the Cotton market. My future backtesting (I know, bad choice of words) will not touch this data. I will begin in 1988 and include a total of 10 markets. But for now, I’ve chosen some data that I don’t mind contaminating.

If you’re suspicious about my worries concerning contamination, suspend your doubts for now. We are participating in best practices; that’s the point of using way out of range data. Though it may not matter now, later on it will be important to keep backtest data separate from walk-forward data. This best practice is designed to help avoid curve fitting.

Alright, we’re now done with the first third of our system development. Next we run the second third, which includes Backtest, Optimize, Walk Forward, or for acronym fans, BOW. Each phase is progressively more interesting, which keeps us doing what we’re doing. During this second phase, we will generate all manner of data that we need to process and understand correctly.

Coding the Bumblebee

June 4, 2009

Ah, the excitement and sheer joy that is before us as we learn to code a trade system. I am using TradersStudio as my EOD trading software. Though it has its own programming language, its language is very similar to Visual Basic. So it helps to get a grasp of the VB language and since I’m not a programmer (just one in the making), I’ve elicited the help of an online video tutorial service to assist me in my endeavors to learn programming. Programming can be fun, just like re-grouting your shower. So let’s get to it!

Before we get into the very exciting world of coding a trade system, let’s review why it’s a good idea, and why paper and pencil methods are inferior.

1. Properly coded programs don’t miss trades or make the wrong trades.

2. Coded programs enable the system to test over a vast range of dates and markets that would take the better part of a century if it were attempted by hand.

3. Testing done with a coded program accumulates all manner of data that adds to different ways of viewing your results.

4. Coded programs enable a trade system developer to draw valid statistical inferences because of the vast data that is accumulated.

5. Minor changes to a trade system can be easily programmed and tests can be re-run with minimal effort and time.

Okay, that enough for now. For the Luddites who insist on paper and pencil methods, let me grant this benefit to their antiquated methods.

Manual backtesting ‘programs’ the trader to take trades when signals are generated and take them off when their system dictates.

Trust me though, after you’ve invested enough time in programming, backtesting, optimizing and walking forward, the last thing you’ll want to do is violate your trading system in real-time, thus nullifying all the time and effort you put into developing the system.

I offer below the program code for Bumblebee:

‘********************************
‘ Three parameter system, not always in

Sub Hysterosis (SlowLength As Integer, FastLength As Integer, StdMult As Double)

‘Declare variables

Dim SlowAve As BarArray
Dim FastAve As BarArray

Dim SlowPlus As BarArray
Dim SlowMinus As BarArray

‘Direct where the variable gets its data

SlowAve = Average(Close, SlowLength,0)
FastAve = Average(Close, FastLength,0)

SlowPlus = UpperBolBand(Close, SlowLength, StdMult, 0)
SlowMinus = LowerBolBand(Close, SlowLength, StdMult, 0)

‘Enter position criteria

If FastAve > SlowPlus And FastAve[1] < SlowPlus[1] Then
Buy(“Buy”,1,0, Market,Day)
End If

If FastAve SlowMinus[1] Then
Sell(“Sell”,1,0,Market,Day)
End If

‘Exit position criteria

If FastAve SlowAve[1]Then
ExitLong(“XLong”, “Buy”, 1, 0, Market, Day)
End If

If FastAve > SlowAve And FastAve[1] < SlowAve[1] Then
ExitShort(“XShort”, “Sell”, 1, 0, Market, Day)
End If

End Sub

Not that complicated, really. But then again, this is a fairly simple system. What we told our good silicon friends to do is the following:

We would like to enter a long trade when the fast moving average closes above the upper Bollinger band of the slow moving average. To exit this trade, we need the fast moving average to close below the slow moving average. And do the same on the short side, please.

This chatty explanation is called pseudo-code. It’s an important step, but then again it’s pretty obvious.

Observing the markets: Bumblebee is born

May 19, 2009

I am developing a trading system from the ground up. It’s called Bumblebee. We’re gonna start with the first phase, which includes three parts. Observe market behavior, Code a plan that exploits market behavior, Test the code’s integrity. For acronym fans, this is the OCT phase. Below we will deal with Observation.

The beginning is important because it involves the logic behind the system. Repeatable patterns can be found in markets, but to trade these patterns, I hold the view that there needs to be an identifiable reason for why the pattern develops. It centers around market behavior and the fact that market participants are human and usually act as humans do. This is in contradiction to Efficient Market theory that suggests that market participants always make rational decisions based on their own self-interest and that information is instantaneously absorbed and priced in. When we observe market behavior, we realize how irrational markets really are. But irrational behavior does not mean unpredictable. An alcoholic acts irrationally when he gets lit up and decides to drive to the liquor store to get more booze. But its a pretty good bet (and predictable) that if an alcoholic has a drink, number two through six is not far behind.

Oftentimes, market behavior is exhibited in technical indicators and price chart patterns. Certain candlestick patterns are visual representations of markets revealing themselves. Moving averages represent a group consensus of value. These are all valid hooks to hang your trading hat on. But you can overdo it and end up in a Poisson Distribution before you know it.

For example, consider the following trade system. When the 14-period RSI crosses above 50 and the Ulcer Index reaches 23, and the (insert fibonacci number here) period moving average is (insert another fibonacci number here) percent of price (insert a third fibonacci number here) bars ago, then enter long. That’s not explaining or exploiting market behavior. That looking for patterns. And you can always find patterns in nature that are completely unrelated.

The bottom line is that there needs to be a logical explanation for why a pattern has developed and how it reveals market behavior.

The Bumblebee System is going to trade off the observations that trends, once they develop, continue in their direction until their momentum has become exhausted. Market participants follow market leaders and as the market leaders gather more followers, the trend continues until there is nobody left to follow. The market then sits around twiddling its thumbs until another leader either takes them higher or lower. The trend is your friend until the end, or something like that.

We are going to use two moving averages to document market consensus of fair value. A fast one will demonstrate a short term notion of value and a slow average will reveal the longer term notion of value. The fast one leads the way, the slow one follows. It’s your plain vanilla dual moving crossover system. Long when fast is above slow, short when fast is below slow. But to make it more interesting, we’re going to add a Bollinger band around the slow average. Why? To avoid whipsaws and to give market participants enough time to make up their minds. We ask the market, ‘Which way are you going?’. And then (by inserting Bollinger bands) we ask it “Are you sure?”. If it answers up and yes I’m sure, we go long.

A Bollinger band consists of two lines wrapped around a moving average. The lines are equidistant on the top and bottom, and their distance is defined by a factor of standard deviations. For example, we can select a 2 standard deviation width to the band and capture much of price action. Not 95% of price action as the standard deviation number suggests, but a lot. We are selecting a much smaller factor, something less than 1. We’re trying to keep our question simple in saying “Are you sure?” and we’re not trying to be annoying and ask “Are you absolutely, positively sure?”.

If you draw a moving average and wrap two lines around it, you essentially have four lanes. Two lanes inside the bands on either side of the moving average, and two lanes outside the bands, one over the top line and one below the bottom line. Usually a dual moving average system triggers when the fast crosses the slow. We’re going to require it to not only cross the slow, but its upper or lower band. Once the fast average crosses the slow average and its upper band, it now resides in the top lane and the signal is triggered to enter long. We have a higher threshold for entry. To exit the long, we wait for the fast average to dip below the slow average and enter into the third lane. Short trades will be taken with the same logic in the opposite direction.