Log in

Reverse optimization and Walk Forward Testing

image

In this short blog post I want to revisit some concepts and to show you some new paths of investigation, I hope so. It is known to us at the expense  of a lot of suffering and/or time (for many), that there is no holy grail indicator, system or EA (Expert Advisor), that will make money consistently with the same set of parameters. That is not due to the fact that all EA sellors are scammers, but to the fact that the market is very complex and reacts to itself. For that reason many systems have been developped helping the professional traders to gain some kind of meta-knowledge for the use of the trading tools. 

Let first examine the The Walk Forward Optimization

1. The Walk Forward Optimization as defind by Wikipedia is:

"is a method used in finance for determining the best parameters to use in a trading strategy. The trading strategy is optimized with in sample data for a time window in a data series. The remainder of the data are reserved for out of sample testing. A small portion of the reserved data following the in sample data is tested with the results recorded. The in sample time window is shifted forward by the period covered by the out of sample test, and the process repeated. At the end, all of the recorded results are used to assess the trading strategy."[1]

And Walk Forward Optimization is a different thing from the backtesting:

"Backtesting is using past data to test a trading system. It's useful because if a system was not profitable in the past it will not be profitable in the future. It refers to applying a trading system to historical data to verify how a system would have performed during the specified time period."[2]


And Forward testing or paper trading is also known as Walk forward testing is "the simulation of the real markets data on paper only. It means that though you are moving along the markets live, but you are not actually putting in real money, but doing virtual trading in lie markets to understand the movements of markets better. Hence, it is also called as the Paper Trading. Forward performance testing is a simulation of actual trading and involves following the system's logic in a live market."[2]

Hower nothing is more usefull than a visual representation. On the site of Amibroker you can find a dynamic representation of the concept.

Those concepts are really some of the most important thing we can learn for trading. And that is because all professional traders use this stuff. So and here we can ask a question.

2. We can rise some question especially regarding this routine face to face of what we know of the true nature of the market

However can we do better?

Yes that is a reasonable question.

What about the reverse. You are going now to ask me, why I am going to reverse it.

Or you may not discuss at and push me over the cliff and not care about this stuff any more.

But if you want to pay a little more attention here is my argument.

From what we know the market is chaotic, there are some measures of this, for example the Lyapunov exponent.

Here I provide several easy to follow and understand links.

"The Lyapunov exponent (L) determines the rate of predictability. A positive Lyapunov exponent indicates chaos and it sets the time scale which makes the state of prediction possible. The bigger the largest positive Lyapunov exponent is, the more rapid the loss of predictive “power”, and the less the prediction time for the time series is. The predictability is calculated as a value  = 1.0 / L."

See this excellent article for more details.

Basically The calculation of Lyapunov exponent or predictability enables us to evaluate the reliability of prediction.

You can see this article too for more easy to understand details on the Lyapunov exponent.

The Lamda (Lyapunov exponent can be lower than zero):The orbit attracts to a stable fixed point or stable periodic orbit. Negative Lyapunov exponents are characteristic of dissipative or non-conservative systems (the damped harmonic oscillator for instance). Such systems exhibit asymptotic stability; the more negative the exponent, the greater the stability.

As for the market this would mean that the market conditions are really excellent for prediction.


The Lamda (Lyapunov exponent can be equal to zero):

A Lyapunov exponent of zero indicates that the system is in some sort of steady state mode. Still cool for market analysis.


The Lamda (Lyapunov exponent can be bigger than zero):

The orbit is unstable and chaotic. Nearby points, no matter how close, will diverge to any arbitrary separation. All neighborhoods in the phase space will eventually be visited. These points are said to be unstable.

The bigger the lamda more chaotic becomes the actual prediction.


3. How that stuff is related with the optimization routine?

Yes that is the question here. How that stuff is related with the optimization routine?

In fact when you use Forward testing or paper trading then use actually optimize on much previous datam, and during the paper trading window actually you use older model that performs very well on the paper trading data. OK, but if the market is chaotic theoretically that means that every model will degrade with every new market information that is coming. (We have a different model based on clusters of volatility and fractal dimension and that some models perform very well on specific clusters).

So when you actually use the model it is based on old data.

Basically the idea is to use the recent data and see if the model that works well on current data performs well on past data.

That is what I call Walk Reverse Testing.

1. In this case our system is optimised on the very recent data.

2. It performs on immediately past data (reverse testing, it answers the question if the system is working now)

3. And we can see even more we can apply this model on all the available data (the ultimate test, it answers the question if the system is sound)

If a system pass well the tests of 1 and 2 and 3, I could rely on it more heavily.


4. Here I would add some more ideas based on the recent developments by Jaguar.

Here on this site the are two shared easy to use instruments for the estimation of the bigger lyapunov exponent.

-The lagspace software

-The new user friendly program

What we did until now is what is described in the books, to detrend the prices.

I use  =ln (F3/F2), the F column (excel) is the close price. That makes the detrending.

However I think that something is wrong here. Still I want to detrend, but I think following this we may distort the data.

So what about the approach of Jaguar to look for the best correlated indicator of Jaguar, or at the normalized oscillator we actually use and make an estimation of the Lyapunov exponent on this.

The issue is open.


References:

1. Expert Advisors Based on Popular Trading Systems and Alchemy of Trading Robot Optimization

2. Wikipedia Walk forward optimization

3. Kirkpatrick, Charles D.; Dahlquist, Julie R. (2010-11-15). Technical Analysis: The Complete Resource for Financial Market Technicians. FT Press. p. 548. ISBN 978-0-13-705944-7. Retrieved 13 June 2011.

4.  a b Investopedia: Backtesting And Forward Testing

5.  ^ Can your system do the walk

6. How Not to Fall into Optimization Traps?

7. Robert Pardo. Design, Testing and Optimization of Trading Systems ISBN-10: 0471554464



Comments