Optimization results differs from backtesting

Created at 09 Mar 2020, 11:54
Subscribe
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
LI

light96

Follow

Joined 09.03.2020

Optimization results differs from backtesting
09 Mar 2020, 11:54

Hi everyone,

During the last days I have run several optimization processes, and at the end of each, I test every bundle of parameters through the Backtesting function: no way to get predicted results.
It seems that the predictions provided by the Optimization tool are completely wrong if compared with the results provided by the Backtest, even after a rigorous check about the period analyzed and the data source.
I even done some research, and it seems to be a problem which persists from 2015.
So my question is: It is  possible to overcome this kind of trouble, or should we even program an own simulator to run optimization?

@light96
Replies

PanagiotisCharalampous
09 Mar 2020, 11:56

Hi light96,

Can you please post the cBot code and steps to reproduce the problem?

Best Regards,

Panagiotis 

Join us on Telegram

 

@PanagiotisCharalampous

light96
09 Mar 2020, 12:29

Yes, of course.

First attempt:

 

Second one:

 

Code:

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class FrancoBot : Robot
    {
        [Parameter("Volume", Group = "Posizione", DefaultValue = 30000)]
        public int Volume { get; set; }

        [Parameter("Take Profit", Group = "Posizione", DefaultValue = 60)]
        public double takeProfit { get; set; }

        [Parameter("Stop & Loss", Group = "Posizione", DefaultValue = 35)]
        public double stopAndLoss { get; set; }

        [Parameter("Pending Order Threshold", Group = "Posizione", DefaultValue = 10)]
        public double pendingOrder_threshold { get; set; }

        [Parameter("Source", Group = "SMA & Mercato")]
        public DataSeries Source { get; set; }

        [Parameter("Periodo SMA 1", Group = "SMA & Mercato", DefaultValue = 1440)]
        public int Periods { get; set; }

        [Parameter("Periodo SMA 2", Group = "SMA & Mercato", DefaultValue = 300)]
        public int Periods2 { get; set; }

        [Parameter("Periodo SMA Small 1", Group = "SMA & Mercato", DefaultValue = 2)]
        public int PeriodsSmall { get; set; }

        [Parameter("Periodo SMA Small 2", Group = "SMA & Mercato", DefaultValue = 30)]
        public int PeriodsSmall2 { get; set; }

        [Parameter("Utente", Group = "Others", DefaultValue = "Gian")]
        public string utente { get; set; }

        [Parameter("Send Email", Group = "Others", DefaultValue = true)]
        public bool emailEnabled { get; set; }

        public string symbol;

        public double divisoreTakeOrStop { get; set; }

        private string emailAddress = "tbotnotifications@gmail.com";
        private string oggettoMail;
        private string testoMail;


        //pending order stuff
        private double pendingOrder_startingPriceValue;
        private bool pendingOrder_isActive = false;
        private TradeType pendingOrder_positionDirection;

        private bool Bool;

        private double previousPriceAsk;
        private double previousPriceBid;



        //CALCOLO I VALORI INIZIALI DI SMA E SMALL SMA 
        //IN BASE AL SEGNO OTTENUTO DALLA DIFFERENZA TRA LE DUE INIZALIZZO LA MIA VARIABILE BOOLEANA
        protected override void OnStart()
        {
            symbol = Symbol.Name;
            pendingOrder_threshold = pendingOrder_threshold / 10000;

            double smaValue = calcolaSMAValue(Source, Periods, Periods2);
            double SMAsmall = calcolaSMAValue(Source, PeriodsSmall, PeriodsSmall2);

            Print("SMAsmall Value => " + SMAsmall);
            Print("SMA => " + smaValue);
            double initialValue = SMAsmall - smaValue;

            //inizializzo i valori previousPriceAsk e previousPriceBid
            previousPriceAsk = Symbol.Ask;
            previousPriceBid = Symbol.Bid;

            if (initialValue > 0)
            {
                Bool = true;
            }
            if (initialValue < 0)
            {
                Bool = false;
            }

            Print("Valore Iniziale Differenza SMA1- SMA2 => " + initialValue);
        }

        protected override void OnTick()
        {
            //ricalcolaTakeProfitStopAndLoss();

            //controllo se è stato innescato il pending order, se è stato innescato controllo se devo aprire la posizione
            if (pendingOrder_isActive)
            {
                //logica se devo aprire in buy
                if (pendingOrder_positionDirection == TradeType.Buy)
                {
                    if (Symbol.Ask >= pendingOrder_startingPriceValue + pendingOrder_threshold)
                    {
                        apriPosizione(pendingOrder_positionDirection);
                        disablePendingOrder();
                    }
                }

                //logica se devo aprire in sell
                if (pendingOrder_positionDirection == TradeType.Sell)
                {
                    if (Symbol.Bid <= pendingOrder_startingPriceValue - pendingOrder_threshold)
                    {
                        apriPosizione(pendingOrder_positionDirection);
                        disablePendingOrder();
                    }
                }

            }

        }

        protected override void OnBar()
        {
            double smaValue = calcolaSMAValue(Source, Periods, Periods2);
            double smaMmallValue = calcolaSMAValue(Source, PeriodsSmall, PeriodsSmall2);

            double diffSMA = smaMmallValue - smaValue;
            Print("Differenza SMASmall - SMA => " + diffSMA);

            if (Bool == false)
            {

                //se le medie si sono rincrociate (quindi la loro differenza ha cambiato di segno), chiudo la vecchia posizione e ne apro una con segno opposto
                if (diffSMA > 0)
                {
                    Bool = true;

                    Position posizione = Positions.Find("Sell" + symbol, symbol, TradeType.Sell);

                    if (posizione != null)
                        ClosePosition(posizione);

                    activatePendingOrder(TradeType.Buy);
                }
            }
            else
            {

                //se le medie si sono rincrociate (quindi la loro differenza ha cambiato di segno), chiudo la vecchia posizione e ne apro una con segno opposto
                if (diffSMA < 0)
                {
                    Bool = false;

                    Position posizione = Positions.Find("Buy" + symbol, symbol, TradeType.Buy);
                    if (posizione != null)
                        ClosePosition(posizione);

                    activatePendingOrder(TradeType.Sell);
                }
            }

        }

        private void activatePendingOrder(TradeType direction)
        {
            pendingOrder_isActive = true;
            pendingOrder_positionDirection = direction;
            pendingOrder_startingPriceValue = direction.ToString() == "Buy" ? Symbol.Ask : Symbol.Bid;
        }

        private void disablePendingOrder()
        {
            pendingOrder_isActive = false;
        }

        //metodo che apre una posizione, controlla se deve impostare TakeProfit e StopAndLoss ed eventualmente manda una Mail
        private void apriPosizione(TradeType direction)
        {
            TradeResult tradeResult;
            string directionString = direction.ToString();

            tradeResult = ExecuteMarketOrder(direction, symbol, Volume, directionString + symbol, stopAndLoss, takeProfit);

            //se il parametro "Send Email" è True mando una mail di notifica
            if (emailEnabled)
                sendEmail(tradeResult, directionString);

        }


        //controllo se la posizione è stata aperta correttamente e invio una mail di successo o errore
        private void sendEmail(TradeResult result, string buyOrSell)
        {
            if (result.IsSuccessful)
            {
                oggettoMail = utente + " Aperta - " + symbol;
                testoMail = "Posizione aperta in " + buyOrSell + " per il mercato " + symbol + " con volume " + Volume;
            }
            else
            {
                oggettoMail = utente + " ERRORE - " + symbol;
                testoMail = "Errore aprendo in " + buyOrSell + " per il mercato " + symbol + " con volume " + Volume;
                testoMail = testoMail + "\n Codice errore -> " + result.Error;
                testoMail = testoMail + "\n Descrizione -> " + result.ToString();
            }

            Notifications.SendEmail(emailAddress, emailAddress, oggettoMail, testoMail);

        }

        private double calcolaSMAValue(DataSeries source, int periods, int periods2)
        {
            SimpleMovingAverage sma = Indicators.SimpleMovingAverage(source, periods);
            sma.Calculate(periods);
            SimpleMovingAverage sma2 = Indicators.SimpleMovingAverage(sma.Result, periods2);
            sma2.Calculate(periods2);
            return sma2.Result.LastValue;
        }

        private Position trovaPosizione()
        {
            Position position = null;

            if (Bool == false)
                position = Positions.Find("Sell" + symbol, symbol, TradeType.Sell);

            if (Bool == true)
                position = Positions.Find("Buy" + symbol, symbol, TradeType.Buy);

            return position;
        }


        private void ricalcolaTakeProfitStopAndLoss()
        {
            double lastPrice = 0;
            double localPriceDiff;

            //la posizione dovrebbe essere in Sell             
            if (Bool == false)
            {
                lastPrice = Symbol.Ask;
                localPriceDiff = lastPrice - previousPriceAsk;
                previousPriceAsk = lastPrice;

                if (localPriceDiff < 0)
                {
                    takeProfit = takeProfit + (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss + (localPriceDiff / divisoreTakeOrStop);
                }

                if (localPriceDiff > 0)
                {
                    takeProfit = takeProfit - (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss - (localPriceDiff / divisoreTakeOrStop);
                }
            }

            //la posizione dovrebbe essere in Buy
            if (Bool == true)
            {
                lastPrice = Symbol.Bid;
                localPriceDiff = lastPrice - previousPriceBid;
                previousPriceBid = lastPrice;

                if (localPriceDiff > 0)
                {
                    takeProfit = takeProfit + (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss + (localPriceDiff / divisoreTakeOrStop);
                }

                if (localPriceDiff < 0)
                {
                    takeProfit = takeProfit - (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss - (localPriceDiff / divisoreTakeOrStop);
                }
            }

        }

    }

}

 

@light96

light96
11 Mar 2020, 15:13

RE:

light96 said:

Yes, of course.

First attempt:

 

Second one:

 

Code:

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class FrancoBot : Robot
    {
        [Parameter("Volume", Group = "Posizione", DefaultValue = 30000)]
        public int Volume { get; set; }

        [Parameter("Take Profit", Group = "Posizione", DefaultValue = 60)]
        public double takeProfit { get; set; }

        [Parameter("Stop & Loss", Group = "Posizione", DefaultValue = 35)]
        public double stopAndLoss { get; set; }

        [Parameter("Pending Order Threshold", Group = "Posizione", DefaultValue = 10)]
        public double pendingOrder_threshold { get; set; }

        [Parameter("Source", Group = "SMA & Mercato")]
        public DataSeries Source { get; set; }

        [Parameter("Periodo SMA 1", Group = "SMA & Mercato", DefaultValue = 1440)]
        public int Periods { get; set; }

        [Parameter("Periodo SMA 2", Group = "SMA & Mercato", DefaultValue = 300)]
        public int Periods2 { get; set; }

        [Parameter("Periodo SMA Small 1", Group = "SMA & Mercato", DefaultValue = 2)]
        public int PeriodsSmall { get; set; }

        [Parameter("Periodo SMA Small 2", Group = "SMA & Mercato", DefaultValue = 30)]
        public int PeriodsSmall2 { get; set; }

        [Parameter("Utente", Group = "Others", DefaultValue = "Gian")]
        public string utente { get; set; }

        [Parameter("Send Email", Group = "Others", DefaultValue = true)]
        public bool emailEnabled { get; set; }

        public string symbol;

        public double divisoreTakeOrStop { get; set; }

        private string emailAddress = "tbotnotifications@gmail.com";
        private string oggettoMail;
        private string testoMail;


        //pending order stuff
        private double pendingOrder_startingPriceValue;
        private bool pendingOrder_isActive = false;
        private TradeType pendingOrder_positionDirection;

        private bool Bool;

        private double previousPriceAsk;
        private double previousPriceBid;



        //CALCOLO I VALORI INIZIALI DI SMA E SMALL SMA 
        //IN BASE AL SEGNO OTTENUTO DALLA DIFFERENZA TRA LE DUE INIZALIZZO LA MIA VARIABILE BOOLEANA
        protected override void OnStart()
        {
            symbol = Symbol.Name;
            pendingOrder_threshold = pendingOrder_threshold / 10000;

            double smaValue = calcolaSMAValue(Source, Periods, Periods2);
            double SMAsmall = calcolaSMAValue(Source, PeriodsSmall, PeriodsSmall2);

            Print("SMAsmall Value => " + SMAsmall);
            Print("SMA => " + smaValue);
            double initialValue = SMAsmall - smaValue;

            //inizializzo i valori previousPriceAsk e previousPriceBid
            previousPriceAsk = Symbol.Ask;
            previousPriceBid = Symbol.Bid;

            if (initialValue > 0)
            {
                Bool = true;
            }
            if (initialValue < 0)
            {
                Bool = false;
            }

            Print("Valore Iniziale Differenza SMA1- SMA2 => " + initialValue);
        }

        protected override void OnTick()
        {
            //ricalcolaTakeProfitStopAndLoss();

            //controllo se è stato innescato il pending order, se è stato innescato controllo se devo aprire la posizione
            if (pendingOrder_isActive)
            {
                //logica se devo aprire in buy
                if (pendingOrder_positionDirection == TradeType.Buy)
                {
                    if (Symbol.Ask >= pendingOrder_startingPriceValue + pendingOrder_threshold)
                    {
                        apriPosizione(pendingOrder_positionDirection);
                        disablePendingOrder();
                    }
                }

                //logica se devo aprire in sell
                if (pendingOrder_positionDirection == TradeType.Sell)
                {
                    if (Symbol.Bid <= pendingOrder_startingPriceValue - pendingOrder_threshold)
                    {
                        apriPosizione(pendingOrder_positionDirection);
                        disablePendingOrder();
                    }
                }

            }

        }

        protected override void OnBar()
        {
            double smaValue = calcolaSMAValue(Source, Periods, Periods2);
            double smaMmallValue = calcolaSMAValue(Source, PeriodsSmall, PeriodsSmall2);

            double diffSMA = smaMmallValue - smaValue;
            Print("Differenza SMASmall - SMA => " + diffSMA);

            if (Bool == false)
            {

                //se le medie si sono rincrociate (quindi la loro differenza ha cambiato di segno), chiudo la vecchia posizione e ne apro una con segno opposto
                if (diffSMA > 0)
                {
                    Bool = true;

                    Position posizione = Positions.Find("Sell" + symbol, symbol, TradeType.Sell);

                    if (posizione != null)
                        ClosePosition(posizione);

                    activatePendingOrder(TradeType.Buy);
                }
            }
            else
            {

                //se le medie si sono rincrociate (quindi la loro differenza ha cambiato di segno), chiudo la vecchia posizione e ne apro una con segno opposto
                if (diffSMA < 0)
                {
                    Bool = false;

                    Position posizione = Positions.Find("Buy" + symbol, symbol, TradeType.Buy);
                    if (posizione != null)
                        ClosePosition(posizione);

                    activatePendingOrder(TradeType.Sell);
                }
            }

        }

        private void activatePendingOrder(TradeType direction)
        {
            pendingOrder_isActive = true;
            pendingOrder_positionDirection = direction;
            pendingOrder_startingPriceValue = direction.ToString() == "Buy" ? Symbol.Ask : Symbol.Bid;
        }

        private void disablePendingOrder()
        {
            pendingOrder_isActive = false;
        }

        //metodo che apre una posizione, controlla se deve impostare TakeProfit e StopAndLoss ed eventualmente manda una Mail
        private void apriPosizione(TradeType direction)
        {
            TradeResult tradeResult;
            string directionString = direction.ToString();

            tradeResult = ExecuteMarketOrder(direction, symbol, Volume, directionString + symbol, stopAndLoss, takeProfit);

            //se il parametro "Send Email" è True mando una mail di notifica
            if (emailEnabled)
                sendEmail(tradeResult, directionString);

        }


        //controllo se la posizione è stata aperta correttamente e invio una mail di successo o errore
        private void sendEmail(TradeResult result, string buyOrSell)
        {
            if (result.IsSuccessful)
            {
                oggettoMail = utente + " Aperta - " + symbol;
                testoMail = "Posizione aperta in " + buyOrSell + " per il mercato " + symbol + " con volume " + Volume;
            }
            else
            {
                oggettoMail = utente + " ERRORE - " + symbol;
                testoMail = "Errore aprendo in " + buyOrSell + " per il mercato " + symbol + " con volume " + Volume;
                testoMail = testoMail + "\n Codice errore -> " + result.Error;
                testoMail = testoMail + "\n Descrizione -> " + result.ToString();
            }

            Notifications.SendEmail(emailAddress, emailAddress, oggettoMail, testoMail);

        }

        private double calcolaSMAValue(DataSeries source, int periods, int periods2)
        {
            SimpleMovingAverage sma = Indicators.SimpleMovingAverage(source, periods);
            sma.Calculate(periods);
            SimpleMovingAverage sma2 = Indicators.SimpleMovingAverage(sma.Result, periods2);
            sma2.Calculate(periods2);
            return sma2.Result.LastValue;
        }

        private Position trovaPosizione()
        {
            Position position = null;

            if (Bool == false)
                position = Positions.Find("Sell" + symbol, symbol, TradeType.Sell);

            if (Bool == true)
                position = Positions.Find("Buy" + symbol, symbol, TradeType.Buy);

            return position;
        }


        private void ricalcolaTakeProfitStopAndLoss()
        {
            double lastPrice = 0;
            double localPriceDiff;

            //la posizione dovrebbe essere in Sell             
            if (Bool == false)
            {
                lastPrice = Symbol.Ask;
                localPriceDiff = lastPrice - previousPriceAsk;
                previousPriceAsk = lastPrice;

                if (localPriceDiff < 0)
                {
                    takeProfit = takeProfit + (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss + (localPriceDiff / divisoreTakeOrStop);
                }

                if (localPriceDiff > 0)
                {
                    takeProfit = takeProfit - (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss - (localPriceDiff / divisoreTakeOrStop);
                }
            }

            //la posizione dovrebbe essere in Buy
            if (Bool == true)
            {
                lastPrice = Symbol.Bid;
                localPriceDiff = lastPrice - previousPriceBid;
                previousPriceBid = lastPrice;

                if (localPriceDiff > 0)
                {
                    takeProfit = takeProfit + (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss + (localPriceDiff / divisoreTakeOrStop);
                }

                if (localPriceDiff < 0)
                {
                    takeProfit = takeProfit - (localPriceDiff / divisoreTakeOrStop);
                    stopAndLoss = stopAndLoss - (localPriceDiff / divisoreTakeOrStop);
                }
            }

        }

    }

}

 

No way to solve the problem?

@light96

PanagiotisCharalampous
12 Mar 2020, 11:46

Hi light96,

I tried reproducing this the last couple of days but could not. All my backtesting runs match the optimization runs. The fact that you are using GA method does not allow me to get the exact same results as you and reproduce. Any change you can narrow down the optimization range, run an exhaustive grid search and provide another example that will allow us to reproduce this?

Best Regards,

Panagiotis 

Join us on Telegram

@PanagiotisCharalampous

light96
14 Mar 2020, 16:20

RE:

PanagiotisCharalampous said:

Hi light96,

I tried reproducing this the last couple of days but could not. All my backtesting runs match the optimization runs. The fact that you are using GA method does not allow me to get the exact same results as you and reproduce. Any change you can narrow down the optimization range, run an exhaustive grid search and provide another example that will allow us to reproduce this?

Best Regards,

Panagiotis 

Join us on Telegram

Surely. These are the results of an optimization process run with Grid method, same trouble:

First attempt:

Second attempt:

 

I hope is something possible to solve.

Thank you.

@light96

PanagiotisCharalampous
16 Mar 2020, 10:49

Hi light96,

Thanks, this has helped reproduce the issue. I will forward to the product team for further investigation.

Best Regards,

Panagiotis 

Join us on Telegram

@PanagiotisCharalampous

light96
16 Mar 2020, 16:04

RE:

PanagiotisCharalampous said:

Hi light96,

Thanks, this has helped reproduce the issue. I will forward to the product team for further investigation.

Best Regards,

Panagiotis 

Join us on Telegram

 

Thank you so much.

I'll wait for a new message, and I hope they can solve it. 
 

See you soon.

 

 

@light96
Copyright © 2024 Spotware Systems Ltd. All rights reserved.
cTrader Ltd offers through its group of companies the cTrader platform. The information on this website is for general informational purposes only and does not constitute financial or investment advice. cTrader does not solicit retail investors. Reliance on this information is at your own risk.