Optimization does not continue when specifying too much parameters to optimize

christoph.keller
19 Apr 2021, 12:14

Additional information about the optimized bot

Here are some additional information about the bot I'm trying to optimize:

- Following indicators are used: 3 x RSI, 6 x SMA
- Inside OnStart, I validate the input parameters (as there are invalid ones), and throw an exception (InvalidOperationException) when the parameters are not correct
- Also tried to just log a message and execute Stop(); instead of throwing an exception, this also does not work
- Running the bot in 15min TimeFrame, opening orders OnBar and closing orders mostly OnTick

PanagiotisCharalampous
19 Apr 2021, 12:33

Hi christoph.keller,

To investigate this further, we will need to have the cBot code and optimization parameters. Can you provide them for us?

Best Regards,

Panagiotis 

Join us on Telegram

christoph.keller
19 Apr 2021, 13:17

RE:

PanagiotisCharalampous said:

Hi christoph.keller,

To investigate this further, we will need to have the cBot code and optimization parameters. Can you provide them for us?

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

Thank you for your reply. Unfortunately, I cannot share the original bot in public, but I created a small test-bot (of course, this one is not really profitable... :)). With this bot, I can see the same issue. When optimizing Parameter1 - Parameter20 (defaults are set including a valid range), the optimization fails (after 50 - 95 passes), when disabling Parameter 11 - Parameter20, the optimization works (with around 1100 passes).

Here is the bot-code:

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class OptimizationTest : Robot
    {
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter1 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter2 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter3 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter4 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter5 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter6 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter7 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter8 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter9 { get; set; }
	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 100, Step = 1)]
	    public int Parameter10 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter11 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter12 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter13 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter14 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter15 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter16 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter17 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter18 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter19 { get; set; }

	    [Parameter(DefaultValue = 1, MinValue = 1, MaxValue = 10, Step = 0.5)]
	    public double Parameter20 { get; set; }

	    private RelativeStrengthIndex _rsi1;
	    private RelativeStrengthIndex _rsi2;
	    private RelativeStrengthIndex _rsi3;
	    private SimpleMovingAverage _sma1a;
	    private SimpleMovingAverage _sma1b;
	    private SimpleMovingAverage _sma2a;
	    private SimpleMovingAverage _sma2b;
	    private SimpleMovingAverage _sma3a;
	    private SimpleMovingAverage _sma3b;

	    protected override void OnStart()
		{
			if (Parameter4 > Parameter5 || Parameter6 > Parameter7 || Parameter8 > Parameter9)
			{
				throw new InvalidOperationException("Wrong input parameters.");
			}

			_rsi1 = Indicators.RelativeStrengthIndex(Bars.ClosePrices, Parameter1);
			_rsi2 = Indicators.RelativeStrengthIndex(Bars.ClosePrices, Parameter2);
			_rsi3 = Indicators.RelativeStrengthIndex(Bars.ClosePrices, Parameter3);

			_sma1a = Indicators.SimpleMovingAverage(_rsi1.Result, Parameter4);
			_sma1b = Indicators.SimpleMovingAverage(_rsi1.Result, Parameter5);

			_sma2a = Indicators.SimpleMovingAverage(_rsi2.Result, Parameter6);
			_sma2b = Indicators.SimpleMovingAverage(_rsi2.Result, Parameter7);

			_sma3a = Indicators.SimpleMovingAverage(_rsi3.Result, Parameter8);
			_sma3b = Indicators.SimpleMovingAverage(_rsi3.Result, Parameter9);
		}

        protected override void OnBar()
		{
			if (Positions.Any())
			{
				return;
			}

			if (_sma1a.Result.LastValue > _sma1b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Buy, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter12, null);
			}
			else if (_sma1a.Result.LastValue < _sma1b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Sell, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter13, null);
			}

			if (_sma2a.Result.LastValue > _sma2b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Buy, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter14, null);
			}
			else if (_sma2a.Result.LastValue < _sma2b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Sell, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter15, null);
			}

			if (_sma3a.Result.LastValue > _sma3b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Buy, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter16, null);
			}
			else if (_sma3a.Result.LastValue < _sma3b.Result.LastValue)
			{
				ExecuteMarketOrder(TradeType.Sell, SymbolName, Symbol.VolumeInUnitsMin, "Bot", Parameter17, null);
			}
		}

        protected override void OnTick()
        {
	        foreach (var position in Positions)
	        {
		        if (position.NetProfit > Parameter11)
		        {
			        position.Close();
		        }

		        if (position.EntryTime > Server.Time.AddHours(Parameter10))
		        {
			        position.Close();
		        }
	        }
        }
    }
}

Please let me know, if you cannot see the mentioned issue. Perhaps multiple optimization restarts are needed.

Thanks in advance and have a great day,
Chris

christoph.keller
19 Apr 2021, 18:01 ( Updated at: 22 Apr 2021, 09:20 )

-- duplicated message, see answer above.

PanagiotisCharalampous
20 Apr 2021, 16:27

Hi christoph.keller,

Can you please also provide optimization method and optimization criteria?

Best Regards,

Panagiotis 

Join us on Telegram

christoph.keller
21 Apr 2021, 08:58

RE:

PanagiotisCharalampous said:

Hi christoph.keller,

Can you please also provide optimization method and optimization criteria?

Best Regards,

Panagiotis 

Join us on Telegram

Good morning Panagiotis,

Thank you for your answer. Here are more info about my environment:

• Optimization method: Genetic algorithm
• Criteria: maximize "net profit", minimize "equity drawdown %", maximize "winning trades" (Default settings)
• I'm working on a notebook with a Intel Core i7-8750H (@2.2GHz) with 16GB RAM
• enabled gcserver-workaround to maximize CPU usage (cTrader.exe.config extended with "<gcServer enabled="true" />")

Hope this helps to find the source of the problem. If you need me to do any tests, just let me know.

Best regards,
Chris

PanagiotisCharalampous
22 Apr 2021, 09:41

Hi Chris,

Thanks for the additional information. One last question, is this reproduced when you switch to grid search algorithm?

Best Regards,

Panagiotis 

Join us on Telegram

christoph.keller
22 Apr 2021, 11:02

RE:

PanagiotisCharalampous said:

Hi Chris,

Thanks for the additional information. One last question, is this reproduced when you switch to grid search algorithm?

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

Thanks for your reply. I retested the issue in grid-search algorithm and it seems that this mode does not have the issue.
I stopped after about 300 passes:

• CPU was constant during the whole test at around 80%
• After stop, CPU usage dropped instantly
• cTrader closes normally after the test (Window closes instantly, process ends after about 5 seconds

Hope this helps. Let me know if you need more information.

Best regards and have a great day,
Chris

PanagiotisCharalampous
22 Apr 2021, 16:44

Hi christoph.keller,

We tried to reproduce this behavior internally. The conclusion is that even thought at some stage the optimization freezes for a while (some minutes depending on the processor), it eventually resumes and terminates. Can you please advise if this is the case for you as well or does it freeze indefinitely?

Best Regards,

Panagiotis 

Join us on Telegram

christoph.keller
26 Apr 2021, 12:14

RE:

PanagiotisCharalampous said:

Hi christoph.keller,

We tried to reproduce this behavior internally. The conclusion is that even thought at some stage the optimization freezes for a while (some minutes depending on the processor), it eventually resumes and terminates. Can you please advise if this is the case for you as well or does it freeze indefinitely?

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

Thanks for your reply. I saw that the optimization some times continues very slowly, so it seems to recover from the freeze. But it makes it nearly impossible to finish the optimization (as it tends to run for days rather than hours, when using less parameters to optimize).

Hope this helps.

Best regards and have a nice day,
Chris

christoph.keller
01 Jul 2021, 14:03

RE: RE:

Is there any update on this issue?

Thanks in advance and have a great day,
Chris

PanagiotisCharalampous
01 Jul 2021, 14:56

Hi Chris,

At the moment there is no open issue. As per our tests, optimization will become slow due to many parameters but it eventually resumes and terminates.

Best Regards,

Panagiotis 

Join us on Telegram and Facebook