martins
03 Dec 2024, 22:36 ( Updated at: 03 Dec 2024, 23:29 )

RE: indicator.Count is 0 when referenced in cBot's first OnBar and changes to correct 201 if referenced again without doing anything else

It's not quite as bad as initially thought, but only accessing non-display indicators (without [Output…] suffixes) in OnBar always gives Count=0 & NaN values.

These demos might be a bit OTT but show the problem (they only really needed Output & NonOutput).  Using the default parameters, it will log NonOutput.Count=0 (and Nan values) in the first 10 OnBar calls, then the 200+ history Calculates will run (because OnBar starts accessing Output.Count too), and it'll be normal from there on (Output & NonOutput having proper values).

Also see that the ‘popup text’ doesn't always change as you move the mouse across different lines - it should say the name of the highlighted line but tends to stay ‘stuck’ on the first one you crossed - this could be related to the other case re ‘popup text apprears for unticked lines’).

#define ONBAR   // include an OnBar method (or comment out this line)
//#define ONBARCLOSED  // include an OnBarClosed method (or comment out this line)

using System;
using cAlgo.API;
using cAlgo.API.Indicators;
//using cAlgo.API.Indicators;
//using cAlgo.API.Internals;
//using cAlgo.Indicators;
//using mjsDemoOutputAndNonOutput;
namespace cAlgo.Robots
{
[Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None, AddIndicators = true)]
public class mjsDemoZeroInIndicatorCount : Robot
{

 [Parameter("robot Sleep (ms)", DefaultValue = 0)]   public int robotSleep { get; set; }  // makes no difference other than speed, eg 50 or 200 millisecs every Bar
 [Parameter("indic Sleep (ms)", DefaultValue = 0)]   public int indicSleep { get; set; }  // makes no difference other than speed, eg 5 or 20, cBot just suspended while Calculates catch up
 [Parameter("Trigger Calculates (call #)", DefaultValue = "10")] public int triggerCalculates { get; set; } // get Output.Count in OnBar call # (to trigger Calculates if not already run)  
 [Parameter("Stop at OnBar (call #)", DefaultValue = "-1")] public int parmStopOnBar { get; set; }  // for example: don't bother running to backtest end-date
 [Parameter("Count of [Output]", DefaultValue = "false")] public bool referOutput { get; set; }  // true will cause Calculates immediately, ie normal indicator operation
 [Parameter("Count of [NonOutput]", DefaultValue = "true")] public bool referNonOutput { get; set; } // if only non-display series are accessed, the indicator doesn't 'get going', just Initialize
 [Parameter("Count of Standard", DefaultValue = "false")] public bool referStandard { get; set; }  // true will cause Calculates immediately, ie normal indicator operation
 [Parameter("Value of [Output]", DefaultValue = "false")] public bool valueOutput { get; set; }  // true will cause Calculates immediately, ie normal indicator operation
 [Parameter("Value of [NonOutput]", DefaultValue = "true")] public bool valueNonOutput { get; set; } // if only non-display series are accessed, the indicator doesn't 'get going', just Initialize
 [Parameter("Value of Standard", DefaultValue = "false")] public bool valueStandard { get; set; }  // true will cause Calculates immediately, ie normal indicator operation

 private DonchianChannel _standardIndic;
 private mjsDemoOutputAndNonOutput _demoIndic;

 protected override void OnStart()
 {
  printMethod = " OnStart: ";
  Logit("started");
  Logit("Server.Time=" + Server.Time + " Bars.Count=" + Bars.Count + " Bars.Last(0).OpenTime=" + Bars.Last(0).OpenTime + " countOnBar=" + countOnBar);
  try
  {
   _standardIndic = Indicators.DonchianChannel(50);
   _demoIndic = Indicators.GetIndicator<mjsDemoOutputAndNonOutput>(1.005, indicSleep);
  }
  catch { Logit("an indicator failed to start"); Stop(); }
  Logit("ending");
 }

 string printMethod;
 void Logit(string msg)
 {
  Print(DateTime.Now.ToString("HH:mm:ss.ffff") + printMethod + msg);
 }
 int countOnBar = 0;

#if ONBAR  // include an OnBar method
 protected override void OnBar()
 {
  printMethod = " OnBar: ";
  Logit("started");
  doBarStuff();
  Logit("ending");
 }
#endif
#if ONBARCLOSED        // include an OnBarClosed method                                     
 protected override void OnBarClosed()
 {
  printMethod = " OnBarClosed: ";
  Logit("started");
  doBarStuff();
  Logit("ending");
 }
#endif
 void doBarStuff()
 {
  Logit("Server.Time=" + Server.Time + " Bars.Count=" + Bars.Count  + " Bars.Last(0).OpenTime=" + Bars.Last(0).OpenTime + " countOnBar=" + countOnBar);
  if (robotSleep > 0)
  {
   Logit("System.Threading.Thread.Sleep(" + robotSleep + ") starting");
   System.Threading.Thread.Sleep(robotSleep);
   Logit("System.Threading.Thread.Sleep(" + robotSleep + ") done");
  }

  if (triggerCalculates >= 0 && countOnBar == triggerCalculates)
  {
   Logit("turn on getting a 'display' Output.Count next in order to trigger history Calculates   (due to countOnBar=" + countOnBar + " == parmStopOnBar=" + triggerCalculates);
   referOutput = true;
  }
//   for (int i=0; i<100; i++) Print("Print some stuff                                                                                                             x"); 
//   if (Bars.Count <= 203)
  {
   if (referNonOutput)
   {
    Logit("get _demoIndic.NonOutput.Count next"); 
    Print("_demoIndic.NonOutput.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput2.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput3.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput4.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput5.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput6.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput7.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput8.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput9.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput10.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput11.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput12.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput13.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput14.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput15.Count=" + _demoIndic.NonOutput.Count);
    Print("_demoIndic.NonOutput16.Count=" + _demoIndic.NonOutput.Count);

    Logit("_demoIndic.NonOutput.Count=" + _demoIndic.NonOutput.Count);
    Logit("_demoIndic.NonOutput.Count=" + _demoIndic.NonOutput.Count);
    Logit("_demoIndic.NonOutput.Count=" + _demoIndic.NonOutput.Count);
   }
   if (referOutput)
   {
    Logit("get _demoIndic.Output.Count next");
    Print("_demoIndic.Output.Count=" + _demoIndic.Output.Count);
    Print("_demoIndic.Output2.Count=" + _demoIndic.Output2.Count);
    Print("_demoIndic.Output3.Count=" + _demoIndic.Output3.Count);
    Print("_demoIndic.Output4.Count=" + _demoIndic.Output4.Count);
    Print("_demoIndic.Output5.Count=" + _demoIndic.Output5.Count);
    Print("_demoIndic.Output6.Count=" + _demoIndic.Output6.Count);
    Print("_demoIndic.Output7.Count=" + _demoIndic.Output7.Count);
    Print("_demoIndic.Output8.Count=" + _demoIndic.Output8.Count);
    Print("_demoIndic.Output9.Count=" + _demoIndic.Output9.Count);
    Print("_demoIndic.Output10.Count=" + _demoIndic.Output10.Count);
    Print("_demoIndic.Output11.Count=" + _demoIndic.Output11.Count);
    Print("_demoIndic.Output12.Count=" + _demoIndic.Output12.Count);
    Print("_demoIndic.Output13.Count=" + _demoIndic.Output13.Count);
    Print("_demoIndic.Output14.Count=" + _demoIndic.Output14.Count);
    Print("_demoIndic.Output15.Count=" + _demoIndic.Output15.Count);
    Print("_demoIndic.Output16.Count=" + _demoIndic.Output16.Count);
    Print("_demoIndic.Output17.Count=" + _demoIndic.Output17.Count);
    Print("_demoIndic.Output18.Count=" + _demoIndic.Output18.Count);
    Print("_demoIndic.Output19.Count=" + _demoIndic.Output19.Count);
    Print("_demoIndic.Output20.Count=" + _demoIndic.Output20.Count);
    Print("_demoIndic.Output21.Count=" + _demoIndic.Output21.Count);
    Print("_demoIndic.Output22.Count=" + _demoIndic.Output22.Count);
    Print("_demoIndic.Output23.Count=" + _demoIndic.Output23.Count);
    Print("_demoIndic.Output24.Count=" + _demoIndic.Output24.Count);

    Logit("_demoIndic.Output.Count=" + _demoIndic.Output.Count);
    Logit("_demoIndic.Output.Count=" + _demoIndic.Output.Count);
    Logit("_demoIndic.Output.Count=" + _demoIndic.Output.Count);
   }
   if (referStandard)
   {
    Logit("get _standardIndic.Middle.Count next");
    Logit("_standardIndic.Middle.Count=" + _standardIndic.Middle.Count);
    Logit("_standardIndic.Middle.Count=" + _standardIndic.Middle.Count);
    Logit("_standardIndic.Middle.Count=" + _standardIndic.Middle.Count);
   }
   int i = 0; // or Count-1
   if (valueNonOutput) // will be all NaN if Count is 0 and Calculates haven't run yet
   {
    Logit("get _demoIndic.NonOutput[" + i + "] next");
    Print("_demoIndic.NonOutput[" + i + "]=" + _demoIndic.NonOutput[i]);
    Print("_demoIndic.NonOutput2[" + i + "]=" + _demoIndic.NonOutput2[i]);
    Print("_demoIndic.NonOutput3[" + i + "]=" + _demoIndic.NonOutput3[i]);
    Print("_demoIndic.NonOutput4[" + i + "]=" + _demoIndic.NonOutput4[i]);
    Print("_demoIndic.NonOutput5[" + i + "]=" + _demoIndic.NonOutput5[i]);
    Print("_demoIndic.NonOutput6[" + i + "]=" + _demoIndic.NonOutput6[i]);
    Print("_demoIndic.NonOutput7[" + i + "]=" + _demoIndic.NonOutput7[i]);
    Print("_demoIndic.NonOutput8[" + i + "]=" + _demoIndic.NonOutput8[i]);
    Print("_demoIndic.NonOutput9[" + i + "]=" + _demoIndic.NonOutput9[i]);
    Print("_demoIndic.NonOutput10[" + i + "]=" + _demoIndic.NonOutput10[i]);
    Print("_demoIndic.NonOutput11[" + i + "]=" + _demoIndic.NonOutput11[i]);
    Print("_demoIndic.NonOutput12[" + i + "]=" + _demoIndic.NonOutput12[i]);
    Print("_demoIndic.NonOutput13[" + i + "]=" + _demoIndic.NonOutput13[i]);
    Print("_demoIndic.NonOutput14[" + i + "]=" + _demoIndic.NonOutput14[i]);
    Print("_demoIndic.NonOutput15[" + i + "]=" + _demoIndic.NonOutput15[i]);
    Print("_demoIndic.NonOutput16[" + i + "]=" + _demoIndic.NonOutput16[i]);

    Logit("_demoIndic.NonOutput[" + i + "]=" + _demoIndic.NonOutput[i]);
    Logit("_demoIndic.NonOutput[" + i + "]=" + _demoIndic.NonOutput[i]);
    Logit("_demoIndic.NonOutput[" + i + "]=" + _demoIndic.NonOutput[i]);
   }
   if (valueOutput)
   {
    Logit("get _demoIndic.Output[" + i + "] next");
    Logit("_demoIndic.Output[" + i + "]=" + _demoIndic.Output[i]);
    Logit("_demoIndic.Output[" + i + "]=" + _demoIndic.Output[i]);
    Logit("_demoIndic.Output[" + i + "]=" + _demoIndic.Output[i]);
   }
   if (valueStandard)
   {
    Logit("get _standardIndic.Middle[" + i + "] next");
    Logit("_standardIndic.Middle[" + i + "]=" + _standardIndic.Middle[i]);
    Logit("_standardIndic.Middle[" + i + "]=" + _standardIndic.Middle[i]);
    Logit("_standardIndic.Middle[" + i + "]=" + _standardIndic.Middle[i]);
   }
  }
  if (parmStopOnBar >= 0 && countOnBar == parmStopOnBar)
  {
   Logit("Stop() due to countOnBar=" + countOnBar + " == parmStopOnBar=" + parmStopOnBar);
   Stop();
  }
  countOnBar++;
 }

}
}

=========================================================

using System;
using cAlgo.API;
using cAlgo.API.Collections;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;

namespace cAlgo
{
   [Indicator(IsOverlay = true, AccessRights = AccessRights.None)]
   public class mjsDemoOutputAndNonOutput : Indicator
   {
       [Parameter("Above/Below Factor", DefaultValue = "1.005")] public string Factor { get; set; }
 [Parameter("Sleep (ms)", DefaultValue = 10)]    public int parmSleep { get; set; }

       [Output("Output")] public IndicatorDataSeries Output { get; set; }
       [Output("Output2")] public IndicatorDataSeries Output2 { get; set; }
       [Output("Output3")] public IndicatorDataSeries Output3 { get; set; }
       [Output("Output4")] public IndicatorDataSeries Output4 { get; set; }
       [Output("Output5")] public IndicatorDataSeries Output5 { get; set; }
       [Output("Output6")] public IndicatorDataSeries Output6 { get; set; }
       [Output("Output7")] public IndicatorDataSeries Output7 { get; set; }
       [Output("Output8")] public IndicatorDataSeries Output8 { get; set; }
       [Output("Output9")] public IndicatorDataSeries Output9 { get; set; }
       [Output("Output10")] public IndicatorDataSeries Output10 { get; set; }
       [Output("Output11")] public IndicatorDataSeries Output11 { get; set; }
       [Output("Output12")] public IndicatorDataSeries Output12 { get; set; }
       [Output("Output13")] public IndicatorDataSeries Output13 { get; set; }
       [Output("Output14")] public IndicatorDataSeries Output14 { get; set; }
       [Output("Output15")] public IndicatorDataSeries Output15 { get; set; }
       [Output("Output16")] public IndicatorDataSeries Output16 { get; set; }
       [Output("Output17")] public IndicatorDataSeries Output17 { get; set; }
       [Output("Output18")] public IndicatorDataSeries Output18 { get; set; }
       [Output("Output19")] public IndicatorDataSeries Output19 { get; set; }
       [Output("Output20")] public IndicatorDataSeries Output20 { get; set; }
       [Output("Output21")] public IndicatorDataSeries Output21 { get; set; }
       [Output("Output22")] public IndicatorDataSeries Output22 { get; set; }
       [Output("Output23")] public IndicatorDataSeries Output23 { get; set; }
       [Output("Output24")] public IndicatorDataSeries Output24 { get; set; }

       public IndicatorDataSeries NonOutput { get; set; }
       public IndicatorDataSeries NonOutput2 { get; set; }
       public IndicatorDataSeries NonOutput3 { get; set; }
       public IndicatorDataSeries NonOutput4 { get; set; }
       public IndicatorDataSeries NonOutput5 { get; set; }
       public IndicatorDataSeries NonOutput6 { get; set; }
       public IndicatorDataSeries NonOutput7 { get; set; }
       public IndicatorDataSeries NonOutput8 { get; set; }
       public IndicatorDataSeries NonOutput9 { get; set; }
       public IndicatorDataSeries NonOutput10 { get; set; }
       public IndicatorDataSeries NonOutput11 { get; set; }
       public IndicatorDataSeries NonOutput12 { get; set; }
       public IndicatorDataSeries NonOutput13 { get; set; }
       public IndicatorDataSeries NonOutput14 { get; set; }
       public IndicatorDataSeries NonOutput15 { get; set; }
       public IndicatorDataSeries NonOutput16 { get; set; }

 protected override void Initialize()
 {
  printMethod = " Initialise: ";
  Logit("started - Factor=" + Factor + " Sleep=" + parmSleep);
  Logit("Server.Time=" + Server.Time + " Bars.Count=" + Bars.Count + " Bars.Last(0).OpenTime=" + Bars.Last(0).OpenTime + " countCalculate=" + countCalculate);
  NonOutput = CreateDataSeries();
  NonOutput2 = CreateDataSeries();
  NonOutput3 = CreateDataSeries();
  NonOutput4 = CreateDataSeries();
  NonOutput5 = CreateDataSeries();
  NonOutput6 = CreateDataSeries();
  NonOutput7 = CreateDataSeries();
  NonOutput8 = CreateDataSeries();
  NonOutput9 = CreateDataSeries();
  NonOutput10 = CreateDataSeries();
  NonOutput11 = CreateDataSeries();
  NonOutput12 = CreateDataSeries();
  NonOutput13 = CreateDataSeries();
  NonOutput14 = CreateDataSeries();
  NonOutput15 = CreateDataSeries();
  NonOutput16 = CreateDataSeries();
  if (parmSleep > 0)
  {
   Logit("System.Threading.Thread.Sleep(" + parmSleep + ") starting");
   System.Threading.Thread.Sleep(parmSleep);
   Logit("System.Threading.Thread.Sleep(" + parmSleep + ") done");
  }
  Logit("ending");
 }

 int countCalculate = 0;
       public override void Calculate(int index)
       {
  printMethod = " Calculate: ";
  Logit("started");
  Logit("Server.Time=" + Server.Time + " index=" + index  + " Bars[index].OpenTime=" + Bars[index].OpenTime + " countCalculate=" + countCalculate);
  if (parmSleep > 0)
  {
   Logit("System.Threading.Thread.Sleep(" + parmSleep + ") starting");
   System.Threading.Thread.Sleep(parmSleep);
   Logit("System.Threading.Thread.Sleep(" + parmSleep + ") done");
  }
  Output[index] = Bars[index].Close * 1.001;
  Output2[index] = Bars[index].Close * 1.002;
  Output3[index] = Bars[index].Close * 1.003;
  Output4[index] = Bars[index].Close * 1.004;
  Output5[index] = Bars[index].Close * 1.005;
  Output6[index] = Bars[index].Close * 1.006;
  Output7[index] = Bars[index].Close * 1.007;
  Output8[index] = Bars[index].Close * 1.008;
  Output9[index] = Bars[index].Close * 1.009;
  Output10[index] = Bars[index].Close * 1.010;
  Output11[index] = Bars[index].Close * 1.011;
  Output12[index] = Bars[index].Close * 1.012;
  Output13[index] = Bars[index].Close * 1.013;
  Output14[index] = Bars[index].Close * 1.014;
  Output15[index] = Bars[index].Close * 1.015;
  Output16[index] = Bars[index].Close * 1.016;
  Output17[index] = Bars[index].Close * 1.017;
  Output18[index] = Bars[index].Close * 1.018;
  Output19[index] = Bars[index].Close * 1.019;
  Output20[index] = Bars[index].Close * 1.020;
  Output21[index] = Bars[index].Close * 1.021;
  Output22[index] = Bars[index].Close * 1.022;
  Output23[index] = Bars[index].Close * 1.023;
  Output24[index] = Bars[index].Close * 1.024;

  NonOutput[index] = Bars[index].Close / 1.001;
  NonOutput2[index] = Bars[index].Close / 1.002;
  NonOutput3[index] = Bars[index].Close / 1.003;
  NonOutput4[index] = Bars[index].Close / 1.004;
  NonOutput5[index] = Bars[index].Close / 1.005;
  NonOutput6[index] = Bars[index].Close / 1.006;
  NonOutput7[index] = Bars[index].Close / 1.007;
  NonOutput8[index] = Bars[index].Close / 1.008;
  NonOutput9[index] = Bars[index].Close / 1.009;
  NonOutput10[index] = Bars[index].Close / 1.010;
  NonOutput11[index] = Bars[index].Close / 1.011;
  NonOutput12[index] = Bars[index].Close / 1.012;
  NonOutput13[index] = Bars[index].Close / 1.013;
  NonOutput14[index] = Bars[index].Close / 1.014;
  NonOutput15[index] = Bars[index].Close / 1.015;
  NonOutput16[index] = Bars[index].Close / 1.016;

   Logit("Server.Time=" + Server.Time + " index=" + index  + " Output.Count=" + Output.Count + "  Output[" + index + "]=" + Output[index] 
   + " NonOutput.Count=" + NonOutput.Count + "  NonOutput[" + index + "]=" + NonOutput[index] + " countCalculate=" + countCalculate);
  countCalculate++;
  Logit("ending");
      }
 string printMethod;
 void Logit(string msg)
 {
  Print(DateTime.Now.ToString("HH:mm:ss.ffff") + printMethod + msg);
 }
   }
}

Later thoughts: Are the Series without an [Output] suffix ignored, by the part of cTrader that initiates running Calculates because, without [Output], they are unknown to that part?

If so, could there be a new keyword added in [Output] to say ‘not to be displayed’ - possibly the equivalent of a (permanently) unticked [Output] IndicatorDataSeries ?

Thanks Martin

martins
03 Dec 2024, 00:46 ( Updated at: 03 Dec 2024, 01:11 )

RE: RE: indicator.Count is 0 when referenced in cBot's first OnBar and changes to correct 201 if referenced again without doing anything else

firemyst said: 

martins said: 

Further tests with Print(DateTime.Now.ToString("HH:mm:ss.ffff") + …) & some System.Threading.Thread.Sleep calls in both an indicator & a cbot shows that making a reference, in the first OnBar call, to the .Count of a NON-DISPLAY IndicatorDataSeries (one WITHOUT a "[Output …] attribute) DOES NOT trigger the indicator to start making Calculate calls - the Calculate calls for the 200 history Bars seem only to be triggered by the first reference to the .Count of any DISPLAY IndicatorDataSeries (one WITH [Output…]) but even then, the initial zero count is returned to the cBot and incorporated into a Print buffer before the Calculates happen.

Then the OnBar pauses while the 200 or so history Calculates happen for the Bars prior to the backtest start time, then the buffer gets put into log.txt and the Log screen, and subsequent references in the first OnBar to .Count of any display or non-display IndicatorDataSeries give correct values (because all the history Calculates have been run).

It seem in we have to make 2 references in a cBot's first OnBar (when index=0) to an indicator's IndicatorDataSeries having the [Output…] clause (making it, if the indicator were run independently, a display line not just a calculated series) in order to get a proper value - is this correct and documented or is it a bug?

I know this means nothing, but I applaud your research in trying to figure it all out and am looking forward to updates. 

It all started with some NaN values causing initial trades which didn't look right!  
It might be deliberate, so that if you ..GetIndicator<…> loads of them but don't actually use them all you don't waste cpu & memory initialising & then maintaining the redundant ones every new Bar or Tick.  If so maybe there's doc about it, but where.

martins
03 Dec 2024, 00:26 ( Updated at: 03 Dec 2024, 00:34 )

Further tests with Print(DateTime.Now.ToString("HH:mm:ss.ffff") + …) & some System.Threading.Thread.Sleep calls in both an indicator & a cbot shows that making a reference, in the first OnBar call, to the .Count of a NON-DISPLAY IndicatorDataSeries (one WITHOUT a "[Output …] attribute) DOES NOT trigger the indicator to start making Calculate calls - the Calculate calls for the 200 history Bars seem only to be triggered by the first reference to the .Count of any DISPLAY IndicatorDataSeries (one WITH [Output…]) but even then, the initial zero count is returned to the cBot and incorporated into a Print buffer before the Calculates happen.

Then the OnBar pauses while the 200 or so history Calculates happen for the Bars prior to the backtest start time, then the buffer gets put into log.txt and the Log screen, and subsequent references in the first OnBar to .Count of any display or non-display IndicatorDataSeries give correct values (because all the history Calculates have been run).

It seem we have to initially make 2 references in a cBot's first OnBar call (i.e. when index=0) to an indicator's IndicatorDataSeries having the [Output…] clause in order to get a proper value - is this correct and documented or is it a bug?

martins
29 Nov 2024, 12:52

RE: RE: Indicators on Chart show popup text for lines that are turn off (not ticked) thus reducing area available for right click menu (zoom etc)

martins said: 

PanagiotisCharalampous said: 

Hi there,

I did not understand what you are trying to describe. Can you visualize this somehow so that we can understand which popup texts you are referring to?

Best regards,

Panagiotis

When you hover the mouse over a line of an indicator it highlight's the whole line as a wide light grey line, and shows a text near the mouse saying the name of the indicator, its parameters in brackets, and the current value at that point. I called that text ‘popup text’. The problem is the text seems to appear regardless of whether the line is ticked (to show normally) or not (only the text 'pops up', the line itself remain correctly hidden).

To reproduce, add any indicator to a chart, say an EMA that only has one line, set its parameters so its line is well away from the candlesticks or any other stuff on the chart, hover on the line, see the ‘popup text’ & ‘fat’ line. All correct so far. Then right click & untick the line, click ok to remove the control box, and hover where the line was - the text will show and a right click there will bring up the control box again, whereas the text should not show and right clicking there should just cause the normal right-click chart menu to show (the same as if you right-clicked on the chart in other free space). The hidden line seems to be contaminating the what-looks-like free areas of the chart. It's probably a one-line fix.

Mouse is hovering over where the Middle line would show if it were ticked - it is unticked so correctly doesn't show, but also should not produce text when hovered over.

martins
28 Nov 2024, 12:37 ( Updated at: 28 Nov 2024, 13:05 )

RE: Indicators on Chart show popup text for lines that are turn off (not ticked) thus reducing area available for right click menu (zoom etc)

PanagiotisCharalampous said: 

Hi there,

I did not understand what you are trying to describe. Can you visualize this somehow so that we can understand which popup texts you are referring to?

Best regards,

Panagiotis

When you hover the mouse over a line of an indicator it highlight's the whole line as a wide light grey line, and shows a text near the mouse saying the name of the indicator, its parameters in brackets, and the current value at that point. I called that text ‘popup text’. The problem is the text seems to appear regardless of whether the line is ticked (to show normally) or not (only the text 'pops up', the line itself remain correctly hidden).

To reproduce, add any indicator to a chart, say an EMA that only has one line, set its parameters so its line is well away from the candlesticks or any other stuff on the chart, hover on the line, see the ‘popup text’ & ‘fat’ line. All correct so far. Then right click & untick the line, click ok to remove the control box, and hover where the line was - the text will show and a right click there will bring up the control box again, whereas the text should not show and right clicking there should just cause the normal right-click chart menu to show (the same as if you right-clicked on the chart in other free space). The hidden line seems to be contaminating the what-looks-like free areas of the chart. It's probably a one-line fix.

martins
27 Nov 2024, 15:33 ( Updated at: 27 Nov 2024, 15:40 )

RE: Forums: why not default to Last Added as the sort order, not Hot Activity which shows ancient posts

firemyst said: 

Exactly. I'm always having to change the setting to “last added” before viewing anything in the forums

Or a new option, “last changed”, meaning either added or commented on, is that what Hot Activity is supposed to do but doesn't?

Update: or maybe it does sometimes - this went to the top of Hot Activity after adding a comment. 
I think the problem is there's something else swamping the ‘new updates go to top’ rule, making old items go to the top maybe just because they've been read again, or have high static views.

martins
27 Nov 2024, 15:14 ( Updated at: 27 Nov 2024, 15:29 )

Best way would be to add a green button to the top of the Parameter tab next to save & load, to have the effect of copying all the current values to replace the optimise values, a bit like the green “Apply” on each line of Optimisation results but in reverse. 
I suppose everything else already in the current optimise parameters could be left unchanged - the min/max/steps and tick boxes - although that still means the 1st time you have to untick loads of them.

(We did add logic to OnStart for creating an .optset file from all the parameter values to allow easy optimising starting with the same parameters as a backtest, but a button to do it would be better and for everyone. And thanks for changing the .optset & .cbotset keywords output by save back to starting with uppercase for backward compatibility, lowercase (of 5.0.39 or .40?) didn't load into v4 during the switchover so we had to write a conversion)

martins
27 Nov 2024, 12:11 ( Updated at: 27 Nov 2024, 12:36 )

RE: Never let "upgrading to a new release" be the reason why "old backtest results cannot be exactly reproduced"

Just need option for a fixed swap rate and maybe a ‘do it the v4 way’ option. Also clarity about how to control what account currency exchange rate is used in backtest.

Commission rate is covered, can set own rate for backtest (I remembered after posting but there's no way to edit while ‘awaiting moderation’, hint!) - it's just swap rate that needs a way for setting own fixed rate and maybe a compatibility mode for “like v4” (where I think swap was not subtracted from equity until the end.

Also, if cTrader changes the way account currency exchange rate is calculated (from ‘fixed rate at backtest end date’ to ‘same calc as live’?) then please: there would need to be a compatibility mode option for 'doing it the old way', otherwise old backtests become useless again. 

Or is that what the button “download for historical data for additional symbols to accurately convert profits to account currency” is for?? 
In which case: Is ‘leaving that button unticked’ effectively the compatibility option? - please be clearer with documentation on anything that affects backtest results.  

martins
26 Nov 2024, 19:57 ( Updated at: 26 Nov 2024, 20:11 )

Agreed. Would be nice to have an easy way to get a number proportional to the TimeFrame period.

I sometimes use this, but if both of the 1st 2 Bars are non-standard, weekend and a gap in data for instance, then it won't work:

deltaBar = Math.Min(Bars[1].OpenTime.ToOADate() - Bars[0].OpenTime.ToOADate(), Bars[2].OpenTime.ToOADate() - Bars[1].OpenTime.ToOADate());   // TimeFrame as days ( * 1440 for minutes) 

Or more definitively, this:

   string timeframeChar = Bars.TimeFrame.ShortName.Substring(1, 1);
  double timeframeNumber = 0, timeframeDaysOA = 0;
  if (timeframeChar == "M") timeframeDaysOA = 20;
  else {
   try { timeframeNumber = Convert.ToInt32(Bars.TimeFrame.ShortName.Substring(2)); }
   catch { timeframeNumber = 0; }
   if (timeframeChar == "m") timeframeDaysOA = timeframeNumber / 3600;
   else if (timeframeChar == "h") timeframeDaysOA = timeframeNumber / 24;
   else if (timeframeChar == "D") timeframeDaysOA = timeframeNumber;
   else if (timeframeChar == "W") timeframeDaysOA = timeframeNumber * 5;
  }
 

(probably should have used chars not strings)

martins
26 Nov 2024, 14:14 ( Updated at: 26 Nov 2024, 15:50 )

RE: Legacy Build with Visual Studio 2019 fails with error "No cTrader or cAlgo was found with version 1.20 or higher"

firemyst said: 

I don't know what the issue is from looking at the files you posted.

However, brute force way to try and fix it (no guarantees!) – take a backup copy of your actual C# code.

Delete the indicator/cbot from cTrader.

Once deleted, go back and create a new indicator/cbot, adding references as appropriate.

Build it in cTrader.

Then select to edit in Visual studio.

Paste in your C# code from your backup.

hopefully it'll then build successfully.

The only drawback with this approach is if you have anything saved in a “template”, it may have to be readded.

Sorry, thanks but I did say “Update: solved by uninstall/reinstall. Don't think it was the dll” - I got the unchanged source to compile/build properly by uninstalling & reinstalling cTrader using the saved ctrader…setup.exe I'd previously used when the 4.9.2 installed. It downloaded the same old version 4.9.2, then proceeded to download the update 5.0.46 whilst displaying an option to “start ctrader” which i clicked and 4.9.2 started up, or by clicking the taskbar icon 5.0.46 starts up. After the install finished VS 2019 build was ok.

However the resultant .algo and other non-recompiled algos immediately crash if run in 4.9.2 on that pc system (log line 1: "…started", line 2: “…crashed with error #3804C3AC”). (whereas on another older pc with 4.9.2 [it also autoupdated to 5.0.46 too but can still start using AppData…cTrader_4.9.2.26009.exe] I can still run them ok, using same .algo files copied from the newer & now ‘bad’ pc). So the remaining problem is it seems something got removed by the uninstall/reinstall (on the newer pc) that is required for 4.9.2 backtest of net4 cBots to work (I also reinstalled the vsix on VS2019, but can't be that else they'd also not work when .algo file copied to other system). 
Process Monitor shows the old system that works is using awsglobal for the tcp, whereas the new one where 4.9.2 backtest fails seems to load the algo file, read the csv data file many times, then some certificates, then gives up soon after looking in several folders for and not finding System.Net.Sockets.resources.dll

I can develop on 5.0.46 fine, the whole point of using 4.9.2 is to EXACTLY reproduce old backtest results.
v5.0 has more accurate treatment of swap and no compatibility option, so nothing reproduces exactly, order timing changes, pseudo random numbers used up at a different rate, so you can't tell whether some change in backtest results was due to a subtle cBot logic alteration, or due to the version of cTrader & its treatment of swap. 
More importantly: without a pre-swap-changes (or whatever, the ‘old’) version of cTrader (or a compatibility mode in the new one) there's no opportunity to examine & UNDERSTAND exactly why the backtest result is different - by rerunning/comparing the old vs the new with extra debug log messages etc (having locked down the price data by using csv files so it can't change which otherwise it seems to) - I though first rule of computing was “only one change at a time” but autoupdating software makes that difficult. 

martins
24 Nov 2024, 14:17 ( Updated at: 25 Nov 2024, 06:26 )

RE: Legacy Build with Visual Studio 2019 fails with error "No cTrader or cAlgo was found with version 1.20 or higher"

firemyst said: 

Open up the Visual Studio “.csproj” file.

You'll see a section similar to:

<ItemGroup>
   <PackageReference Include="cTrader.Automate" Version="1.*" />
 </ItemGroup>

Or maybe even:

 <ItemGroup>
   <PackageReference Include="cTrader.Automate" Version="*" />
 </ItemGroup>

In previous incarnations, I think the version was hard coded like such:

 <ItemGroup>
   <PackageReference Include="cTrader.Automate" Version="1.1" />
 </ItemGroup>

Thanks for the suggestion, might that be for 2022 projects using Automate rather 2019 with cAlgo.API?

An example of VS 2022 project I have is short and contains the 1.* Automate reference:

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
 <PropertyGroup>
   <TargetFramework>net472</TargetFramework>
 </PropertyGroup>
 <ItemGroup>
   <PackageReference Include="cTrader.Automate" Version="1.*" />
 </ItemGroup>
</Project>

whereas the 2019 version of the same indicator is much longer (similar for other legacy stuff) containing a dll reference to cAlgo.API, Version=1.0.0.0 and its folder, but no Automate (unless that's what the ProjectTypeGuids DD87… & FAE0… are for):

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
 <PropertyGroup>
   <LangVersion>7.2</LangVersion>
   <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
   <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
   <ProjectGuid>{50097A1D-91A2-47D8-99D4-800ECC741F63}</ProjectGuid>
   <ProjectTypeGuids>{DD87C1B2-3799-4CA2-93B6-5288EE928820};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
   <OutputType>Library</OutputType>
   <AppDesignerFolder>Properties</AppDesignerFolder>
   <RootNamespace>cAlgo</RootNamespace>
   <AssemblyName>mjsEMA_2019net4</AssemblyName>
   <TargetFrameworkVersion>v4.0</TargetFrameworkVersion>
   <TargetFrameworkProfile>Client</TargetFrameworkProfile>
   <FileAlignment>512</FileAlignment>
   <CheckForOverflowUnderflow>True</CheckForOverflowUnderflow>
 </PropertyGroup>
 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
   <DebugSymbols>true</DebugSymbols>
   <DebugType>full</DebugType>
   <Optimize>false</Optimize>
   <OutputPath>bin\Debug\</OutputPath>
   <DefineConstants>DEBUG;TRACE</DefineConstants>
   <ErrorReport>prompt</ErrorReport>
   <WarningLevel>4</WarningLevel>
   <WarningsAsErrors>CS0108,CS0162,CS0109,CS0169,CS0628</WarningsAsErrors>
 </PropertyGroup>
 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
   <DebugType>pdbonly</DebugType>
   <Optimize>true</Optimize>
   <OutputPath>bin\Release\</OutputPath>
   <DefineConstants>TRACE</DefineConstants>
   <ErrorReport>prompt</ErrorReport>
   <WarningLevel>4</WarningLevel>
   <WarningsAsErrors>CS0108,CS0162,CS0109,CS0169,CS0628</WarningsAsErrors>
 </PropertyGroup>
 <ItemGroup>
   <Reference Include="System" />
   <Reference Include="System.Core" />
   <Reference Include="System.Xml.Linq" />
   <Reference Include="System.Data.DataSetExtensions" />
   <Reference Include="System.Data" />
   <Reference Include="System.Xml" />
   <Reference Include="cAlgo.API, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3499da3018340880, processorArchitecture=MSIL">
     <SpecificVersion>False</SpecificVersion>
     <HintPath>..\..\..\..\API\cAlgo.API.dll</HintPath>
   </Reference>
 </ItemGroup>
 <ItemGroup>
   <Compile Include="mjsEMA_2019net4.cs" />
   <Compile Include="Properties\AssemblyInfo.cs" />
 </ItemGroup>
 <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
 <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
      Other similar extension points exist, see Microsoft.Common.targets.
 <Target Name="BeforeBuild">
 </Target>
 <Target Name="AfterBuild">
 </Target>
 -->
</Project>

The strange thing is the reinstall didn't seem to change cAlgo.API.dll, unless it came with the same date but possibly contained a different Registry entry name or direct link to a different broker's AppData/…/cTrader/hexfolder copy of the app?

martins
16 Nov 2024, 23:53 ( Updated at: 17 Nov 2024, 08:37 )

If you save the current parms as a new set d'you see that as one to load? Normally they show as  *.optset files.
Could the others be in a different folder?

martins
16 Nov 2024, 22:45 ( Updated at: 17 Nov 2024, 08:37 )

RE: RE: RE: How is the new EntryPrice (& thus Pips etc) of a modified or partially closed Position calculated?

Thank you that's very helpful. I hadn't fully realized “[modified CFD] Positions are just collections of deals traded in a FIFO manner”, even for netting accounts it seems (per https://match-trade.com/retail-hedge-account-vs-netting/ ), unlike stock trading trading where reducing a holding is often considered not to alter the average price (as in https://economics.stackexchange.com/questions/20464/how-to-calculate-average-buy-price-when-you-buy-sell-and-rebuy ).

I did try something experimental, not wanting real takeprofit or stoploss action, by keeping a calculated average EntryPrice as an override encoded in the StopLoss or TakeProfit (as 4 times the actual value so as not to trigger a real take or stop) and only needing to be maintained for volume alterations from the 1st time each position is reduced, so the adjustment to its pips/profit & balance could be made from then on. Probably not particularly useful, but can work without storing any ‘state data’ outside of cTrader if not wanting real take or stop, and without needing to track or read the whole history. 

A modifyable Position.Comment would be better, and useful for other purposes too, has it been considered?

martins
14 Nov 2024, 17:28 ( Updated at: 15 Nov 2024, 00:28 )

RE: How is the new EntryPrice (& thus Pips etc) of a modified or partially closed Position calculated?

PanagiotisCharalampous said: 

Hi there,

The entry price of the position is the average entry price of the deals composing the position. If the position has only one opening deal, then the opening price should not change. If more opening deals are involved e.g. you have increased the size of the position after entering it, the entry price might change as partial volume is closed and deals removed from the position.

Best regards,

Panagiotis

Ok, thanks, that's helpful. I can see from logging that when the volume of a position is INCREASED the calculation of its new Position.Pips value is exactly:
 (oldPips * oldVol - spreadAsPips * increaseInVol) / newVol
which is correctly distributing the pips on the old portion plus the immediate pip loss of the new portion (due to spread) across the new larger volume (and so the EntryPrice will be that many pips from the current price and represent the weighted average). Rounding of the calculation seems usually to the nearest 0.1 pip but sometimes goes the other way, possibly to correct for accumulated rounding error. That's all fine.

HOWEVER, I'm seeing the remaining portion of a position after a REDUCTION in volume ALSO has its Position.Pips altered. Why is that?
Possibly the position was not combined really (in the live server or backtest memory) but rather kept as a collection of parts from each modification that increased volume, so that when it is later reduced some specific part is closed, in a LIFO or FIFO order or some other rule? Is this broker driven? 

So maybe ModifyPosition does not really modify a position in the sense of ending up with one homogeneous position with a single EntryPrice & Pips profit? Can't brokers running cTrader handle merging modification deals into one net holding?

If not, please could the doc say so, and how do we know what the separate parts are so we can predict partial closing results (without laboriously keeping track of all modifications)? Is there any information in a Position that can tell a cBot the position's stats will not react in a predictable way if later PARTIALLY closed ('predictable' meaning Entry & Pips stay unchanged, profit prorata etc)? 

I'm coming to the conclusion it's not good to mix increasing and decreasing volume via ModifyPosition; one or the other yes, predictable, but not both in the same cBot if it's basing decisions on Position.Pips or Account.UnrealizedProfit, or on almost anything other than Equity.

Thanks.

Update re "laboriously keeping track": not that hard, but involves keeping ‘account state’ data outside the cTrader account, like in a local file or cloud, not self-contained in the positions - unless perhaps cTrader could add a large Account.Comment field that could store arbitrary data, or make the Positions.Comment modifyable & bigger?? :) 

…
modifyDeals.Count=95 i=58 modifyDeals[i].Id=6 modifyDeals[i].Vol=19000 modifyDeals[i].Bid=1.36653 modifyDeals[i].Ask=1.36656
modifyDeals.Count=95 i=59 modifyDeals[i].Id=6 modifyDeals[i].Vol=13000 modifyDeals[i].Bid=1.36404 modifyDeals[i].Ask=1.36407
modifyDeals.Count=95 i=60 modifyDeals[i].Id=5 modifyDeals[i].Vol=4000 modifyDeals[i].Bid=1.36399 modifyDeals[i].Ask=1.36402
modifyDeals.Count=95 i=61 modifyDeals[i].Id=6 modifyDeals[i].Vol=-230000 modifyDeals[i].Bid=1.3605 modifyDeals[i].Ask=1.36053
modifyDeals.Count=95 i=62 modifyDeals[i].Id=5 modifyDeals[i].Vol=-230000 modifyDeals[i].Bid=1.3605 modifyDeals[i].Ask=1.36053
modifyDeals.Count=95 i=63 modifyDeals[i].Id=6 modifyDeals[i].Vol=-77000 modifyDeals[i].Bid=1.36007 modifyDeals[i].Ask=1.3601
modifyDeals.Count=95 i=64 modifyDeals[i].Id=5 modifyDeals[i].Vol=-77000 modifyDeals[i].Bid=1.36007 modifyDeals[i].Ask=1.3601
modifyDeals.Count=95 i=65 modifyDeals[i].Id=6 modifyDeals[i].Vol=10000 modifyDeals[i].Bid=1.35909 modifyDeals[i].Ask=1.35912
modifyDeals.Count=95 i=66 modifyDeals[i].Id=5 modifyDeals[i].Vol=19000 modifyDeals[i].Bid=1.36651 modifyDeals[i].Ask=1.36654
modifyDeals.Count=95 i=67 modifyDeals[i].Id=5 modifyDeals[i].Vol=10000 modifyDeals[i].Bid=1.36892 modifyDeals[i].Ask=1.36895
modifyDeals.Count=95 i=68 modifyDeals[i].Id=5 modifyDeals[i].Vol=13000 modifyDeals[i].Bid=1.37044 modifyDeals[i].Ask=1.37047
…

  struct modifyDeal {     // https://stackoverflow.com/questions/33809867/c-sharp-struct-in-a-list
  public int Id { get; set; }
  public double Vol { get; set; }
  public double Bid { get; set; }
  public double Ask { get; set; }
  public modifyDeal(int id, double vol, double bid, double ask)
  { Id = id; Vol = vol; Bid = bid; Ask = ask; }
 }

 List<modifyDeal> modifyDeals = new List<modifyDeal>(); // 14nov24
…
    var deal = new modifyDeal(position.Id, newVolume - position.VolumeInUnits, Symbol.Bid, Symbol.Ask);
   //or deal.Id = position.Id; etc
   modifyDeals.Add(deal); 

Might be able to extract same info using the History interface possibly (how far back does it go?), but haven't found all the doc about it, for instance where is History.OrderByDescending() other than in the example https://help.ctrader.com/ctrader-algo/references/Trading/History/History/#examples 

Later: Actually only one adjustment number per position might need remembering in order to emulate proportional netting when partially closing a position that has been increased earlier - the difference between ‘the proportional Pips or Entry expected before the partial close is done’ and ‘the actual new Pips or new Entry that appears in the Position afterwards’ - so the Bot can apply that offset to later Pips & profit (with opposite adjustment to Balance).
Is there any way of storing a small amount of data per position on the server, for instance a modifyable Comment? 
 

martins
11 Nov 2024, 22:38 ( Updated at: 11 Nov 2024, 22:54 )

RE: Backtester/Optimiser not matching with same parameters

PanagiotisCharalampous said: 

Hi there,

We have received the information and the issue will be investigated soon.

Best regards,

Panagiotis

I have found that it can be ‘correct’ for there to be differences between backtests on v4 vs v5 because v5 handes swap differently - it deducts it from the Account.Equity around 22:00 on days it is due, whereas v4 didn't seem to ever deduct swap (it just reported an accumulated amount at the end) - as far as I can see by comparing detailed logs of same test on both versions. So if decisions are based on equity, they will be different.

Whilst having more accurate swap treatment is commendable, it is highly inconvenient to not be able to exactly recreate old saved results in order to know whether a change to the bot has improved it or not. An option to again NOT deduct swap would also act as a sanity checker / confidence booster for our migration to v5.

To address the above, please can there be consideration of adding an option to disable the newly imposed mandatory deduction of swap from equity in backtest & optimise, and an option to specify a fixed custom rate (so old tests can still be recreated even if the automatic rate changes), in much the same way as there is already an option to set commission & spread.  Thanks for reading, Martin

martins
06 Nov 2024, 14:42 ( Updated at: 06 Nov 2024, 21:05 )

RE: RE: .algo indicators don't reload

I have the same problem on 5.0.40 - if you're looking at an indicator instance (in Indicators tab of Algo) and rebuild it (in visual studio 2022) it unloads but does not reload. The instance remains but does not show its lines until the whole cTrader disktop app is restarted. Doesn't seem to make a difference whether it's .NET 6 or Framework. Might make a difference if it's 2019, I've only noticed this happening with newly created indicators for which ‘edit in visual studio’ starts up 2022. 

This is worse than 4.9.2, where although the instance dissappears, it can be re-added without having to restart cTrader. 

Update: Building the same new source renamed as another solution configured for and using Visual Studio 2019 makes the same (but differently named) indicator just reload, no problems, on either 4.9.2 or 5.0.40 

(To make the 2019 .net4 version: I copied the 2022 .net6 indicator's folder, rename the inner folder too [I added suffix _2019net4], made the long winded .cproj file by editing a copy of a genuine old indicator, added & edited a Properties/AssemblyInfo.cs from same place & an edited copy of its .sln, then opened that in 2019 for a rebuild.)
(To switch between 4.9.2 & 5.0.40: I see 2 folders in x:\Users\…\AppData\Local\Spotware\cTrader with long hex names - one came from installing ICMarket's download & contains 4.9.2, one came from downloading from Spotware and contains 5.0.40 [having auto updated from 5.4.39 & still has 2 inner folders but only the cTrader.exe directly in the 2 hex folders will run, bringing up either 4.9.2 or 5.0.40 (not the hidden-by-update 5.0.39), or can run both at same time]).

martins
17 Oct 2024, 14:05 ( Updated at: 18 Oct 2024, 05:19 )

RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

PanagiotisCharalampous said: 

Hi there,

The OnException method was designed to inform developer that something is wrong in the algo. It is not supposed to be used in the execution flow of the algo. The order of execution of the OnException is not guaranteed and can be changed later. If you want to handle exceptions in predicted way you should use try catch {} blocks explicitly and do not rely on the OnException handler.

Best regards,

Panagiotis

Fair enough, for Indicators possibly, although if a Robot relies on them they need the same fault tolerance. 
Does it work differently for Robots - reading this https://help.ctrader.com/ctrader-algo/fault-tolerance/ it presents OnException as if it can take the place of individual error handling: “you can also customise how your algos react to certain exceptions, or overwrite the default fault tolerance behaviors entirely. To do so, use the OnException() method. It acts as the dedicated handler for any unhandled exceptions.”

martins
16 Oct 2024, 21:43

RE: RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

Did you see the same - it's as if, when Initialize runs, the 200 history calls to Calculate are already on whatever .NET scheduling queue c# uses, and any unhandled errors (i.e. without a Try/Catch) result in OnException calls aded to the end of the queue, whereas one might expect OnException to at least run in front of all pre-scheduled stuff, or ideally as an interrupt to the current task.

martins
13 Oct 2024, 22:37

RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

PanagiotisCharalampous said: 

Hi there,

Can you share a code example so that we can understand the issue?

Best regards,

Panagiotis

Try using the demonstator below: Clear the Trade Chart's Log after starting this with the defaults (which is no exceptions & display a green line .002 above the highs). Then click its Properties or right click the line, and change ‘Exception In Init?’ to Yes. The display will show Highs without the offset added (because the offset isn't copied from the parm until after the error point). Click ok to get rid of the parameter window so you can see the log. On my system the ‘OnException:’ messages don't appear until after all the history Calculates. Similarly, if I instead set ‘Exception In Calc’ to say 1140 (having seen there were 1150 history bars in the earlier log) then the OnException log for that doesn't appear until after all the history Calculates, ie around 1150 (visible on the screen because the default is to leave the indicator Result below Low, rather than above High, for Calculate calls that have exceptions (can make all of them fail by specifying -1 for ‘Exception In Calc’, or only all calls startinf at the 1140th by setting -1140).

Same happens on Backtesting chart, with the indicator added to an already complete backtest. 
The original place I noticed this was with an Indicator used by a Robot in Backtesting - in that case, the OnException happens after the default 200-bar history has been run through Calculate, i.e. when the (backtest) Server.Date as seen by the Robot is at the start date of the backtest.       

As an aside, why does the display line disappear if its IndicatorDataSeries is explicitly CreateDataSeries'ed? (the ‘call CreateDataSeries in Init?’ parm stuff). I was originally doing that in Initialize in another indicator in order to Print its .Count (even though the values would get overwritten later by Calculate) which - the use of name.Count in Initialize without a prior CreateDataSeries - causes an exception (that's what started all this!)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using cAlgo.API;
using cAlgo.API.Collections;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;

namespace cAlgo
{
    [Indicator(IsOverlay = true, AutoRescale = false, AccessRights = AccessRights.None)]
    public class mjsDemo : Indicator
    {
        [Parameter("Offset to add to High", DefaultValue = "0.002")]
        public double offsetParm { get; set; }
        [Parameter("Exception in Init?", DefaultValue = "false")]
        public bool doExceptionInInit { get; set; }
        [Parameter("Exception in Calc # (0=never 3=third etc or -n = all starting at n)", DefaultValue = "0")]
        public int doExceptionInCalc { get; set; }
        [Parameter("call CreateDataSeries in Init?", DefaultValue = "false")]
        public bool doCreateDataSeries { get; set; }
        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }
        double offset = 0;
        string whereAreWe = "nowhere";
        protected override void Initialize()
        {
            Print("Initialize: started");
            whereAreWe = "Initialize";  // or System.Reflection.MethodBase.GetCurrentMethod().Name;
            Print("Initialize: whereAreWe=" + whereAreWe);
            if (doExceptionInInit) {
                Print("Initialize: throw exception next");
                throw new DivideByZeroException();
            }
            Print("Initialize: copy offsetParm to offset next");
            offset = offsetParm;
            Print("Initialize: offset=" + offset);
            if (doCreateDataSeries) {
                Print("Calculate: CreateDataSeries next");
                Result = CreateDataSeries();
                Print("Calculate: CreateDataSeries done");
            }
            whereAreWe = "nowhere";
            Print("Initialize: ending");
        }
        int countCalcs = 0;
        public override void Calculate(int index)
        {
            Print("Calculate: started index=" + index + " Bars[index].OpenTime=" + Bars[index].OpenTime + " offset=" + offset);
            if (whereAreWe != "nowhere") Print("Calculate: not nowhere so OnException hasn't run yet after an error in... whereAreWe=" + whereAreWe);
            whereAreWe = "Calculate" + index;   // or System.Reflection.MethodBase.GetCurrentMethod().Name + index;
            Print("Calculate: whereAreWe=" + whereAreWe);
            countCalcs++;
            Result[index] = Bars[index].Low - offset;   // will be overwritten if no exception
            Print("Calculate: temp Result[index]=" + Result[index]);
            if (countCalcs == doExceptionInCalc || (doExceptionInCalc < 0 && countCalcs > -doExceptionInCalc)) {
                Print("Calculate: throw exception next because doExceptionInCalc=" + doExceptionInCalc + " and countCalcs=" + countCalcs);
                throw new DivideByZeroException();
            }
            Result[index] = Bars[index].High + offset;
            Print("Calculate: offset Result[index]=" + Result[index]);
            whereAreWe = "nowhere";
            Print("Calculate: ending index=" + index);
        }
        protected override void OnException(Exception e)
        {
            Print("OnException: started exception=" + e);
            Print("OnException: last set whereAreWe=" + whereAreWe);
            Print("OnException: What can we do in here if history calcs are done already?");
            Print("OnException: Better to put try/catch around Init logic?");
            Print("OnException: copy offsetParm to offset next");
            offset = offsetParm;
            Print("Initialize: offset=" + offset);
            whereAreWe = "nowhere";
            Print("OnException: ending");
        }
    }
}

martins
01 Oct 2024, 15:47 ( Updated at: 01 Oct 2024, 17:52 )

RE: Backtest - same code gets different trade results by changing end date

Hi,
I've jusy spent a few hours trying to debug why a cBot logic change altered the trade pattern, but couldn't use the feature of clicking on the yellow balance line, or a trade in the history, to redraw the upper chart with the price & trades, because (I think) you can only go back a 1000 or 2000 trades from the end (is that right?) - so I needed to shorten the backtest period in order to find the relevant part of the chart that matched the history. 

BUT, because of the 'feature' mentioned in this forum item, the equity after the early trades was completly different (due to, as I now know, it using currency conversion rates from different end dates) so the trades soon became completly different too - in other words I couldn't easily find the relevant parts of the longer before & after charts by shortening the period of backtest. The fact that the program change hadn't actually caused a difference in early trades was additionally confusing - it made me wonder whether the change was in effect or not! 

Is there any way of avoiding this effect - say, by using the beginning date for currency rates (since you'd expect trades to alter with a different start date anyway), or some specified date, or even better when the currency pair actually contains the currency rate needed (eg GBPUSD with an account in GBP) then use the proper dynamic rate?

(I suppose another way would be to make cBot decisions based on some sort of overall pips*volume profit measure, to avoid the arbitary exchange rate, but that seems overkill! [and wouldn't include commission or swap])

I suppose another, easier, way, using cTrader as it is, would be to have a cBot parameter to say if/when to stop trading earlier than the backtest end date (using the latter to set the same currency rate and the parameter to control the number of trades or early end date), so there are few enough trades to allow the chart scrolling to work from the yellow balance & the history. BTW, what is the limit for that?

(using cTrader Desktop v4.9.2)