ctid2032775
16 Jul 2020, 11:54

RE: RE: RE: RE: RE:

prosteel1 said:

cTrader cBot Memory Manager

I’ve been testing this and seems good to manage memory.

I’ve setup a loadmorehistory routine that gets the date of the first bar on the highest timeframe and then loads more bars on lower timeframes.

I haven’t had to use the Ticks as you mentioned as yet, rather, I use bars of Tick10 etc. using a workaround I posed in this linked post below.

When I set the higher timeframe as 4 hour it went back almost a year, then the other 6 timeframes down to Tick5 loaded history back to there which resulted in 1,700,000+ bars on Tick 5 and 1 minute and yes it took a very long time to load and analyse on i7 with 32GB ram haha

cTDN Forum - Tick timeframes not part of TimeFrame Class

 

Many thanks for your suggestion!

Based on this - and as I need ask and bid prices to calculate mean prices for the automatic optimization and for the machine learning algorithm - I changed the logic to "reading bars" and calculating the prices (bid = bar.Open, ask = bid + (fix) spread) as it's also done by the internal cTrader optimization. This runs (a lot) faster and takes (extremely) less memory.

ctid2032775
10 Jul 2020, 11:29

RE: RE: RE:

prosteel1 said:

ctid2032775 said:

PanagiotisCharalampous said:

Hi Christian,

Unfortunately there is no feature for automatic optimization at the moment neither we have plans for this in the near future. If you need this, it is something you will need to develop yourself.

Best Regards,

Panagiotis 

Join us on Telegram

 

Hi,

I made it to develop a module for "automatic" optimization but have a quick question to finish it...

Right now this module reads historic data from a text file. This file is created by running a (small) cBot in backtesting mode that just gets the data (OnTick/OnBar) and writes it into the file. This requires some manual steps and to not loose any data (i. e. missing some ticks/bars) this can only be done during weekends!

I would like to fetch the data (OHLC, Ask, Bid) for a specific time frame (e. g. last 2 years) every time the cBot is started (OnStart) and run the optimization with this historic data - is there a way to realize this (maybe with FIX API or Open API)? Can you provide some code snippets and/or examples where this was already done?

Many thanks for your support.

BR,
Christian

I use a for loop OnStart to cycle through each bar from the current bar count back to count = 1. There is also a newish  'Bars.LoadMoreHistory' command to get a specified number of more bars. OnTick I check for a new Bar and I run it again but only go back 2 bars.  I think this is what you are looking for as the data available to Backtesting lags a day, whereas this accesses the live data :) Current bar data can be accessed using Bars.LastBar.Low etc and current bid/ask using Symbol.Bid and Symbol.Ask.

I run OnTick so I can check for the price reaching take profit 1 price to close half the position and to move the stoploss to entry and checking that the Symbol.Ask is not above the short stoploss when placing an order (as that prevents the stoploss from being set), etc etc.

OnStart()

Bars series = MarketData.GetBars(frame);
int i = series.Count - 1;

BarLast = i - 2;
for (int ii = i - 2; ii >= 1; ii--)
{

      // Write OHLC Data

}

OnTick()

// Check for a new bar on specific timeframe

Bars series = MarketData.GetBars(frame);
if (series.Count - 1 > BarLast)
{

     for (int ii = i - 2; ii >= x; ii--)
     {

           // Write OHLC Data

     }

}

 

In regards to your question above "Part of the bot is a machine learning algorithm that is already existing in C# - I guess it's not a problem to include this in a cBot, right?", I am quite a noob but when I investigated this I think the answer was C# was not suitable so it would need to use another language suited to machine learning and so going through the FIX API would be required. Possibly making an API to connect the ML code to the C# code?

First of all many thanks for your reply!

As there are no problems but a lot of challenges ;-) - my challenge is to get the historical ask and bid prices for about one year for my automatic optimization algorithm (please keep in mind that I cannot use the built-in optimization for this part of the cBot; see above). Unfortunately the bars do not contain this information...

But, I found a way to get the information while cBot starts - by fetching the ticks (Ticks ticks = MarketData.GetTicks(); ... i = ticks.LoadMoreHistory();) I get the required information but this process is extremely memory consuming. This leads to slowing down the system even if the garbage collector removes the array(s) from the memory!

Any idea to get the data without such a high memory consumption is welcome! :-)

The machine learning part is finished - I am using the C4.5 algorithm in the accord.net framework to create a decision tree based on some (calculated) attributes. The prediction of the tree with current attributes is then used to calculate the size of the opened orders (i. e. money management)...

ctid2032775
26 Jun 2020, 10:25

RE:

PanagiotisCharalampous said:

Hi Christian,

Unfortunately there is no feature for automatic optimization at the moment neither we have plans for this in the near future. If you need this, it is something you will need to develop yourself.

Best Regards,

Panagiotis 

Join us on Telegram

 

Hi,

I made it to develop a module for "automatic" optimization but have a quick question to finish it...

Right now this module reads historic data from a text file. This file is created by running a (small) cBot in backtesting mode that just gets the data (OnTick/OnBar) and writes it into the file. This requires some manual steps and to not loose any data (i. e. missing some ticks/bars) this can only be done during weekends!

I would like to fetch the data (OHLC, Ask, Bid) for a specific time frame (e. g. last 2 years) every time the cBot is started (OnStart) and run the optimization with this historic data - is there a way to realize this (maybe with FIX API or Open API)? Can you provide some code snippets and/or examples where this was already done?

Many thanks for your support.

BR,
Christian