Poor performance retrieving multiple time-frames and symbols

Created at 27 Mar 2016, 09:36
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
MA

march.tim

Joined 08.03.2016

Poor performance retrieving multiple time-frames and symbols
27 Mar 2016, 09:36


Hey Guys,

I'm building a version of the Hector DeVille 3SMA filter dashboard for cTrader. The purpose is to retrieve data for multiple time-frames across multiple symbols and indicate whether different simple moving averages are aligned correctly to indicate a trend.

The indicator works, but is TERRIBLY slow to load. I've moved the main code up to the Initialise class so it's only running once at startup, but still takes 3-4 minutes to retrieve data for 10 pairs. Ideally I'd like to monitor up to 32 symbols but it's not viable with the current performance.

If anyone could suggest possible options for performance enhancements I'd be very grateful...

using System;
using cAlgo.API;
using cAlgo.API.Internals;
using cAlgo.API.Indicators;
using cAlgo.Indicators;

namespace cAlgo
{
    [Indicator(IsOverlay = false, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class NewIndicator05 : Indicator
    {
        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }

        [Parameter("Symbols", DefaultValue = "AUDCAD,AUDCHF,AUDJPY")]
        public string symbols { get; set; }

        [Parameter("SMA 1", DefaultValue = 30, MinValue = 1)]
        public int per1 { get; set; }

        [Parameter("SMA 2", DefaultValue = 50, MinValue = 1)]
        public int per2 { get; set; }

        [Parameter("SMA 3", DefaultValue = 100, MinValue = 1)]
        public int per3 { get; set; }

        private int[] per;

        private TimeFrame[] tf;
        private MovingAverage[,] sma;
        private MarketSeries[] series;

        // private Symbol[] sym;


        protected override void Initialize()
        {
            // Initialize and create nested indicators
            tf = new TimeFrame[4];
            tf[0] = TimeFrame.Hour;
            tf[1] = TimeFrame.Hour4;
            tf[2] = TimeFrame.Daily;
            tf[3] = TimeFrame.Weekly;

            per = new int[3];
            per[0] = per1;
            per[1] = per2;
            per[2] = per3;


            string[] separators = 
            {
                ","
            };
            string[] symbol = symbols.Split(separators, StringSplitOptions.RemoveEmptyEntries);


            // Ghetto nasty way of counting symbols. Find a nicer way to do this...
            int symcount = 0;
            foreach (var sym in symbol)
            {
                symcount++;
            }

            int ii = 0;
            foreach (var sym in symbol)
            {
                // Get market data for each timeframe for the symbol...
                series = new MarketSeries[4];
                for (int i = 0; i < 4; i++)
                {
                    series[i] = MarketData.GetSeries(sym, tf[i]);
                }

                // Get moving agerages for each timeframe...
                sma = new MovingAverage[3, 4];
                for (int i = 0; i < 3; i++)
                {
                    for (int j = 0; j < 4; j++)
                    {
                        sma[i, j] = Indicators.MovingAverage(series[j].Close, per[i], MovingAverageType.Simple);
                    }
                }

                // Print the symbol name...
                string str = "";
                str = xySpace(0, ii) + sym;
                ChartObjects.DrawText("sym_" + sym, str, StaticPosition.TopLeft, Colors.Green);


                for (int i = 0; i < 4; i++)
                {
                    if ((sma[0, i].Result.LastValue) > (sma[1, i].Result.LastValue) && (sma[1, i].Result.LastValue) > (sma[2, i].Result.LastValue))
                    {
                        str = xySpace((i + 2), ii) + "U";
                        ChartObjects.DrawText("trend_u_" + sym + i, str, StaticPosition.TopLeft, Colors.Green);
                    }
                    else if ((sma[0, i].Result.LastValue) < (sma[1, i].Result.LastValue) && (sma[1, i].Result.LastValue) < (sma[2, i].Result.LastValue))
                    {
                        str = xySpace((i + 2), ii) + "D";
                        ChartObjects.DrawText("trend_d_" + sym + i, str, StaticPosition.TopLeft, Colors.Red);
                    }
                    else
                    {
                        str = xySpace((i + 2), ii) + "N";
                        ChartObjects.DrawText("trend_n_" + sym + i, str, StaticPosition.TopLeft, Colors.Blue);
                    }
                }

                ii++;
                // }
            }
        }
        // END Initialise


        public override void Calculate(int index)
        {
            // Calculate value at specified index
            // Result[index] = ...

            // DATA FORMAT IS...
            // sma[PERIOD, TIMEFRAME]
            // xySpace[COLUMN, ROW]

            if (!IsLastBar)
                return;
        }
        // END Calculate


        private string xySpace(int x, int y)
        {
            if (x < 0 || y < 0)
                return ("");
            string str = "";
            for (int i = 0; i < y; i++)
                str += "\n";
            for (int i = 0; i < x; i++)
                str += "\t";
            return (str);
        }
        // END xySpace


    }
    // END Indicator


}
// END cAlgo

 

 


@march.tim
Replies

march.tim
27 Mar 2016, 10:11

OK... I did some more digging and came across this thread /forum/indicator-support/2759.

I've implemented the same timing code in the indicator and measured it's performance on a call-by-call basis. It looks like it's the same issue and the MarketData.GetSeries call takes anywhere up to 6.7 seconds to complete for each time frame. The quickest calls I've observed are around 1.5 seconds.

 

                // Get market data for each timeframe for the symbol...
                series = new MarketSeries[4];
                for (int i = 0; i < 4; i++)
                {
                    var start = DateTime.Now;
                    series[i] = MarketData.GetSeries(sym, tf[i]);
                    var end = DateTime.Now;
                    Print((end - start).TotalMilliseconds + "\t" + sym + "\t" + tf[i]);
                }

There's a quote in the thread above from SpotWare saying "Because of technical reasons GetSeries returns value when next tick comes to the platform. We plan to change this behavior in the future." I'm currently testing in off-market conditions, perhaps we'll see better performance when all the markets are open during the week.

Does anyone know if there's a way to limit the number of bars MarketData.GetSeries returns to speed it up? For example, if the longest SMA I'm trying to calculate is 100 bars, could I only retrieve that many from the server somehow?


@march.tim

Spotware
29 Mar 2016, 12:29

Dear Trader,

Currently there is no parameter to specify the amount of the historical data to retrieve from the GetSeries() method. We will consider providing it in the future.

It's also in our plans to further optimize the performance on all cTrader platforms in the future. Stay tuned. Additionally, you can post your ideas/suggestions to http://vote.spotware.com/


@Spotware