single positional indexer is out-of-bounds

[PhilippeDo]

I have now this error when I tried to use prophet

{"d2b9be2a-6d27-4e61-a577-3d99892e94e0": {"dimVal": null, "error": "{\"message\": \"single positional indexer is out-of-bounds\", \"stackTrace\": \"Traceback (most recent call last):\\n File \\\"/code/ops/tasks/anomalyDetection.py\\\", line 56, in anomalyService\\n result = detect(df, granularity, detectionRuleType, anomalyDef)\\n File \\\"/code/ops/tasks/anomalyDetection.py\\\", line 29, in detect\\n return prophetDetect(df, granularity)\\n File \\\"/code/ops/tasks/detectionTypes/prophet.py\\\", line 47, in prophetDetect\\n lastISO = df.iloc[-1][\\\"ds\\\"]\\n File \\\"/opt/venv/lib/python3.7/site-packages/pandas/core/indexing.py\\\", line 895, in __getitem__\\n return self._getitem_axis(maybe_callable, axis=axis)\\n File \\\"/opt/venv/lib/python3.7/site-packages/pandas/core/indexing.py\\\", line 1501, in _getitem_axis\\n self._validate_integer(key, axis)\\n File \\\"/opt/venv/lib/python3.7/site-packages/pandas/core/indexing.py\\\", line 1444, in _validate_integer\\n raise IndexError(\\\"single positional indexer is out-of-bounds\\\")\\nIndexError: single positional indexer is out-of-bounds\\n\"}", "success": false}}

I used the following in DATASET section

select DATE_FORMAT(date, '%Y-%m-%d %H') as Date,
pagelt from pageloadtime2

and then when I run, the dataset looks liks this. It is a minimal dataset withou dimension, with only a measure and a timestamp

[ankitkpandey]

Hi @PhilippeDo could you elaborate a bit on the exact anomaly definition you created and also the dataframe output of the dataset query. It will help me to recreate the issue at my end.
Thank you

[PhilippeDo]

Hi @ankitkpandey,

You will find mys anomaly definition here

thanks

[sachinkbansal]

@PhilippeDo this is a bug. have created an issue #179

[PhilippeDo]

Hey @PhilippeDo So the error is a pandas indexing error and is caused by the following line, lastISO = df.iloc[-1]["ds"] This is happening because of the previous line df = df[df["ds"] < todayISO] which basically removes all data from the current date, because of an assumption we had while building the product. The assumption was that all daily data of the current date will be incomplete and will become an anomaly automatically. A false anomaly. So we intentionally ignore the data for today's date.

I'm guessing your dataset's granularity is hourly and there we should only remove the data from the latest hour. Let me get back to you after discussing these changes with the team.

Would love to know your ideas about this assumption as well.

Hi @ankitkpandey

Thanks for your investigation and efforts. Regarding your asumption, I really don't understand the rationales here to consider that " all daily data of the current date will be incomplete" neither "we should only remove the data from the latest hour." All the data that I send to Cueobserve are good data and are not corrupted neither incomplete. So in my point of view all the data should be taken into account.

Best Regards

[sachinkbansal]

@PhilippeDo If you notice the sample dataset SQLs in CueObserve (sample below), these are group by queries that bucket the data into time buckets. The data can be limited by a WHERE clause on the timestamp column that fetches data until now(). Since this would result in the last time bucket being incomplete almost always, we ignore the last data point when generating data frames for anomaly detection. Else the last point, being partial, might be detected as an anomaly.

SELECT
DATE_FORMAT(CreatedTS, '%Y-%m-%d %H') as OrderDate,
City, State, -- dimensions
COUNT(1) as Orders, SUM(IFNULL(Order_Amount,0)) as OrderAmount -- measures
FROM ORDERS
WHERE CreatedTS >= DATE_SUB(DATE_FORMAT(now(), '%Y-%m-%d %H'), INTERVAL 21 DAY) -- limit historical data to use for forecasting
GROUP BY 1, 2, 3
ORDER BY 1

For scenarios where the last data point is complete, this is a limitation. Until we handle this scenario, as a workaround, you might want to insert an additional data point in the dataset.

[PhilippeDo]

Thanks for the explanation @sachinkbansal. I succeed to have some result in Cueobserve with more than 24 records. But I didn't manage the buckets with GROUP BY in my sql query, Is it mandatory to GROUP by the records ? and what happen if not ? I notice also that in anomaly detection the data dysplayed stop at last day at 11:00 pm while we have more recent records in database. Does it means the buvket is a one day buvket and CueObserve dysplay only full buckets ?

Thanks
Best Regards

[sachinkbansal]

@PhilippeDo GROUP BY is not mandatory. It's a way to reduce the dataset size since the query gets executed on the source database or datawarehouse. The data points stop at 11pm, probably because we are ignoring the entire date, instead of just ignoring the last point. There's an existing issue #179 that should fix this.