hourly error

[jithendra945]

Im getting this error, when i am trying to run anomaly definition.

Im not getting why it is having None in it.

[sachinkbansal]

created issue #79

[jithendra945]

big query database

SELECT
TIMESTAMP_TRUNC(created_on, HOUR) as Date,
plan,
COUNT(1) as count, -- measures
FROM table
WHERE created_on >= TIMESTAMP_SUB(TIMESTAMP_TRUNC(CURRENT_TIMESTAMP(), DAY), INTERVAL 15 DAY)
GROUP BY 1,2
ORDER BY 1,2

[jithendra945]

If u are able to reproduce the issue,
Im Waiting for a solution.

[jithendra945]

I tried with another table,
To reproduce :-
First I dint kept where clause, granuality is Day
and it Succeded.

Now, I kept Where Clause limiting the data to 30days and 60days, and the same Error is Coming.

[ankitkpandey]

Hi @jithendra945 , thanks for helping us find this issue. Actually, the anomaly service was returning None object in case of insufficient data. Right now it expects individual dataframes for each dimension value to meet certain conditions. As you can see, we have currently hardcoded the df to have atleast 20 datapoints for prophet needs it to have a meaningful trend analysis.

def dataFrameEmpty(df):
    """Checks whether dataFrame has enough data for prophet"""
    if df is None:
        return True
    if df.empty:
        return True
    if df.shape[0] < 20:
        return True
    return False

[ankitkpandey]

I'm currently in the process of handling it as an error and logging it as Insufficient data, later we could maybe make the hardcoded 20 rows condition variable.
Anyway, thanks again. I'd recommend running it on data with more than 20 days of data for each dimension in the meantime

[jithendra945]

For each dimension, there should be at least 20 days of Data. Got it Ankit. Thanks. I'm facing this issue for a couple of days. If I will check before giving the query, then it shouldn't be a problem Right. ?

…

On Fri, 6 Aug 2021 at 14:45, Ankit Kumar Pandey ***@***.***> wrote: I'm currently in the process of handling it as an error and logging it as Insufficient data, later we could maybe make the hardcoded 20 rows condition variable. Anyway, thanks again. I'd recommend running it on data with more than 20 days of data for each dimension in the meantime — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#75 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT5VCGAVI4QZ4MNVRBIILVDT3OR4FANCNFSM5BTLOARQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[ankitkpandey]

Yess @jithendra945 , It should work in that case. Also I will raise a PR by tonight which handles the case for data less than 20 days and adds it to the lag instead of returning None