Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interpolated_state_timeline producing incorrect results #793

Open
KamalAman opened this issue Feb 28, 2024 · 1 comment
Open

interpolated_state_timeline producing incorrect results #793

KamalAman opened this issue Feb 28, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@KamalAman
Copy link

Relevant system information:

  • OS: OSX
  • PostgreSQL version (output of SELECT version();): 15.4-ts2.12.0
  • TimescaleDB Toolkit version (output of \dx timescaledb_toolkit in psql): 1.17
  • Installation method: docker

Describe the bug
Events are getting doubled up and not merging together properly when they span buckets, this is a similar query as what is documented here, just with a 60 minute window. The second OK record here passes the 3rd, creating extra records with invalid timeframes

To Reproduce

SET TIME ZONE 'UTC';
drop table states_test_k;
CREATE TABLE states_test_k(ts TIMESTAMPTZ, state TEXT);
INSERT INTO states_test_k VALUES
    ('2020-01-01 01:00:00+00', 'OK'),
    ('2020-01-01 01:00:01+00', 'START'),
    ('2020-01-01 01:00:03+00', 'OK'),
    ('2020-01-01 01:00:30+00', 'START'),
    ('2020-01-01 01:00:59+00', 'OK'),
    ('2020-01-01 01:01:30+00', 'START'),
    ('2020-01-01 01:01:59+00', 'OK'),
    ('2020-01-01 01:02:59+00', 'START'),
    ('2020-01-01 01:03:59+00', 'OK'),
    ('2020-01-01 02:01:59+00', 'START');

SELECT
    bucket,
    (interpolated_state_timeline(
        summary,
        bucket,
        '60 min',
        LAG(summary) OVER (ORDER by bucket)
    )).*
FROM (
    SELECT
        time_bucket('1 min'::interval, ts) AS bucket,
        state_agg(ts, state) AS summary
    FROM states_test_k
    GROUP BY time_bucket('1 min'::interval, ts)
) t
order by bucket;

Expected behavior
The results should be

OK	2020-01-01 01:00:00+00	2020-01-01 01:00:01+00
START	2020-01-01 01:00:01+00	2020-01-01 01:00:03+00
OK	2020-01-01 01:00:03+00	2020-01-01 01:00:30+00
START	2020-01-01 01:00:30+00	2020-01-01 01:00:59+00
OK	2020-01-01 01:00:59+00	2020-01-01 01:01:30+00
START	2020-01-01 01:01:30+00	2020-01-01 01:01:59+00
OK	2020-01-01 01:01:59+00	2020-01-01 01:02:59+00
START	2020-01-01 01:02:59+00	2020-01-01 01:03:59+00
OK	2020-01-01 01:03:59+00	2020-01-01 02:01:59+00
START	2020-01-01 02:01:59+00	2020-01-01 02:01:59+00

Actual behavior
The results are incorrect creating invalid time ranges that pass "overlapping" time ranges. All of the double-up are correct OK-OK and START-START

2020-01-01 01:00:00+00	OK	2020-01-01 01:00:00+00	2020-01-01 01:00:01+00
2020-01-01 01:00:00+00	START	2020-01-01 01:00:01+00	2020-01-01 01:00:03+00
2020-01-01 01:00:00+00	OK	2020-01-01 01:00:03+00	2020-01-01 01:00:30+00
2020-01-01 01:00:00+00	START	2020-01-01 01:00:30+00	2020-01-01 01:00:59+00
2020-01-01 01:00:00+00	OK	2020-01-01 01:00:59+00	2020-01-01 02:00:00+00
2020-01-01 01:01:00+00	OK	2020-01-01 01:01:00+00	2020-01-01 01:01:30+00
2020-01-01 01:01:00+00	START	2020-01-01 01:01:30+00	2020-01-01 01:01:59+00
2020-01-01 01:01:00+00	OK	2020-01-01 01:01:59+00	2020-01-01 02:01:00+00
2020-01-01 01:02:00+00	OK	2020-01-01 01:02:00+00	2020-01-01 01:02:59+00
2020-01-01 01:02:00+00	START	2020-01-01 01:02:59+00	2020-01-01 02:02:00+00
2020-01-01 01:03:00+00	START	2020-01-01 01:03:00+00	2020-01-01 01:03:59+00
2020-01-01 01:03:00+00	OK	2020-01-01 01:03:59+00	2020-01-01 02:03:00+00
2020-01-01 02:01:00+00	OK	2020-01-01 02:01:00+00	2020-01-01 02:01:59+00
2020-01-01 02:01:00+00	START	2020-01-01 02:01:59+00	2020-01-01 03:01:00+00

Screenshots
image

Additional context
interpolated_state_timeline has the potential to be a very powerful feature. However it is producing unexpected results and I am wary of its actual performance on large datasets even if was functioning as expected.

@KamalAman KamalAman added the bug Something isn't working label Feb 28, 2024
@jamessewell
Copy link

Just adding a live link https://dbfiddle.uk/L4WfMlIK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants