Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

train on normal data? #30

Open
JokerDu opened this issue Jun 22, 2019 · 1 comment
Open

train on normal data? #30

JokerDu opened this issue Jun 22, 2019 · 1 comment

Comments

@JokerDu
Copy link

JokerDu commented Jun 22, 2019

Hi, thank you for sharing the anomaly detection code. I notice that author Pankaj Malhotra uses only the normal sequences for training in paper LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection. But I found you use not only normal sequences but anomaly sequences for training.
in code 0_download_dataset.py, you tag label of anomaly data as 1, label of normal data as 0.

tokens.append(1) if 150 < i < 250 or   \
                            5970 < i < 6050 or \
                            8500 < i < 8650 or \
                            8750 < i < 8890 or \
                            10000 < i < 10200 or \
                            14700 < i < 14800 \
                          else tokens.append(0)

train data is labeled_data[:13104], which including normal sequences and anomaly sequences, could you explain that, thank you so much~

@chickenbestlover
Copy link
Owner

As you said, I included anomaly points in the trainset in my code. Ideally, the trainset would contain only normal points, but in the nyc_taxi dataset, it was almost impossible to construct a trainset with only normal points because the anomaly points were included in several parts. That's why we had to include anomaly points in the trainset. But because most of the trainsets are normal points, this was not a big problem for learning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants