/
TODO.txt
94 lines (78 loc) · 4.41 KB
/
TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
[X] Finish tasks from previous meeting (import command-line switches).
[X] Fix bug in energy export.
[ ] Verify bug is fixed.
[X] Use end time, not start time.
Egauge: [X] Store raw data for energy for appliances, not 1 or 0.
[X] Create separate sources for each appliance and whole house.
[X] Reading is average power during interval, not meter reading.
[X] Store average power in powerConsumed, also create "odometer" reading.
[X] Determine which accounts have multiple MTUs associated with them.
[X] See if all non-monotonic data is from multiple MTUs.
[X] Provide in summary the number of violations of each validation rule occurred.
[X] Work on exporter that exports multiple sources at same (computed) timestamp.
[X] Import the sample TED and Hobo files when received from Eileen.
[X] Import the sample Egauge files.
[X] Export CSV file containing multiple MTU IDs and data that is non-monotonically increasing in HneiCsvImporter.
[X] Start on CSV export
[-] Importer should take option that specifies file type (FC, TED, Hobo, etc.)
[X] Auto infer file type from file name or file contents.
To accomplish this objective, turn all main methods in the importers into methods that is called by the one main method.
[X] Start writing to CSV file as soon as sensor data is grabbed from server instead of putting them in lists.
[ ] Finish printStats methods in importers.
[X] Count of number of new sources.
[X] Count of number of existing sources.
[X] Count of new data and pre-existing data.
[X] Average number of data items imported per second.
[X] The name of the file that was processed.
[X] How many entries were processed.
[X] How many entries were not valid.
[X] The percentage of invalid entries.
[X] The time interval of the run (first entry date and last entry date).
[X] The total elapsed time for the run.
[X] Print summary statistics to standard output and log file.
[ ] Individual entry feedback is a "verbose" setting.
What is the best possible design relationship between TabularFileSensor and HNEITabularFileSensor so that:
[ ] they do not produce any replicated code
[ ] they support the next person who comes along and wants a tabular file sensor.
Work with Robert to create the right structure.
[X] Use Ivy to download opencsv.
[X] Convert Eclipse build library references to project-relative locations.
Validator
Please create a flexible, "smart import" facility that will subject each entry record to a set of validation functions, for example:
[X] Entry is numeric.
[X] Entry has no missing values.
[X] Entry is not lower than last entry value.
[X] Entry is less than 20kW * time since last entry's value.
[X] Apply validation to MTU ID, Port, Meter Type, etc. and record errors in log file.
[X] These validation functions may change, and there may be more of them over time, so define a Java interface called "Validator" that
has a boolean function called "validate" that accepts an entry and returns true if the entry passes that particular validation function.
Then the system can implement as many Validator classes as we want, and we just make an instance of each of them, store them in a List,
and apply each of them to each entry.
[X] If any return false, then print out (or log) the problematic entry along with which validator(s) failed. There needs to be a getExplanation()
method that returns a String for each Validator class so the system can describe what went wrong in easily understandable terms.
Package structure:
[X] org.wattdepot.hnei.csvimport.*
[X] Contains all classes to read in the CSV files and send to a server.
[X] Validator interface
[X] NumericValue implements Validator
[X] NoMissingFields implements Validator
[X] MonotonicallyIncreasing implements Validator
HNEIImporter
[ ] has the main method.
[ ] may need some Logging classes.
[ ] Set a property for each source called samplingInterval to be either 'daily' or 'hourly' based upon which it is.
org.wattdepot.hnei.export.*
[ ] Contains classes to read data from a server and output data in a format that they like.
HNEIExporter
[ ] has a main method.
[ ] takes arguments:
[ ] start day
[ ] end day
[ ] sampling interval [daily | hourly]
[ ] source(s)
org.wattdepot.hnei.analysis.*
(Future work)
Google Project Hosting:
[ ] Convert to wattdepot-hnei
[ ] Import into SVN
- Be careful not to import bin, build, or lib directories into SVN.