Datawave 825 part2 - Modification to the MetadataHelper class required to datawave issue #825 #5
Conversation
jzgithub1
force-pushed
the
datawave-825-part2
branch
from
5c6994e
to
4ae6957
Compare
ivakegg
force-pushed
the
datawave-825-part2
branch
from
b368e9b
to
5712282
Compare
…h is very compact
…when I want it to
ivakegg
force-pushed
the
datawave-825-part2
branch
from
5712282
to
cd88bd6
Compare
| @@ -209,6 +209,68 @@ public Boolean isIndexed(Text colf, Entry<String,Entry<String,Set<String>>> key) | |||
| return result; | |||
| } | |||
|
|
|||
| /** | |||
| * Method that fetches fetches a DataFrequencyValue for an indexed rowid. | |||
|
|
||
| Range range = new Range(upCaseFieldName); | ||
| scanner.setRange(range); | ||
| scanner.fetchColumnFamily(colf); |
There was a problem hiding this comment.
why pass in the colf when we know it should be "i" ?
| private static final Logger log = LoggerFactory.getLogger(IndexedDatesValue.class); | ||
|
|
||
| private YearMonthDay startDay; | ||
| private TreeSet<YearMonthDay> indexedDatesSet = new TreeSet<>(); |
There was a problem hiding this comment.
I thought we were going to internalize this as a set of ranges instead of distinct YearMonthDays. Doing it this way will create a ton of objects. Also, we need to figure out how to deal with that fact that data is indexed already but has no bits set. So either we need a way of updating legacy metadata to presume that it has been indexed since the beginning of time, or we assume that we have indexed data from the beginning of time until the first bit set.
There was a problem hiding this comment.
I am thinking the storage of a bitset is excellent, we simply should transform that into a set of FieldIndexHoles instead of a large set of YearMonthDay objects directly.
jzgithub1
force-pushed
the
datawave-825-part2
branch
from
67c2c0a
to
33666c8
Compare
Updates to the MetadataHelper.isIndexed and getIndexFields function required for use of the new IndexColumnIterator in Datawave