Feature/fi reindex integration

[FineAndDandy]

Added support for batch processing, fixed bugs with tf processing offsets for multivalued fields, and extended unit test coverage

[MiguelRicardos]

Is there a better way than catching one exception just to throw another?

[jschmidt10]

I am okay with this pattern since we're converting a checked exception into an unchecked one. This is us saying that if the table fails, there is no reasonable recovery. The conversion to an unchecked means you don't have to string throws TableNotFoundException up your entire call stack.

One thing I would do is add the originating exception to the causedBy of the one we're throwing for stacktrace traceability.

[MiguelRicardos]

sounds reasonable to me.

[MiguelRicardos]

I wonder if it would be valuable to have a getKeyType(cf) that returns a string to assign to keyType and then do switch statements on this like:

switch (keyType) {
    case "d":
        processDKey(key, value, context);
        break;
    case "tf":
        processTFKey(key, value, context);
        break;
    case "fi":
        processFIKey(key, value, context);
        break;
    default:
        processEventKey(key, value, context);
        break;
}

or even better if the case of the d, tf, and fi keys are just an integer we can do the case statement on the length of the bytes in the cf:

int keyTypeCode = cf.length()

switch (keyTypeCode) {
    case 1:
        processDKey(key, value, context);
        break;
    case 2:
        processTFKey(key, value, context);
        break;
    case 4:
        //possible extra check on starting bytes
        processFIKey(key, value, context);
        break;
    default:
        processEventKey(key, value, context);
        break;

It could eliminate the need for the isKey* methods and might be more efficient.

[MiguelRicardos]

Consider changing to:

if (key.isDeleted()) {
    if (!this.propagateDeletes) {
        incrementCounter("deletes", "skipped");
        return;
    }
    incrementCounter("deletes", "propagated");
}

[MiguelRicardos]

You might want to move:

(!this.reprocessEvents || helper.isIndexOnlyField(this.normalizedFieldName))

to a single if statment tha encompasses the (helper.isIndexedField(this.normalizedFieldName) and (helper.isReverseIndexedField(this.normalizedFieldName) checks.

[MiguelRicardos]

Might want to use java.time instead of Calendar.

private long floorTimestamp(long timestamp) {
    if (!this.floorTimestamps) {
        return timestamp;
    }

    return java.time.Instant.ofEpochMilli(timestamp)
            .atZone(java.time.ZoneId.systemDefault())  // Adjust the zone as necessary
            .toLocalDate()
            .atStartOfDay(java.time.ZoneId.systemDefault())
            .toInstant()
            .toEpochMilli();
}