Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added QueryLogIterator class #2322

Draft
wants to merge 15 commits into
base: integration
Choose a base branch
from

Conversation

SethSmucker
Copy link

Added a QueryLogIterator class. This iterator logs the QueryID and method name called before and after each method is executed by the following iterator.

protected static final Logger log = Logger.getLogger(QueryLogIterator.class);

protected SortedKeyValueIterator<Key,Value> source;
protected SortedKeyValueIterator<Key,Value> sourceForDeepCopies;
Copy link
Collaborator

@ivakegg ivakegg Mar 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sourceForDeepCopies not needed, not used. Same goes for initKeySource, seekKeySource, myEnvironment, and queryOptions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we should be doing is attempting to pull the query id from the queryOptions for use in log messages.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be better than stoiring the entire map of options.

protected NestedIterator<Key> initKeySource, seekKeySource;
protected IteratorEnvironment myEnvironment;

protected Range range;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for Range, Key, Value, and yield

public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> options, IteratorEnvironment env) throws IOException {

try{
if (log.isInfoEnabled()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a "start" method that takes in the name of the method . That would make the code a lot simpler. Also this method could be the one that sets the thread name.

this.source = source;
}
finally{
if (log.isInfoEnabled()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps an "end" method akin to the start method. Also this method could be the one that resets the thread name to its original form.

try {
if(log.isInfoEnabled()){
log.info("QueryLogIterator: deepCopy() Started QueryID: " + this.queryOptions.get(QUERY_ID));
newQLI = new QueryLogIterator(this, this.myEnvironment);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move outside the if clause here.

try {
if(log.isInfoEnabled()){
log.info("QueryLogIterator: seek() Started QueryID: " + this.queryOptions.get(QUERY_ID));
this.source.seek(range, collection, b);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move outside of the if clause

@@ -467,9 +472,14 @@ protected CloseableIterable<QueryData> process(ScannerFactory scannerFactory, Me
cfg = getQueryIterator(metadataHelper, config, settings, "", false, false);
}
configureIterator(config, cfg, newQueryString, isFullTable);

while (null == logCfg) {
logCfg = getQueryIterator(metadataHelper, config, settings, "", false, false);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be getQueryLogIterator

@@ -437,6 +439,7 @@ public class QueryOptions implements OptionDescriber {

private Map<String,Long> fieldCounts;
private Map<String,Long> termCounts;
private boolean activeTserverLogging = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets make this "true" by default

@@ -469,6 +469,8 @@ public class ShardQueryConfiguration extends GenericQueryConfiguration implement
*/
private boolean useTermCounts = false;

private boolean isTserverLoggingActive = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets make this true by default

@lbschanno lbschanno linked an issue Apr 11, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

QueryIterators/ScanIterators should log query id first
2 participants