MiddleManager errors while executing the peon task

[soullkk]

MiddleManager errors while executing the peon task

Affected Version

28.0.1

Description

Please include as much detailed information about the problem as possible.

• Cluster size
  3-node cluster
• Configurations in use
  peon task context :

{"druid.indexer.fork.property.druid.processing.buffer.sizeBytes":33554432,"druid.indexer.fork.property.druid.processing.numThreads":3,"druid.indexer.runner.javaOpts":"-server -Xms256m -Xmx768m -XX:MaxDirectMemorySize=1g"}

• Steps to reproduce the problem
  When executing the Poen task, set druid-indexer.fork.property.druid.processing.buffer.sizeBytes to an integer

• The error message or stack traces encountered. Providing more context, such as nearby log messages or even entire logs, can be helpful.

2024-04-11 20:20:19,572 INFO  [forking-task-runner-35][][org.apache.druid.indexing.overlord.ForkingTaskRunner] Exception caught during execution
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
        at org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:324) ~[druid-indexing-service-28.0.1-htrunk7.jar:?]
        at org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:176) ~[druid-indexing-service-28.0.1-htrunk7.jar:?]
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131) ~[guava-31.1-jre.jar:?]
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74) ~[guava-31.1-jre.jar:?]
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82) ~[guava-31.1-jre.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_402]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_402]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_402]

• Any debugging that you have already done
  When the return value of task.getContextValue(propName) is an Integer instead of an int, "public CommandListBuilder addSystemProperty(String property, String value)" will be called and an error will occur: java. lang. ClassCastException: Java. lang. Integer cannot be cast to Java. lang. String.
  I think it is necessary to change "public CommandListBuilder addSystemProperty(String property, int value)" to "public CommandListBuilder addSystemProperty(String property, Object value)" , and I will try to fix this bug soon

public class ForkingTaskRunner
                        final Map<String, Object> context = task.getContext();
                        if (context != null) {
                          for (String propName : context.keySet()) {
                            if (propName.startsWith(CHILD_PROPERTY_PREFIX)) {
                              command.addSystemProperty(
                                  propName.substring(CHILD_PROPERTY_PREFIX.length()),
                                  task.getContextValue(propName)
                              );
                            }
                          }
                        }
}

public static class CommandListBuilder
  {
    ArrayList<String> commandList = new ArrayList<>();

    public CommandListBuilder add(String arg)
    {
      commandList.add(arg);
      return this;
    }

    public CommandListBuilder addSystemProperty(String property, int value)
    {
      return addSystemProperty(property, String.valueOf(value));
    }

    public CommandListBuilder addSystemProperty(String property, long value)
    {
      return addSystemProperty(property, String.valueOf(value));
    }

    public CommandListBuilder addSystemProperty(String property, boolean value)
    {
      return addSystemProperty(property, String.valueOf(value));
    }

    public CommandListBuilder addSystemProperty(String property, String value)
    {
      return add(StringUtils.format("-D%s=%s", property, value));
    }

    public CommandListBuilder addAll(Iterable<String> args)
    {
      for (String arg : args) {
        add(arg);
      }
      return this;
    }

    public ArrayList<String> getCommandList()
    {
      return commandList;
    }

  }

[soullkk]

    private static final Integer num = 1;

    public static void main(String[] args) {
        add(get());
    }

    public static void add(int obj) {
        System.out.println("int");
    }

    public static void add(String obj) {
        System.out.println("string");
    }

    public static void add(long obj) {
        System.out.println("long");
    }

    public static <T> T get() {
        return (T) num;
    }

result is :
Exception in thread "main" java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at lkk.util.Main.main(Main.java:136)

[kfaraz]

@soullkk , does it work if you pass the sizeBytes as a String, i.e. "33554432" instead of 33554432?

[soullkk]

@soullkk , does it work if you pass the sizeBytes as a String, i.e. "33554432" instead of 33554432?

Yes, using "33554432" is not a problem, but this value is an int instead of a string, and it does not match the guidance provided by the Druid. Additionally, in lower versions of the Druid, such as 24.0.1, using 33554432 is not a problem. So I think this is a bug and an incompatible change point in higher versions of the Druid.