关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 #89

JiaRG · 2023-03-28T14:28:40Z

问题描述

目前源码中假如检测到类名和方法名中存在 $ 符号就会跳过注入

MyPerf4J/MyPerf4J-Base/src/main/java/cn/myperf4j/base/config/ProfilingFilter.java

Lines 87 to 97 in 4c0a98d

    
           public static boolean isNotNeedInject(String innerClassName) { 
        
               if (innerClassName == null) { 
        
                   return false; 
        
               } 
        
               if (innerClassName.indexOf('$') >= 0) { 
        
                   return true; 
        
               } 
        
               return isMatch(innerClassName, excludePackagePrefix, excludePackageExp); 
        
           }

MyPerf4J/MyPerf4J-Base/src/main/java/cn/myperf4j/base/config/ProfilingFilter.java

Lines 182 to 190 in 4c0a98d

    
           private static boolean isSpecialMethod(String methodName) { 
        
               int symbolIndex = methodName.indexOf('$'); 
        
               if (symbolIndex < 0) { 
        
                   return false; 
        
               } 
        
               int leftParenIndex = methodName.indexOf('('); 
        
               return leftParenIndex < 0 || symbolIndex < leftParenIndex; 
        
           }

JiaRG · 2023-03-28T14:36:17Z

cn.myperf4j.asm.aop.ProfilingTransformer#getBytes

  private byte[] getBytes(ClassLoader loader,
                          String className,
                          byte[] classFileBuffer) {
      ClassReader cr = new ClassReader(classFileBuffer);
      ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_MAXS) {
          @Override
          protected ClassLoader getClassLoader() {
              return Thread.currentThread().getContextClassLoader();
          }
      };
      ClassVisitor cv = new ProfilingClassAdapter(cw, className);
      cr.accept(cv, ClassReader.EXPAND_FRAMES);
      return cw.toByteArray();
  }

采用线程本地类加载器后，自测在 tomcat 8.5 上可以监控 tomcat 本身和内部类

此处的改动就限制死了加载的class文件必须是使用 jdk 1.7 以上版本编译过的

see: https://www.java67.com/2020/04/javalangverifyerror-expecting-stack-map.html

LinShunKang · 2023-03-29T14:25:39Z

现在 MyPerf4J 只支持 1.7 及其之上的版本，这个改动的影响面比较大，你测试了哪些场景？

这块代码逻辑兼容了 PlainJava、Tomcat、SpringBoot、Flink 等场景，如果你的解决方案同样适用这些场景的话，欢迎提交 PR

JiaRG · 2023-04-01T13:23:12Z

前三者没问题，Flink 没环境测。。。

LinShunKang · 2023-04-01T15:25:02Z

这种写法你是在哪儿看到的？当年我写这段代码的时候对于使用 COMPUTE_MAXS 还是 COMPUTE_FRAMES 还是比较困惑的

JiaRG · 2023-04-01T15:58:32Z

堆栈显示这报的错，看了下里面有获取 ClassLoader 的方法，就试了下

那俩变量就 google 下，看到了那篇文章的内容😁

LinShunKang · 2023-04-01T16:35:28Z

有想法、运气不错；不过，你这个改动得进行全面的测试才行

JiaRG · 2023-04-02T16:42:46Z

翻了下文档，读的时候不需要计算，要不会有性能损耗，写的时候才重新计算

    private byte[] getBytes(ClassLoader loader,
                            String className,
                            byte[] classFileBuffer) {
        ClassReader cr = new ClassReader(classFileBuffer);
        ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES) {
            @Override
            protected ClassLoader getClassLoader() {
                return Thread.currentThread().getContextClassLoader();
            }
        };
        ClassVisitor cv = new ProfilingClassAdapter(cw, className);
        cr.accept(cv, ClassReader.SKIP_FRAMES);
        return cw.toByteArray();
    }

LinShunKang · 2023-04-09T07:22:28Z

翻了下文档，读的时候不需要计算，要不会有性能损耗，写的时候才重新计算

这个方案有问题：

JiaRG · 2023-04-10T14:41:53Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

JiaRG · 2023-04-10T14:46:03Z

出现了个这问题😓

2023-04-10 22:38:00.014 [MyPerf4J] ERROR [MyPerf4J-LightWeightScheduler-0] JvmMetricsScheduler.processThreadMetrics(1681137420014, 1681137420014, 1681137480014) Could not initialize class cn.myperf4j.base.metric.collector.JvmThreadCollector
java.lang.NoClassDefFoundError: Could not initialize class cn.myperf4j.base.metric.collector.JvmThreadCollector
        at cn.myperf4j.core.scheduler.JvmMetricsScheduler.processThreadMetrics(JvmMetricsScheduler.java:139)
        at cn.myperf4j.core.scheduler.JvmMetricsScheduler.run(JvmMetricsScheduler.java:72)
        at cn.myperf4j.core.LightWeightScheduler.runTask(LightWeightScheduler.java:100)
        at cn.myperf4j.core.LightWeightScheduler.runAllTasks(LightWeightScheduler.java:93)
        at cn.myperf4j.core.LightWeightScheduler.access$200(LightWeightScheduler.java:18)
        at cn.myperf4j.core.LightWeightScheduler$1.run(LightWeightScheduler.java:85)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.reflect.InaccessibleObjectException: Unable to make private static native java.lang.Thread[] java.lang.Thread.getThreads() accessible: module java.base does not "opens java.lang" to unnamed module @6a41eaa2 [in thread "MyPerf4J-LightWeightScheduler-1"]
        at java.base/java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:387)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:363)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:311)
        at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:201)
        at java.base/java.lang.reflect.Method.setAccessible(Method.java:195)
        at cn.myperf4j.base.metric.collector.JvmThreadCollector.reflectGetThreadsMethod(JvmThreadCollector.java:34)
        at cn.myperf4j.base.metric.collector.JvmThreadCollector.<clinit>(JvmThreadCollector.java:27)
        ... 12 more

JiaRG · 2023-04-10T14:50:45Z

ThreadMXBean 里面不是有现成的方法可以获取到数据么，为啥要反射呢？

LinShunKang · 2023-04-10T16:25:46Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

不是

LinShunKang · 2023-04-10T16:27:40Z

出现了个这问题😓

2023-04-10 22:38:00.014 [MyPerf4J] ERROR [MyPerf4J-LightWeightScheduler-0] JvmMetricsScheduler.processThreadMetrics(1681137420014, 1681137420014, 1681137480014) Could not initialize class cn.myperf4j.base.metric.collector.JvmThreadCollector
java.lang.NoClassDefFoundError: Could not initialize class cn.myperf4j.base.metric.collector.JvmThreadCollector
        at cn.myperf4j.core.scheduler.JvmMetricsScheduler.processThreadMetrics(JvmMetricsScheduler.java:139)
        at cn.myperf4j.core.scheduler.JvmMetricsScheduler.run(JvmMetricsScheduler.java:72)
        at cn.myperf4j.core.LightWeightScheduler.runTask(LightWeightScheduler.java:100)
        at cn.myperf4j.core.LightWeightScheduler.runAllTasks(LightWeightScheduler.java:93)
        at cn.myperf4j.core.LightWeightScheduler.access$200(LightWeightScheduler.java:18)
        at cn.myperf4j.core.LightWeightScheduler$1.run(LightWeightScheduler.java:85)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1623)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.reflect.InaccessibleObjectException: Unable to make private static native java.lang.Thread[] java.lang.Thread.getThreads() accessible: module java.base does not "opens java.lang" to unnamed module @6a41eaa2 [in thread "MyPerf4J-LightWeightScheduler-1"]
        at java.base/java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:387)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:363)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:311)
        at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:201)
        at java.base/java.lang.reflect.Method.setAccessible(Method.java:195)
        at cn.myperf4j.base.metric.collector.JvmThreadCollector.reflectGetThreadsMethod(JvmThreadCollector.java:34)
        at cn.myperf4j.base.metric.collector.JvmThreadCollector.<clinit>(JvmThreadCollector.java:27)
        ... 12 more

注意看：module java.base does not "opens java.lang" to unnamed module

LinShunKang · 2023-04-10T16:28:24Z

ThreadMXBean 里面不是有现成的方法可以获取到数据么，为啥要反射呢？

性能差，线程数很多时会影响应用

JiaRG · 2023-04-11T05:28:39Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

不是

有复现的 demo 么，图里也看不出啥来，或者加载的那个类有啥特别的

LinShunKang · 2023-04-22T10:15:03Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

不是

有复现的 demo 么，图里也看不出啥来，或者加载的那个类有啥特别的

从代码上看没什么特别的：

class A extends B
在加载 B 时抛出 java.lang.ClassCircularityError: A

LinShunKang · 2023-04-22T10:58:34Z

你试试这个方案：

    private byte[] getBytes(final ClassLoader loader,
                            String className,
                            byte[] classFileBuffer) {
        final ClassReader cr = new ClassReader(classFileBuffer);
        final ClassWriter cw = new ClassWriter(cr, needComputeMaxs(loader) ? COMPUTE_MAXS : COMPUTE_FRAMES) {
            @Override
            protected ClassLoader getClassLoader() {
                return loader != null ? loader : super.getClassLoader();
            }
        };
        final ClassVisitor cv = new ProfilingClassAdapter(cw, className);
        cr.accept(cv, EXPAND_FRAMES);
        return cw.toByteArray();
    }

JiaRG · 2023-04-22T13:32:15Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

不是

有复现的 demo 么，图里也看不出啥来，或者加载的那个类有啥特别的

从代码上看没什么特别的：

class A extends B

在加载 B 时抛出 java.lang.ClassCircularityError: A

那说明 B extends A ? 要么类库出现了多个版本了

LinShunKang · 2023-04-22T13:46:27Z

这是跟其它的 agent 冲突了吧，我本地用 jdk20 起了一个 springboot2.7.8 的 demo 没问题

不是

有复现的 demo 么，图里也看不出啥来，或者加载的那个类有啥特别的

从代码上看没什么特别的：

class A extends B

在加载 B 时抛出 java.lang.ClassCircularityError: A

那说明 B extends A ? 要么类库出现了多个版本了

B 不会继承 A 否则编译都通不过；还是类加载的问题

JiaRG · 2023-04-22T14:44:10Z

-XX:+TraceClassLoading 看下没 agent 的时候和有 agent 的时候那个类是从哪加载的。。。剩下我能想到的就只有那个类所在的类库出现了多个版本了，最好提供个 demo 吧😢

LinShunKang · 2023-05-07T13:23:29Z

@JiaRG
如果以这个为 v0.1:

    private byte[] getBytes(final ClassLoader loader,
                            String className,
                            byte[] classFileBuffer) {
        final ClassReader cr = new ClassReader(classFileBuffer);
        final ClassWriter cw = new ClassWriter(cr, needComputeMaxs(loader) ? COMPUTE_MAXS : COMPUTE_FRAMES) {
            @Override
            protected ClassLoader getClassLoader() {
                return loader != null ? loader : super.getClassLoader();
            }
        };
        final ClassVisitor cv = new ProfilingClassAdapter(cw, className);
        cr.accept(cv, EXPAND_FRAMES);
        return cw.toByteArray();
    }

以这个为 v1:

    private byte[] getBytes(ClassLoader loader,
                            String className,
                            byte[] classFileBuffer) {
        ClassReader cr = new ClassReader(classFileBuffer);
        ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES) {
            @Override
            protected ClassLoader getClassLoader() {
                return Thread.currentThread().getContextClassLoader();
            }
        };
        ClassVisitor cv = new ProfilingClassAdapter(cw, className);
        cr.accept(cv, ClassReader.SKIP_FRAMES);
        return cw.toByteArray();
    }

那么，在没有 MyPerf4J 和有 MyPerf4J v0.1 与 v1 的版本在添加了 -XX:+TraceClassLoading 参数后的日志（做了脱敏与过滤）为：

你可以看到有问题的 v1 版本的类加载与另外两个明显不同，v1 版本里 MqttWireMessage 加载了 2 次

LinShunKang · 2023-05-07T13:25:25Z

你试试这个方案：

    private byte[] getBytes(final ClassLoader loader,
                            String className,
                            byte[] classFileBuffer) {
        final ClassReader cr = new ClassReader(classFileBuffer);
        final ClassWriter cw = new ClassWriter(cr, needComputeMaxs(loader) ? COMPUTE_MAXS : COMPUTE_FRAMES) {
            @Override
            protected ClassLoader getClassLoader() {
                return loader != null ? loader : super.getClassLoader();
            }
        };
        final ClassVisitor cv = new ProfilingClassAdapter(cw, className);
        cr.accept(cv, EXPAND_FRAMES);
        return cw.toByteArray();
    }

@JiaRG 这个 v0.1 的版本，你试了么？

JiaRG added the question Further information is requested label Mar 28, 2023

JiaRG added a commit to JiaRG/MyPerf4J that referenced this issue Apr 5, 2023

Refactor: LinShunKang#89

58f791d

JiaRG changed the title ~~关于内部类、匿名类的监控问题~~ 关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 Apr 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 #89

关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 #89

JiaRG commented Mar 28, 2023

JiaRG commented Mar 28, 2023 •

edited

LinShunKang commented Mar 29, 2023

JiaRG commented Apr 1, 2023

LinShunKang commented Apr 1, 2023

JiaRG commented Apr 1, 2023

LinShunKang commented Apr 1, 2023

JiaRG commented Apr 2, 2023

LinShunKang commented Apr 9, 2023

JiaRG commented Apr 10, 2023

JiaRG commented Apr 10, 2023 •

edited

JiaRG commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

JiaRG commented Apr 11, 2023

LinShunKang commented Apr 22, 2023

LinShunKang commented Apr 22, 2023 •

edited

JiaRG commented Apr 22, 2023

LinShunKang commented Apr 22, 2023

JiaRG commented Apr 22, 2023 •

edited

LinShunKang commented May 7, 2023 •

edited

LinShunKang commented May 7, 2023 •

edited

关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 #89

关于ASM COMPUTE_FRAMES COMPUTE_MAXS的讨论 #89

Comments

JiaRG commented Mar 28, 2023

JiaRG commented Mar 28, 2023 • edited

LinShunKang commented Mar 29, 2023

JiaRG commented Apr 1, 2023

LinShunKang commented Apr 1, 2023

JiaRG commented Apr 1, 2023

LinShunKang commented Apr 1, 2023

JiaRG commented Apr 2, 2023

LinShunKang commented Apr 9, 2023

JiaRG commented Apr 10, 2023

JiaRG commented Apr 10, 2023 • edited

JiaRG commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

LinShunKang commented Apr 10, 2023

JiaRG commented Apr 11, 2023

LinShunKang commented Apr 22, 2023

LinShunKang commented Apr 22, 2023 • edited

JiaRG commented Apr 22, 2023

LinShunKang commented Apr 22, 2023

JiaRG commented Apr 22, 2023 • edited

LinShunKang commented May 7, 2023 • edited

LinShunKang commented May 7, 2023 • edited

JiaRG commented Mar 28, 2023 •

edited

JiaRG commented Apr 10, 2023 •

edited

LinShunKang commented Apr 22, 2023 •

edited

JiaRG commented Apr 22, 2023 •

edited

LinShunKang commented May 7, 2023 •

edited

LinShunKang commented May 7, 2023 •

edited