性能调优之JMH必知必会2：JMH的基本用法
JMH必知必会系列文章（持续更新）
一、前言
在前面文章《性能调优之JMH必知必会1：什么是JMH》中介绍了JMH的基本概述和简单用法。现在来介绍JMH的一些基本用法。【单位换算：1秒(s)=1000000微秒(us)=1000000000纳秒(ns)】
官方JMH源码（包含样例，在jmh-samples包里）下载地址：https://github.com/openjdk/jmh/tags。
官方JMH样例在线浏览地址：http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/。
本文内容参考书籍《Java高并发编程详解：深入理解并发核心库》，作者为 汪文君 ，读者有需要可以去购买正版书籍。
本文由 @大白有点菜 原创，请勿盗用，转载请说明出处！如果觉得文章还不错，请点点赞，加关注，谢谢！
二、JMH的基本用法
1、添加JMH依赖包
在Maven仓库中搜索依赖包jmh-core 和 jmh-generator-annprocess ，版本为 1.36 。需要注释 jmh-generator-annprocess 包中的“<scope>test</scope>”，不然项目运行会报错。
<!-- https://mvnrepository.com/artifact/org.openjdk.jmh/jmh-core -->
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>1.36</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.openjdk.jmh/jmh-generator-annprocess -->
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>1.36</version>
<!-- <scope>test</scope>-->
</dependency>
2、@Benchmark
只有标记了@Benchmark 注解的方法，JMH才会对其执行基准测试，不然就是一个普通方法。如果一个类中没有任何被@Benchmark标记的方法，那么对其进行基准测试则会出现异常。例如，方法 whatAboutDBYDC 没有加@Benchmark注解，运行时报“No benchmarks to run”的错误。
【核心代码 JmhTestApp2】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试2
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class JmhTestApp2 {
/**
* 没有添加 @Benchmark 方法
* @return
*/
public void whatAboutDBYDC() {
System.out.println("大白其实没有那么菜");
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp2.class.getSimpleName())
.forks(1)
.measurementIterations(10)
.warmupIterations(10)
.build();
new Runner(opts).run();
}
}
【运行结果】
Exception in thread "main" No benchmarks to run; check the include/exclude regexps.
at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:258)
at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
at cn.zhuangyt.javabase.jmh.JmhTestApp2.main(JmhTestApp2.java:34)
Process finished with exit code 1
2、@Warmup和@Measurement
（1）@Warmup：预热。在基准测试代码正式度量之前，先对其进行预热，使得代码的执行是经历过了类的早期优化、JVM运行期编译、JIT优化之后的最终状态，从而能够获得代码真实的性能数据。注解中主要参数如下：
- iterations：预热迭代次数。
- time：每次预热迭代的时间。
- timeUnit：预热迭代持续时间的时间单位。
- batchSize：批处理大小，每个操作的基准方法调用数。
package org.openjdk.jmh.annotations;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;
/**
* <p>Warmup annotation allows to set the default warmup parameters for the benchmark.</p>
*
* <p>This annotation may be put at {@link Benchmark} method to have effect on that method
* only, or at the enclosing class instance to have the effect over all {@link Benchmark}
* methods in the class. This annotation may be overridden with the runtime options.</p>
*
* @see Measurement
*/
@Target({ElementType.METHOD,ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Inherited
public @interface Warmup {
int BLANK_ITERATIONS = -1;
int BLANK_TIME = -1;
int BLANK_BATCHSIZE = -1;
/** @return Number of warmup iterations */
int iterations() default BLANK_ITERATIONS;
/** @return Time for each warmup iteration */
int time() default BLANK_TIME;
/** @return Time unit for warmup iteration duration */
TimeUnit timeUnit() default TimeUnit.SECONDS;
/** @return batch size: number of benchmark method calls per operation */
int batchSize() default BLANK_BATCHSIZE;
}
（2）@Measurement：度量。在每一轮的度量中，所有的度量数据会被纳入统计之中（预热数据不会纳入统计之中）。注解中主要参数如下：
- iterations：度量迭代次数。
- time：每次度量迭代的时间。
- timeUnit：度量迭代持续时间的时间单位。
- batchSize：批处理大小，每个操作的基准方法调用数。
package org.openjdk.jmh.annotations;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;
/**
* <p>Measurement annotations allows to set the default measurement parameters for
* the benchmark.</p>
*
* <p>This annotation may be put at {@link Benchmark} method to have effect on that
* method only, or at the enclosing class instance to have the effect over all
* {@link Benchmark} methods in the class. This annotation may be overridden with
* the runtime options.</p>
*
* @see Warmup
*/
@Inherited
@Target({ElementType.METHOD,ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
public @interface Measurement {
int BLANK_ITERATIONS = -1;
int BLANK_TIME = -1;
int BLANK_BATCHSIZE = -1;
/** @return Number of measurement iterations */
int iterations() default BLANK_ITERATIONS;
/** @return Time of each measurement iteration */
int time() default BLANK_TIME;
/** @return Time unit for measurement iteration duration */
TimeUnit timeUnit() default TimeUnit.SECONDS;
/** @return Batch size: number of benchmark method calls per operation */
int batchSize() default BLANK_BATCHSIZE;
}
（3）注解 @Warmup 和 @Measurement 的全局设置和局部设置
如果在类上添加@Warmup和@Measurement注解，代表全局设置；如果在基准测试方法中添加@Warmup和@Measurement注解，代表局部设置。在方法上添加注解的优先级高于在类上添加注解，是这样吗？我们来验证一下。类上@Warmup注解中设置预热次数为4，@Measurement注解中设置度量次数为4。方法上@Warmup注解中设置预热次数为2，@Measurement注解中设置度量次数为2。
【核心代码 JmhTestApp3】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试3
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 4, time = 100, timeUnit = TimeUnit.MICROSECONDS)
@Measurement(iterations = 4, time = 50, timeUnit = TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class JmhTestApp3 {
/**
* 大白有点菜怎么样 方法
* @return
*/
@Benchmark
@Warmup(iterations = 2, time = 200, timeUnit = TimeUnit.MICROSECONDS)
@Measurement(iterations = 2, time = 100, timeUnit = TimeUnit.MICROSECONDS)
public void whatAboutDBYDC() {
System.out.println("大白其实没有那么菜");
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp3.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【运行结果】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5492:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 2 iterations, 200 us each
# Measurement: 2 iterations, 100 us each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp3.whatAboutDBYDC
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration 1: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
114.435 us/op
# Warmup Iteration 2: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
93.810 us/op
Iteration 1: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
91.572 us/op
Iteration 2: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
51.695 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp3.whatAboutDBYDC":
71.633 us/op
# Run complete. Total time: 00:00:01
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp3.whatAboutDBYDC avgt 2 71.633 us/op
Process finished with exit code 0
从运行结果看，whatAboutDBYDC 方法度量次数（Cnt）为2，说明在方法上配置的@Warmup和@Measurement会覆盖在类上配置的@Warmup和@Measurement。
（4）构造 Options 时对 Warmup 和 Measurement 全局设置。
【Options中设置 Warmup 参数】：
- warmupIterations(int value) 等同于 @Warmup 的 iterations
- warmupTime(TimeValue value) 等同于 @Warmup 的 time
- TimeValue 等同于 @Warmup 的 timeUnit
【Options中设置 Measurement 参数】：
- measurementIterations(int value) 等同于 @Measurement 的 iterations
- measurementTime(TimeValue value) 等同于 @Measurement 的 time
- TimeValue 等同于 @Measurement 的 timeUnit
【核心代码 JmhTestApp4】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import java.util.concurrent.TimeUnit;
/**
* JMH测试4
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class JmhTestApp4 {
/**
* 大白有点菜怎么样 方法
* @return
*/
@Benchmark
public void whatAboutDBYDC() {
System.out.println("大白其实没有那么菜");
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp4.class.getSimpleName())
.forks(1)
.warmupIterations(2)
.warmupTime(TimeValue.microseconds(60))
.measurementIterations(2)
.measurementTime(TimeValue.microseconds(30))
.build();
new Runner(opts).run();
}
}
【运行结果】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5822:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 2 iterations, 60 us each
# Measurement: 2 iterations, 30 us each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp4.whatAboutDBYDC
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration 1: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
85.974 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp4.whatAboutDBYDC":
76.850 us/op
# Run complete. Total time: 00:00:01
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp4.whatAboutDBYDC avgt 2 76.850 us/op
Process finished with exit code 0
（5）通过 Options 配置的全局设置（4个方法warmupIterations、warmupTime、measurementIterations、measurementTime）优先级高于在类和方法上添加 @Warmup 与 @Measurement 注解吗？实践出真理，靠猜没有用。在类上的预热次数为4，度量次数为4；在方法上的预热次数为2，度量次数为2；构造Options中，预热次数为6，度量次数为6。
【核心代码 JmhTestApp5】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import java.util.concurrent.TimeUnit;
/**
* JMH测试5
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 4, time = 100, timeUnit = TimeUnit.MICROSECONDS)
@Measurement(iterations = 4, time = 50, timeUnit = TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class JmhTestApp5 {
/**
* 大白有点菜怎么样 方法
* @return
*/
@Benchmark
@Warmup(iterations = 2, time = 200, timeUnit = TimeUnit.MICROSECONDS)
@Measurement(iterations = 2, time = 100, timeUnit = TimeUnit.MICROSECONDS)
public void whatAboutDBYDC() {
System.out.println("大白其实没有那么菜");
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp5.class.getSimpleName())
.forks(1)
.warmupIterations(6)
.warmupTime(TimeValue.microseconds(300))
.measurementIterations(6)
.measurementTime(TimeValue.microseconds(150))
.build();
new Runner(opts).run();
}
}
【运行结果】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=6085:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 6 iterations, 300 us each
# Measurement: 6 iterations, 150 us each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp5.whatAboutDBYDC
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration 1: 大白其实没有那么菜
大白其实没有那么菜
......
大白其实没有那么菜
67.766 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp5.whatAboutDBYDC":
58.009 ±(99.9%) 37.652 us/op [Average]
(min, avg, max) = (39.799, 58.009, 77.847), stdev = 13.427
CI (99.9%): [20.357, 95.661] (assumes normal distribution)
# Run complete. Total time: 00:00:01
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp5.whatAboutDBYDC avgt 6 58.009 ± 37.652 us/op
Process finished with exit code 0
从结果可以看出，whatAboutDBYDC 方法的度量次数（Cnt）为6，说明在 Options 中设置 Warmup 和 Measurement 优先级是最高的，即使方法上添加了注解 @Warmup 与 @Measurement 也无法将其覆盖。
3、输出日志中一些中文解释
// JMH版本 1.36
# JMH version: 1.36
// JDK版本
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
// Java命令目录
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
// JVM运行时指定的参数
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=6085:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
// Blackhole 模式，默认配置
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
// 预热迭代6次，每300微秒（us）执行一次
# Warmup: 6 iterations, 300 us each
// 度量迭代6次，每150微秒（us）执行一次
# Measurement: 6 iterations, 150 us each
// 每一次迭代的超时时间
# Timeout: 10 min per iteration
// 执行基准测试的线程数量
# Threads: 1 thread, will synchronize iterations
// Benchmark的Mode，统计平均时间，方法调用一次所耗费的单位时间
# Benchmark mode: Average time, time/op
// 执行基准测试的方法
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp5.whatAboutDBYDC
// 执行进度
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
// 每预热迭代一次耗费的时间，此处是预热迭代6次
# Warmup Iteration 1: 464.600 us/op
# Warmup Iteration 2: 105.989 us/op
# Warmup Iteration 3: 104.560 us/op
# Warmup Iteration 4: 51.880 us/op
# Warmup Iteration 5: 40.179 us/op
# Warmup Iteration 6: 60.855 us/op
// 每度量迭代一次耗费的时间，此处是度量迭代6次
Iteration 1: 77.847 us/op
Iteration 2: 55.029 us/op
Iteration 3: 58.097 us/op
Iteration 4: 49.513 us/op
Iteration 5: 39.799 us/op
Iteration 6: 67.766 us/op
// 最终的统计结果
Result "cn.zhuangyt.javabase.jmh.JmhTestApp5.whatAboutDBYDC":
58.009 ±(99.9%) 37.652 us/op [Average]
// 小、平均、最大以及标准误差
(min, avg, max) = (39.799, 58.009, 77.847), stdev = 13.427
CI (99.9%): [20.357, 95.661] (assumes normal distribution)
// 执行完成耗费的总时间
# Run complete. Total time: 00:00:01
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
// 执行基准测试的方法名 模式 次数 分数 误差 执行时间单位
Benchmark Mode Cnt Score Error Units
JmhTestApp5.whatAboutDBYDC avgt 6 58.009 ± 37.652 us/op
4、@BenchmarkMode
（1）JMH使用 @BenchmarkMode 注解来声明使用哪一种模式来运行，主要是5种模式，在枚举类Mode中有定义 。如下：
- Throughput：
吞吐量，每单位时间的操作。通过连续调用 Benchmark 方法运行，计算所有工作线程的总吞吐量。 这种模式是基于时间的，它会一直运行到迭代时间到期。
- AverageTime：
平均时间，每次操作的平均时间。通过连续调用 Benchmark 方法运行，计算调用所有工作线程的平均时间。 这与吞吐量相反，但具有不同的聚合策略。 这种模式是基于时间的，它会一直运行到迭代时间到期。
- SampleTime：
采样时间，采样每个操作的时间。通过连续调用 Benchmark 方法运行，并随机采样调用所需的时间。 此模式会自动调整采样频率，但可能会省略一些错过采样测量的停顿。 这种模式是基于时间的，它会一直运行到迭代时间到期。
- SingleShotTime：
测量单次操作的时间。通过调用 Benchmark 一次并测量其时间来运行。 当您不想隐藏预热调用，或者如果您想要查看调用之间的进度，或者您想要记录每个样本时，此模式可用于估计“冷”性能。 这种模式是基于工作的，并且只会运行一次基准方法调用。注意事项：a）通常需要更多的预热/度量迭代。b）如果基准很小，定时器开销可能会很大； 如果这是一个问题，请切换到 SampleTime 模式。
- All：Meta-mode（元模式）：所有基准模式。 这主要用于内部 JMH 测试。
【Mode枚举类源代码】
package org.openjdk.jmh.annotations;
import java.util.ArrayList;
import java.util.List;
/**
* Benchmark mode.
*/
public enum Mode {
/**
* <p>Throughput: operations per unit of time.</p>
*
* <p>Runs by continuously calling {@link Benchmark} methods,
* counting the total throughput over all worker threads. This mode is time-based, and it will
* run until the iteration time expires.</p>
*/
Throughput("thrpt", "Throughput, ops/time"),
/**
* <p>Average time: average time per per operation.</p>
*
* <p>Runs by continuously calling {@link Benchmark} methods,
* counting the average time to call over all worker threads. This is the inverse of {@link Mode#Throughput},
* but with different aggregation policy. This mode is time-based, and it will run until the iteration time
* expires.</p>
*/
AverageTime("avgt", "Average time, time/op"),
/**
* <p>Sample time: samples the time for each operation.</p>
*
* <p>Runs by continuously calling {@link Benchmark} methods,
* and randomly samples the time needed for the call. This mode automatically adjusts the sampling
* frequency, but may omit some pauses which missed the sampling measurement. This mode is time-based, and it will
* run until the iteration time expires.</p>
*/
SampleTime("sample", "Sampling time"),
/**
* <p>Single shot time: measures the time for a single operation.</p>
*
* <p>Runs by calling {@link Benchmark} once and measuring its time.
* This mode is useful to estimate the "cold" performance when you don't want to hide the warmup invocations, or
* if you want to see the progress from call to call, or you want to record every single sample. This mode is
* work-based, and will run only for a single invocation of {@link Benchmark}
* method.</p>
*
* Caveats for this mode include:
* <ul>
* <li>More warmup/measurement iterations are generally required.</li>
* <li>Timers overhead might be significant if benchmarks are small; switch to {@link #SampleTime} mode if
* that is a problem.</li>
* </ul>
*/
SingleShotTime("ss", "Single shot invocation time"),
/**
* Meta-mode: all the benchmark modes.
* This is mostly useful for internal JMH testing.
*/
All("all", "All benchmark modes"),
;
private final String shortLabel;
private final String longLabel;
Mode(String shortLabel, String longLabel) {
this.shortLabel = shortLabel;
this.longLabel = longLabel;
}
public String shortLabel() {
return shortLabel;
}
public String longLabel() {
return longLabel;
}
public static Mode deepValueOf(String name) {
try {
return Mode.valueOf(name);
} catch (IllegalArgumentException iae) {
Mode inferred = null;
for (Mode type : values()) {
if (type.shortLabel().startsWith(name)) {
if (inferred == null) {
inferred = type;
} else {
throw new IllegalStateException("Unable to parse benchmark mode, ambiguous prefix given: \"" + name + "\"\n" +
"Known values are " + getKnown());
}
}
}
if (inferred != null) {
return inferred;
} else {
throw new IllegalStateException("Unable to parse benchmark mode: \"" + name + "\"\n" +
"Known values are " + getKnown());
}
}
}
public static List<String> getKnown() {
List<String> res = new ArrayList<>();
for (Mode type : Mode.values()) {
res.add(type.name() + "/" + type.shortLabel());
}
return res;
}
}
（2）运行官方提供的样例代码，由于官方样例把所有模式都放在一个类里面运行，不太好看运行结果，所以笔者就逐一剥离并单独运行。
1）Throughput模式：吞吐量，每单位时间的操作。它的输出信息表明了在单位时间内可以对该方法调用多少次。
【官方样例（JMHSample_02_BenchmarkModes）1：Mode.Throughput 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的Throughput模式
* @author 大白有点菜
*/
public class JmhTestApp7_Mode_Throughput {
/**
* Mode.Throughput, as stated in its Javadoc, measures the raw throughput by
* continuously calling the benchmark method in a time-bound iteration, and
* counting how many times we executed the method.
*
* Mode.Throughput，如其Javadoc中所述，通过在有时间限制的迭代中连续调用基准方法，并计算我们执行该方法的次数，来测量原始吞吐量。
*
* We are using the special annotation to select the units to measure in,
* although you can use the default.
*
* 我们使用特殊注释来选择要测量的单位，尽管您可以使用默认值。
*/
@Benchmark
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
public void measureThroughput() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_Throughput.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）1运行结果：Mode.Throughput 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=4874:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_Throughput.measureThroughput
# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 9.152 ops/s
# Warmup Iteration 2: 9.183 ops/s
# Warmup Iteration 3: 9.184 ops/s
# Warmup Iteration 4: 9.199 ops/s
# Warmup Iteration 5: 9.200 ops/s
Iteration 1: 9.189 ops/s
Iteration 2: 9.208 ops/s
Iteration 3: 9.176 ops/s
Iteration 4: 9.153 ops/s
Iteration 5: 9.165 ops/s
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_Throughput.measureThroughput":
9.178 ±(99.9%) 0.081 ops/s [Average]
(min, avg, max) = (9.153, 9.178, 9.208), stdev = 0.021
CI (99.9%): [9.097, 9.259] (assumes normal distribution)
# Run complete. Total time: 00:01:41
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Mode_Throughput.measureThroughput thrpt 5 9.178 ± 0.081 ops/s
2）AverageTime模式：平均时间，每次操作的平均时间。它主要用于输出基准测试方法每调用一次所耗费的时间，也就是elapsed time/operation。
【官方样例（JMHSample_02_BenchmarkModes）2：Mode.AverageTime 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的AverageTime模式
* @author 大白有点菜
*/
public class JmhTestApp7_Mode_AverageTime {
/**
* Mode.AverageTime measures the average execution time, and it does it
* in the way similar to Mode.Throughput.
*
* Mode.AverageTime 测量平均执行时间，其方式与Mode.Throughput类似。
*
* Some might say it is the reciprocal throughput, and it really is.
* There are workloads where measuring times is more convenient though.
*
* 有些人可能会说这是吞吐量的倒数，事实上确实如此。但在某些工作负载中，测量时间更方便。
*/
@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void measureAvgTime() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp7_Mode_AverageTime.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）2运行结果：Mode.AverageTime 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=4801:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_AverageTime.measureAvgTime
# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 108779.123 us/op
# Warmup Iteration 2: 108878.502 us/op
# Warmup Iteration 3: 108750.412 us/op
# Warmup Iteration 4: 109019.321 us/op
# Warmup Iteration 5: 109039.535 us/op
Iteration 1: 108635.140 us/op
Iteration 2: 108453.894 us/op
Iteration 3: 108628.090 us/op
Iteration 4: 108665.567 us/op
Iteration 5: 108589.455 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_AverageTime.measureAvgTime":
108594.429 ±(99.9%) 320.009 us/op [Average]
(min, avg, max) = (108453.894, 108594.429, 108665.567), stdev = 83.105
CI (99.9%): [108274.420, 108914.438] (assumes normal distribution)
# Run complete. Total time: 00:01:41
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Mode_AverageTime.measureAvgTime avgt 5 108594.429 ± 320.009 us/op
3）SampleTime模式：采样时间，采样每个操作的时间。采用一种抽样的方式来统计基准测试方法的性能结果，与直方图几乎一样，它会收集所有的性能数据，并且将其分布在不同的区间中。
【官方样例（JMHSample_02_BenchmarkModes）3：Mode.SampleTime 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的SampleTime模式
* @author 大白有点菜
*/
public class JmhTestApp7_Mode_SampleTime {
/**
* Mode.SampleTime samples the execution time. With this mode, we are
* still running the method in a time-bound iteration, but instead of
* measuring the total time, we measure the time spent in *some* of
* the benchmark method calls.
*
* Mode.SampleTime 对执行时间进行采样。使用这种模式，我们仍然在有时间限制的迭代中运行该方法，
* 但我们不是测量总时间，而是测量一些基准方法调用所花费的时间。
*
* This allows us to infer the distributions, percentiles, etc.
*
* 这允许我们推断分布、百分位数等。
*
* JMH also tries to auto-adjust sampling frequency: if the method
* is long enough, you will end up capturing all the samples.
*
* JMH还尝试自动调整采样频率：如果方法足够长，您将最终捕获所有样本。
*/
@Benchmark
@BenchmarkMode(Mode.SampleTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void measureSamples() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp7_Mode_SampleTime.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）3运行结果：Mode.SampleTime 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=4961:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Sampling time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_SampleTime.measureSamples
# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 108969.272 ±(99.9%) 1055.430 us/op
# Warmup Iteration 2: 108657.279 ±(99.9%) 545.605 us/op
# Warmup Iteration 3: 108444.463 ±(99.9%) 365.195 us/op
# Warmup Iteration 4: 108354.263 ±(99.9%) 465.995 us/op
# Warmup Iteration 5: 108406.410 ±(99.9%) 393.365 us/op
Iteration 1: 109007.738 ±(99.9%) 448.031 us/op
measureSamples·p0.00: 106430.464 us/op
measureSamples·p0.50: 108920.832 us/op
measureSamples·p0.90: 111017.984 us/op
measureSamples·p0.95: 111588.147 us/op
measureSamples·p0.99: 111673.344 us/op
measureSamples·p0.999: 111673.344 us/op
measureSamples·p0.9999: 111673.344 us/op
measureSamples·p1.00: 111673.344 us/op
Iteration 2: 108700.969 ±(99.9%) 427.887 us/op
measureSamples·p0.00: 106692.608 us/op
measureSamples·p0.50: 108658.688 us/op
measureSamples·p0.90: 110519.910 us/op
measureSamples·p0.95: 111188.378 us/op
measureSamples·p0.99: 111804.416 us/op
measureSamples·p0.999: 111804.416 us/op
measureSamples·p0.9999: 111804.416 us/op
measureSamples·p1.00: 111804.416 us/op
Iteration 3: 108324.666 ±(99.9%) 329.715 us/op
measureSamples·p0.00: 106299.392 us/op
measureSamples·p0.50: 108265.472 us/op
measureSamples·p0.90: 109445.120 us/op
measureSamples·p0.95: 109838.336 us/op
measureSamples·p0.99: 111280.128 us/op
measureSamples·p0.999: 111280.128 us/op
measureSamples·p0.9999: 111280.128 us/op
measureSamples·p1.00: 111280.128 us/op
Iteration 4: 108724.929 ±(99.9%) 424.264 us/op
measureSamples·p0.00: 105906.176 us/op
measureSamples·p0.50: 108920.832 us/op
measureSamples·p0.90: 109969.408 us/op
measureSamples·p0.95: 111004.877 us/op
measureSamples·p0.99: 111673.344 us/op
measureSamples·p0.999: 111673.344 us/op
measureSamples·p0.9999: 111673.344 us/op
measureSamples·p1.00: 111673.344 us/op
Iteration 5: 108500.838 ±(99.9%) 403.344 us/op
measureSamples·p0.00: 105119.744 us/op
measureSamples·p0.50: 108396.544 us/op
measureSamples·p0.90: 109916.979 us/op
measureSamples·p0.95: 110664.090 us/op
measureSamples·p0.99: 111804.416 us/op
measureSamples·p0.999: 111804.416 us/op
measureSamples·p0.9999: 111804.416 us/op
measureSamples·p1.00: 111804.416 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_SampleTime.measureSamples":
N = 464
mean = 108651.061 ±(99.9%) 180.687 us/op
Histogram, us/op:
[105000.000, 105500.000) = 1
[105500.000, 106000.000) = 1
[106000.000, 106500.000) = 7
[106500.000, 107000.000) = 23
[107000.000, 107500.000) = 47
[107500.000, 108000.000) = 53
[108000.000, 108500.000) = 89
[108500.000, 109000.000) = 73
[109000.000, 109500.000) = 79
[109500.000, 110000.000) = 40
[110000.000, 110500.000) = 16
[110500.000, 111000.000) = 12
[111000.000, 111500.000) = 10
Percentiles, us/op:
p(0.0000) = 105119.744 us/op
p(50.0000) = 108527.616 us/op
p(90.0000) = 110231.552 us/op
p(95.0000) = 110985.216 us/op
p(99.0000) = 111673.344 us/op
p(99.9000) = 111804.416 us/op
p(99.9900) = 111804.416 us/op
p(99.9990) = 111804.416 us/op
p(99.9999) = 111804.416 us/op
p(100.0000) = 111804.416 us/op
# Run complete. Total time: 00:01:41
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Mode_SampleTime.measureSamples sample 464 108651.061 ± 180.687 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.00 sample 105119.744 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.50 sample 108527.616 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.90 sample 110231.552 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.95 sample 110985.216 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.99 sample 111673.344 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.999 sample 111804.416 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p0.9999 sample 111804.416 us/op
JmhTestApp7_Mode_SampleTime.measureSamples:measureSamples·p1.00 sample 111804.416 us/op
4）SingleShotTime模式：测量单次操作的时间。可用来进行冷测试，不论是Warmup还是Measurement，在每一个批次中基准测试方法只会被执行一次，一般情况下，我们会将Warmup的批次设置为0。
【官方样例（JMHSample_02_BenchmarkModes）4：Mode.SingleShotTime 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的SingleShotTime模式
* @author 大白有点菜
*/
public class JmhTestApp7_Mode_SingleShotTime {
/**
* Mode.SingleShotTime measures the single method invocation time. As the Javadoc
* suggests, we do only the single benchmark method invocation. The iteration
* time is meaningless in this mode: as soon as benchmark method stops, the
* iteration is over.
*
* Mode.SingleShotTime 测量单个方法调用时间。正如Javadoc所建议的，我们只执行单个基准方法调用。
* 在这种模式下，迭代时间是没有意义的：一旦基准方法停止，迭代就结束了。
*
* This mode is useful to do cold startup tests, when you specifically
* do not want to call the benchmark method continuously.
*
* 当您特别不想连续调用基准方法时，此模式对于执行冷启动测试非常有用。
*/
@Benchmark
@BenchmarkMode(Mode.SingleShotTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void measureSingleShot() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp7_Mode_SingleShotTime.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）4运行结果：Mode.SingleShotTime 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=4700:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: <none>
# Measurement: 1 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_SingleShotTime.measureSingleShot
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
Iteration 1: 104832.800 us/op
# Run complete. Total time: 00:00:01
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Mode_SingleShotTime.measureSingleShot ss 104832.800 us/op
5）All模式：所有基准模式。也可以设置多个模式。
【官方样例（JMHSample_02_BenchmarkModes）5：Mode.All 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的All模式
* @author 大白有点菜
*/
public class JmhTestApp7_Mode_All {
/**
* Or even...
*/
@Benchmark
@BenchmarkMode(Mode.All)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void measureAll() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp7_Mode_All.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）5运行结果：Mode.All 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5009:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll
# Run progress: 0.00% complete, ETA 00:05:00
# Fork: 1 of 1
# Warmup Iteration 1: ≈ 10⁻⁵ ops/us
# Warmup Iteration 2: ≈ 10⁻⁵ ops/us
# Warmup Iteration 3: ≈ 10⁻⁵ ops/us
# Warmup Iteration 4: ≈ 10⁻⁵ ops/us
# Warmup Iteration 5: ≈ 10⁻⁵ ops/us
Iteration 1: ≈ 10⁻⁵ ops/us
Iteration 2: ≈ 10⁻⁵ ops/us
Iteration 3: ≈ 10⁻⁵ ops/us
Iteration 4: ≈ 10⁻⁵ ops/us
Iteration 5: ≈ 10⁻⁵ ops/us
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll":
≈ 10⁻⁵ ops/us
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5009:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll
# Run progress: 33.33% complete, ETA 00:03:23
# Fork: 1 of 1
# Warmup Iteration 1: 108604.346 us/op
# Warmup Iteration 2: 108580.663 us/op
# Warmup Iteration 3: 108595.484 us/op
# Warmup Iteration 4: 108302.799 us/op
# Warmup Iteration 5: 108270.811 us/op
Iteration 1: 108260.180 us/op
Iteration 2: 108307.660 us/op
Iteration 3: 108350.144 us/op
Iteration 4: 108562.173 us/op
Iteration 5: 108278.643 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll":
108351.760 ±(99.9%) 471.425 us/op [Average]
(min, avg, max) = (108260.180, 108351.760, 108562.173), stdev = 122.428
CI (99.9%): [107880.335, 108823.185] (assumes normal distribution)
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5009:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Sampling time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll
# Run progress: 66.67% complete, ETA 00:01:41
# Fork: 1 of 1
# Warmup Iteration 1: 108093.528 ±(99.9%) 406.919 us/op
# Warmup Iteration 2: 108630.500 ±(99.9%) 470.883 us/op
# Warmup Iteration 3: 108440.235 ±(99.9%) 354.199 us/op
# Warmup Iteration 4: 108458.557 ±(99.9%) 336.270 us/op
# Warmup Iteration 5: 108502.247 ±(99.9%) 428.759 us/op
Iteration 1: 108156.950 ±(99.9%) 321.022 us/op
measureAll·p0.00: 106168.320 us/op
measureAll·p0.50: 108134.400 us/op
measureAll·p0.90: 109445.120 us/op
measureAll·p0.95: 109576.192 us/op
measureAll·p0.99: 110755.840 us/op
measureAll·p0.999: 110755.840 us/op
measureAll·p0.9999: 110755.840 us/op
measureAll·p1.00: 110755.840 us/op
Iteration 2: 108531.844 ±(99.9%) 398.651 us/op
measureAll·p0.00: 104071.168 us/op
measureAll·p0.50: 108396.544 us/op
measureAll·p0.90: 109969.408 us/op
measureAll·p0.95: 110441.267 us/op
measureAll·p0.99: 111673.344 us/op
measureAll·p0.999: 111673.344 us/op
measureAll·p0.9999: 111673.344 us/op
measureAll·p1.00: 111673.344 us/op
Iteration 3: 108369.766 ±(99.9%) 373.598 us/op
measureAll·p0.00: 106299.392 us/op
measureAll·p0.50: 108265.472 us/op
measureAll·p0.90: 109707.264 us/op
measureAll·p0.95: 110310.195 us/op
measureAll·p0.99: 112066.560 us/op
measureAll·p0.999: 112066.560 us/op
measureAll·p0.9999: 112066.560 us/op
measureAll·p1.00: 112066.560 us/op
Iteration 4: 108689.694 ±(99.9%) 442.667 us/op
measureAll·p0.00: 105512.960 us/op
measureAll·p0.50: 108527.616 us/op
measureAll·p0.90: 110703.411 us/op
measureAll·p0.95: 111149.056 us/op
measureAll·p0.99: 111804.416 us/op
measureAll·p0.999: 111804.416 us/op
measureAll·p0.9999: 111804.416 us/op
measureAll·p1.00: 111804.416 us/op
Iteration 5: 108674.360 ±(99.9%) 458.997 us/op
measureAll·p0.00: 106299.392 us/op
measureAll·p0.50: 108396.544 us/op
measureAll·p0.90: 110716.518 us/op
measureAll·p0.95: 111326.003 us/op
measureAll·p0.99: 112328.704 us/op
measureAll·p0.999: 112328.704 us/op
measureAll·p0.9999: 112328.704 us/op
measureAll·p1.00: 112328.704 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll":
N = 464
mean = 108484.114 ±(99.9%) 176.978 us/op
Histogram, us/op:
[104000.000, 104500.000) = 1
[104500.000, 105000.000) = 0
[105000.000, 105500.000) = 0
[105500.000, 106000.000) = 1
[106000.000, 106500.000) = 6
[106500.000, 107000.000) = 19
[107000.000, 107500.000) = 64
[107500.000, 108000.000) = 75
[108000.000, 108500.000) = 89
[108500.000, 109000.000) = 78
[109000.000, 109500.000) = 51
[109500.000, 110000.000) = 36
[110000.000, 110500.000) = 15
[110500.000, 111000.000) = 9
[111000.000, 111500.000) = 12
[111500.000, 112000.000) = 6
[112000.000, 112500.000) = 2
Percentiles, us/op:
p(0.0000) = 104071.168 us/op
p(50.0000) = 108265.472 us/op
p(90.0000) = 109969.408 us/op
p(95.0000) = 110755.840 us/op
p(99.0000) = 111673.344 us/op
p(99.9000) = 112328.704 us/op
p(99.9900) = 112328.704 us/op
p(99.9990) = 112328.704 us/op
p(99.9999) = 112328.704 us/op
p(100.0000) = 112328.704 us/op
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5009:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: <none>
# Measurement: 1 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Mode_All.measureAll
# Run progress: 100.00% complete, ETA 00:00:00
# Fork: 1 of 1
Iteration 1: 104234.900 us/op
# Run complete. Total time: 00:05:06
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Mode_All.measureAll thrpt 5 ≈ 10⁻⁵ ops/us
JmhTestApp7_Mode_All.measureAll avgt 5 108351.760 ± 471.425 us/op
JmhTestApp7_Mode_All.measureAll sample 464 108484.114 ± 176.978 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.00 sample 104071.168 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.50 sample 108265.472 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.90 sample 109969.408 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.95 sample 110755.840 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.99 sample 111673.344 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.999 sample 112328.704 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p0.9999 sample 112328.704 us/op
JmhTestApp7_Mode_All.measureAll:measureAll·p1.00 sample 112328.704 us/op
JmhTestApp7_Mode_All.measureAll ss 104234.900 us/op
【官方样例（JMHSample_02_BenchmarkModes）6：Mode.All 模式的另外一种写法 {Mode.Throughput, Mode.AverageTime, Mode.SampleTime, Mode.SingleShotTime}】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试7：BenchmarkMode中的All模式的另外一种实现 {Mode.Throughput, Mode.AverageTime, Mode.SampleTime, Mode.SingleShotTime}
* @author 大白有点菜
*/
public class JmhTestApp7_Multiple {
/**
* We can also ask for multiple benchmark modes at once. All the tests
* above can be replaced with just a single test like this:
*
* 我们还可以同时要求多个基准模式。上面的所有测试都可以用这样的单个测试来代替：
*/
@Benchmark
@BenchmarkMode({Mode.Throughput, Mode.AverageTime, Mode.SampleTime, Mode.SingleShotTime})
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void measureMultiple() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp7_Multiple.class.getSimpleName())
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_02_BenchmarkModes）6运行结果：Mode.All 模式的另外一种写法 {Mode.Throughput, Mode.AverageTime, Mode.SampleTime, Mode.SingleShotTime}】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5362:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple
# Run progress: 0.00% complete, ETA 00:05:00
# Fork: 1 of 1
# Warmup Iteration 1: ≈ 10⁻⁵ ops/us
# Warmup Iteration 2: ≈ 10⁻⁵ ops/us
# Warmup Iteration 3: ≈ 10⁻⁵ ops/us
# Warmup Iteration 4: ≈ 10⁻⁵ ops/us
# Warmup Iteration 5: ≈ 10⁻⁵ ops/us
Iteration 1: ≈ 10⁻⁵ ops/us
Iteration 2: ≈ 10⁻⁵ ops/us
Iteration 3: ≈ 10⁻⁵ ops/us
Iteration 4: ≈ 10⁻⁵ ops/us
Iteration 5: ≈ 10⁻⁵ ops/us
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple":
≈ 10⁻⁵ ops/us
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5362:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple
# Run progress: 33.33% complete, ETA 00:03:23
# Fork: 1 of 1
# Warmup Iteration 1: 108449.824 us/op
# Warmup Iteration 2: 108303.587 us/op
# Warmup Iteration 3: 108324.440 us/op
# Warmup Iteration 4: 108565.077 us/op
# Warmup Iteration 5: 108615.320 us/op
Iteration 1: 108481.380 us/op
Iteration 2: 108280.342 us/op
Iteration 3: 108210.744 us/op
Iteration 4: 108302.180 us/op
Iteration 5: 108525.647 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple":
108360.058 ±(99.9%) 524.232 us/op [Average]
(min, avg, max) = (108210.744, 108360.058, 108525.647), stdev = 136.141
CI (99.9%): [107835.826, 108884.291] (assumes normal distribution)
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5362:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Sampling time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple
# Run progress: 66.67% complete, ETA 00:01:41
# Fork: 1 of 1
# Warmup Iteration 1: 108132.991 ±(99.9%) 403.858 us/op
# Warmup Iteration 2: 108595.266 ±(99.9%) 484.952 us/op
# Warmup Iteration 3: 108288.022 ±(99.9%) 328.665 us/op
# Warmup Iteration 4: 108585.400 ±(99.9%) 384.368 us/op
# Warmup Iteration 5: 108600.904 ±(99.9%) 452.161 us/op
Iteration 1: 108368.356 ±(99.9%) 326.058 us/op
measureMultiple·p0.00: 106430.464 us/op
measureMultiple·p0.50: 108265.472 us/op
measureMultiple·p0.90: 109654.835 us/op
measureMultiple·p0.95: 110139.802 us/op
measureMultiple·p0.99: 110493.696 us/op
measureMultiple·p0.999: 110493.696 us/op
measureMultiple·p0.9999: 110493.696 us/op
measureMultiple·p1.00: 110493.696 us/op
Iteration 2: 108476.878 ±(99.9%) 368.247 us/op
measureMultiple·p0.00: 106430.464 us/op
measureMultiple·p0.50: 108265.472 us/op
measureMultiple·p0.90: 109969.408 us/op
measureMultiple·p0.95: 110664.090 us/op
measureMultiple·p0.99: 111673.344 us/op
measureMultiple·p0.999: 111673.344 us/op
measureMultiple·p0.9999: 111673.344 us/op
measureMultiple·p1.00: 111673.344 us/op
Iteration 3: 109012.013 ±(99.9%) 444.440 us/op
measureMultiple·p0.00: 106168.320 us/op
measureMultiple·p0.50: 109051.904 us/op
measureMultiple·p0.90: 110847.590 us/op
measureMultiple·p0.95: 111542.272 us/op
measureMultiple·p0.99: 111804.416 us/op
measureMultiple·p0.999: 111804.416 us/op
measureMultiple·p0.9999: 111804.416 us/op
measureMultiple·p1.00: 111804.416 us/op
Iteration 4: 109298.376 ±(99.9%) 432.264 us/op
measureMultiple·p0.00: 106692.608 us/op
measureMultiple·p0.50: 109182.976 us/op
measureMultiple·p0.90: 111149.056 us/op
measureMultiple·p0.95: 111588.147 us/op
measureMultiple·p0.99: 111673.344 us/op
measureMultiple·p0.999: 111673.344 us/op
measureMultiple·p0.9999: 111673.344 us/op
measureMultiple·p1.00: 111673.344 us/op
Iteration 5: 108406.410 ±(99.9%) 347.513 us/op
measureMultiple·p0.00: 106430.464 us/op
measureMultiple·p0.50: 108396.544 us/op
measureMultiple·p0.90: 109654.835 us/op
measureMultiple·p0.95: 110572.339 us/op
measureMultiple·p0.99: 111542.272 us/op
measureMultiple·p0.999: 111542.272 us/op
measureMultiple·p0.9999: 111542.272 us/op
measureMultiple·p1.00: 111542.272 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple":
N = 463
mean = 108710.494 ±(99.9%) 177.069 us/op
Histogram, us/op:
[106000.000, 106500.000) = 4
[106500.000, 107000.000) = 17
[107000.000, 107500.000) = 47
[107500.000, 108000.000) = 66
[108000.000, 108500.000) = 75
[108500.000, 109000.000) = 76
[109000.000, 109500.000) = 71
[109500.000, 110000.000) = 53
[110000.000, 110500.000) = 15
[110500.000, 111000.000) = 17
[111000.000, 111500.000) = 7
Percentiles, us/op:
p(0.0000) = 106168.320 us/op
p(50.0000) = 108658.688 us/op
p(90.0000) = 110231.552 us/op
p(95.0000) = 110886.912 us/op
p(99.0000) = 111673.344 us/op
p(99.9000) = 111804.416 us/op
p(99.9900) = 111804.416 us/op
p(99.9990) = 111804.416 us/op
p(99.9999) = 111804.416 us/op
p(100.0000) = 111804.416 us/op
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=5362:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: <none>
# Measurement: 1 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp7_Multiple.measureMultiple
# Run progress: 100.00% complete, ETA 00:00:00
# Fork: 1 of 1
Iteration 1: 109474.600 us/op
# Run complete. Total time: 00:05:06
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp7_Multiple.measureMultiple thrpt 5 ≈ 10⁻⁵ ops/us
JmhTestApp7_Multiple.measureMultiple avgt 5 108360.058 ± 524.232 us/op
JmhTestApp7_Multiple.measureMultiple sample 463 108710.494 ± 177.069 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.00 sample 106168.320 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.50 sample 108658.688 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.90 sample 110231.552 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.95 sample 110886.912 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.99 sample 111673.344 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.999 sample 111804.416 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p0.9999 sample 111804.416 us/op
JmhTestApp7_Multiple.measureMultiple:measureMultiple·p1.00 sample 111804.416 us/op
JmhTestApp7_Multiple.measureMultiple ss 109474.600 us/op
5、@OutputTimeUnit
OutputTimeUnit提供了统计结果输出时的默认单位时间，比如，调用一次该方法将会耗费多少个单位时间，或者在单位时间内对该方法进行了多少次的调用，同样，OutputTimeUnit既可以设置在class上，也可以设置在method上，还可以在Options中进行设置，它们的覆盖次序与BenchmarkMode一致。
【@OutputTimeUnit 样例代码】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试8
* @author 大白有点菜
*/
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Measurement(iterations = 5)
@Warmup(iterations = 2)
@State(Scope.Thread)
public class JmhTestApp8 {
@Benchmark
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void testOutputTimeUnit() throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(100);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp8.class.getSimpleName())
.forks(1)
.timeUnit(TimeUnit.NANOSECONDS)
.build();
new Runner(opts).run();
}
}
【@OutputTimeUnit 样例运行结果】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=6661:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 2 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp8.testOutputTimeUnit
# Run progress: 0.00% complete, ETA 00:01:10
# Fork: 1 of 1
# Warmup Iteration 1: ≈ 10⁻⁸ ops/ns
# Warmup Iteration 2: ≈ 10⁻⁸ ops/ns
Iteration 1: ≈ 10⁻⁸ ops/ns
Iteration 2: ≈ 10⁻⁸ ops/ns
Iteration 3: ≈ 10⁻⁸ ops/ns
Iteration 4: ≈ 10⁻⁸ ops/ns
Iteration 5: ≈ 10⁻⁸ ops/ns
Result "cn.zhuangyt.javabase.jmh.JmhTestApp8.testOutputTimeUnit":
≈ 10⁻⁸ ops/ns
# Run complete. Total time: 00:01:12
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp8.testOutputTimeUnit thrpt 5 ≈ 10⁻⁸ ops/ns
6、@State
（1）JMH使用 @BenchmarkMode 注解来标记状态对象。【State接口官方注解 - 谷歌翻译 - start】 State 对象自然地封装了基准测试所处理的状态。 State 对象的范围定义了它在工作线程之间共享的程度。State 对象通常作为参数注入到 Benchmark 方法中，JMH 负责它们的实例化和共享。 State 对象也可以注入到其他 State 对象的 Setup 和 TearDown 方法中，以进行分阶段初始化。 那么State-s之间的依赖图应该是有向无环图。State 对象可以被继承：你可以将 State 放在超类上，并使用子类作为状态。【State接口官方注解 - 谷歌翻译 - end】 主要包括 Scope 的3个枚举值，如下：
- Benchmark：使用基准范围，相同类型的所有实例将在所有工作线程之间共享。此状态对象上的 Setup 和 TearDown 方法将由其中一个工作线程执行，并且每个级别仅执行一次。 没有其它线程会触及状态对象。
- Group：使用组范围，同一类型的所有实例将在同一组内的所有线程之间共享。 每个线程组都将提供自己的状态对象。
此状态对象上的 Setup 和 TearDown 方法将由其中一个组线程执行，并且每个级别仅执行一次。 没有其它线程会触及状态对象。
- Thread：使用线程范围，即使在同一个基准测试中注入了多个状态对象，相同类型的所有实例都是不同的。此状态对象上的 Setup 和 TearDown 方法将由单个工作线程独占执行，并且每个级别仅执行一次。 没有其它线程会触及状态对象。
【State接口源码】
package org.openjdk.jmh.annotations;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* <p>Marks the state object.</p>
*
* <p>State objects naturally encapsulate the state on which benchmark is working on.
* The {@link Scope} of state object defines to which extent it is shared among the
* worker threads.</p>
*
* <p>State objects are usually injected into {@link Benchmark} methods as arguments,
* and JMH takes care of their instantiation and sharing. State objects may also be
* injected into {@link Setup} and {@link TearDown} methods of other {@link State}
* objects to get the staged initialization. In that case, the dependency graph
* between the {@link State}-s should be directed acyclic graph.</p>
*
* <p>State objects may be inherited: you can place {@link State} on a super class and
* use subclasses as states.</p>
*/
@Inherited
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface State {
/**
* State scope.
* @return state scope
* @see Scope
*/
Scope value();
}
【Scope枚举类源码】
package org.openjdk.jmh.annotations;
/**
* {@link State} scope.
*/
public enum Scope {
/**
* <p>Benchmark state scope.</p>
*
* <p>With benchmark scope, all instances of the same type will be shared across all
* worker threads.</p>
*
* <p>{@link Setup} and {@link TearDown} methods on this state object would be performed
* by one of the worker threads, and only once per {@link Level}.
* No other threads would ever touch the state object.</p>
*/
Benchmark,
/**
* <p>Group state scope.</p>
*
* <p>With group scope, all instances of the same type will be shared across all
* threads within the same group. Each thread group will be supplied with its own
* state object.</p>
*
* <p>{@link Setup} and {@link TearDown} methods on this state object would be performed
* by one of the group threads, and only once per {@link Level}.
* No other threads would ever touch the state object.</p>
*
* @see Group
*/
Group,
/**
* <p>Thread state scope.</p>
*
* <p>With thread scope, all instances of the same type are distinct, even if multiple
* state objects are injected in the same benchmark</p>
*
* <p>{@link Setup} and {@link TearDown} methods on this state object would be performed
* by single worker thread exclusively, and only once per {@link Level}.
* No other threads would ever touch the state object.</p>
*
*/
Thread,
}
（2）Thread独享的State
所谓线程独享的 State 是指，每一个运行基准测试方法的线程都会持有一个独立的对象实例，该实例既可能是作为基准测试方法参数传入的，也可能是运行基准方法所在的宿主class，将 State 设置为 Scope.Thread 一般主要是针对非线程安全的类。
【官方样例（JMHSample_03_States）1：Scope.Thread 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
/**
* JMH测试9：@State中的Scope.Thread模式
* @author 大白有点菜
*/
public class JmhTestApp9_Scope_Thread {
@State(Scope.Thread)
public static class ThreadState {
volatile double x = Math.PI;
}
/**
* Benchmark methods can reference the states, and JMH will inject the
* appropriate states while calling these methods. You can have no states at
* all, or have only one state, or have multiple states referenced. This
* makes building multi-threaded benchmark a breeze.
*
* 基准方法可以引用状态，JMH将在调用这些方法时注入适当的状态。您可以完全没有状态，或者只有一个状态，或者引用多个状态。
* 这使得构建多线程基准测试变得轻而易举。
*
* For this exercise, we have two methods.
*
* 对于这个练习，我们有两种方法。
*/
@Benchmark
public void measureUnshared(ThreadState state) {
// All benchmark threads will call in this method.
//
// 所有基准测试线程都将调用此方法。
//
// However, since ThreadState is the Scope.Thread, each thread
// will have it's own copy of the state, and this benchmark
// will measure unshared case.
//
// 然而，由于 ThreadState 是 Scope.Thread，每个线程都有自己的状态副本，并且此基准测试将衡量未共享的情况。
state.x++;
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp9_Scope_Thread.class.getSimpleName())
.threads(4)
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_03_States）1运行结果：Scope.Thread 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=7174:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 4 threads, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp9_Scope_Thread.measureUnshared
# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 505696718.944 ops/s
# Warmup Iteration 2: 506686232.238 ops/s
# Warmup Iteration 3: 513358631.861 ops/s
# Warmup Iteration 4: 504199913.029 ops/s
# Warmup Iteration 5: 518579614.668 ops/s
Iteration 1: 527669924.138 ops/s
Iteration 2: 524116235.523 ops/s
Iteration 3: 514012396.968 ops/s
Iteration 4: 513185321.114 ops/s
Iteration 5: 523309077.183 ops/s
Result "cn.zhuangyt.javabase.jmh.JmhTestApp9_Scope_Thread.measureUnshared":
520458590.985 ±(99.9%) 24952115.677 ops/s [Average]
(min, avg, max) = (513185321.114, 520458590.985, 527669924.138), stdev = 6479985.202
CI (99.9%): [495506475.308, 545410706.662] (assumes normal distribution)
# Run complete. Total time: 00:01:42
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp9_Scope_Thread.measureUnshared thrpt 5 520458590.985 ± 24952115.677 ops/s
【书本样例1：Scope.Thread 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试10：@State中的Scope.Thread模式
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
@Warmup(iterations = 5)
@Measurement(iterations = 10)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
// 设置5个线程运行基准测试方法
@Threads(5)
public class JmhTestApp10_Scope_Thread {
/**
* 5个运行线程，每一个线程都会持有一个Test的实例
*/
@State(Scope.Thread)
public static class Test
{
public Test()
{
System.out.println("大白有点菜");
}
public void method()
{
}
}
/**
* 通过基准测试将State引用传入
* @param test
*/
@Benchmark
public void test(Test test)
{
test.method();
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp10_Scope_Thread.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书本样例1运行结果：Scope.Thread 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=7513:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 5 threads, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Thread.test
# Run progress: 0.00% complete, ETA 00:02:30
# Fork: 1 of 1
# Warmup Iteration 1: 大白有点菜
大白有点菜
大白有点菜
大白有点菜
大白有点菜
0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 2: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 3: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 4: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 5: 0.001 ±(99.9%) 0.001 us/op
Iteration 1: 0.001 ±(99.9%) 0.001 us/op
Iteration 2: 0.001 ±(99.9%) 0.001 us/op
Iteration 3: 0.001 ±(99.9%) 0.001 us/op
Iteration 4: 0.001 ±(99.9%) 0.001 us/op
Iteration 5: 0.001 ±(99.9%) 0.001 us/op
Iteration 6: 0.001 ±(99.9%) 0.001 us/op
Iteration 7: 0.001 ±(99.9%) 0.001 us/op
Iteration 8: 0.001 ±(99.9%) 0.001 us/op
Iteration 9: 0.001 ±(99.9%) 0.001 us/op
Iteration 10: 0.001 ±(99.9%) 0.001 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Thread.test":
0.001 ±(99.9%) 0.001 us/op [Average]
(min, avg, max) = (0.001, 0.001, 0.001), stdev = 0.001
CI (99.9%): [0.001, 0.001] (assumes normal distribution)
# Run complete. Total time: 00:02:31
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp10_Scope_Thread.test avgt 10 0.001 ± 0.001 us/op
运行书中样例1，会看到“大白有点菜”字样出现了5次。
（3）Thread共享的State
有时我们需要测试在多线程的情况下某个类被不同线程操作时的性能，比如，多线程访问某个共享数据时，我们需要让多个线程使用同一个实例才可以。因此JMH提供了多线程共享的一种状态Scope.Benchmark。
【官方样例（JMHSample_03_States）2：Scope.Benchmark 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
/**
* JMH测试9：@State中的Scope.Benchmark模式
* @author 大白有点菜
*/
public class JmhTestApp9_Scope_Benchmark {
@State(Scope.Benchmark)
public static class BenchmarkState {
volatile double x = Math.PI;
}
/**
* Benchmark methods can reference the states, and JMH will inject the
* appropriate states while calling these methods. You can have no states at
* all, or have only one state, or have multiple states referenced. This
* makes building multi-threaded benchmark a breeze.
*
* 基准方法可以引用状态，JMH将在调用这些方法时注入适当的状态。您可以完全没有状态，或者只有一个状态，或者引用多个状态。
* 这使得构建多线程基准测试变得轻而易举。
*
*/
@Benchmark
public void measureShared(BenchmarkState state) {
// All benchmark threads will call in this method.
//
// 所有基准测试线程都将调用此方法。
//
// Since BenchmarkState is the Scope.Benchmark, all threads
// will share the state instance, and we will end up measuring
// shared case.
//
// 由于 BenchmarkState 是 Scope.Benchmark，所有线程将共享状态实例，我们将最终测量共享情况。
state.x++;
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp9_Scope_Benchmark.class.getSimpleName())
.threads(4)
.forks(1)
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_03_States）2运行结果：Scope.Benchmark 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=8054:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 4 threads, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp9_Scope_Benchmark.measureShared
# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 58510065.637 ops/s
# Warmup Iteration 2: 56490358.586 ops/s
# Warmup Iteration 3: 57563927.347 ops/s
# Warmup Iteration 4: 59405519.820 ops/s
# Warmup Iteration 5: 58989965.640 ops/s
Iteration 1: 60619868.940 ops/s
Iteration 2: 58317280.827 ops/s
Iteration 3: 57503349.240 ops/s
Iteration 4: 58527274.114 ops/s
Iteration 5: 58634007.609 ops/s
Result "cn.zhuangyt.javabase.jmh.JmhTestApp9_Scope_Benchmark.measureShared":
58720356.146 ±(99.9%) 4430943.982 ops/s [Average]
(min, avg, max) = (57503349.240, 58720356.146, 60619868.940), stdev = 1150702.081
CI (99.9%): [54289412.164, 63151300.128] (assumes normal distribution)
# Run complete. Total time: 00:01:41
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp9_Scope_Benchmark.measureShared thrpt 5 58720356.146 ± 4430943.982 ops/s
【书本样例2：Scope.Benchmark 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试10：@State中的Scope.Thread模式
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
@Warmup(iterations = 5)
@Measurement(iterations = 10)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
// 设置5个线程运行基准测试方法
@Threads(5)
public class JmhTestApp10_Scope_Thread {
/**
* 5个运行线程，每一个线程都会持有一个Test的实例
*/
@State(Scope.Thread)
public static class Test
{
public Test()
{
System.out.println("大白有点菜");
}
public void method()
{
}
}
/**
* 通过基准测试将State引用传入
* @param test
*/
@Benchmark
public void test(Test test)
{
test.method();
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp10_Scope_Thread.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书本样例2运行结果：Scope.Benchmark 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=8523:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 5 threads, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Benchmark.test
# Run progress: 0.00% complete, ETA 00:02:30
# Fork: 1 of 1
# Warmup Iteration 1: 大白有点菜
0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 2: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 3: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 4: 0.001 ±(99.9%) 0.001 us/op
# Warmup Iteration 5: 0.001 ±(99.9%) 0.001 us/op
Iteration 1: 0.001 ±(99.9%) 0.001 us/op
Iteration 2: 0.001 ±(99.9%) 0.001 us/op
Iteration 3: 0.001 ±(99.9%) 0.001 us/op
Iteration 4: 0.001 ±(99.9%) 0.001 us/op
Iteration 5: 0.001 ±(99.9%) 0.001 us/op
Iteration 6: 0.001 ±(99.9%) 0.001 us/op
Iteration 7: 0.001 ±(99.9%) 0.001 us/op
Iteration 8: 0.001 ±(99.9%) 0.001 us/op
Iteration 9: 0.001 ±(99.9%) 0.001 us/op
Iteration 10: 0.001 ±(99.9%) 0.001 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Benchmark.test":
0.001 ±(99.9%) 0.001 us/op [Average]
(min, avg, max) = (0.001, 0.001, 0.001), stdev = 0.001
CI (99.9%): [0.001, 0.001] (assumes normal distribution)
# Run complete. Total time: 00:02:31
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp10_Scope_Benchmark.test avgt 10 0.001 ± 0.001 us/op
运行书中样例2，会看到“大白有点菜”字样只出现了1次。
（4）线程组共享的State
目前为止，我们所编写的基准测试方法都会被JMH框架根据方法名的字典顺序排序后按照顺序逐个地调用执行，因此不存在两个方法同时运行的情况，如果想要测试某个共享数据或共享资源在多线程的情况下同时被读写的行为，是没有办法进行的，比如，在多线程高并发的环境中，多个线程同时对一个ConcurrentHashMap进行读写。
简单地归纳出：第一，是在多线程情况下的单个实例；第二，允许一个以上的基准测试方法并发并行地运行。
所幸的是，Scope.Group可以帮助我们实现这一点，先来看一个简单的例子。
【书本样例3：Scope.Benchmark 模式】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试10：@State中的Scope.Thread模式
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
@Warmup(iterations = 5)
@Measurement(iterations = 10)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
// 设置5个线程运行基准测试方法
@Threads(5)
public class JmhTestApp10_Scope_Thread {
/**
* 5个运行线程，每一个线程都会持有一个Test的实例
*/
@State(Scope.Thread)
public static class Test
{
public Test()
{
System.out.println("大白有点菜");
}
public void method()
{
}
}
/**
* 通过基准测试将State引用传入
* @param test
*/
@Benchmark
public void test(Test test)
{
test.method();
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp10_Scope_Thread.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书本样例2运行结果：Scope.Benchmark 模式】
# JMH version: 1.36
# VM version: JDK 1.8.0_281, Java HotSpot(TM) 64-Bit Server VM, 25.281-b09
# VM invoker: D:\Develop\JDK\jdk1.8.0_281\jre\bin\java.exe
# VM options: -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\lib\idea_rt.jar=9146:C:\Program Files\JetBrains\IntelliJ IDEA 2021.1.3\bin -Dfile.encoding=UTF-8
# Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 10 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 6 threads (1 group; 3x "testRead", 3x "testWrite" in each group), will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Group.test
# Run progress: 0.00% complete, ETA 00:02:30
# Fork: 1 of 1
# Warmup Iteration 1: 大白有点菜
.....省略
读取
读取
读取
写入
写入
读取
写入
177.955 ±(99.9%) 111.590 us/op
testRead: 179.907 ±(99.9%) 998.243 us/op
testWrite: 176.003 ±(99.9%) 563.344 us/op
Result "cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Group.test":
150.983 ±(99.9%) 18.471 us/op [Average]
(min, avg, max) = (139.016, 150.983, 177.955), stdev = 12.218
CI (99.9%): [132.512, 169.454] (assumes normal distribution)
Secondary result "cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Group.test:testRead":
147.630 ±(99.9%) 35.905 us/op [Average]
(min, avg, max) = (113.613, 147.630, 179.907), stdev = 23.749
CI (99.9%): [111.725, 183.536] (assumes normal distribution)
Secondary result "cn.zhuangyt.javabase.jmh.JmhTestApp10_Scope_Group.test:testWrite":
154.335 ±(99.9%) 38.908 us/op [Average]
(min, avg, max) = (109.814, 154.335, 187.453), stdev = 25.735
CI (99.9%): [115.427, 193.243] (assumes normal distribution)
# Run complete. Total time: 00:02:32
Benchmark Mode Cnt Score Error Units
JmhTestApp10_Scope_Group.test avgt 10 150.983 ± 18.471 us/op
JmhTestApp10_Scope_Group.test:testRead avgt 10 147.630 ± 35.905 us/op
JmhTestApp10_Scope_Group.test:testWrite avgt 10 154.335 ± 38.908 us/op
从运行结果看出，总共6个线程会执行基准测试方法，这6个线程都在同一个group中，其中，testRead方法会被3个线程执行，testWrite方法会被3个线程执行。read和write分别交替输出，因此testRead和testWrite是交替执行的。
7、@Param
写一个例子来测试 ConcurrentHashMap 和 SynchronizedMap 的 put 方法的性能。
【书中样例1，照搬会报OOM错误，修改为指定的运行时长】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
/**
* JMH测试11：@Param用法1
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
// 此处修改：指定运行时长
@Warmup(iterations = 5, time = 6000, timeUnit = TimeUnit.MICROSECONDS)
@Measurement(iterations = 10, time = 6000, timeUnit = TimeUnit.MICROSECONDS)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
// 设置5个线程运行基准测试方法
@Threads(5)
@State(Scope.Benchmark)
public class JmhTestApp11_Param1 {
private Map<Long, Long> concurrentMap;
private Map<Long, Long> synchronizedMap;
@Setup
public void setUp()
{
concurrentMap = new ConcurrentHashMap<>();
synchronizedMap = Collections.synchronizedMap(new HashMap<>());
}
@Benchmark
public void testConcurrencyMap()
{
this.concurrentMap.put(System.nanoTime(), System.nanoTime());
}
@Benchmark
public void testSynchronizedMap()
{
this.synchronizedMap.put(System.nanoTime(), System.nanoTime());
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp11_Param1.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书中样例1运行结果】
Benchmark Mode Cnt Score Error Units
JmhTestApp11_Param.testConcurrencyMap avgt 10 1.151 ± 2.155 us/op
JmhTestApp11_Param.testSynchronizedMap avgt 10 102.052 ± 462.463 us/op
通过基准测试，ConcurrentHashMap 比 SynchronizedMap 的表现要优秀很多（在多线程同时对其进行put操作时）。
Java提供的具备线程安全的Map接口实现除了 ConcurrentHashMap 和 SynchronizedMap，还有 ConcurrentSkipListMap 和 Hashtable 可以选择。如果要对其进行测试，那么需要再增加两个不同类型的Map和两个针对这两个Map实现的基准测试方法。但是很显然，这种方式存在大量的代码冗余，因此JMH为我们提供了一个@Param的注解，它使得参数可配置，也就是说一个参数在每一次的基准测试时都会有不同的值与之对应。
【书中样例2，照搬会报OOM错误，修改为指定的运行时长】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.Collections;
import java.util.HashMap;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.TimeUnit;
/**
* JMH测试11：@Param用法2
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@Fork(1)
@Warmup(iterations = 5, time = 2, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 10, time = 2, timeUnit = TimeUnit.SECONDS)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
// 设置5个线程运行基准测试方法
@Threads(5)
@State(Scope.Benchmark)
public class JmhTestApp11_Param2 {
/**
* 为type提供了四种可配置的参数值
*/
@Param({"1", "2", "3", "4"})
private int type;
private Map<Long, Long> map;
@Setup
public void setUp()
{
switch (type)
{
case 1:
this.map = new ConcurrentHashMap<>();
break;
case 2:
this.map = new ConcurrentSkipListMap<>();
break;
case 3:
this.map = new Hashtable<>();
break;
case 4:
this.map = Collections.synchronizedMap(
new HashMap<>());
break;
default:
throw new IllegalArgumentException("Illegal map type.");
}
}
/**
* 只需要一个基准测试方法即可
*/
@Benchmark
public void test()
{
this.map.put(System.nanoTime(), System.nanoTime());
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp11_Param2.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书中样例2运行结果】
Benchmark (type) Mode Cnt Score Error Units
JmhTestApp11_Param2.test 1 avgt 5 9.847 ± 19.892 us/op
JmhTestApp11_Param2.test 2 avgt 5 92.409 ± 511.893 us/op
JmhTestApp11_Param2.test 3 avgt 5 8.416 ± 21.854 us/op
JmhTestApp11_Param2.test 4 avgt 5 17.567 ± 52.462 us/op
在setUp方法中，我们分别实例化了四种不同类型的Map实现类，分别对应于@Param的不同参数。Param与不同类型Map的对应关系如下表格所示。
| Param 值 | Map接口的实现 |
|---|---|
| 1 | ConcurrentHashMap |
| 2 | ConcurrentSkipListMap |
| 3 | Hashtable |
| 4 | Collections.synchronizedMap |
从运行结果中，type为2对应 ConcurrentSkipListMap 的结果和其它三个差异大，并不能就此说明它put方法的性能就很差。经过笔者多次运行，发现这4个每次的结果互相之间都会有波动，数值大小对比并不稳定。书中的例子笔者直接去运行，报 GC overhead limited 错误，那是垃圾回收不及时，占用大量内存导致的，笔者发现电脑内存一下子飙升得很高。
在前面的测试中，对所有线程安全Map的基准测试都是基于put方法进行的，也就是说并没有同时进行读写、修改、删除等动作，因此单凭对一个方法的基准测试就下定论说哪个性能好，哪个性能不好这种说法是不够严谨的。
【附上官方的一个样例（JMHSample_27_Params）】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.math.BigInteger;
import java.util.concurrent.TimeUnit;
/**
* JMH测试11：@Param用法3
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@State(Scope.Benchmark)
public class JmhTestApp11_Param3 {
/**
* In many cases, the experiments require walking the configuration space
* for a benchmark. This is needed for additional control, or investigating
* how the workload performance changes with different settings.
*
* 在许多情况下，实验需要在配置空间中进行基准测试。这是额外控制或调查工作负载性能如何随不同设置而变化所必需的。
*/
@Param({"1", "31", "65", "101", "103"})
public int arg;
@Param({"0", "1", "2", "4", "8", "16", "32"})
public int certainty;
@Benchmark
public boolean bench() {
return BigInteger.valueOf(arg).isProbablePrime(certainty);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp11_Param3.class.getSimpleName())
// .param("arg", "41", "42") // Use this to selectively constrain/override parameters
.build();
new Runner(opts).run();
}
}
【官方样例（JMHSample_27_Params）运行结果，总耗时6分25秒，茶都凉了】
# Run complete. Total time: 00:06:25
Benchmark (arg) (certainty) Mode Cnt Score Error Units
JmhTestApp11_Param3.bench 1 0 avgt 5 3.334 ± 0.286 ns/op
JmhTestApp11_Param3.bench 1 1 avgt 5 5.176 ± 0.399 ns/op
JmhTestApp11_Param3.bench 1 2 avgt 5 5.499 ± 1.102 ns/op
JmhTestApp11_Param3.bench 1 4 avgt 5 5.155 ± 0.138 ns/op
JmhTestApp11_Param3.bench 1 8 avgt 5 5.184 ± 0.614 ns/op
JmhTestApp11_Param3.bench 1 16 avgt 5 5.214 ± 0.502 ns/op
JmhTestApp11_Param3.bench 1 32 avgt 5 5.316 ± 0.798 ns/op
JmhTestApp11_Param3.bench 31 0 avgt 5 4.896 ± 1.177 ns/op
JmhTestApp11_Param3.bench 31 1 avgt 5 517.796 ± 112.742 ns/op
JmhTestApp11_Param3.bench 31 2 avgt 5 487.609 ± 67.261 ns/op
JmhTestApp11_Param3.bench 31 4 avgt 5 937.554 ± 129.048 ns/op
JmhTestApp11_Param3.bench 31 8 avgt 5 1766.238 ± 87.655 ns/op
JmhTestApp11_Param3.bench 31 16 avgt 5 3501.604 ± 157.523 ns/op
JmhTestApp11_Param3.bench 31 32 avgt 5 7025.745 ± 552.950 ns/op
JmhTestApp11_Param3.bench 65 0 avgt 5 4.887 ± 0.948 ns/op
JmhTestApp11_Param3.bench 65 1 avgt 5 1174.681 ± 137.184 ns/op
JmhTestApp11_Param3.bench 65 2 avgt 5 1108.444 ± 107.148 ns/op
JmhTestApp11_Param3.bench 65 4 avgt 5 1191.372 ± 70.621 ns/op
JmhTestApp11_Param3.bench 65 8 avgt 5 1205.699 ± 71.732 ns/op
JmhTestApp11_Param3.bench 65 16 avgt 5 1195.987 ± 62.160 ns/op
JmhTestApp11_Param3.bench 65 32 avgt 5 1333.416 ± 218.306 ns/op
JmhTestApp11_Param3.bench 101 0 avgt 5 4.961 ± 1.316 ns/op
JmhTestApp11_Param3.bench 101 1 avgt 5 629.886 ± 54.217 ns/op
JmhTestApp11_Param3.bench 101 2 avgt 5 631.038 ± 23.492 ns/op
JmhTestApp11_Param3.bench 101 4 avgt 5 1203.518 ± 22.493 ns/op
JmhTestApp11_Param3.bench 101 8 avgt 5 2357.052 ± 84.173 ns/op
JmhTestApp11_Param3.bench 101 16 avgt 5 4848.529 ± 378.556 ns/op
JmhTestApp11_Param3.bench 101 32 avgt 5 9260.880 ± 563.534 ns/op
JmhTestApp11_Param3.bench 103 0 avgt 5 4.431 ± 0.411 ns/op
JmhTestApp11_Param3.bench 103 1 avgt 5 559.861 ± 29.160 ns/op
JmhTestApp11_Param3.bench 103 2 avgt 5 571.062 ± 53.429 ns/op
JmhTestApp11_Param3.bench 103 4 avgt 5 1074.069 ± 38.266 ns/op
JmhTestApp11_Param3.bench 103 8 avgt 5 2212.901 ± 261.150 ns/op
JmhTestApp11_Param3.bench 103 16 avgt 5 4136.388 ± 248.863 ns/op
JmhTestApp11_Param3.bench 103 32 avgt 5 8159.819 ± 548.510 ns/op
8、JMH的测试套件（Fixture）：@Setup、@TearDown
在使用Junit编写单元测试的时候，我们可以使用的套件有@Before、@After、@BeforeClass、@AfterClass等。在JMH中，一样有测试套件，那就来具体说说。
（1）Setup（设置） 以及 TearDown（拆卸）
JMH提供了两个注解 @Setup 和 @TearDown 用于套件测试，其中 @Setup 会在每一个基准测试方法执行前被调用，通常用于资源的初始化，@TearDown 则会在基准测试方法被执行之后被调用，通常可用于资源的回收清理工作。@Setup 和 @TearDown 都用到 Level 这个枚举去定义运行级别。
【@Setup 源码】
package org.openjdk.jmh.annotations;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* <p>Setup marks the fixture method to be run before the benchmark.</p>
*
* <p>Since fixture methods manage the {@link State} lifecycles, {@link Setup}
* can only be declared in {@link State} classes. The {@link Setup} method will
* be executed by a thread which has the access to {@link State}, and it is not
* defined which thread exactly. Note that means {@link TearDown} may be executed
* by a different thread, if {@link State} is shared between the threads.</p>
*
* <p>Uses may optionally provide the {@link Level} at which the fixture method
* should run.</p>
*
* @see State
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Setup {
/**
* @return Level of this method.
* @see Level
*/
Level value() default Level.Trial;
}
【翻译@Setup 源码中的注解，同时参考了谷歌翻译和百度翻译】
Setup（设置） 标记的 fixture（套件） 方法会在在基准测试之前运行。
由于 fixture（套件） 方法管理 State 生命周期，因此只能在 State 类中声明 Setup（设置） 。Setup（设置） 方法将由一个具有 State 访问权限的线程执行，并且没有准确定义哪个线程。注意，如果线程之间共享 State ，则 TearDown（拆卸） 可能由不同的线程执行。
当 fixture（套件） 方法运行时使用可以选择提供 Level。
谷歌和百度两者翻译的有些语句都不是那么通顺，笔者英语烂，无法将其更好地优化，读者可自行去阅读原注解。
【@TearDown 源码】
package org.openjdk.jmh.annotations;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* <p>TearDown marks the fixture method to be run after the benchmark.</p>
*
* <p>Since fixture methods manage the {@link State} lifecycles, {@link TearDown}
* can only be declared in {@link State} classes. The {@link TearDown} method will
* be executed by a thread which has the access to {@link State}, and it is not
* defined which thread exactly. Note that means {@link Setup} may be executed
* by a different thread, if {@link State} is shared between the threads.</p>
*
* <p>Uses may optionally provide the {@link Level} at which the fixture method
* should run.</p>
*
* @see State
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface TearDown {
/**
* @return At which level to run this fixture.
* @see Level
*/
Level value() default Level.Trial;
}
【翻译@TearDown 源码中的注解，同时参考了谷歌翻译和百度翻译】
TearDown（拆卸） 标记的 fixture（套件） 方法会在在基准测试之前运行。
由于 fixture（套件） 方法管理 State 生命周期，因此只能在 State 类中声明 TearDown（拆卸） 。TearDown（拆卸） 方法将由一个具有 State 访问权限的线程执行，并且没有准确定义哪个线程。注意，如果线程之间共享 State ，则 Setup（设置） 可能由不同的线程执行。
当 fixture（套件） 方法运行时使用可以选择提供 Level。
【书本样例，有改动，指定运行时长】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
/**
* JMH测试12：@Setup用法
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@State(Scope.Thread)
public class JmhTestApp12_Setup {
/**
* 定义了一个List<String>，但是没有对其进行初始化
*/
private List<String> list;
/**
* 将方法标记为@Setup，执行初始化操作
*/
@Setup
public void setUp()
{
this.list = new ArrayList<>();
}
/**
* 简单地调用list的add 方法
*/
@Benchmark
public void measureRight()
{
this.list.add("Test");
}
/**
* 该方法什么都不做
*/
@Benchmark
public void measureWrong()
{
// do nothing
}
/**
* 将方法标记为@TearDown,运行资源回收甚至断言的操作
*/
@TearDown
public void tearDown() {
// 断言list中的元素个数大于0，很明显，measureWrong基准测试将会失败
assert this.list.size() > 0 : "The list elements must greater than zero";
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp12_Setup.class.getSimpleName())
// .jvmArgs("-ea") 激活断言，enable assertion的意思
.jvmArgs("-ea")
.build();
new Runner(opts).run();
}
}
【运行结果】
java.lang.AssertionError: The list elements must greater than zero
at cn.zhuangyt.javabase.jmh.JmhTestApp12_Setup.tearDown(JmhTestApp12_Setup.java:63)
at cn.zhuangyt.javabase.jmh.jmh_generated.JmhTestApp12_Setup_measureWrong_jmhTest.measureWrong_AverageTime(JmhTestApp12_Setup_measureWrong_jmhTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475)
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Result "cn.zhuangyt.javabase.jmh.JmhTestApp12_Setup.measureWrong":
≈ 10⁻⁴ us/op
# Run complete. Total time: 00:00:26
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JmhTestApp12_Setup.measureWrong avgt 4 ≈ 10⁻⁴ us/op
运行上面的基准测试程序，measureRight 基准测试方法能够正确地执行，但是 measureWrong 却会失败。
（2）Level（级别）
在默认情况下，Setup（设置） 和 TearDown（拆卸） 会在一个基准方法的所有批次执行前后分别执行，如果需要在每一个批次或者每一次基准方法调用执行的前后执行对应的套件方法，则需要对 @Setup 和 @TearDown 进行简单的配置。
【Level 枚举类源码】
package org.openjdk.jmh.annotations;
/**
* Control when to run the fixture methods.
*
* @see Setup
* @see TearDown
*/
public enum Level {
/**
* Trial level: to be executed before/after each run of the benchmark.
*
* <p>Trial is the set of benchmark iterations.</p>
*/
Trial,
/**
* Iteration level: to be executed before/after each iteration of the benchmark.
*
* <p>Iteration is the set of benchmark invocations.</p>
*/
Iteration,
/**
* Invocation level: to be executed for each benchmark method execution.
*
* <p><b>WARNING: HERE BE DRAGONS! THIS IS A SHARP TOOL.
* MAKE SURE YOU UNDERSTAND THE REASONING AND THE IMPLICATIONS
* OF THE WARNINGS BELOW BEFORE EVEN CONSIDERING USING THIS LEVEL.</b></p>
*
* <p>This level is only usable for benchmarks taking more than a millisecond
* per single {@link Benchmark} method invocation. It is a good idea to validate
* the impact for your case on ad-hoc basis as well.</p>
*
* <p>WARNING #1: Since we have to subtract the setup/teardown costs from
* the benchmark time, on this level, we have to timestamp *each* benchmark
* invocation. If the benchmarked method is small, then we saturate the
* system with timestamp requests, which introduce artificial latency,
* throughput, and scalability bottlenecks.</p>
*
* <p>WARNING #2: Since we measure individual invocation timings with this
* level, we probably set ourselves up for (coordinated) omission. That means
* the hiccups in measurement can be hidden from timing measurement, and
* can introduce surprising results. For example, when we use timings to
* understand the benchmark throughput, the omitted timing measurement will
* result in lower aggregate time, and fictionally *larger* throughput.</p>
*
* <p>WARNING #3: In order to maintain the same sharing behavior as other
* Levels, we sometimes have to synchronize (arbitrage) the access to
* {@link State} objects. Other levels do this outside the measurement,
* but at this level, we have to synchronize on *critical path*, further
* offsetting the measurement.</p>
*
* <p>WARNING #4: Current implementation allows the helper method execution
* at this Level to overlap with the benchmark invocation itself in order
* to simplify arbitrage. That matters in multi-threaded benchmarks, when
* one worker thread executing {@link Benchmark} method may observe other
* worker thread already calling {@link TearDown} for the same object.</p>
*/
Invocation,
}
【Level 枚举类中参数的定义，主要是翻译注解，同时参考了谷歌翻译和百度翻译】
Trial：试验级别。在每次运行基准测试之前/之后执行。试验是一组基准迭代。
Iteration：迭代级别。在基准测试的每次迭代之前/之后执行。迭代是一组基准调用。
Invocation：调用级别。去执行每个基准方法执行。此级别仅适用于每次基准测试方法调用花费超过 1 毫秒的基准测试。最好也临时验证对您的案例的影响。此级别有几个警告需要注意：
警告1）由于我们必须从基准时间中减去 Setup/Teardown 成本，因此在这个级别上，我们必须为每个基准调用设置时间戳。如果基准方法很小，那么我们会用时间戳请求使系统饱和，这会引入人为的延迟、吞吐量和可扩展性瓶颈。
警告2）由于我们用这个级别来衡量单独的调用时间，我们可能会为（协调的）遗漏做好准备。这意味着测量中的小问题可以从时序测量中隐藏起来，并且可以带来令人惊讶的结果。例如，当我们使用时序来了解基准吞吐量时，省略时序测量将导致聚合时间减少，并虚构更大的吞吐量。
警告3）为了保持与其他级别相同的共享行为，我们有时必须同步（套利）对 State 对象的访问。其他级别在度量（measurement）之外执行此操作，但在此级别，我们必须在关键路径上同步，从而进一步抵消度量（measurement）。
警告4）当前实现允许此级别的 辅助（helper）方法执行与基准调用本身重叠，以简化 套利（arbitrage）。这在多线程基准测试中很重要，当一个工作线程执行 Benchmark 方法时可能会观察到其他工作线程已经为同一对象调用 TearDown（拆卸）。
1）Trial：Setup和TearDown默认的配置，该套件方法会在每一个基准测试方法的所有批次执行的前后被执行。
@Setup(Level.Trial)
public void setUp()
2）Iteration：由于我们可以设置Warmup和Measurement，因此每一个基准测试方法都会被执行若干个批次，如果想要在每一个基准测试批次执行的前后调用套件方法，则可以将Level设置为Iteration。
@Setup(Level.Iteration)
public void setUp()
3）Invocation：将Level设置为Invocation意味着在每一个批次的度量过程中，每一次对基准方法的调用前后都会执行套件方法。
@Setup(Level.Invocation)
public void setUp()
9、@CompilerControl
JVM真的会对我们的代码进行相关的优化吗？那就通过一个简单的例子来验证一下优化是否存在。
【书中样例1：@CompilerControl】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
import static java.lang.Math.PI;
import static java.lang.Math.log;
/**
* JMH测试13：@CompilerControl用法1
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@State(Scope.Thread)
public class JmhTestApp13_CompilerControl1 {
@Benchmark
public void test1()
{
}
@Benchmark
public void test2()
{
log(PI);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp13_CompilerControl1.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书中样例1运行结果：@CompilerControl】
Benchmark Mode Cnt Score Error Units
JmhTestApp13_CompilerControl1.test1 avgt 5 ≈ 10⁻⁴ us/op
JmhTestApp13_CompilerControl1.test2 avgt 5 ≈ 10⁻⁴ us/op
JmhTestApp13_CompilerControl1包含两个基准测试方法test1和test2，但是test1方法中并未运行任何计算，而test2方法中进行了 Math.log 的运算，根据常识，Math.log方法的CPU耗时肯定要高于一个空方法，但是运行上面的基准测试方法之后得出的性能数据表明的结果是两者几乎不相上下。
由于test2方法中存在 Dead Code，JVM在运行test2方法时对程序进行了优化，具体来说就是将log运算的相关代码进行了运行期擦除，下面通过 CompilerControl 禁止JVM运行时优化和编译之后再来执行一下基准测试方法，然后进行对比。
【书中样例2：@CompilerControl】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
import static java.lang.Math.PI;
import static java.lang.Math.log;
/**
* JMH测试13：@CompilerControl用法2
* @author 大白有点菜
*/
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@State(Scope.Thread)
public class JmhTestApp13_CompilerControl2 {
/**
* test1测试：禁止优化
*/
@CompilerControl(CompilerControl.Mode.EXCLUDE)
@Benchmark
public void test1()
{
}
/**
* test2测试：禁止优化
*/
@CompilerControl(CompilerControl.Mode.EXCLUDE)
@Benchmark
public void test2()
{
log(PI);
}
public static void main(String[] args) throws RunnerException {
final Options opts = new OptionsBuilder()
.include(JmhTestApp13_CompilerControl2.class.getSimpleName())
.build();
new Runner(opts).run();
}
}
【书中样例2运行结果：@CompilerControl】
Benchmark Mode Cnt Score Error Units
JmhTestApp13_CompilerControl2.test1 avgt 5 0.011 ± 0.002 us/op
JmhTestApp13_CompilerControl2.test2 avgt 5 0.031 ± 0.004 us/op
运行上面的基准测试方法之后，结果表明两者的差别就出来了。
如果想在自己的应用程序中杜绝JVM运行期的优化，那么可以通过如下的方式来实现（这种情况并不推荐）。
- 通过编写程序的方式禁止JVM运行期动态编译和优化
java.lang.Compiler.disable()。- 在启动JVM时增加参数
-Djava.compiler=NONE。
【官方样例（JMHSample_16_CompilerControl）：@CompilerControl】
package cn.zhuangyt.javabase.jmh;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
/**
* JMH测试13：@CompilerControl用法3
* @author 大白有点菜
*/
@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
public class JmhTestApp13_CompilerControl3 {
/**
* These are our targets:
* - first method is prohibited from inlining
* 禁止第一个方法内联
* - second method is forced to inline
* 第二个方法强制内联
* - third method is prohibited from compiling
* 第三种方法禁止编译
*
* We might even place the annotations directly to the benchmarked
* methods, but this expresses the intent more clearly.
*
* 我们甚至可以将注释直接放在基准方法上，但这更清楚地表达了意图。
*/
public void target_blank() {
// this method was intentionally left blank
}
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
public void target_dontInline() {
// this method was intentionally left blank
}
@CompilerControl(CompilerControl.Mode.INLINE)
public void target_inline() {
// this method was intentionally left blank
}
@CompilerControl(CompilerControl.Mode.EXCLUDE)
public void target_exclude() {
// this method was intentionally left blank
}
/*
* These method measures the calls performance.
*/
@Benchmark
public void baseline() {
// this method was intentionally left blank
}
@Benchmark
public void blank() {
target_blank();
}
@Benchmark
public void dontinline() {
target_dontInline();
}
@Benchmark
public void inline() {
target_inline();
}
@Benchmark
public void exclude() {
target_exclude();
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(JmhTestApp13_CompilerControl3.class.getSimpleName())
.warmupIterations(0)
.measurementIterations(3)
.forks(1)
.build();
new Runner(opt).run();
}
}
【官方样例（JMHSample_16_CompilerControl）运行结果：@CompilerControl】
Benchmark Mode Cnt Score Error Units
JmhTestApp13_CompilerControl3.baseline avgt 3 0.263 ± 0.068 ns/op
JmhTestApp13_CompilerControl3.blank avgt 3 0.280 ± 0.351 ns/op
JmhTestApp13_CompilerControl3.dontinline avgt 3 1.696 ± 3.774 ns/op
JmhTestApp13_CompilerControl3.exclude avgt 3 11.516 ± 16.560 ns/op
JmhTestApp13_CompilerControl3.inline avgt 3 0.263 ± 0.020 ns/op