Bootstrap

生产环境WAS产生javacore、dmp、dump文件分析

目录

一、分析工具

二、Java 转储(Java dump)

Java 转储内容(Java dump contents)

标题(TITLE)

GP信息(GPINFO)

环境信息(ENVINFO)

本地内存信息(NATIVEMEMINFO)

内存信息(MEMINFO)

锁(LOCKS)

线程(THREADS)

钩子(HOOKS)

共享类(SHARED CLASSES)

类(CLASSES)

场景(Scenarios)

一般保护故障(General Protection Fault)

Java 内存溢出错误(Java OutOfMemoryError)

本机 OutOfMemoryError(Native OutOfMemoryError)

死锁(Deadlock)

挂起(Hang)


一、分析工具

构件类型

问题类型

典型输入

可用的工具

Verbose Garbage Collection 日志 (verbosegGC)

内存泄漏、内存不足情况

诸如 native_stdout.log 等 JVM 日志文件中的 verbosegc 语句

1. IBM Monitoring and Diagnostic Tools for Java - Garbage Collection and Memory Visualizer (GCMV)

2. IBM Pattern Modeling and Analysis Tool for Java Garbage Collector (PMAT)

3. Diagnostic Tool for Java Garbage Collector

Java 转储javacore

崩溃、挂起、性能瓶颈、JVM 意外终止

javacore.*.txt

javacorePID.*.txt

4. IBM Thread and Monitor Dump Analyzer for Java (TMDA)

5. Thread Analyzer

线程

锁分析

运行的 JVM 的连接

6. IBM Lock Analyzer for Java

堆转储

内存泄漏、内存不足情况

IBM 可移植堆转储 (heapdump.phd)

IBM 文本堆转储 (heapdump.txt)

HPROF 堆转储格式 (hprof.txt)

7. Memory Dump Diagnostic For Java (MDD4J)

8. HeapAnalyzer

9. Heaproots

系统或核心转储

系统状况的一般分析;检测异常;系统状态的深入分析。

特殊情况:意外崩溃。

文件名:与操作系统相关(示例包括 core.dmp、user.dmp 或者只是“core”)。

在将该文件用作分析的输入之前,必须使用 jextract 工具处理该文件,从而产生 core.dmp.zip 文件(IBM JVM 5.0 及更高版本)或 core.sdff 文件 (IBM JVM 1.4.2)。

注意:仅适用于 IBM JVM。

IBM Monitoring and Diagnostic Tools for Java - Dump Analyzer及其在WebSphere Application Server modules for Dump Analyzer中的扩展

二、Java 转储(Java dump)

Java dumps(有时称为Java 核心)是在 VM 因操作系统信号或用户启动的按键组合而意外结束时生成的。您还可以通过从应用程序中以编程方式调用 Dump API 或在命令行上OutOfMemoryError指定选项来生成 Java 转储。-Xdump:java

如果您的 Java 应用程序崩溃或挂起,Java 转储可以提供有用的信息来帮助您诊断根本原因。

  • 如果您的应用程序崩溃,将针对以下类型的故障自动生成 Java 转储:
    • VM 收到意外信号或断言失败
    • VM 内存不足
  • kill -3如果您的应用程序挂起,您可以通过向 VM发送 SIGQUIT 信号 ( ) 来触发 Java 转储的生成。

注意:在 Windows® 上,如果您在控制台窗口中启动 VM,则可以强制 VM 响应 SIGBREAK 信号(Ctrl-Break 键盘组合)生成 Java 转储。如果您没有在控制台窗口中启动,则 Windows 上没有与 Linux 命令等效的用于发送信号的命令。这里唯一的选项是通过在 Windows 任务管理器的“进程”kill选项卡中找到 VM 进程并单击 “创建转储文件”来触发完整系统转储。

为了帮助您了解 Java 转储如何帮助您进行问题诊断,本主题包含了一些场景来帮助您解释数据:

Java 转储内容(Java dump contents)

Java dumps总结了事件发生时虚拟机的状态,其中大部分信息与虚拟机的组件有关。该文件由多个部分组成,提供不同类型的信息。

标题(TITLE)

Java dump file 的第一部分提供了有关触发转储生成的事件的信息。在下面的示例中,您可以看到某个vmstop事件在指定的日期和时间触发了转储。

0SECTION       TITLE subcomponent dump routine
NULL           ===============================
1TICHARSET     UTF-8
1TISIGINFO     Dump Event "vmstop" (00000002) Detail "#0000000000000000" received
1TIDATETIMEUTC Date: 2021/04/23 at 18:02:44:017 (UTC)
1TIDATETIME    Date: 2021/04/23 at 14:02:44:017
1TITIMEZONE    Timezone: UTC-4 (EDT)
1TINANOTIME    System nanotime: 379202644260787
1TIFILENAME    Javacore filename:    /home/doc-javacore/javacore.20210423.140244.1175.0001.txt
1TIREQFLAGS    Request Flags: 0x81 (exclusive+preempt)
1TIPREPSTATE   Prep State: 0x106 (vm_access+exclusive_vm_access+trace_disabled)

GP信息(GPINFO)

GPINFO 部分提供了有关 VM 所运行系统的一般信息。以下示例取自在 Linux 系统上生成的 Java 转储。

NULL           ------------------------------------------------------------------------
0SECTION       GPINFO subcomponent dump routine
NULL           ================================
2XHOSLEVEL     OS Level         : Linux 3.10.0-862.11.6.el7.x86_64
2XHCPUS        Processors -
3XHCPUARCH       Architecture   : amd64
3XHNUMCPUS       How Many       : 4
3XHNUMASUP       NUMA is either not supported or has been disabled by user
NULL
1XHERROR2      Register dump section only produced for SIGSEGV, SIGILL or SIGFPE.
NULL

此部分的内容可能有所不同,具体取决于转储的原因。例如,如果转储是由一般保护故障 (gpf) 引起的,则还会记录发生崩溃的库以及显示为 的值VM flags。此值可以提供一些线索,说明可能涉及 VM 的哪个组件。在输出中查找以下行:

1XHFLAGS       VM flags:0000000000000000

记录的十六进制数VM flags以 MSSSS 结尾,其中 M 是 VM 组件,SSSS 是组件特定代码,如下表所示:

成分

代码值

翻译员

0x10000

气相色谱

0x20000

增长堆栈

0x30000

杰尼提

0x40000

代码生成

0x50000

BC验证

0x60000

实时验证

0x70000

共享类

0x80000

0000000000000000(0x00000) 表示崩溃发生在虚拟机外部。

环境信息(ENVINFO)

本节包含有关崩溃发生环境的有用信息,包括以下数据:

  • Java 版本 ( 1CIJAVAVERSION)
  • Eclipse OpenJ9™ VM 和子组件版本信息(1CIVMVERSION、、、、)1CIJ9VMVERSION1CIJITVERSION1CIOMRVERSION1CIJCLVERSION
  • VM 启动时间(1CISTARTTIME)和进程信息(1CIPROCESSID
  • Java 主目录 ( 1CIJAVAHOMEDIR) 和 DLL ( 1CIJAVADLLDIR) 目录
  • 命令行上传递的用户参数(1CIUSERARGS),标识那些被忽略的参数(1CIIGNOREDARGS
  • 系统对用户的限制(1CIUSERLIMITS
  • 环境变量已就位(1CIENVVARS
  • 系统信息(1CISYSINFO
  • CPU 信息(1CICPUINFO
  • 控制组 (Cgroup) 信息 ( 1CICGRPINFO)

为了清楚起见,以下示例显示了本节的缩短版本,其中...表示已删除的行:

NULL           ------------------------------------------------------------------------
0SECTION       ENVINFO subcomponent dump routine
NULL           =================================
1CIJAVAVERSION JRE 9 Linux amd64-64 (build 9.0.4-internal+0-adhoc..openj9-openjdk-jdk9)
1CIVMVERSION   20180830_000000
1CIJ9VMVERSION 8e7c6ec
1CIJITVERSION  8e7c6ec
1CIOMRVERSION  553811b_CMPRSS
1CIJCLVERSION  ec1d223 based on jdk-9.0.4+12
1CIJITMODES    JIT enabled, AOT enabled, FSD disabled, HCR enabled
1CIRUNNINGAS   Running as a standalone JVM
1CIVMIDLESTATE VM Idle State: ACTIVE
1CICONTINFO    Running in container : FALSE
1CICGRPINFO    JVM support for cgroups enabled : TRUE
1CISTARTTIME   JVM start time: 2018/08/30 at 21:55:47:387
1CISTARTNANO   JVM start nanotime: 22012135233549
1CIPROCESSID   Process ID: 30285 (0x764D)
1CICMDLINE     [not available]
1CIJAVAHOMEDIR Java Home Dir:   /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk
1CIJAVADLLDIR  Java DLL Dir:    /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/bin
1CISYSCP       Sys Classpath:
1CIUSERARGS    UserArgs:
2CIUSERARG               -Xoptionsfile=/home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/options.default
...
1CIIGNOREDARGS Ignored Args:
2CIIGNOREDARG            -XX:+UseCompressedOop
2CIIGNOREDARG            -XX:CompressedClassSpaceSize=528482304
NULL
1CIUSERLIMITS  User Limits (in bytes except for NOFILE and NPROC)
NULL           ------------------------------------------------------------------------
NULL           type                            soft limit           hard limit
2CIUSERLIMIT   RLIMIT_AS                        unlimited            unlimited
2CIUSERLIMIT   RLIMIT_CORE                              0            unlimited
2CIUSERLIMIT   RLIMIT_CPU                       unlimited            unlimited
2CIUSERLIMIT   RLIMIT_DATA                      unlimited            unlimited
2CIUSERLIMIT   RLIMIT_FSIZE                     unlimited            unlimited
2CIUSERLIMIT   RLIMIT_LOCKS                     unlimited            unlimited
2CIUSERLIMIT   RLIMIT_MEMLOCK                       65536                65536
2CIUSERLIMIT   RLIMIT_NOFILE                         4096                 4096
2CIUSERLIMIT   RLIMIT_NPROC                          4096                30592
2CIUSERLIMIT   RLIMIT_RSS                       unlimited            unlimited
2CIUSERLIMIT   RLIMIT_STACK                       8388608            unlimited
2CIUSERLIMIT   RLIMIT_MSGQUEUE                     819200               819200
2CIUSERLIMIT   RLIMIT_NICE                              0                    0
2CIUSERLIMIT   RLIMIT_RTPRIO                            0                    0
2CIUSERLIMIT   RLIMIT_SIGPENDING                    30592                30592
NULL
1CIENVVARS     Environment Variables
NULL           ------------------------------------------------------------------------
2CIENVVAR      XDG_VTNR=1
2CIENVVAR      SSH_AGENT_PID=2653
...
NULL
1CISYSINFO     System Information
NULL           ------------------------------------------------------------------------
2CISYSINFO     /proc/sys/kernel/core_pattern = core
2CISYSINFO     /proc/sys/kernel/core_uses_pid = 1
NULL
1CICPUINFO     CPU Information
NULL           ------------------------------------------------------------------------
2CIPHYSCPU     Physical CPUs: 8
2CIONLNCPU     Online CPUs: 8
2CIBOUNDCPU    Bound CPUs: 8
2CIACTIVECPU   Active CPUs: 0
2CITARGETCPU   Target CPUs: 8
2CIJITFEATURE  CPU features (JIT): fpu cx8 cmov mmx sse sse2 ssse3 fma sse4_1 popcnt aesni osxsave avx avx2 rdt_m
2CIAOTFEATURE  CPU features (AOT): fpu cx8 cmov mmx sse sse2 ssse3 fma sse4_1 popcnt aesni osxsave avx avx2 rdt_m
NULL
1CICGRPINFO    Cgroup Information
NULL           ------------------------------------------------------------------------
2CICGRPINFO    subsystem : cpu
2CICGRPINFO    cgroup name : /
3CICGRPINFO    CPU Period : 100000 microseconds
3CICGRPINFO    CPU Quota : Not Set
3CICGRPINFO    CPU Shares : 1024
3CICGRPINFO    Period intervals elapsed count : 0
3CICGRPINFO    Throttled count : 0
3CICGRPINFO    Total throttle time : 0 nanoseconds
2CICGRPINFO    subsystem : cpuset
2CICGRPINFO    cgroup name : /
3CICGRPINFO    CPU exclusive : 1
3CICGRPINFO    Mem exclusive : 1
3CICGRPINFO    CPUs : 0-7
3CICGRPINFO    Mems : 0
2CICGRPINFO    subsystem : memory
2CICGRPINFO    cgroup name : /
3CICGRPINFO    Memory Limit : Not Set
3CICGRPINFO    Memory + Swap Limit : Not Set
3CICGRPINFO    Memory Usage : 5363396608 bytes
3CICGRPINFO    Memory + Swap Usage : 5363396608 bytes
3CICGRPINFO    Memory Max Usage : 0 bytes
3CICGRPINFO    Memory + Swap Max Usage : 0 bytes
3CICGRPINFO    Memory limit exceeded count : 0
3CICGRPINFO    Memory + Swap limit exceeded count : 0
3CICGRPINFO    OOM Killer Disabled : 0
3CICGRPINFO    Under OOM : 0
NULL

本地内存信息(NATIVEMEMINFO)

malloc()此部分记录使用库函数(例如和)请求的本机内存的信息mmap()。值按组件细分提供,指示分配的总字节数和本机内存分配数。在以下示例中,为 VM 类分配了 4,682,840 字节的本机内存(但尚未释放),这对应于 141 个分配。

NULL           ------------------------------------------------------------------------
0SECTION       NATIVEMEMINFO subcomponent dump routine
NULL           =================================
0MEMUSER
1MEMUSER       JRE: 2,569,088,312 bytes / 4653 allocations
1MEMUSER       |
2MEMUSER       +--VM: 2,280,088,336 bytes / 2423 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Classes: 4,682,840 bytes / 141 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Memory Manager (GC): 2,054,966,784 bytes / 433 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Heap: 2,014,113,792 bytes / 1 allocation
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 40,852,992 bytes / 432 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Threads: 10,970,016 bytes / 156 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Stack: 197,760 bytes / 16 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Native Stack: 10,616,832 bytes / 17 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 155,424 bytes / 123 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Trace: 180,056 bytes / 263 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JVMTI: 17,776 bytes / 13 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JNI: 36,184 bytes / 52 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Port Library: 208,179,632 bytes / 72 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Unused <32bit allocation regions: 208,168,752 bytes / 1 allocation
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 10,880 bytes / 71 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Other: 1,055,048 bytes / 1293 allocations
1MEMUSER       |
2MEMUSER       +--JIT: 288,472,816 bytes / 140 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JIT Code Cache: 268,435,456 bytes / 1 allocation
2MEMUSER       |  |
3MEMUSER       |  +--JIT Data Cache: 2,097,216 bytes / 1 allocation
2MEMUSER       |  |
3MEMUSER       |  +--Other: 17,940,144 bytes / 138 allocations
1MEMUSER       |
2MEMUSER       +--Class Libraries: 13,432 bytes / 25 allocations
2MEMUSER       |  |
3MEMUSER       |  +--VM Class Libraries: 13,432 bytes / 25 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--sun.misc.Unsafe: 3,184 bytes / 13 allocations
4MEMUSER       |  |  |  |
5MEMUSER       |  |  |  +--Direct Byte Buffers: 1,056 bytes / 12 allocations
4MEMUSER       |  |  |  |
5MEMUSER       |  |  |  +--Other: 2,128 bytes / 1 allocation
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 10,248 bytes / 12 allocations
1MEMUSER       |
2MEMUSER       +--Unknown: 513,728 bytes / 2065 allocations
NULL

此部分不记录由应用程序或 JNI 代码分配的内存,并且通常比操作系统工具记录的值略小。

内存信息(MEMINFO)

本节涉及内存管理,以十进制和十六进制格式提供虚拟机中对象堆、内部内存、用于类的内存、JIT 代码缓存和 JIT 数据缓存的内存使用情况明细。您还可以在生成转储时找出正在使用的垃圾回收策略。

对象内存区域 ( 1STHEAPTYPE) 记录正在使用的每个内存区域、其起始和结束地址以及区域大小。还记录了用于内部内存、类内存、JIT 代码缓存和 JIT 数据缓存的内存段的更多信息 ( 1STSEGMENT)。此信息包括段控制数据结构的地址、本机内存段的起始和结束地址以及段大小。

为了清楚起见,以下示例显示了本节的缩短版本,其中...表示已删除的行:

NULL           ------------------------------------------------------------------------
0SECTION       MEMINFO subcomponent dump routine
NULL           =================================
NULL
1STHEAPTYPE    Object Memory
NULL           id                 start              end                size               space/region
1STHEAPSPACE   0x00007FF4F00744A0         --                 --                 --         Generational
1STHEAPREGION  0x00007FF4F0074CE0 0x0000000087F40000 0x0000000088540000 0x0000000000600000 Generational/Tenured Region
1STHEAPREGION  0x00007FF4F0074930 0x00000000FFE00000 0x00000000FFF00000 0x0000000000100000 Generational/Nursery Region
1STHEAPREGION  0x00007FF4F0074580 0x00000000FFF00000 0x0000000100000000 0x0000000000100000 Generational/Nursery Region
NULL
1STHEAPTOTAL   Total memory:                     8388608 (0x0000000000800000)
1STHEAPINUSE   Total memory in use:              2030408 (0x00000000001EFB48)
1STHEAPFREE    Total memory free:                6358200 (0x00000000006104B8)
NULL
1STSEGTYPE     Internal Memory
NULL           segment            start              alloc              end                type       size
1STSEGMENT     0x00007FF4F004CBC8 0x00007FF4CD33C000 0x00007FF4CD33C000 0x00007FF4CE33C000 0x01000440 0x0000000001000000
1STSEGMENT     0x00007FF4F004CB08 0x00007FF4DE43D030 0x00007FF4DE517770 0x00007FF4DE53D030 0x00800040 0x0000000000100000
NULL
1STSEGTOTAL    Total memory:                    17825792 (0x0000000001100000)
1STSEGINUSE    Total memory in use:               894784 (0x00000000000DA740)
1STSEGFREE     Total memory free:               16931008 (0x00000000010258C0)
NULL
1STSEGTYPE     Class Memory
NULL           segment            start              alloc              end                type       size
1STSEGMENT     0x00007FF4F03B5638 0x0000000001053D98 0x000000000105BD98 0x000000000105BD98 0x00010040 0x0000000000008000
1STSEGMENT     0x00007FF4F03B5578 0x0000000001048188 0x0000000001050188 0x0000000001050188 0x00010040 0x0000000000008000
...
NULL
1STSEGTOTAL    Total memory:                     3512520 (0x00000000003598C8)
1STSEGINUSE    Total memory in use:              3433944 (0x00000000003465D8)
1STSEGFREE     Total memory free:                  78576 (0x00000000000132F0)
NULL
1STSEGTYPE     JIT Code Cache
NULL           segment            start              alloc              end                type       size
1STSEGMENT     0x00007FF4F00961F8 0x00007FF4CE43D000 0x00007FF4CE445790 0x00007FF4DE43D000 0x00000068 0x0000000010000000
NULL
1STSEGTOTAL    Total memory:                   268435456 (0x0000000010000000)
1STSEGINUSE    Total memory in use:                34704 (0x0000000000008790)
1STSEGFREE     Total memory free:              268400752 (0x000000000FFF7870)
1STSEGLIMIT    Allocation limit:               268435456 (0x0000000010000000)
NULL
1STSEGTYPE     JIT Data Cache
NULL           segment            start              alloc              end                type       size
1STSEGMENT     0x00007FF4F0096668 0x00007FF4CC553030 0x00007FF4CC753030 0x00007FF4CC753030 0x00000048 0x0000000000200000
NULL
1STSEGTOTAL    Total memory:                     2097152 (0x0000000000200000)
1STSEGINUSE    Total memory in use:              2097152 (0x0000000000200000)
1STSEGFREE     Total memory free:                      0 (0x0000000000000000)
1STSEGLIMIT    Allocation limit:               402653184 (0x0000000018000000)
NULL
1STGCHTYPE     GC History
NULL

在本例中,GC 历史记录 ( 1STGCHTYPE) 部分为空白。如果使用跟踪工具诊断的虚拟机中发生了垃圾收集周期,则此部分将被填充。

锁(LOCKS)

Java 转储的这一部分提供了有关锁的信息,锁可防止共享资源被多个实体同时访问。这些信息在死锁情况下至关重要,死锁情况下两个线程尝试在对象上同步并锁定类的实例。记录了导致问题的线程的精确信息,使您能够确定根本原因。

以下示例显示了典型的 LOCKS 部分,其中触发转储时不存在死锁。为清楚起见,以下示例显示了此部分的缩短版本,其中...表示删除了以下行:

NULL           ------------------------------------------------------------------------
0SECTION       LOCKS subcomponent dump routine
NULL           ===============================
NULL
1LKPOOLINFO    Monitor pool info:
2LKPOOLTOTAL     Current total number of monitors: 3
NULL
1LKMONPOOLDUMP Monitor Pool Dump (flat & inflated object-monitors):
2LKMONINUSE      sys_mon_t:0x00007FF4B0001D78 infl_mon_t: 0x00007FF4B0001DF8:
3LKMONOBJECT       java/lang/ref/ReferenceQueue@0x00000000FFE26A10: <unowned>
3LKNOTIFYQ            Waiting to be notified:
3LKWAITNOTIFY            "Common-Cleaner" (J9VMThread:0x0000000000FD0100)
NULL
1LKREGMONDUMP  JVM System Monitor Dump (registered monitors):
2LKREGMON          Thread global lock (0x00007FF4F0004FE8): <unowned>
2LKREGMON          &(PPG_mem_mem32_subAllocHeapMem32.monitor) lock (0x00007FF4F0005098): <unowned>
2LKREGMON          NLS hash table lock (0x00007FF4F0005148): <unowned>
...
NULL

线程(THREADS)

Java 转储文件的 THREADS 部分提供了有关 VM 线程池的摘要信息以及有关 Java 线程、本机线程和堆栈跟踪的详细信息。了解此部分的内容可以帮助您诊断由阻塞或等待线程引起的问题。

Java 线程在本机线程上运行。该Thread Details小节为每个 Java 线程记录了几行内容,其中包括以下关键信息:

  • 3XMTHREADINFO:线程名称、VM线程结构和Java线程对象的地址信息、线程状态、线程优先级。
  • 3XMJAVALTHREAD:来自线程对象的 Java 线程 ID 和守护进程状态。
  • 3XMTHREADINFO1:本机操作系统线程 ID、优先级、调度策略、内部 VM 线程状态和 VM 线程标志。
  • 3XMTHREADINFO2:本机堆栈地址范围。
  • 3XMTHREADINFO3:Java调用堆栈信息(4XESTACKTRACE)或Native调用堆栈信息(4XENATIVESTACK)。
  • 5XESTACKTRACE:此行表示是否通过特定方法获取锁。

Java 线程优先级与操作系统优先级值相对应。线程状态如下表所示:

线程状态值

地位

描述

R

可运行

该线程可以运行

连续波

条件等待

线程正在等待

年代

暂停

线程被另一个线程挂起

僵尸

线程被销毁

停放

该主题由java.util.concurrent

已阻止

线程正在等待获取锁

对于处于停放状态 (P)、阻塞状态 (B) 或等待状态 (CW) 的线程,3XMTHREADBLOCK输出中会包含一个附加行 ( ),显示线程停放、阻塞或等待的位置。对于等待类初始化锁 ( java/lang/J9VMInternals$ClassInitializationLock) 的线程,此行包含当前正在处理类初始化的线程的名称。您可以使用此信息来诊断由类初始化引起的死锁,这些死锁可能无法在 LOCKS 部分中检测和报告。

为了清楚起见,以下示例显示了典型 THREADS 部分的缩短版本,其中...表示已删除的行:

NULL           ------------------------------------------------------------------------
0SECTION       THREADS subcomponent dump routine
NULL           =================================
NULL
1XMPOOLINFO    JVM Thread pool info:
2XMPOOLTOTAL       Current total number of pooled threads: 19
2XMPOOLLIVE        Current total number of live threads: 18
2XMPOOLDAEMON      Current total number of live daemon threads: 15
NULL
1XMTHDINFO     Thread Details
NULL
...
3XMTHREADINFO      "JIT Diagnostic Compilation Thread-007 Suspended" J9VMThread:0x0000000000035200, omrthread_t:0x00007F3F8C0D02C8, java/lang/Thread:0x00000000FFF42120, state:R, prio=10
3XMJAVALTHREAD            (java/lang/Thread getId:0x9, isDaemon:true)
3XMJAVALTHRCCL            sun/misc/Launcher$AppClassLoader(0x00000000FFF3BF98)
3XMTHREADINFO1            (native thread ID:0x618F, native priority:0xB, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000081)
3XMTHREADINFO2            (native stack address range from:0x00007F3F879C5000, to:0x00007F3F87AC5000, size:0x100000)
3XMCPUTIME               CPU usage total: 0.052410771 secs, current category="JIT"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           No Java callstack associated with this thread
...
NULL
...
3XMTHREADINFO      "Class Initialization Thread 2" J9VMThread:0x0000000000124D00, omrthread_t:0x00007F3F8C1494C8, java/lang/Thread:0x00000000FFF53EE8, state:CW, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x13, isDaemon:false)
3XMJAVALTHRCCL            sun/misc/Launcher$AppClassLoader(0x00000000FFF3BF98)
3XMTHREADINFO1            (native thread ID:0x6199, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000181)
3XMTHREADINFO2            (native stack address range from:0x00007F3F74AB4000, to:0x00007F3F74AF4000, size:0x40000)
3XMCPUTIME               CPU usage total: 0.008712260 secs, current category="Application"
3XMTHREADBLOCK     Waiting on: java/lang/J9VMInternals$ClassInitializationLock@0x00000000FFF61C90 Owned by: <unowned> Initializing thread: "Class Initialization Thread 1"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=4096 (0x1000)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at java/lang/Class.forNameImpl(Native Method)
4XESTACKTRACE                at java/lang/Class.forName(Class.java:339)
4XESTACKTRACE                at ClassInitLockBug$ClassInitThread.run(ClassInitLockBug.java:16)
...
NULL
...
NULL
3XMTHREADINFO      "Class Initialization Thread 1" J9VMThread:0x0000000000124100, omrthread_t:0x00007F3F8C148F50, java/lang/Thread:0x00000000FFF53D80, state:CW, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x12, isDaemon:false)
3XMJAVALTHRCCL            sun/misc/Launcher$AppClassLoader(0x00000000FFF3BF98)
3XMTHREADINFO1            (native thread ID:0x6198, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000481)
3XMTHREADINFO2            (native stack address range from:0x00007F3F74AF5000, to:0x00007F3F74B35000, size:0x40000)
3XMCPUTIME               CPU usage total: 0.010221701 secs, current category="Application"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=12736 (0x31C0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at java/lang/Thread.sleepImpl(Native Method)
4XESTACKTRACE                at java/lang/Thread.sleep(Thread.java:983)
4XESTACKTRACE                at java/lang/Thread.sleep(Thread.java:966)
4XESTACKTRACE                at TestClass.<clinit>(ClassInitLockBug.java:29)
4XESTACKTRACE                at java/lang/Class.forNameImpl(Native Method)
4XESTACKTRACE                at java/lang/Class.forName(Class.java:339)
4XESTACKTRACE                at ClassInitLockBug$ClassInitThread.run(ClassInitLockBug.java:16)
...
NULL
...
NULL
1XMTHDSUMMARY  Threads CPU Usage Summary
NULL           =========================
NULL
1XMTHDCATINFO  Warning: to get more accurate CPU times for the GC, the option -XX:-ReduceCPUMonitorOverhead can be used. See the user guide for more information.
NULL
1XMTHDCATEGORY All JVM attached threads: 0.698865000 secs
1XMTHDCATEGORY |
2XMTHDCATEGORY +--System-JVM: 0.653723000 secs
2XMTHDCATEGORY |  |
3XMTHDCATEGORY |  +--GC: 0.047248000 secs
2XMTHDCATEGORY |  |
3XMTHDCATEGORY |  +--JIT: 0.512971000 secs
1XMTHDCATEGORY |
2XMTHDCATEGORY +--Application: 0.045142000 secs

钩子(HOOKS)

本节介绍内部 VM 事件回调,用于诊断 VM 中的性能问题。列出了多个挂钩接口,其中包括各自的挂钩事件。

以下示例显示了 的数据J9VMHookInterface,包括所有先前事件的总时间、调用点位置 (<源文件>:<行号>)、开始时间以及最后一个回调和最长回调的持续时间(所有时间以微秒为单位)。每次 Java 转储后都会重置钩子数据。

NULL           ------------------------------------------------------------------------
SECTION HOOK   subcomponent dump routine
NULL           =========================
1NOTE          These data are reset every time a javacore is taken
1HKINTERFACE   MM_OMRHookInterface
NULL           ------------------------------------------------------------------------
1HKINTERFACE   MM_PrivateHookInterface
NULL           ------------------------------------------------------------------------
1HKINTERFACE   MM_HookInterface
NULL           ------------------------------------------------------------------------
1HKINTERFACE   J9VMHookInterface
NULL           ------------------------------------------------------------------------
2HKEVENTID     1
3HKCALLCOUNT       1239
3HKTOTALTIME       219564us
3HKLAST            Last Callback
4HKCALLSITE            trcengine.c:395
4HKSTARTTIME           Start Time: 2019-10-18T00:15:14.664
4HKDURATION            Duration : 16us
3HKLONGST          Longest Callback
4HKCALLSITE            trcengine.c:395
4HKSTARTTIME           Start Time: 2019-10-18T21:28:34.895
4HKDURATION            Duration : 5012us
NULL
...
1HKINTERFACE   J9VMZipCachePoolHookInterface
NULL           ------------------------------------------------------------------------
1HKINTERFACE   J9JITHookInterface
NULL           ------------------------------------------------------------------------
2HKEVENTID     3
3HKCALLCOUNT       3113
3HKTOTALTIME       4904us
3HKLAST            Last Callback
4HKCALLSITE            common/mgmtinit.c:193
4HKSTARTTIME           Start Time: 2019-10-18T16:04:15.320
4HKDURATION            Duration : 3us
3HKLONGST          Longest Callback
4HKCALLSITE            common/mgmtinit.c:193
4HKSTARTTIME           Start Time: 2019-10-18T16:37:17.633
4HKDURATION            Duration : 27us
NULL
...

共享类(SHARED CLASSES)

如果在运行时启用了共享类缓存,则 Java 转储文件中提供的信息将描述创建缓存时使用的设置,以及有关缓存的大小和内容的摘要信息。

在以下示例中,使用类调试区域 ( ) 创建了共享类缓存-Xnolinenumbers=false。默认启用字节码检测 (BCI),并且允许共享缓存的虚拟机存储类路径,这也是默认设置。

显示Cache Summary缓存大小 ( 2SCLTEXTCSZ) 为 16776608 字节,软最大大小 ( 2SCLTEXTSMB) 也为 16776608 字节,剩余 12691668 字节可用空间 ( 2SCLTEXTFRB)。类调试区域 ( ) 的大小2SCLTEXTDAS为 1331200 字节,仅使用了其中 11% 的空间。

在该Cache Memory Status小节中,该行2SCLTEXTCMDT指示共享缓存的名称和位置,并cr表明该缓存是 64 位压缩引用缓存。

NULL           ------------------------------------------------------------------------
0SECTION       SHARED CLASSES subcomponent dump routine
NULL           ========================================
NULL
1SCLTEXTCRTW   Cache Created With
NULL           ------------------
NULL
2SCLTEXTXNL        -Xnolinenumbers       = false
2SCLTEXTBCI        BCI Enabled           = true
2SCLTEXTBCI        Restrict Classpaths   = false
NULL
1SCLTEXTCSUM   Cache Summary
NULL           ------------------
NULL
2SCLTEXTNLC        No line number content                    = false
2SCLTEXTLNC        Line number content                       = true
NULL
2SCLTEXTRCS        ROMClass start address                    = 0x00007F423061C000
2SCLTEXTRCE        ROMClass end address                      = 0x00007F42307B9A28
2SCLTEXTMSA        Metadata start address                    = 0x00007F42313D42FC
2SCLTEXTCEA        Cache end address                         = 0x00007F4231600000
2SCLTEXTRTF        Runtime flags                             = 0x00102001ECA6028B
2SCLTEXTCGN        Cache generation                          = 35
NULL
2SCLTEXTCSZ        Cache size                                = 16776608
2SCLTEXTSMB        Softmx bytes                              = 16776608
2SCLTEXTFRB        Free bytes                                = 12691668
2SCLTEXTRCB        ROMClass bytes                            = 1694248
2SCLTEXTAOB        AOT code bytes                            = 0
2SCLTEXTADB        AOT data bytes                            = 0
2SCLTEXTAHB        AOT class hierarchy bytes                 = 32
2SCLTEXTATB        AOT thunk bytes                           = 0
2SCLTEXTARB        Reserved space for AOT bytes              = -1
2SCLTEXTAMB        Maximum space for AOT bytes               = -1
2SCLTEXTJHB        JIT hint bytes                            = 308
2SCLTEXTJPB        JIT profile bytes                         = 2296
2SCLTEXTJRB        Reserved space for JIT data bytes         = -1
2SCLTEXTJMB        Maximum space for JIT data bytes          = -1
2SCLTEXTNOB        Java Object bytes                         = 0
2SCLTEXTZCB        Zip cache bytes                           = 919328
2SCLTEXTSHB        Startup hint bytes                        = 0
2SCLTEXTRWB        ReadWrite bytes                           = 114080
2SCLTEXTJCB        JCL data bytes                            = 0
2SCLTEXTBDA        Byte data bytes                           = 0
2SCLTEXTMDA        Metadata bytes                            = 23448
2SCLTEXTDAS        Class debug area size                     = 1331200
2SCLTEXTDAU        Class debug area % used                   = 11%
2SCLTEXTDAN        Class LineNumberTable bytes               = 156240
2SCLTEXTDAV        Class LocalVariableTable bytes            = 0
NULL
2SCLTEXTNRC        Number ROMClasses                         = 595
2SCLTEXTNAM        Number AOT Methods                        = 0
2SCLTEXTNAD        Number AOT Data Entries                   = 0
2SCLTEXTNAH        Number AOT Class Hierarchy                = 1
2SCLTEXTNAT        Number AOT Thunks                         = 0
2SCLTEXTNJH        Number JIT Hints                          = 14
2SCLTEXTNJP        Number JIT Profiles                       = 20
2SCLTEXTNCP        Number Classpaths                         = 1
2SCLTEXTNUR        Number URLs                               = 0
2SCLTEXTNTK        Number Tokens                             = 0
2SCLTEXTNOJ        Number Java Objects                       = 0
2SCLTEXTNZC        Number Zip Caches                         = 5
2SCLTEXTNSH        Number Startup Hint Entries               = 0
2SCLTEXTNJC        Number JCL Entries                        = 0
2SCLTEXTNST        Number Stale classes                      = 0
2SCLTEXTPST        Percent Stale classes                     = 0%
NULL
2SCLTEXTCPF        Cache is 24% full
NULL
1SCLTEXTCMST   Cache Memory Status
NULL           ------------------
1SCLTEXTCNTD       Cache Name                    Feature                  Memory type              Cache path
NULL
2SCLTEXTCMDT       sharedcc_doc-javacore         CR                       Memory mapped file       /tmp/javasharedresources/C290M4F1A64P_sharedcc_doc-javacore_G35
NULL
1SCLTEXTCMST   Cache Lock Status
NULL           ------------------
1SCLTEXTCNTD       Lock Name                     Lock type                TID owning lock
NULL
2SCLTEXTCWRL       Cache write lock              File lock                Unowned
2SCLTEXTCRWL       Cache read/write lock         File lock                Unowned
NULL

以下示例显示了分层缓存的信息:

NULL           ------------------------------------------------------------------------
0SECTION       SHARED CLASSES subcomponent dump routine
NULL           ========================================
NULL
1SCLTEXTCSTL   Cache Statistics for Top Layer
NULL
1SCLTEXTCRTW       Cache Created With
NULL               ------------------
NULL
2SCLTEXTXNL            -Xnolinenumbers       = false
2SCLTEXTBCI            BCI Enabled           = true
2SCLTEXTBCI            Restrict Classpaths   = false
NULL
1SCLTEXTCSUM       Cache Summary
NULL               ------------------
NULL
2SCLTEXTNLC            No line number content                    = false
2SCLTEXTLNC            Line number content                       = false
NULL
2SCLTEXTRCS            ROMClass start address                    = 0x00007F0EDB567000
2SCLTEXTRCE            ROMClass end address                      = 0x00007F0EDB567000
2SCLTEXTMSA            Metadata start address                    = 0x00007F0EDC40241C
2SCLTEXTCEA            Cache end address                         = 0x00007F0EDC54B000
2SCLTEXTRTF            Runtime flags                             = 0x80102001ECA602BB
2SCLTEXTCGN            Cache generation                          = 41
2SCLTEXTCLY            Cache layer                               = 1
NULL
2SCLTEXTCSZ            Cache size                                = 16776608
2SCLTEXTSMB            Softmx bytes                              = 16776608
2SCLTEXTFRB            Free bytes                                = 15315996
2SCLTEXTARB            Reserved space for AOT bytes              = -1
2SCLTEXTAMB            Maximum space for AOT bytes               = -1
2SCLTEXTJRB            Reserved space for JIT data bytes         = -1
2SCLTEXTJMB            Maximum space for JIT data bytes          = -1
2SCLTEXTRWB            ReadWrite bytes                           = 114080
2SCLTEXTDAS            Class debug area size                     = 1331200
2SCLTEXTDAU            Class debug area % used                   = 0%
2SCLTEXTDAN            Class LineNumberTable bytes               = 0
2SCLTEXTDAV            Class LocalVariableTable bytes            = 0
NULL
2SCLTEXTCPF            Cache is 8% full
NULL
1SCLTEXTCMST       Cache Memory Status
NULL               ------------------
1SCLTEXTCNTD           Cache Name                    Feature                  Memory type              Cache path
NULL
2SCLTEXTCMDT           Cache1                        CR                       Memory mapped file       /tmp/javasharedresources/C290M4F1A64P_Cache1_G41L01
NULL
1SCLTEXTCMST       Cache Lock Status
NULL               ------------------
1SCLTEXTCNTD           Lock Name                     Lock type                TID owning lock
NULL
2SCLTEXTCWRL           Cache write lock              File lock                Unowned
2SCLTEXTCRWL           Cache read/write lock         File lock                Unowned
NULL
1SCLTEXTCSAL   Cache Statistics for All Layers
NULL
2SCLTEXTRCB            ROMClass bytes                            = 1459040
2SCLTEXTAOB            AOT code bytes                            = 57624
2SCLTEXTADB            AOT data bytes                            = 272
2SCLTEXTAHB            AOT class hierarchy bytes                 = 1840
2SCLTEXTATB            AOT thunk bytes                           = 632
2SCLTEXTJHB            JIT hint bytes                            = 484
2SCLTEXTJPB            JIT profile bytes                         = 0
2SCLTEXTNOB            Java Object bytes                         = 0
2SCLTEXTZCB            Zip cache bytes                           = 1134016
2SCLTEXTSHB            Startup hint bytes                        = 0
2SCLTEXTJCB            JCL data bytes                            = 0
2SCLTEXTBDA            Byte data bytes                           = 0
NULL
2SCLTEXTNRC            Number ROMClasses                         = 503
2SCLTEXTNAM            Number AOT Methods                        = 16
2SCLTEXTNAD            Number AOT Data Entries                   = 1
2SCLTEXTNAH            Number AOT Class Hierarchy                = 28
2SCLTEXTNAT            Number AOT Thunks                         = 11
2SCLTEXTNJH            Number JIT Hints                          = 15
2SCLTEXTNJP            Number JIT Profiles                       = 0
2SCLTEXTNCP            Number Classpaths                         = 1
2SCLTEXTNUR            Number URLs                               = 0
2SCLTEXTNTK            Number Tokens                             = 0
2SCLTEXTNOJ            Number Java Objects                       = 0
2SCLTEXTNZC            Number Zip Caches                         = 21
2SCLTEXTNSH            Number Startup Hint Entries               = 0
2SCLTEXTNJC            Number JCL Entries                        = 0
2SCLTEXTNST            Number Stale classes                      = 0
2SCLTEXTPST            Percent Stale classes                     = 0%

 

类(CLASSES)

类部分显示有关类加载器的信息。第一部分是摘要,记录每个可用的类加载器 ( ),后面是它加载的库和类的数量。此信息后面是已加载的库 ( ) 和类 ( )2CLTEXTCLLOADER的更详细列表。1CLTEXTCLLIB1CLTEXTCLLO

在示例中,您可以看到java/lang/InternalAnonymousClassLoader已加载两个类,jdk/internal/loader/BuiltinClassLoader$$Lambda$2/00000000F03876A0(0x0000000001030F00)jdk/internal/loader/BuiltinClassLoader$$Lambda$1/00000000F00D2460(0x0000000001018A00)

NULL           ------------------------------------------------------------------------
0SECTION       CLASSES subcomponent dump routine
NULL           =================================
1CLTEXTCLLOS    Classloader summaries
1CLTEXTCLLSS        12345678: 1=primordial,2=extension,3=shareable,4=middleware,5=system,6=trusted,7=application,8=delegating
2CLTEXTCLLOADER     p---st-- Loader *System*(0x00000000FFE1D258)
3CLNMBRLOADEDLIB        Number of loaded libraries 5
3CLNMBRLOADEDCL         Number of loaded classes 638
2CLTEXTCLLOADER     -x--st-- Loader jdk/internal/loader/ClassLoaders$PlatformClassLoader(0x00000000FFE1D4F0), Parent *none*(0x0000000000000000)
3CLNMBRLOADEDLIB        Number of loaded libraries 0
3CLNMBRLOADEDCL         Number of loaded classes 0
2CLTEXTCLLOADER     ----st-- Loader java/lang/InternalAnonymousClassLoader(0x00000000FFE1DFD0), Parent *none*(0x0000000000000000)
3CLNMBRLOADEDLIB        Number of loaded libraries 0
3CLNMBRLOADEDCL         Number of loaded classes 2
2CLTEXTCLLOADER     -----ta- Loader jdk/internal/loader/ClassLoaders$AppClassLoader(0x00000000FFE1DAD0), Parent jdk/internal/loader/ClassLoaders$PlatformClassLoader(0x00000000FFE1D4F0)
3CLNMBRLOADEDLIB        Number of loaded libraries 0
3CLNMBRLOADEDCL         Number of loaded classes 0
1CLTEXTCLLIB    ClassLoader loaded libraries
2CLTEXTCLLIB        Loader *System*(0x00000000FFE1D258)
3CLTEXTLIB              /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/compressedrefs/jclse9_29
3CLTEXTLIB              /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/java
3CLTEXTLIB              /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/compressedrefs/j9jit29
3CLTEXTLIB              /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/zip
3CLTEXTLIB              /home/me/openj9-openjdk-jdk9/build/linux-x86_64-normal-server-release/images/jdk/lib/nio
1CLTEXTCLLOD    ClassLoader loaded classes
2CLTEXTCLLOAD       Loader *System*(0x00000000FFE1D258)
3CLTEXTCLASS            [Ljava/lang/Thread$State;(0x0000000001056400)
...
2CLTEXTCLLOAD       Loader jdk/internal/loader/ClassLoaders$PlatformClassLoader(0x00000000FFE1D4F0)
2CLTEXTCLLOAD       Loader java/lang/InternalAnonymousClassLoader(0x00000000FFE1DFD0)
3CLTEXTCLASS            jdk/internal/loader/BuiltinClassLoader$$Lambda$2/00000000F03876A0(0x0000000001030F00)
3CLTEXTCLASS            jdk/internal/loader/BuiltinClassLoader$$Lambda$1/00000000F00D2460(0x0000000001018A00)
2CLTEXTCLLOAD       Loader jdk/internal/loader/ClassLoaders$AppClassLoader(0x00000000FFE1DAD0)

场景(Scenarios)

一般保护故障(General Protection Fault)

在这种情况下,Java 应用程序由于通用保护错误 (GPF) 而崩溃,并自动生成 Java 转储文件。

文件的第一部分(TITLE)告诉您 GPF 触发了 Java 转储。

0SECTION       TITLE subcomponent dump routine
NULL           ===============================
1TICHARSET     UTF-8
1TISIGINFO     Dump Event "gpf" (00002000) received
1TIDATETIMEUTC Date: 2021/04/23 at 18:02:44:017 (UTC)
1TIDATETIME    Date: 2021/04/23 at 14:02:44:017
1TITIMEZONE    Timezone: UTC-4 (EDT)
1TINANOTIME    System nanotime: 379202644260787
1TIFILENAME    Javacore filename:    /home/test/JNICrasher/javacore.20210423.140244.29399.0002.txt
1TIREQFLAGS    Request Flags: 0x81 (exclusive+preempt)
1TIPREPSTATE   Prep State: 0x100 (trace_disabled)
1TIPREPINFO    Exclusive VM access not taken: data may not be consistent across javacore sections

要解决此问题,您需要知道哪个线程导致了 GPF 发生。崩溃时正在运行的线程在 Java 转储的 THREADS 部分中被报告为当前线程 。以下是从 THREADS 部分摘录的:

NULL           ------------------------------------------------------------------------
0SECTION       THREADS subcomponent dump routine
NULL           =================================
NULL
1XMPOOLINFO    JVM Thread pool info:
2XMPOOLTOTAL       Current total number of pooled threads: 16
2XMPOOLLIVE        Current total number of live threads: 15
2XMPOOLDAEMON      Current total number of live daemon threads: 14
NULL
1XMCURTHDINFO  Current thread
3XMTHREADINFO      "main" J9VMThread:0xB6B60E00, omrthread_t:0xB6B049D8, java/lang/Thread:0xB55444D0, state:R, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x1, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x72D8, native priority:0x5, native policy:UNKNOWN, vmstate:R, vm thread flags:0x00000000)
3XMTHREADINFO2            (native stack address range from:0xB6CE3000, to:0xB74E4000, size:0x801000)
3XMCPUTIME               CPU usage total: 0.319865924 secs, current category="Application"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=778008 (0xBDF18)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at JNICrasher.doSomethingThatCrashes(Native Method)
4XESTACKTRACE                at JNICrasher.main(JNICrasher.java:7)
3XMTHREADINFO3           Native callstack:
4XENATIVESTACK               (0xB6C6F663 [libj9prt29.so+0x3b663])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB6C6F1CE [libj9prt29.so+0x3b1ce])
4XENATIVESTACK               (0xB6C6F2C6 [libj9prt29.so+0x3b2c6])
4XENATIVESTACK               (0xB6C6ED93 [libj9prt29.so+0x3ad93])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB6C6ED07 [libj9prt29.so+0x3ad07])
4XENATIVESTACK               (0xB6C6AA3D [libj9prt29.so+0x36a3d])
4XENATIVESTACK               (0xB6C6C3A4 [libj9prt29.so+0x383a4])
4XENATIVESTACK               (0xB667FA19 [libj9dmp29.so+0xfa19])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB66878CF [libj9dmp29.so+0x178cf])
4XENATIVESTACK               (0xB6688083 [libj9dmp29.so+0x18083])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB6680C0D [libj9dmp29.so+0x10c0d])
4XENATIVESTACK               (0xB667F9D7 [libj9dmp29.so+0xf9d7])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB668B02F [libj9dmp29.so+0x1b02f])
4XENATIVESTACK               (0xB668B4D3 [libj9dmp29.so+0x1b4d3])
4XENATIVESTACK               (0xB66740F1 [libj9dmp29.so+0x40f1])
4XENATIVESTACK               (0xB66726FA [libj9dmp29.so+0x26fa])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB66726A9 [libj9dmp29.so+0x26a9])
4XENATIVESTACK               (0xB6676AE4 [libj9dmp29.so+0x6ae4])
4XENATIVESTACK               (0xB668D75A [libj9dmp29.so+0x1d75a])
4XENATIVESTACK               (0xB6A28DD4 [libj9vm29.so+0x81dd4])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB6A289EE [libj9vm29.so+0x819ee])
4XENATIVESTACK               (0xB6A29A40 [libj9vm29.so+0x82a40])
4XENATIVESTACK               (0xB6C52B6A [libj9prt29.so+0x1eb6a])
4XENATIVESTACK               __kernel_rt_sigreturn+0x0 (0xB7747410)
4XENATIVESTACK               (0xB75330B6 [libffi29.so+0x50b6])
4XENATIVESTACK               ffi_raw_call+0xad (0xB7531C53 [libffi29.so+0x3c53])
4XENATIVESTACK               (0xB69BE4AB [libj9vm29.so+0x174ab])
4XENATIVESTACK               (0xB6A665BC [libj9vm29.so+0xbf5bc])
4XENATIVESTACK               (0xB6A15552 [libj9vm29.so+0x6e552])
4XENATIVESTACK               (0xB6A30894 [libj9vm29.so+0x89894])
4XENATIVESTACK               (0xB6A6F169 [libj9vm29.so+0xc8169])
4XENATIVESTACK               (0xB6C52F6E [libj9prt29.so+0x1ef6e])
4XENATIVESTACK               (0xB6A6F1FA [libj9vm29.so+0xc81fa])
4XENATIVESTACK               (0xB6A30994 [libj9vm29.so+0x89994])
4XENATIVESTACK               (0xB6A2CE4C [libj9vm29.so+0x85e4c])
4XENATIVESTACK               (0xB770487D [libjli.so+0x787d])
4XENATIVESTACK               (0xB7719F72 [libpthread.so.0+0x6f72])
4XENATIVESTACK               clone+0x5e (0xB763543E [libc.so.6+0xee43e])

摘录告诉您当前线程为,并提供了有关崩溃发生时java/lang/ThreadJava 调用堆栈和本机调用堆栈 ( ) 的信息。为了模拟由应用程序中的错误导致的崩溃,此示例调用了一个 JNI 方法,其本机实现导致崩溃。Java 调用堆栈显示对 JNI 本机方法 ( ) 的调用,本机调用堆栈显示故障点。在此示例中,本机调用堆栈不包含任何函数名称来帮助您隔离本机代码中的错误。您可以从系统转储中获取此信息,系统转储通常与 Java 转储一起生成。使用Dump 查看器打开系统转储并使用命令打印当前线程的 Java 和本机堆栈。3XMTHREADINFO3JNIcrasherinfo thread

下次运行应用程序时,您可以使用-XX:+ShowNativeStackSymbols=all命令行选项显示本机调用堆栈中相应的函数名称。

4XENATIVESTACK               protectedBacktrace+0x12 (0x00007F3F9213E312 [libj9prt29.so+0x25312])
4XENATIVESTACK               omrsig_protect+0x1e3 (0x00007F3F92142AD3 [libj9prt29.so+0x29ad3])
4XENATIVESTACK               omrintrospect_backtrace_thread_raw+0xbf (0x00007F3F9213E80F [libj9prt29.so+0x2580f])
4XENATIVESTACK               omrsig_protect+0x1e3 (0x00007F3F92142AD3 [libj9prt29.so+0x29ad3])
4XENATIVESTACK               omrintrospect_backtrace_thread+0x70 (0x00007F3F9213E1D0 [libj9prt29.so+0x251d0])
4XENATIVESTACK               setup_native_thread+0x1d2 (0x00007F3F9213F652 [libj9prt29.so+0x26652])
4XENATIVESTACK               omrintrospect_threads_startDo_with_signal+0x474 (0x00007F3F921403F4 [libj9prt29.so+0x273f4])
4XENATIVESTACK               omrsig_protect+0x1e3 (0x00007F3F92142AD3 [libj9prt29.so+0x29ad3])

Java 内存溢出错误(Java OutOfMemoryError)

在这种情况下,Java 堆内存不足,导致OutOfMemoryError,并自动生成 Java 转储文件。

文件的第一部分(TITLE)告诉您systhrow由于java/lang/OutOfMemoryErrorJava 堆空间出现 OOM(),某个事件触发了 Java 转储。

0SECTION       TITLE subcomponent dump routine
NULL           ===============================
1TICHARSET     UTF-8
1TISIGINFO     Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Java heap space" received
1TIDATETIMEUTC Date: 2021/04/23 at 18:02:44:017 (UTC)
1TIDATETIME    Date: 2021/04/23 at 14:02:44:017
1TITIMEZONE    Timezone: UTC-4 (EDT)
1TINANOTIME    System nanotime: 379202644260787
1TIFILENAME    Javacore filename:    /home/cheesemp/test/javacore.20210423.140244.18885.0003.txt
1TIREQFLAGS    Request Flags: 0x81 (exclusive+preempt)
1TIPREPSTATE   Prep State: 0x104 (exclusive_vm_access+trace_disabled)

MEMINFO 部分记录了分配给 Java 堆 ( 1STHEAPTYPE Object Memory) 的内存量、正在使用的内存量以及空闲的内存量。解决问题可能很简单,只需在启动应用程序时设置更大的堆大小即可。

如果您不知道 Java 堆的大小设置是多少,您可以在 ENVINFO 部分中找到该信息,该部分记录了应用程序启动时使用的命令行选项。查找或搜索字符串1CIUSERARGS UserArgs:并查看以 开头的所有行记录的条目2CIUSERARG。Java 堆大小由 选项设置-Xmx。如果未在命令行上通过 设置大小,则应用默认值,您可以在默认设置-Xmx中找到该默认值。

在这种情况下,解决问题的方法不是调整 Java 堆大小。以下是 MEMINFO 部分:

0SECTION       MEMINFO subcomponent dump routine
NULL           =================================
NULL
1STHEAPTYPE    Object Memory
NULL           id         start      end        size       space/region
1STHEAPSPACE   0xB6B49D20     --         --         --     Generational
1STHEAPREGION  0xB6B4A078 0x95750000 0xB5470000 0x1FD20000 Generational/Tenured Region
1STHEAPREGION  0xB6B49F10 0xB5470000 0xB54C0000 0x00050000 Generational/Nursery Region
1STHEAPREGION  0xB6B49DA8 0xB54C0000 0xB5750000 0x00290000 Generational/Nursery Region
NULL
1STHEAPTOTAL   Total memory:         536870912 (0x20000000)
1STHEAPINUSE   Total memory in use:  302603160 (0x12095B98)
1STHEAPFREE    Total memory free:    234267752 (0x0DF6A468)

输出显示只有 56% 的 Java 堆正在使用中,因此这表明应用程序正在尝试执行一些不太理想的操作。要进一步调查,您需要找出发生 OOM 时哪个线程是当前线程,以查看它正在尝试执行的操作。与上一个场景一样,您可以在 THREADS 部分中找到 当前线程。以下是输出的摘录:

0SECTION       THREADS subcomponent dump routine
NULL           =================================
NULL
1XMPOOLINFO    JVM Thread pool info:
2XMPOOLTOTAL       Current total number of pooled threads: 16
2XMPOOLLIVE        Current total number of live threads: 16
2XMPOOLDAEMON      Current total number of live daemon threads: 15
NULL
1XMCURTHDINFO  Current thread
3XMTHREADINFO      "main" J9VMThread:0xB6B60C00, omrthread_t:0xB6B049D8, java/lang/Thread:0x95764520, state:R, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x1, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x49C6, native priority:0x5, native policy:UNKNOWN, vmstate:R, vm thread flags:0x00001020)
3XMTHREADINFO2            (native stack address range from:0xB6CB5000, to:0xB74B6000, size:0x801000)
3XMCPUTIME               CPU usage total: 8.537823831 secs, current category="Application"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at java/lang/StringBuffer.ensureCapacityImpl(StringBuffer.java:696)
4XESTACKTRACE                at java/lang/StringBuffer.append(StringBuffer.java:486(Compiled Code))
5XESTACKTRACE                   (entered lock: java/lang/StringBuffer@0x957645B8, entry count: 1)
4XESTACKTRACE                at java/lang/StringBuffer.append(StringBuffer.java:428(Compiled Code))
4XESTACKTRACE                at HeapBreaker.main(HeapBreaker.java:34(Compiled Code))
3XMTHREADINFO3           Native callstack:
4XENATIVESTACK               (0xB6C535B3 [libj9prt29.so+0x3b5b3])
4XENATIVESTACK               (0xB6C36F3E [libj9prt29.so+0x1ef3e])
4XENATIVESTACK               (0xB6C5311E [libj9prt29.so+0x3b11e])
4XENATIVESTACK               (0xB6C53216 [libj9prt29.so+0x3b216])
4XENATIVESTACK               (0xB6C52CE3 [libj9prt29.so+0x3ace3])
4XENATIVESTACK               (0xB6C36F3E [libj9prt29.so+0x1ef3e])
4XENATIVESTACK               (0xB6C52C57 [libj9prt29.so+0x3ac57])
4XENATIVESTACK               (0xB6C4E9CD [libj9prt29.so+0x369cd])
4XENATIVESTACK               (0xB6C502FA [libj9prt29.so+0x382fa])

为了模拟 Java OutOfMemoryError,此示例应用程序StringBuffer在无限循环中反复将字符附加到对象。Java 调用堆栈显示该HeapBreaker.main方法附加字符(java/lang/StringGuffer.append),直到该方法java/lang/StringBuffer.ensureCapacityImpl()抛出OutOfMemoryError

StringBuffer 对象是字符数组 ( ) 的包装器char[],当达到底层数组的容量时,内容会自动复制到新的更大的数组中。新数组在StringBuffer.ensureCapacity()方法中创建,其大小或多或少是旧数组的两倍。在这种情况下,数组会占用 Java 堆中的所有剩余空间。

Java 转储文件的 MEMINFO 部分还可以告诉您何时意外的大型分配请求导致 OOM。查找 GC 历史记录 ( 1STGCHTYPE) 部分,其中详细说明了触发 GC 活动的分配请求。在示例输出中,您可以看到大型分配请求 ( requestedbytes=603979784) 触发了全局 GC。当 GC 无法在堆中释放足够的空间来满足请求时,分配失败会生成 OOM。

1STGCHTYPE     GC History
3STHSTTYPE     14:29:29:580239000 GMT j9mm.101 -   J9AllocateIndexableObject() returning NULL! 0 bytes requested for object of class B6BBC300 from memory space 'Generational' id=B6B49D20
3STHSTTYPE     14:29:29:579916000 GMT j9mm.134 -   Allocation failure end: newspace=2686912/3014656 oldspace=231597224/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:579905000 GMT j9mm.470 -   Allocation failure cycle end: newspace=2686912/3014656 oldspace=231597224/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:579859000 GMT j9mm.475 -   GlobalGC end: workstackoverflow=0 overflowcount=0 memory=234284136/536870912
3STHSTTYPE     14:29:29:579807000 GMT j9mm.90 -   GlobalGC collect complete
3STHSTTYPE     14:29:29:579776000 GMT j9mm.137 -   Compact end: bytesmoved=301989896
3STHSTTYPE     14:29:29:313899000 GMT j9mm.136 -   Compact start: reason=compact to meet allocation
3STHSTTYPE     14:29:29:313555000 GMT j9mm.57 -   Sweep end
3STHSTTYPE     14:29:29:310772000 GMT j9mm.56 -   Sweep start
3STHSTTYPE     14:29:29:310765000 GMT j9mm.94 -   Class unloading end: classloadersunloaded=0 classesunloaded=0
3STHSTTYPE     14:29:29:310753000 GMT j9mm.60 -   Class unloading start
3STHSTTYPE     14:29:29:310750000 GMT j9mm.55 -   Mark end
3STHSTTYPE     14:29:29:306013000 GMT j9mm.54 -   Mark start
3STHSTTYPE     14:29:29:305957000 GMT j9mm.474 -   GlobalGC start: globalcount=9
3STHSTTYPE     14:29:29:305888000 GMT j9mm.475 -   GlobalGC end: workstackoverflow=0 overflowcount=0 memory=234284136/536870912
3STHSTTYPE     14:29:29:305837000 GMT j9mm.90 -   GlobalGC collect complete
3STHSTTYPE     14:29:29:305808000 GMT j9mm.137 -   Compact end: bytesmoved=189784
3STHSTTYPE     14:29:29:298042000 GMT j9mm.136 -   Compact start: reason=compact to meet allocation
3STHSTTYPE     14:29:29:297695000 GMT j9mm.57 -   Sweep end
3STHSTTYPE     14:29:29:291696000 GMT j9mm.56 -   Sweep start
3STHSTTYPE     14:29:29:291692000 GMT j9mm.55 -   Mark end
3STHSTTYPE     14:29:29:284994000 GMT j9mm.54 -   Mark start
3STHSTTYPE     14:29:29:284941000 GMT j9mm.474 -   GlobalGC start: globalcount=8
3STHSTTYPE     14:29:29:284916000 GMT j9mm.135 -   Exclusive access: exclusiveaccessms=0.016 meanexclusiveaccessms=0.016 threads=0 lastthreadtid=0xB6B61100 beatenbyotherthread=0
3STHSTTYPE     14:29:29:284914000 GMT j9mm.469 -   Allocation failure cycle start: newspace=2678784/3014656 oldspace=80601248/533856256 loa=5338112/5338112 requestedbytes=603979784
3STHSTTYPE     14:29:29:284893000 GMT j9mm.470 -   Allocation failure cycle end: newspace=2678784/3014656 oldspace=80601248/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:284858000 GMT j9mm.560 -   LocalGC end: rememberedsetoverflow=0 causedrememberedsetoverflow=0 scancacheoverflow=0 failedflipcount=0 failedflipbytes=0 failedtenurecount=0 failedtenurebytes=0 flipcount=2 flipbytes=64 newspace=2678784/3014656 oldspace=80601248/533856256 loa=5338112/5338112 tenureage=0
3STHSTTYPE     14:29:29:284140000 GMT j9mm.140 -   Tilt ratio: 89
3STHSTTYPE     14:29:29:283160000 GMT j9mm.64 -   LocalGC start: globalcount=8 scavengecount=335 weakrefs=0 soft=0 phantom=0 finalizers=0
3STHSTTYPE     14:29:29:283123000 GMT j9mm.135 -   Exclusive access: exclusiveaccessms=0.016 meanexclusiveaccessms=0.016 threads=0 lastthreadtid=0xB6B61100 beatenbyotherthread=0
3STHSTTYPE     14:29:29:283120000 GMT j9mm.469 -   Allocation failure cycle start: newspace=753616/3014656 oldspace=80601248/533856256 loa=5338112/5338112 requestedbytes=603979784
3STHSTTYPE     14:29:29:283117000 GMT j9mm.133 -   Allocation failure start: newspace=753616/3014656 oldspace=80601248/533856256 loa=5338112/5338112 requestedbytes=603979784
3STHSTTYPE     14:29:29:269762000 GMT j9mm.134 -   Allocation failure end: newspace=2686928/3014656 oldspace=80601248/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:269751000 GMT j9mm.470 -   Allocation failure cycle end: newspace=2686976/3014656 oldspace=80601248/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:269718000 GMT j9mm.560 -   LocalGC end: rememberedsetoverflow=0 causedrememberedsetoverflow=0 scancacheoverflow=0 failedflipcount=0 failedflipbytes=0 failedtenurecount=0 failedtenurebytes=0 flipcount=0 flipbytes=0 newspace=2686976/3014656 oldspace=80601248/533856256 loa=5338112/5338112 tenureage=0
3STHSTTYPE     14:29:29:268981000 GMT j9mm.140 -   Tilt ratio: 89
3STHSTTYPE     14:29:29:268007000 GMT j9mm.64 -   LocalGC start: globalcount=8 scavengecount=334 weakrefs=0 soft=0 phantom=0 finalizers=0
3STHSTTYPE     14:29:29:267969000 GMT j9mm.135 -   Exclusive access: exclusiveaccessms=0.016 meanexclusiveaccessms=0.016 threads=0 lastthreadtid=0xB6B61100 beatenbyotherthread=0
3STHSTTYPE     14:29:29:267966000 GMT j9mm.469 -   Allocation failure cycle start: newspace=0/3014656 oldspace=80601248/533856256 loa=5338112/5338112 requestedbytes=48
3STHSTTYPE     14:29:29:267963000 GMT j9mm.133 -   Allocation failure start: newspace=0/3014656 oldspace=80601248/533856256 loa=5338112/5338112 requestedbytes=48
3STHSTTYPE     14:29:29:249015000 GMT j9mm.134 -   Allocation failure end: newspace=2686928/3014656 oldspace=80601248/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:249003000 GMT j9mm.470 -   Allocation failure cycle end: newspace=2686976/3014656 oldspace=80601248/533856256 loa=5338112/5338112
3STHSTTYPE     14:29:29:248971000 GMT j9mm.560 -   LocalGC end: rememberedsetoverflow=0 causedrememberedsetoverflow=0 scancacheoverflow=0 failedflipcount=0 failedflipbytes=0 failedtenurecount=0 failedtenurebytes=0 flipcount=0 flipbytes=0 newspace=2686976/3014656 oldspace=80601248/533856256 loa=5338112/5338112 tenureage=0

尽管在这种情况下使用的 Java 代码故意OutOfMemoryError以明显的方式触发,但在处理诸如 XML 文件之类的大型数据集时,可能会发生类似的分配问题。

诊断问题的下一步是打开发生问题时自动生成的系统转储。使用Eclipse Memory Analyzer™ 工具 (MAT)OutOfMemoryError打开转储并搜索对象,这应该会提供有关问题所在的更多信息。一个常见的例子是看到相同的内容一遍又一遍地重复,这可能表明代码陷入了循环。StringBufferString

注意:如果要使用 MAT 分析系统转储,必须在 Eclipse IDE 中安装 Java 诊断工具框架 (DTFJ) 插件。选择以下菜单项:

Help > Install New Software > Work with "IBM Diagnostic Tool Framework for Java" >

如果与上一种情况不同,您收到一个错误OutOfMemoryError,并且 MEMINFO 部分显示 Java 堆上剩余的空间非常小,则当前线程信息通常并不重要。当前线程只是空间用尽时碰巧是当前的线程。在这种情况下,您可能需要增加 Java 堆大小。有关此任务的帮助,请参阅如何进行堆大小调整

本机 OutOfMemoryError(Native OutOfMemoryError)

在这种情况下,VM 会耗尽本机内存。本机内存是 VM 用于存储 VM 操作所需的所有虚拟化资源和数据的内存。VM 进程可用的本机内存受操作系统限制。VM 可用的本机内存还可能受到操作系统(例如 Unix)施加的其他限制ulimits

发生时NativeOutOfMemoryError,默认情况下会生成 Java 转储。文件的第一部分 (TITLE) 告诉您,由于java/lang/OutOfMemoryError本机内存的 OOM (),systhrow 事件触发了 Java 转储。

0SECTION       TITLE subcomponent dump routine
NULL           ===============================
1TICHARSET     UTF-8
1TISIGINFO     Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "native memory exhausted" received
1TIDATETIMEUTC Date: 2021/04/23 at 18:02:44:017 (UTC)
1TIDATETIME    Date: 2021/04/23 at 14:02:44:017
1TITIMEZONE    Timezone: UTC-4 (EDT)
1TINANOTIME    System nanotime: 379202644260787
1TIFILENAME    Javacore filename:    /home/cheesemp/test/javacore.20210423.140244.19708.0003.txt
1TIREQFLAGS    Request Flags: 0x81 (exclusive+preempt)
1TIPREPSTATE   Prep State: 0x104 (exclusive_vm_access+trace_disabled)

有时,当前线程是导致 的原因NativeOutOfMemoryError。有关当前线程的信息可以在 THREADS 部分中找到,如以下输出所示。

0SECTION       THREADS subcomponent dump routine
NULL           =================================
NULL
1XMPOOLINFO    JVM Thread pool info:
2XMPOOLTOTAL       Current total number of pooled threads: 16
2XMPOOLLIVE        Current total number of live threads: 16
2XMPOOLDAEMON      Current total number of live daemon threads: 15
NULL
1XMCURTHDINFO  Current thread
3XMTHREADINFO      "main" J9VMThread:0xB6C60C00, omrthread_t:0xB6C049D8, java/lang/Thread:0xB55E3C10, state:R, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x1, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x4CFD, native priority:0x5, native policy:UNKNOWN, vmstate:R, vm thread flags:0x00001020)
3XMTHREADINFO2            (native stack address range from:0xB6D4E000, to:0xB754F000, size:0x801000)
3XMCPUTIME               CPU usage total: 3.654896026 secs, current category="Application"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at sun/misc/Unsafe.allocateDBBMemory(Native Method)
4XESTACKTRACE                at java/nio/DirectByteBuffer.<init>(DirectByteBuffer.java:127(Compiled Code))
4XESTACKTRACE                at java/nio/ByteBuffer.allocateDirect(ByteBuffer.java:311)
4XESTACKTRACE                at NativeHeapBreaker.main(NativeHeapBreaker.java:9)
3XMTHREADINFO3           Native callstack:
4XENATIVESTACK               (0xB6A9F5B3 [libj9prt29.so+0x3b5b3])
...
4XENATIVESTACK               (0xB582CC9C [libjclse7b_29.so+0x40c9c])
4XENATIVESTACK               Java_sun_misc_Unsafe_allocateDBBMemory+0x88 (0xB5827F6B [libjclse7b_29.so+0x3bf6b])
4XENATIVESTACK               (0x94A2084A [<unknown>+0x0])
4XENATIVESTACK               (0xB6B2538B [libj9vm29.so+0x6c38b])
4XENATIVESTACK               (0xB6B4074C [libj9vm29.so+0x8774c])
4XENATIVESTACK               (0xB6B7F299 [libj9vm29.so+0xc6299])
4XENATIVESTACK               (0xB6A82F3E [libj9prt29.so+0x1ef3e])
4XENATIVESTACK               (0xB6B7F32A [libj9vm29.so+0xc632a])
4XENATIVESTACK               (0xB6B4084C [libj9vm29.so+0x8784c])
4XENATIVESTACK               (0xB6B3CD0C [libj9vm29.so+0x83d0c])
4XENATIVESTACK               (0xB776F87D [libjli.so+0x787d])
4XENATIVESTACK               (0xB7784F72 [libpthread.so.0+0x6f72])
4XENATIVESTACK               clone+0x5e (0xB76A043E [libc.so.6+0xee43e])

为了输出更清晰Native callstack...指示删除了一些行。

Java 调用堆栈显示从 Java 到本机代码的转换(sun/misc/Unsafe.allocateDBBMemory(Native Method)),表明请求直接字节缓冲区 (DBB) 存储。DBB 存储由本机内存支持,Java 堆仅包含对本机堆缓冲区的引用。在这种情况下,DBB 存储可能是导致此问题的罪魁祸首NativeOutOfMemoryError

下一步是调查 Java 转储文件的 NATIVEMEMINFO 部分,该部分报告 JRE 进程使用的内存量,并分解为各个组成区域。

0SECTION       NATIVEMEMINFO subcomponent dump routine
NULL           =================================
0MEMUSER
1MEMUSER       JRE: 3,166,386,688 bytes / 4388 allocations
1MEMUSER       |
2MEMUSER       +--VM: 563,176,824 bytes / 1518 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Classes: 3,104,416 bytes / 120 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Memory Manager (GC): 548,181,888 bytes / 398 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Heap: 536,932,352 bytes / 1 allocation
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 11,249,536 bytes / 397 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Threads: 10,817,120 bytes / 147 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Stack: 115,584 bytes / 16 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Native Stack: 10,616,832 bytes / 17 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 84,704 bytes / 114 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Trace: 163,688 bytes / 268 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JVMTI: 17,320 bytes / 13 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JNI: 23,296 bytes / 55 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Port Library: 8,576 bytes / 74 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Other: 860,520 bytes / 443 allocations
1MEMUSER       |
2MEMUSER       +--JIT: 3,744,728 bytes / 122 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JIT Code Cache: 2,097,152 bytes / 1 allocation
2MEMUSER       |  |
3MEMUSER       |  +--JIT Data Cache: 524,336 bytes / 1 allocation
2MEMUSER       |  |
3MEMUSER       |  +--Other: 1,123,240 bytes / 120 allocations
1MEMUSER       |
2MEMUSER       +--Class Libraries: 2,599,463,024 bytes / 2732 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Harmony Class Libraries: 1,024 bytes / 1 allocation
2MEMUSER       |  |
3MEMUSER       |  +--VM Class Libraries: 2,599,462,000 bytes / 2731 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--sun.misc.Unsafe: 2,598,510,480 bytes / 2484 allocations
4MEMUSER       |  |  |  |
5MEMUSER       |  |  |  +--Direct Byte Buffers: 2,598,510,480 bytes / 2484 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 951,520 bytes / 247 allocations
1MEMUSER       |
2MEMUSER       +--Unknown: 2,112 bytes / 16 allocations
NULL

在 部分中,显示了VM Class Libraries分配的内存量。由于 是在小型 32 位系统上接收的,因此 的值表示操作系统内存不足。在较大的 UNIX® 系统上,由于 的设置,进程可能内存不足。增加 的值可能会避免出现错误,您可以通过在当前会话中设置来暂时避免此错误。Direct Byte BuffersNativeOutOfMemoryError2,598,510,480 bytesulimitulimitulimit -f unlimited

32 位进程的理论最大大小是 32 位地址空间的大小,即 4 GB。在大多数操作系统上,每个进程的一部分地址空间由内核使用,因此 32 位进程的实际限制实际上远小于 4 GB。因此,使用 32 位 VM 时,本机内存耗尽的情况相当常见。

如果您使用的是带有压缩引用的 64 位 VM,则相同的 4 GB 限制也很重要。在压缩引用模式下,出于性能原因,对对象、类、线程和监视器的所有引用都由 32 位值表示,因此这些结构只能分配在 32 位地址上。但是,操作系统可能会在此 4 GB 地址空间内放置其他分配,如果此区域变得足够满或碎片化,VM 会抛出本机NativeOutOfMemoryError错误。这些错误通常在 VM 尝试创建新线程或加载类时发生。当前线程历史记录部分应包含有关发生错误时线程在 VM 级别执行的操作的更多信息NativeOutOfMemoryError

-Xmcrs您通常可以通过在虚拟机启动时选择在最低 4 GB 内存中保留连续的内存区域来避免此类问题。

导致 的另一个常见原因NativeOutOfMemoryError是应用程序加载了重复的类。类在本机内存中的 Java 堆之外分配。如果ClassesNATIVEMEMINFO 部分中报告的值非常大,则重复的类可能是导致问题的原因。Eclipse内存分析器工具 (MAT)可以使用Class Loader Explorer功能告诉您是否有重复的类。由于系统转储和 Java 转储都会自动生成以响应NativeOutOfMemoryError,因此只需在 MAT 中打开系统转储即可继续诊断。

死锁(Deadlock)

当两个线程尝试同步一个对象并锁定一个类的实例时,就会发生死锁。发生这种情况时,您的应用程序会停止响应并挂起。生成 Java 转储文件将快速告诉您是否有死锁情况。通过向kill -3VM 发送 SIGQUIT 信号 ( ) 来触发 Java 转储。

VM 可以检测涉及 Java 监视器的最常见死锁类型。如果检测到这种类型的死锁,则会在 LOCKS 部分提供信息。无法检测到更复杂的死锁,包括涉及本机互斥锁和 Java 监视器混合的死锁。

以下是用于导致常见死锁情况的代码的输出:

NULL
1LKDEADLOCK    Deadlock detected !!!
NULL           ---------------------
NULL
2LKDEADLOCKTHR  Thread "Worker Thread 2" (0x94501D00)
3LKDEADLOCKWTR    is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x08C2B344 infl_mon_t: 0x08C2B384:
4LKDEADLOCKOBJ      java/lang/Object@0xB5666698
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "Worker Thread 3" (0x94507500)
3LKDEADLOCKWTR    which is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x08C2B3A0 infl_mon_t: 0x08C2B3E0:
4LKDEADLOCKOBJ      java/lang/Object@0xB5666678
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "Worker Thread 1" (0x92A3EC00)
3LKDEADLOCKWTR    which is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x08C2B2E8 infl_mon_t: 0x08C2B328:
4LKDEADLOCKOBJ      java/lang/Object@0xB5666688
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "Worker Thread 2" (0x94501D00)

此输出告诉您Worker Thread 2正在等待Worker Thread 3,即 正在等待Worker Thread 1。由于Worker Thread 1也在等待Worker Thread 2,因此存在死锁。下一个要查看的地方是 THREADS 部分中 Java 和本机堆栈的输出。通过查看每个工作线程的堆栈,您可以将问题追溯到应用程序代码中的特定行。

在此示例中,您可以从以下输出中看到,对于所有工作线程,堆栈跟踪(4XESTACKTRACE/ 5XESTACKTRACE)指示应用程序第 35 行存在问题DeadLockTest.java

3XMTHREADINFO      "Worker Thread 1" J9VMThread:0x92A3EC00, omrthread_t:0x92A3C2B0, java/lang/Thread:0xB5666778, state:B, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x13, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x52CF, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
3XMTHREADINFO2            (native stack address range from:0x9297E000, to:0x929BF000, size:0x41000)
3XMCPUTIME               CPU usage total: 0.004365543 secs, current category="Application"
3XMTHREADBLOCK     Blocked on: java/lang/Object@0xB5666688 Owned by: "Worker Thread 2" (J9VMThread:0x94501D00, java/lang/Thread:0xB56668D0)
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at WorkerThread.run(DeadLockTest.java:35)
5XESTACKTRACE                   (entered lock: java/lang/Object@0xB5666678, entry count: 1)
...
3XMTHREADINFO      "Worker Thread 2" J9VMThread:0x94501D00, omrthread_t:0x92A3C8F0, java/lang/Thread:0xB56668D0, state:B, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x14, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x52D0, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
3XMTHREADINFO2            (native stack address range from:0x946BF000, to:0x94700000, size:0x41000)
3XMCPUTIME               CPU usage total: 0.004555580 secs, current category="Application"
3XMTHREADBLOCK     Blocked on: java/lang/Object@0xB5666698 Owned by: "Worker Thread 3" (J9VMThread:0x94507500, java/lang/Thread:0xB5666A18)
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at WorkerThread.run(DeadLockTest.java:35)
5XESTACKTRACE                   (entered lock: java/lang/Object@0xB5666688, entry count: 1)
...
3XMTHREADINFO      "Worker Thread 3" J9VMThread:0x94507500, omrthread_t:0x92A3CC10, java/lang/Thread:0xB5666A18, state:B, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x15, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x52D1, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
3XMTHREADINFO2            (native stack address range from:0x9467E000, to:0x946BF000, size:0x41000)
3XMCPUTIME               CPU usage total: 0.003657010 secs, current category="Application"
3XMTHREADBLOCK     Blocked on: java/lang/Object@0xB5666678 Owned by: "Worker Thread 1" (J9VMThread:0x92A3EC00, java/lang/Thread:0xB5666778)
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at WorkerThread.run(DeadLockTest.java:35)
5XESTACKTRACE                   (entered lock: java/lang/Object@0xB5666698, entry count: 1)

挂起(Hang)

应用程序可能因多种原因而挂起,但最常见的原因是过多的全局垃圾收集 (GC) 活动,即由于 Java 堆几乎耗尽内存,应用程序会反复暂停。您可以通过查看详细的 GC 输出来识别此问题。通过指定-verbose:gc选项来收集此输出。

死锁情况也可能表现为挂起。有关从 Java 转储诊断此类问题的更多信息,请参阅死锁场景。

如果您已经消除了冗长的 GC 活动和死锁,另一种常见的挂起情况涉及线程竞争并等待 Java 对象锁。这种类型的问题通常可以通过检查 Java 转储来诊断。涉及 Java 对象锁的最简单的挂起情况是,一个线程获取了其他线程正在等待的锁,但由于某种原因它没有释放该锁。

Java 转储输出中首先要查看的是LOCKS部分。此部分列出了所有监视器,并显示哪些线程已获取锁以及哪些线程正在等待。如果挂起是由于某个线程未释放其他线程所需的锁而导致的,您可以在输出中看到等待线程的列表。

在此示例场景中,Java 转储LOCKS部分显示Worker Thread 03LKMONOBJECT)已获取锁,并且有 19 个其他工作线程正在等待获取锁。

NULL           ------------------------------------------------------------------------
0SECTION       LOCKS subcomponent dump routine
NULL           ===============================
NULL
1LKPOOLINFO    Monitor pool info:
2LKPOOLTOTAL     Current total number of monitors: 1
NULL
1LKMONPOOLDUMP Monitor Pool Dump (flat & inflated object-monitors):
2LKMONINUSE      sys_mon_t:0x92711200 infl_mon_t: 0x92711240:
3LKMONOBJECT       java/lang/Object@0xB56658D8: Flat locked by "Worker Thread 0" (J9VMThread:0x92A3EC00), entry count 1
3LKWAITERQ            Waiting to enter:
3LKWAITER                "Worker Thread 1" (J9VMThread:0x92703F00)
3LKWAITER                "Worker Thread 2" (J9VMThread:0x92709C00)
3LKWAITER                "Worker Thread 3" (J9VMThread:0x92710A00)
3LKWAITER                "Worker Thread 4" (J9VMThread:0x92717F00)
3LKWAITER                "Worker Thread 5" (J9VMThread:0x9271DC00)
3LKWAITER                "Worker Thread 6" (J9VMThread:0x92723A00)
3LKWAITER                "Worker Thread 7" (J9VMThread:0x92729800)
3LKWAITER                "Worker Thread 8" (J9VMThread:0x92733700)
3LKWAITER                "Worker Thread 9" (J9VMThread:0x92739400)
3LKWAITER                "Worker Thread 10" (J9VMThread:0x92740200)
3LKWAITER                "Worker Thread 11" (J9VMThread:0x92748100)
3LKWAITER                "Worker Thread 12" (J9VMThread:0x9274DF00)
3LKWAITER                "Worker Thread 13" (J9VMThread:0x92754D00)
3LKWAITER                "Worker Thread 14" (J9VMThread:0x9275AA00)
3LKWAITER                "Worker Thread 15" (J9VMThread:0x92760800)
3LKWAITER                "Worker Thread 16" (J9VMThread:0x92766600)
3LKWAITER                "Worker Thread 17" (J9VMThread:0x9276C300)
3LKWAITER                "Worker Thread 18" (J9VMThread:0x92773100)
3LKWAITER                "Worker Thread 19" (J9VMThread:0x92778F00)
NULL

下一步是确定为什么Worker Thread 0没有释放锁。最好的起点是此线程的堆栈跟踪,您可以通过在THREADS部分中搜索线程名称或 J9VMThread ID 来找到它。

以下摘录显示了 的详细信息"Worker Thread 0" (J9VMThread:0x92A3EC00)

NULL
3XMTHREADINFO      "Worker Thread 0" J9VMThread:0x92A3EC00, omrthread_t:0x92A3C280, java/lang/Thread:0xB56668B8, state:CW, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x13, isDaemon:false)
3XMTHREADINFO1            (native thread ID:0x511F, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000401)
3XMTHREADINFO2            (native stack address range from:0x9297E000, to:0x929BF000, size:0x41000)
3XMCPUTIME               CPU usage total: 0.000211878 secs, current category="Application"
3XMHEAPALLOC             Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at java/lang/Thread.sleep(Native Method)
4XESTACKTRACE                at java/lang/Thread.sleep(Thread.java:941)
4XESTACKTRACE                at WorkerThread.doWork(HangTest.java:37)
4XESTACKTRACE                at WorkerThread.run(HangTest.java:31)
5XESTACKTRACE                   (entered lock: java/lang/Object@0xB56658D8, entry count: 1)

在此输出的最后一行中,您可以看到线程获取锁的位置。从此行往上看,您可以看到调用WorkerThread.run了,而后者又调用了WorkerThread.doWork。堆栈显示线程随后java/lang/Thread.sleep在第 37 行的 HangTest.java 中进入了对的调用,这阻止了线程完成其工作并释放锁。在此示例中,sleep添加调用是为了引发挂起,但在实际场景中,原因可能是任何阻塞操作,例如从输入流或套接字读取。另一种可能性是线程正在等待另一个线程拥有的另一个锁。

重要的是要记住,每个 Java 转储都代表一个时间快照。您应该生成至少三个 Java 转储,中间间隔一个短暂的停顿(例如 30 秒),然后比较输出。通过比较,您可以了解所涉及的线程是卡在固定状态还是正在移动。

在这个例子中,线程没有移动,调查需要重点关注其中的逻辑,以WorkerThread.doWork了解为什么Worker Thread 0进入java/lang/Thread.sleep调用。

另一个常见情况是,每个 Java 转储都显示许多线程正在等待另一个线程拥有的锁,但等待线程列表和拥有锁的线程会随时间而变化。在这种情况下,原因可能是线程争用导致的瓶颈,其中线程不断争用同一个锁。在严重的情况下,锁只被持有一小段时间,但有许多线程试图获取它。由于处理锁和调度线程所花费的时间比执行应用程序代码所花费的时间更多,因此性能下降表现为挂起。线程争用通常是由应用程序设计问题引起的。您可以使用与此场景中使用的方法类似的方法来确定哪些代码行导致了争用。

;