Reading a thread dump with jstack
When a JVM application freezes, burns all the CPU for no obvious reason, or slows down all at once, the first question is simple: what are the threads doing right now? jstack answers that question. It is a tool shipped with the JDK that prints the call stack of every thread in a live Java process. No agent to install, no restart, no dependency. You have the PID, you have the dump.
A thread dump is a photo. It freezes the state of all threads at one instant: what they are running, which lock they are waiting on, and which state they are in. Read well, it tells you in seconds whether the application is stuck on a lock, waiting on I/O, or actually doing work.
Find the right process
jstack needs the PID of the target JVM. jps, also part of the JDK, lists Java processes with their main class:
$ jps -l
9850 DeadlockDemo
9123 org.springframework.boot.loader.JarLauncher
Then attach jstack to the PID:
jstack 9850
The dump goes to standard output. Usually you redirect it to a file so you can read it back calmly:
jstack -l 9850 > dump-1.txt
The -l flag (long listing) adds lock information. We will see below why it is worth it almost every time.
Anatomy of a thread line
Each thread starts with a header line, followed by its stack. Here is a real application thread:
"worker-1" #25 [25091] prio=5 os_prio=31 cpu=0.18ms elapsed=13.48s tid=0x0000000810743800 nid=25091 waiting for monitor entry [0x00000001719c2000]
java.lang.Thread.State: BLOCKED (on object monitor)
at DeadlockDemo.lambda$main$0(DeadlockDemo.java:8)
- waiting to lock <0x00000003ce7631c0> (a java.lang.Object)
- locked <0x00000003ce7631b0> (a java.lang.Object)
at java.lang.Thread.run(java.base@25.0.1/Thread.java:1474)
Let us decode the header, field by field:
"worker-1": the thread name. Naming your threads changes everything for diagnosis. A pool calledhttp-nio-8080-exec-3stands out at a glance,Thread-47does not.#25: the thread’s internal id inside the JVM.[25091]andnid=25091: the thread id at the operating system level. This number is what lets you tie a Java thread to a line oftop -H. Depending on the JDK and OS it can be printed in hexadecimal, so convert when needed.prioandos_prio: Java and system priorities. Rarely useful.cpu: CPU time this thread has used since it started.elapsed: how long it has existed.tid: the address of the thread’s internal structure in the JVM.waiting for monitor entry: the summary state, repeated in full just below.
The next line, java.lang.Thread.State, is the most important of all.
Understanding the states
A thread is always in one of these states, and each points the diagnosis in a different direction.
RUNNABLE: the thread is running Java or native code. Careful though, a thread blocked on a network read often shows up as RUNNABLE, because the JVM does not know the native call is waiting. A RUNNABLE stack parked onsocketRead0means “waiting on the network”, not “busy computing”.BLOCKED (on object monitor): the thread wants to enter asynchronizedblock but the monitor is held by another thread. Several threads BLOCKED on the same lock is a contention point.WAITING (on object monitor): the thread calledObject.wait(), or is waiting on ajava.util.concurrentlock. It sleeps until something wakes it up.TIMED_WAITING: the same, but with a timeout (Thread.sleep,wait(timeout),park(timeout)).
The reflex on a frozen-application dump: look first for the BLOCKED threads, then check who holds the lock they are waiting on.
The -l flag: see the locks
Back to the worker-1 stack. Two lines starting with a dash slipped into it:
- waiting to lock <0x00000003ce7631c0> (a java.lang.Object)
- locked <0x00000003ce7631b0> (a java.lang.Object)
locked <...> means “this thread holds this monitor”. waiting to lock <...> means “it wants that one but does not have it yet”. The identifiers in angle brackets (0x00000003ce7631c0) are the addresses of the objects used as locks. They are how you connect two threads: if thread A waits on the object that thread B has locked, you have the thread to pull.
-l also adds, under each thread, the list of “ownable synchronizers”, that is the java.util.concurrent locks (like ReentrantLock) held:
Locked ownable synchronizers:
- None
Without -l, those locks do not appear, and contention on a ReentrantLock goes unnoticed. Hence the advice: get into the habit of always passing -l.
Detecting a deadlock
This is one of the cases where jstack saves the most time. The JVM detects monitor deadlocks itself and writes them out in plain text, at the end of the dump:
Found one Java-level deadlock:
=============================
"worker-1":
waiting to lock monitor 0x00000008107657a0 (object 0x00000003ce7631c0, a java.lang.Object),
which is held by "worker-2"
"worker-2":
waiting to lock monitor 0x00000008107656c0 (object 0x00000003ce7631b0, a java.lang.Object),
which is held by "worker-1"
Found 1 deadlock.
It is all there: worker-1 waits on an object held by worker-2, and worker-2 waits on an object held by worker-1. Neither will let go. The JVM names both threads, both objects, then prints the full stacks so you can trace back to the offending line of code. Here, the two threads take two locks in opposite order, the classic bug.
One limit to know: this automatic detection only covers deadlocks on synchronized monitors and on java.util.concurrent locks. A logical block, for example two threads waiting on each other through a queue or a condition, will not be reported as a “deadlock”. You will have to read it by hand from the WAITING states.
The -e flag: extended information
On a recent JDK, -e adds fields to each thread’s header:
"worker-1" #25 [25091] prio=5 os_prio=31 cpu=0.18ms elapsed=20.60s allocated=1336B defined_classes=2 tid=0x0000000810743800 nid=25091 ...
allocated gives the total this thread has allocated on the heap. Cross-checked against several dumps spread over time, it helps spot the thread producing the most garbage and putting the GC under pressure.
One dump is not enough
A thread dump is a photo, not a film. A thread seen RUNNABLE once may well be finishing its work normally. To tell what is genuinely stuck from what is moving, take several dumps a few seconds apart:
for i in 1 2 3 4 5; do
jstack -l 9850 > dump-$i.txt
sleep 2
done
Then compare them. A thread that stays on the same stack across all five dumps is blocked or very slow. A thread whose stack changes every time is working. It is that comparison, more than the single dump, that points at the real problem.
For a CPU spike, combine it with the system tooling. On Linux, top -H -p <pid> shows per-thread usage and gives the system thread id. Convert it (often to hexadecimal), then look for the matching nid in the dump: that thread’s stack shows the loop eating the CPU.
Patterns that keep coming back
After a few diagnoses, you start recognising shapes:
- All threads of an HTTP pool in BLOCKED or WAITING on the same resource: a downstream resource is saturated, for example an exhausted database connection pool. Requests pile up waiting on
getConnection. - A single RUNNABLE thread burning CPU, the others idle: a CPU hot spot, a loop spinning. That is where a profiler takes over.
- Many threads in TIMED_WAITING on
park: a pool at rest waiting for work. That is normal, not a symptom. - A queue of threads growing dump after dump on the same entry point: a bottleneck that is not draining.
jstack or jcmd?
jstack does one thing and does it well. The tool the OpenJDK team now puts forward as the general entry point is jcmd, which groups many diagnostics together. For a thread dump:
jcmd 9850 Thread.print
jcmd 9850 Thread.print -l # with the ownable synchronizers
The output is the same as jstack’s. jcmd has the advantage of giving access to other commands (heap info, GC, loaded classes) from the same tool. Both work, pick the one you prefer.
A few traps
- Run jstack with the same JDK and the same user as the target JVM. A version or permission mismatch, and the attach fails.
- The old
-Fflag (force, through the Serviceability Agent) is gone from recent JDKs. On a process that is truly frozen and no longer answers the attach, you go through a core dump andjhsdb. - In a container, jstack has to run in the same PID namespace as the JVM, often from inside the container itself. The PID seen from the host is not the one seen inside the container.
- Taking a thread dump brings the JVM to a safepoint. The impact is small, but on a JVM already on the edge, dumps fired in a very tight burst add a bit of pressure. A few seconds apart is plenty.
Going further
jstack answers the question “who is blocked and on what”. When it becomes “who is burning the CPU and in which loop”, a sampling profiler fits better: see Profiling the JVM with async-profiler and How to read a flamegraph.