Start by writing down the symptom precisely. “The service is slow” is not yet useful; “requests wait five seconds before the first response byte” gives you a boundary to investigate.
Observe before changing
Capture the process state, recent logs, resource pressure, and network connections. Avoid restarting immediately when the evidence is ephemeral. A small snapshot often saves hours later.
ps -o pid,ppid,state,etime,%cpu,%mem,cmd -p "$PID"
cat /proc/"$PID"/status
ss -tpn
strace -f -p "$PID"Follow the blocked resource
High CPU suggests profiles and hot loops. Low CPU with long latency often means waiting: disk, DNS, locks, sockets, or an upstream service. Tools such as strace, lsof, ss, and perf reveal different boundaries.
Prove the hypothesis
A correlation is a lead, not a conclusion. Change one variable, predict the result, then observe it. Keep the diagnostic command and result with the incident notes so the conclusion can be challenged later.
The durable debugging skill is not memorizing commands. It is moving methodically from symptom to boundary, from boundary to evidence, and from evidence to a testable explanation.