Chasing Glitches: From Crash Report to Root Cause on macOS

For a few days my Mac had two problems I assumed were unrelated. Messages wouldn’t open: the icon bounced, no window showed up, and eventually it disappeared. Safari would sometimes refuse to load a site until I quit and relaunched it.

The usual advice is to restart the Mac. I had a lot of long-running work open and didn’t want to, so I sat down with Claude Code and traced it instead. The cause was one stuck system service that neither app’s name would ever point you to. The fix was one command, and no reboot.

The specific fix is at the end. Most of this post is the method, because it works for any app that freezes, beachballs or silently fails, not only these two.

The core idea

On macOS, apps do very little on their own. Messages leans on imagent, which leans on identityservicesd, which leans on security daemons, and so on down. When an app hangs while using almost no CPU, it’s nearly always waiting on another process that never answers.

So the job isn’t to fix the app. You walk down the chain of who is waiting on whom until you reach the process that isn’t waiting on anything it can get. That one is the culprit.

The pipeline

Step 1. Read what macOS already recorded

Crash and hang reports live in two folders:

ls -lt ~/Library/Logs/DiagnosticReports /Library/Logs/DiagnosticReports | head -30
Bash

Look for the app’s name and for .ips or .hang files. Three things in a report tell you what kind of problem you have:

  • Exception code. 0x8BADF00D (“ate bad food”) means the watchdog killed the app for being unresponsive. That’s a hang, not a crash. A real crash looks like EXC_BAD_ACCESS or EXC_CRASH (SIGABRT).
  • CPU time. If the report says the app used almost no CPU over many minutes, it was waiting, not working.
  • The triggered thread’s stack. The last few frames before the blocking call usually name the subsystem it was waiting on.

Also check whether the reports started on a particular day. A sudden start points at an event, not at gradual decay.

In my case there were five Messages reports, all identical:

EXC_CRASH (SIGKILL)  code 0x8BADF00D
scene-create watchdog transgression: exhausted real (wall clock) time allowance of 1199.97 seconds
Elapsed application CPU time (seconds): 0.576
Plaintext

That’s twenty minutes of waiting and half a second of work. The stack ended in -[CKCoreChatController conversationAllowedByScreenTime] → IMCore → _dispatch_sync_f_slow. In other words, a synchronous call into the Messages framework that never returned.

Step 2. Sample the live process

Crash reports are history. To see what’s stuck right now, reproduce the hang and use sample, which ships with macOS. It snapshots every thread and shows where each one is sitting:

/usr/bin/sample <pid> 3 -file /tmp/app.txt
Bash

(Use the full path. Some Homebrew packages install their own sample script that shadows it. Same with log: in zsh it’s a shell builtin, so use /usr/bin/log.)

In the output, look for threads parked in blocking calls:

FrameWhat it means
xpc_connection_send_message_with_reply_sync, __NSXPCCONNECTION_IS_WAITING_FOR_A_SYNCHRONOUS_REPLY__Waiting for another process to answer
_dispatch_sync_f_slow, __DISPATCH_WAIT_FOR_QUEUE__Waiting for a queue another thread holds
__ulock_wait, __psynch_mutexwaitWaiting for a lock
__semwait_signal inside a loopSleeping and retrying something that keeps failing

Then read the frames just above the blocking call. Names like IDSDaemonController, CNContactStore, SecKeyCreateSignature or LAContext tell you which service is on the other end.

Step 3. Work out who’s on the other end, then sample that

Mach messages don’t show the peer process directly, so you work it out from three clues:

  • Framework names in the stack. IDS points to identityservicesd, Contacts to contactsd, TKRemoteSEPKey to ctkd, and LAClient to coreauthd.
  • The service’s queue names. This is the best trick I learned. When you sample a server process, its threads often run on queues named after the client, like com.apple.NSXPCConnection.user.anonymous.<pid>. That number is the PID of the process being served. One sample of the server tells you everyone waiting on it.
  • ps -Ao pid,etime,comm. It shows how long each daemon has been running. A daemon that started right when your problems began deserves a close look.

Sample the next process and repeat until you hit a thread waiting on something that will never answer.

My chain:

  1. Messages was waiting on imagent.
  2. imagent was waiting on identityservicesd (iMessage and FaceTime identity).
  3. identityservicesd was stuck inside a Secure Enclave signing call to ctkd:
  4. ctkd (CryptoTokenKit, which handles keys stored in the Secure Enclave) looked like this:

That’s the bottom of the chain. One signing request asked LocalAuthentication for an auth context, and the reply never came. It was holding a lock while it waited, so every later request queued behind it. One of those queues carried the PID of an identityservicesd I had already killed. ps -o lstart= showed this copy of ctkd started about four days earlier, right when the problems began.

Step 4. Check the blast radius

Now look at everything else in that queue, not only the app you started with. Each queued client is something else on the Mac that’s quietly broken.

This is where Safari came in. networkserviceproxy, which handles iCloud Private Relay and some of the proxying Safari relies on, was stuck in the same queue. Two “unrelated” problems had one cause.

Step 5. Don’t trust the loudest log message

The system log will be noisy. My imagent was printing hundreds of these:

Sandbox: imagent deny(1) mach-lookup com.apple.contactsd.persistence
CoreData: Unable to create token NSXPCConnection
Plaintext

That looked exactly like a broken Contacts database, and I spent real time on it. It was a side effect. Here’s the test that saves you: restart the process you suspect, and sample the fresh copy. If the new one gets stuck at the same spot within seconds, the problem is further down the chain. Restarting imagent and then identityservicesd changed nothing, because each new copy joined the same queue behind ctkd.

Step 6. Restart the lowest safe piece, then verify

Before killing anything, check that it’s a per-user, on-demand service:

ps -o user=,comm= -p <pid>              # runs as you, not root?
launchctl print user/$(id -u)/<label>   # does launchd relaunch it on demand?
Bash

If it runs as you and launchd relaunches it on demand, killing it is safe and doesn’t need sudo. Restart the bottom of the chain first, then the layers above it, so they reconnect to a fresh copy.

Then verify with the same tool you used to diagnose it. Sample the new daemon and confirm the stuck threads are gone, then confirm the app actually works. “I restarted it” isn’t the same as “it’s fixed.”

The fix in my case

killall -9 ctkd; sleep 3; killall -9 identityservicesd imagent
Bash

This only kills your user’s copy of ctkd. The root-owned one is left alone, so no sudo is needed.

Afterward the new ctkd had no stuck threads, and Messages opened in a few seconds. networkserviceproxy recovered without its own restart, because the request it was waiting on finally failed and it moved on. I expect Safari to stop needing relaunches too. I’ll know after a few days of normal use, since that problem only showed up now and then.

What I couldn’t find

I never found out why LocalAuthentication didn’t reply. The logs from that minute are nearly empty, and both coreauthd processes were idle when I looked. The most likely explanation is that the request went to a copy of coreauthd that exited without answering, which left ctkd waiting forever. That’s a macOS bug, and this machine was still on Sequoia 15.1. The real long-term fix is updating, which needs the reboot I was dodging. At least now I can schedule it.

The pipeline on one screen

  1. Reports first. Check DiagnosticReports: is it a hang (0x8BADF00D, near-zero CPU) or a crash? When did the reports start?
  2. Sample the live process. Find threads in sync XPC calls, dispatch_sync, locks or retry loops.
  3. Identify the peer. Use framework names in the stack, ...anonymous.<pid> queue names in the server, and daemon start times.
  4. Recurse until the waiting stops at something that can’t answer.
  5. Check the blast radius. Everything else queued behind the culprit is broken too.
  6. Distrust loud logs. If restarting a suspect doesn’t help and the fresh copy hangs in the same spot, look deeper.
  7. Restart the lowest safe piece: per-user, on-demand, bottom first.
  8. Verify by sampling again, then by using the app.

Posted

in

,

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *