Accessibility

idb can read the entire accessibility hierarchy of an iOS Simulator, and act on the elements in it. The commands that do so are documented in UI Automation; this page covers what sits behind them — which backend serves a read, what shape the result comes back in, and what to do when a read comes back emptier than the screen looks.

These primitives support a number of scenarios:

• UI testing without XCTest, by reading the UI state and then driving the screen.
• Remote access to an iOS Simulator, and screencasting.
• Accessibility auditing, by reading the whole hierarchy and applying heuristics to it.

Backends

A read is served by one of three backends, selected with --api on idb ui describe-all, idb ui describe-point and idb ui describe. They differ in where the read runs, which is what determines how much of the screen they can see and how much each read costs.

--api ax — the accessibility backend

The default, and what idb has always used. The companion asks the Simulator's accessibility server, from the host, for the frontmost application's elements.

This is the same view an assistive technology gets. It is cheap and needs nothing installed in the Simulator, but it is a summary rather than a transcript: a view that draws its own content and describes itself with a single label — a custom collection view cell, a composed row of text and images — is reported as that one element, not as the parts it was built from.

--api axbridge — the guest reader

The read runs inside the Simulator. The companion spawns the bundled SimulatorFrameworkBridge helper, which attaches to the target application's own in-process accessibility server, walks its element tree, and returns it as JSON.

Because it reads the application from the inside, it sees the structure the application actually built, at the granularity XCUITest would see — typed, labelled elements where the host-side view reports one composite. It needs no test bundle, no test runner and no automation daemon: the helper is spawned, answers one read, and exits.

The cost is that spawn. The helper has to start and load the accessibility frameworks on every read, which dominates the time a single read takes — roughly 600ms, against roughly 20ms for the walk itself.

--api axbridge-persistent — the guest reader, kept warm

The same reader, spawned once instead of once per read. The companion starts SimulatorFrameworkBridge in serve mode and talks to it over a Unix domain socket, so the framework loading is paid once and every subsequent read is just the walk — around 20ms, roughly 30x faster than re-spawning.

That amortization only pays off across reads by a long-lived process holding one connection: a driver, a REPL, an interactive session. A single idb ui describe-all from a shell spawns the reader, reads once and tears it down, which is strictly slower than --api axbridge doing the same thing without the socket. Use axbridge for one-shot commands and axbridge-persistent for a process that will read repeatedly.

The connection is self-healing: if the reader dies — the Simulator shut down, the process was killed — the next read re-establishes it and retries once, rather than wedging on a dead socket.

Choosing between them

	ax	axbridge	axbridge-persistent
Where the read runs	Host	Inside the Simulator	Inside the Simulator
Detail	Assistive-technology view	Full application tree	Full application tree
Cost per read	Low	~600ms, spawn-dominated	~20ms warm, after a one-off start
Needs SimulatorFrameworkBridge	No	Yes	Yes
Best for	Quick checks; anything the composite view is enough for	A one-shot read that needs full detail	Repeated reads from one long-lived process

Compatibility

--api can never break an existing workflow. Omitting it preserves the historical behaviour exactly, and the field is only set on the wire when you actually ask for a backend. A companion that predates backend selection ignores the field entirely and serves the read as if it were unset, so a new client against an old companion behaves precisely as it did before.

When a backend is selected but cannot run, the read fails with a clear error rather than quietly falling back to a different one — a silent downgrade would report a smaller tree as if it were the whole screen.

Requirements

The axbridge backends are iOS Simulator only, and need the SimulatorFrameworkBridge helper alongside the companion binary. A companion built as a distribution (see Development) has it in the sibling Resources directory, which is the layout idb_companion expects at runtime; a bare binary moved out of that layout does not, and a read reports:

The SimulatorFrameworkBridge guest binary was not found in the companion Resources directory

Read bounds

Whole-tree reads are bounded, so a pathological hierarchy cannot hang a read or return an unbounded payload. The host sets the bounds — a maximum depth of 50 and a budget of 3000 nodes — and both backends truncate at the same point, so a tree read over one is comparable with the same tree read over the other. A read that hit those bounds reports truncated in the complete format below.

Output formats

--format selects the shape of the result, on the same three read commands.

--format default

The historical format, and what you get if you ask for nothing: a flat JSON array of every element, each a dictionary of attributes under their original AX-prefixed names.

$ idb ui describe-point 201 286
{"AXFrame":"{{20, 264}, {362, 44}}","AXUniqueId":"com.apple.settings.general","frame":{"y":264,"x":20,"width":362,"height":44},"role_description":"button","AXLabel":"General","content_required":false,"type":"Button","title":null,"help":null,"custom_actions":[],"AXValue":"","enabled":true,"role":"AXButton","subrole":null}

A whole-screen read emits an array; a point or marker read emits a bare object, or null for a point with nothing under it.

The attributes are AXLabel, AXFrame, AXValue, AXUniqueId, type, title, frame, help, enabled, custom_actions, role, role_description, subrole, content_required, pid, traits, expanded, placeholder, hidden, focused and is_remote. An attribute you did not ask for with --key is absent; one that was asked for but has no value is present and null.

--format nested

The same elements and the same attribute names, but each element carries its descendants under a children key, so the output is one tree rather than a flat list. --nested is a deprecated alias for this; pass one or the other, not both.

--format complete

A consolidated document: the elements, plus everything the read learned about the state they were read in.

$ idb ui describe General --format complete | jq '{backend, truncated, target, screen}'
{
  "backend": "axbridge",
  "truncated": false,
  "target": {
    "kind": "marker",
    "pid": null,
    "x": null,
    "y": null,
    "value": "General",
    "match_key": "AXLabel"
  },
  "screen": { "width": 402, "height": 874, "coordinate_space": "screen" }
}

Field	Description
elements	The elements, always an array — even for a single-element read, which the other formats emit as a bare object
backend	Which backend served the read: ax, axbridge, axbridge-persistent or testmanagerd
target	What was asked for: kind is frontmost, application, point or marker, with the pid, x/y or value/match_key that named it
screen	The bounds element frames are relative to
truncated	Whether the read hit the depth or node bounds and stopped short of the whole tree
modal	The blocking alert on screen, if there is one: whether it belongs to the system or the app, its element type, and its title
coverage	How much of the screen the read's element frames cover, along several dimensions
profile	Timing and call counts for the read, where the backend collects them

Two things make this format easier to consume than the others. Its key set is fixed: every field is always present, null where it does not apply, so one parser handles every read command — a point read and a whole-screen read differ in their values, never in their shape, and target rather than the shape tells you which command produced the document. And its element names are the clean ones: label rather than AXLabel, identifier rather than AXUniqueId, role_description rather than a mix. The two attributes that merely restated another are gone — the stringified AXFrame, since frame carries the same rectangle structurally, and the raw role, since type is its normalised form.

The document is expected to grow. New fields are added additively, and a consumer should ignore fields it does not know rather than reject the document; there is deliberately no version to check.

A companion that predates format selection does not recognise complete and answers in the default format. idb detects that by the response shape and warns on stderr:

warning: the companion does not support --format complete (it predates format selection); the read was served in the legacy format by the default backend

When a read comes back empty

A read that reports far fewer elements than the screen shows is usually one of these:

• The application does not expose its content. A WKWebView or other remote content is a single element to the accessibility system unless the application marks up what is inside it. In the complete format this shows as a large gap between the leaf and content coverage ratios: area the application draws, but does not describe.

• The backend is the composite one. A custom-drawn view described by one label reports as one element over --api ax. Read the same screen with --api axbridge to see whether the detail is there and just not surfaced host-side.

• Accessibility is off for the application. Reads depend on the target's accessibility server having started. If a read reports nothing at all, check that ApplicationAccessibilityEnabled is set in the com.apple.Accessibility domain on the target:

  idb get --domain com.apple.Accessibility ApplicationAccessibilityEnabled
  idb set --domain com.apple.Accessibility ApplicationAccessibilityEnabled --type bool true

  Relaunch the application afterwards; the setting is read at startup.

• The read was truncated. A very deep or very large hierarchy stops at the bounds above. --format complete reports truncated: true when that happened, which distinguishes "this is the whole screen" from "this is as much of it as was read".