FBDeviceControl

FBDeviceControl is the macOS Framework that implements all functionality associated with iOS Devices within idb. It can be used independently of idb as it is a standalone Framework.

This page contains information about the implementation details of how iOS Device access works from macOS.

MobileDevice.framework#

MobileDevice.framework is a System Framework for macOS. It's default location is at /System/Library/PrivateFrameworks/MobileDevice.framework/MobileDevice. This is a Private Framework, there is no Apple-provided documentation for it. Most of it's API is of the CoreFoundation style, which means that most of it's surface is made up of plain C symbols with CF objects passed in or out. As such, this makes integration into an Objective-C Framework (like FBDeviceControl) relitavely simple, since CF objects are often Toll-Free Bridged to Objective-C.

At a first glance, this Framework is difficult to deal with since Private APIs are typically easier to work with if they export Objective-C classes. However, most of the usual patterns around how CoreFoundation APIs do apply. MobileDevice.framework is used extensively across Apple's macOS Applications that interact with iOS Devices (Finder/iTunes, Xcode, Accessibility Inspector, Apple Configurator 2, Photos). If an Application uses an iOS Device on macOS, it is extremely likely to leverage MobileDevice.framework.

This wealth of "clients" of MobileDevice.framework has made it possible to understand how each of the API calls work in terms of their inputs and outputs. As a result, there's a header that defines all of the calls that FBDeviceControl uses.

FBDeviceControl aims to use these APIs as far as possible, MobileDevice.framework is assumed to be the "canonical" way to interact with iOS Devices on macOS. The libimobiledevice project is essentially a re-implementation of MobileDevice.framework that runs on different host operating systems. There's also a range of other projects that use MobileDevice.framework including SDMMobileDevice, MobileDevice and pymobiledevice and more.

AMDevice#

AMDevice is a CF type defined in MobileDevice.framework. It is essentially the object that represents a single iOS Device attached to the host. All functions on an AMDevice start with the prefix AMDevice and typically take an AMDevice as the first argument. An AMDevice will only be present if the device is both booted and attached to the host. It cannot be in DFU or Restore mode.

To operate on an AMDevice, the phone must "trust" the host. You might recognize this as the "Trust Dialog" that appears when connecting an iOS Device to a Mac. Most calls will fail if this trust exchange has not been performed. The Mac maintains a local store of the cryptographic components that are used as part of this trust exchange. This local store means that an iOS Device will not constantly require trust to be authorized every time that the iOS Device is connected to the same host. If this process did not take place, then it would be completely infeasible for iOS Devices to be used in a Continuous Integration environment.

Since this is such an important component of how to interact with iOS Devices, it is backed by an Objective-C FBAMDevice class.

The process of discovering devices is asynchronous, which means that fetching the list of AMDevices at a snapshot in time is going to be unreliable. FBDeviceControl instead uses an API within MobileDevice.framework for recieving an AMDevice instance every time there is a state change in the availability of AMDevice instances. This property is also true of Apple's tools that build on top of MobileDevice.framework; xcodebuild has a -destination-timeout parameter and Apple Configurator's cfgutil has a --timeout parameter since device discovery is delivered asynchronously. You might never notice this in Xcode's "Devices and Simulators" window, but it is still there too.

AMRestorableDevice#

AMRestorableDevice is also part of MobileDevice.framework, it represents any iOS Device that is attached to the host and powered on. This includes iOS Devices that are booting, booted or in DFU/Restore mode.

This API is extremely limited and only exists to describe devices and perform some actions that are only relevant for devices that are not yet in a booted state. This is used to move a booted device to a DFU/Restore state and back out again. It is useful for understanding why a device that appears to be connected may not be represented by an AMDevice instance.

This is why FBDeviceControl can represent a single FBDevice instance to be backed either or both of an AMDevice/AMRestorableDevice, for the sake of full observability into connected iOS Devices. FBDevice instances will also fail appropriately, for instance when attempting to install an Application to an iOS Device that is in DFU/Restore mode.

AMDServiceConnection#

This is another CoreFoundation type that represents a connection to a "lockdown service". In order to get basically anything done on an iOS Device, a connection to service running on the iOS Device is required. There are a huge range of these services for a variety of cases. For example com.apple.syslog_relay is a lockdown service that is used for relaying system logs from the iOS Device to the attached host. You might have seen this used in practice within the "Simulators and Devices" window within Xcode. Connections are created via the AMDeviceSecureStartService call to an AMDevice, returning an AMDServiceConnection type.

There are a number of function calls relevant to this type, dealing with sending and recieving binary data over the connection as if it was a file descriptor. AMDServiceConnection can optionally contain a "secure context", which is cryptographic information required for sending data over a TLS'd connection. This is why it is important to use AMDServiceConnection(Send|Recieve) instead of raw read/write syscalls, sending unencrypted information over an encrypted transport will mean that the recieving side is incapable of reading the same data. The presence of a "secure context" can be detected at runtime by inspecting the connection value. In more recent iOS versions, some services start requiring usage of encrypted IO so detection and usage of these calls is very important.

It is important to stress that this types is just a "Transport" rather than a "Protocol". Each "lockdown service" may have it's own very different binary protocol for sending and recieving data. In the simple case of com.apple.syslog_relay, the service just repeatedly sends text over the connection. Other protocols, for instance those used by Instruments are far more complicated. There is no single Protocol that is used by all lockdown services.

There is one exception to this, the "Plist Protocol". This is implemented in AMDServiceConnection(Send|Recieve)Message calls. This is common across a range of services, such as the screenshot service and SpringBoard service. It's essentially wiring a length header followed by a binary plist, this is used on both the send and recieve sides.

AFC: "Apple File Connection"#

AFC is also common throughout the MobileDevice.framework API. This is a set of APIs for file manipulation on an iOS Devices. It is another example of a protocol that wraps an AMDServiceConnection transport. It has a number of functions for dealing with reading, writing, listing directories, moving, copying and deleting files. These operations are performed on the AFCConnection type.

On non-jailbroken devices, there is no way of getting an AFC connection for the entire root filesystem of the Device. Instead, there are different lockdown services corresponding to various containers or sandboxes within the iPhone's operating system. Access to Photos/Media (com.apple.afc), Application Sandboxes (AMDeviceCreateHouseArrestService) and crash logs (com.apple.crashreportcopymobile) are all examples of AFC services.

Developer Disk Images#

Whilst iOS by default has a number of different services, not all of them are present on the device. In order to augment the iOS Device with additional functionality to an attached macOS host, Xcode bundles a "Developer Disk Image".

This is a regular .dmg file, which can be opened on macOS. Within this disk image is a number of executables, libraries and plists describing the lockdown services that get added to the iOS Device upon mounting them on the device. FBDeviceControl provides an API for manipulating the disk image manipulation functions in MobileDevice.framework. It is not possible to mount arbitrary disk image on an iOS Device. Every disk image and it's binaries are signed by Apple. There are also different disk images for different versions of iOS, presumably because the lockdown services interact with APIs that can change over iOS versions.

The services that are contained within the Developer Disk Image are associated with functionality that is specific to Xcode, Instruments and Accessibility Inspector. This also means that the usage of these services can vary over versions of Xcode (therefore the Disk Images) instead of over iOS versions. As a result, there may be differences in the client-side implementation of the client depending on the protocol version of the service to which the client is communicating.

Instruments Service#

The "Instruments Service", which is a service within the "Developer Disk Image" is a very important one with respect to iOS Device automation. Since Instruments.app and the instruments commandline offers a lot of functionality for launching and profiling Applications and iOS Devices, it is integral to tasks such as app launching and process listing on iOS Devices.

The client-side implementation of this protocol is provided via DTXConnectionServices, with a provisional re-implementation within FBDeviceControl. There are more details about the makeup of this protocol within the ios_instruments_client project.

Video Encoding#

One of the key features of FBDeviceControl is the ability to stream the iPhone's screen to the host over USB. You might be familiar with this within QuickTime's "Screen Recording" feature, where you can record video from a connected iOS Device to an .mp4 file on your Mac.

Access to this is provided via AVFoundation as a "Capture Device". Usage of "Capture Devices" on macOS also requires that the hosting process (The process using FBDeviceControl) has system-level permissions for this on more recent versions of macOS. With a Capture Device for the iOS Device screen, it is then possible to create a "Capture Session" with the device. A "Capture Session" can thent be established, with relevant configuration so that frames are recieved in the most optimal format for the consumer. FBDeviceControl recieves frame samples from the device and these samples are either re-encoded or not before being passed on to a stream of data.

FBDeviceControl supports writing to an mp4 video file or as a stream of encoded data. For streams of data, these can be passed to other Applications for cases like webRTC or HTTP Live-Streaming.