This new iOS 17 API can turn iPhones into subject-tracking cameras using motorized stands

This new iOS 17 API can turn iPhones into subject-tracking cameras using motorized stands 1

Apple is finding ways to make its iPhones more useful for its users. As such, the brand revealed recently that the new iOS 17 comes with a new API called DockKit. It should allow manufacturers and app developers to create apps and accessories (specifically stands) that will turn iPhones into moving cameras with options for customizations and more.

DockKit is one of the best highlights of Apple’s new iOS 17 update. It is dedicated to app developers and accessory manufacturers, allowing them to innovate iPhones into more useful devices. Apple also made the API more convenient by allowing it to work with apps already using the native iPhone camera API, removing the need for an update.

During the WWDC 2023, Apple showed how the API would work by attaching an iPhone to a stand. By integrating the motorized stand into the iPhone’s system, the camera app could control the movement of the stand holding the device and follow certain subjects, such as a person or animal.

“Your app can automatically track subjects in live video across a 360-degree field of view, take direct control of the stand to customize framing, directly control the motors, and provide your own inference model for tracking other objects,” Apple shared.

It can use Apple’s Vision framework, which allows face/face landmark detection, text detection, barcode recognition, image registration, and general feature tracking. This means developers who like to take advantage of the API can create apps that allow users to choose what subject to track using the camera.

By introducing the new API, Apple is opening a whole bunch of possibilities on how Apple could be used by its customers in the future. Initially, the API could be useful for making pet or child-tracking apps at home. Nonetheless, Apple wants to stress to developers that the API can be used for other purposes, including security, education, healthcare, business, content creation, and more. To prove this, Apple showed during the event how the API could be used to make the motorized stand move based on the hand movements of the user.

“DockKit interfaces with articulating stands known as dock accessories to track the location of human faces and bodies that appear in video frames,” Apple wrote in its support document for the API. “It determines how to best position the camera to frame and track those subjects. This feature, known as system tracking, is useful for content creators who want the camera to follow them while they move around their space, or instructors on a video call who are walking around a classroom.”

DockKit’s arrival, however, doesn’t mean you’ll be able to use your iPhone as a subject-tracking camera right away once the iOS 17 officially releases this fall. The update is still in beta, and it will take time for developers and accessory manufacturers to create apps and stands fit for this job. Yet, in the future, the API will make iPhones continuously relevant in the smartphone market by making them more than just simple handheld devices for your pockets.