Apple adds insights for developers ahead of Siri’s AI upgrade

Apple’s push into AI continues as the iPhone maker prepares developers for new tools like Siri screen awareness features.

These features will be part of Apple Intelligence, and the hope is that they will significantly boost the digital assistant’s contextual understanding capabilities.

The company’s new developer tools are starting to be added to Apple Developer, including new ‘App Intent APIs’ that will allow apps’ onscreen content to be accessible to Siri and Apple Intelligence.

Apple showcased onscreen intelligence in June 2024, which is a good hint that it’s still in development.

In the explanation of the feature, Apple writes: “When a user asks a question about onscreen content or wants to perform an action on it, Siri and Apple Intelligence can retrieve the content to respond to the question and perform the action.

“If the user explicitly requests it, Siri and Apple Intelligence can send content to supported third-party services. For example, someone could view a website and use Siri to provide a summary by saying or typing a phrase like ‘Hey Siri, what’s this document about?’”

To do this, a November 2024 update has been added which states developers can make onscreen content available to Siri and Apple Intelligence by describing it as an App Entity and adopting an assistant schema.

The full details about this feature have not yet been shared, but it was in iOS 18.2 beta 1 where ChatGPT was integrated with Siri.

This comes just days after Apple released iOS 18.2 beta 2 for developers which will see several changes brought in for device users.

While the previous variant saw ChatGPT be integrated into Siri, the second edition will allow people to subscribe to the Plus version directly in Settings.

Apple Intelligence will also be able to create an image from notes, with this option to be added to the popup menu when text is highlighted.

The ‘Find My’ tool will also see an upgrade as the location of a lost item will soon be able to be shared to someone else, with a link able to be generated. The recipient doesn’t have to have an iPhone to be able to view it, as the link can be opened on the web.

Smaller changes will be seen in the Mail app with its notification badges, as well as the icons in Settings and camera control upgrades.

Featured Image: AI-generated via Ideogram

The post Apple adds insights for developers ahead of Siri’s AI upgrade appeared first on ReadWrite.