The headline feature Apple unveiled at its Worldwide Developers Conference today is undoubtedly Siri AI, but the company has also revealed some other Apple Intelligence features that are coming to iOS 27, iPadOS 27, macOS 27, watchOS 27, and visionOS 27.
The Photos app now “taps into more powerful image models” to let you make edits, “while respecting the original moment as it was captured”, Apple says. With Spatial Reframing, you can touch and drag a photo and preview a perspective shift in real time, thanks to “Apple’s deep understanding of spatial models” on the Vision Pro headset.
Basically, you can ‘edit’ a picture as if you’d repositioned the camera in the original scene. Spatial Reframing will only generate new content where the perspective has been shifted, thus ensuring that the photo stays consistent with the original scene. You can also expand images with the Extend tool – straightening a crooked horizon without cropping out anything important, or adjusting the aspect ratio.
The Clean Up tool has received a major upgrade, and Apple now promises it removes distractions from your photos “with better quality and more realistic infill”, even for complex scenes.
Safari is getting a bunch of AI enhancements too. It can automatically organize your tabs into relevant topics. As you browse, it will put new tabs into existing topics or create new ones as it sees fit. You can even ask Safari to monitor a web page for changes with the new Notify Me feature, and when a change is detected, it will tell you. This helps with tracking restocks and price drops, for example.
Describe an Extension is a new AI feature that lets you do just that – describe a Safari extension you’d like to exist, and Safari will generate the custom extension right in the toolbar for you.
Passwords can now automatically fix your weak and compromised passwords with one tap, “using Apple Intelligence and Safari to agentically take action” on your behalf, navigating through websites to sign in and change your passwords to stronger ones that it then remembers for you.
Image Playground now lets you create “high-quality images in virtually any style”, including photorealistic, thanks to a new generative model that runs on Apple’s Private Cloud Compute, which ensures that your private data stays private and no one, not even Apple, can access it. Image Playground also lets you easily edit images with prompts, or by tapping, circling, or brushing to highlight an object in order to move or resize it.
Image Playground can also generate Lock Screen wallpapers and Contact Posters, and when you generate an image you can pick its aspect ratio so it’s fit for purpose.
The Messages app offers one-tap suggestions based on the context of your conversations – think creating a reminder or a note. Or if someone asks you for photos, it can help you find the right ones, by recognizing keywords, locations, and people in your library. Suggestions in Mail are now even more capable since they’ve gained the ability to take action with third-party apps, and Smart Reply in Mail and Messages now draws on your personalized writing style.
When you call a business, Call Context provides relevant information that you may have from that business – like a confirmation code or reservation number. This is surfaced directly in the Phone app. Call Context works based on who you’re calling, not what you’re saying, and runs entirely on device, so nothing is shared with Apple or anyone else.
Describe a Shortcut lets you use the powerful Shortcuts automation app by just telling it what you want. It will assemble all of the necessary steps on your behalf. If you want to tweak or add something, you can describe the change, and it will be applied.
The Home app gains generated video descriptions, which let you quickly understand what happened across a sequence of video clips from your smart home devices, even without watching them. You are also able to search through camera clips, to find specific things like a package delivery for example. At the top of the Search page, the app will feature “noteworthy clips” that you may want to review.
Voice Control, which lets you navigate your iPhone or iPad entirely by voice, is more intuitive, letting you simply describe on-screen buttons and controls instead of “memorizing exact labels or numbers”. Accessibility Reader can provide on-demand summaries and translation.
You can also get intelligent suggestions for names of files and folders, based on their contents, and Genmoji quality is “even better”. You can describe the changes you want to make to Genmoji.
Apple Intelligence is available on the iPhone 16 and later, iPhone 15 Pro, iPhone 15 Pro Max, iPad mini (A17 Pro), iPad models with M1 or later, MacBook Neo, Macs with M1 or later, the Vision Pro, Apple Watch Series 10 or later, Apple Watch Ultra 2 or later, and Apple Watch SE 3 when paired with an Apple Intelligence-enabled iPhone.
Some features, including image generation, have daily usage limits which can be increased with iCloud+ subscription plans, and these also include Apple Intelligence features for cameras compatible with the Home app.
