How to Use Apple Visual Intelligence on iPhone 16
Posted on
by
Kirk McElhearn
Visual Intelligence is one of the Apple Intelligence features that lets you send identify images. For example, you could be in front of a landmark or piece of art, invoke Visual Intelligence, and get some information about that item. This mostly uses ChatGPT to give you information, but you can also query Google to see if it can match the image.
Visual Intelligence is available only on iPhone 16 models and requires the Camera Control, a new button on these iPhones. If you press and hold that control while pointing your device at something, the interface displays two buttons: Ask and Search. If you tap Ask, your iPhone queries ChatGPT about what it sees. If you tap Search, it searches Google for similar images.
The results you get from Visual Intelligence vary. Without being able to test it extensively, I have found that it does a pretty good job of recognizing landmarks in photos, as well as miscellaneous items. I tried using it with one of my cats, and it was surprisingly good at figuring out the cat’s breed.
When you invoke Apple Intelligence, the interface stays active, as it would when you’ve opened the camera app. (You can also open the Camera app, then press the camera control to invoke this feature.) You can move your iPhone around and tap the Ask and Search buttons as you move around a scene.
You can use Visual Intelligence to get information about books, objects, pieces of artwork, or landmarks. It can identify plants and animals, it can recognize the faces of well-known people, movie posters, record albums, and much more.
Visual Intelligence is designed to work in portrait mode, but if you turn your iPhone you can trigger it in landscape mode; however, the answer it returns is designed to be read in portrait mode.
Visual Intelligence can give you information about a business, though this feature is currently only available in the US. Depending on the business, you may see opening hours, a telephone number, and a website, and you may be able to make a reservation or place an order.
Visual Intelligence and text
You can also use Visual Intelligence with text on a sign, billboard, or document. Here’s how it works with the back cover of a book.
Press and hold the camera control, and then press the round button at the bottom of the window. Your iPhone recognizes that it is looking at text, and displays two new buttons at the bottom: Summarize and Read Aloud.
Tap Summarize to get a brief summary of the text; the screen only allows a few lines of text, as in the examples above, so the summary won’t be very long. If you want to have the text read to you, tap Read Aloud.
I would have expected Translate to also be an option here. You can use the Translate app on your iPhone to take a photo of text and have it translated, and it would make sense for that feature to be available within the Visual Intelligence interface. In fact, this translation feature seems a lot more intelligent than many of the Apple Intelligence features available.
In some cases, you will be able to interact with the text, such as calling a phone number, composing an email, or going to a website. If there are multiple options, a … button displays; tap this to see which options are available. This is a very easy way to go to a website address you see on a document; it can sometimes be easier than typing a URL.
Apple’s Visual Intelligence features are quite powerful, but only available on iPhone 16 models because it requires the camera control. It’s not clear why Apple hasn’t enabled these features on iPhone 15 Pro models, which support other Apple Intelligence features. It also seems less versatile than Google Lens, a feature available in the Google app for iPhone and Android, and that works on iPhones and iPads running iOS 15 / iPadOS 15 or later. But if you have an iPhone 16, this is a useful feature that you should learn how to use: it can save you time, and give you useful information.
How can I learn more?
Each week on the Intego Mac Podcast, Intego’s Mac security experts discuss the latest Apple news, security and privacy stories, and offer practical advice on getting the most out of your Apple devices. Be sure to follow the podcast to make sure you don’t miss any episodes.
You can also subscribe to our e-mail newsletter and keep an eye here on The Mac Security Blog for the latest Apple security and privacy news. And don’t forget to follow Intego on your favorite social media channels: 
