Starting in iOS 14 and macOS Big Sur, developers will be able to add the capability to detect human body and hand poses in photos and videos to their apps using Apple's updated Vision framework, as explained in this WWDC 2020 session.
This functionality will allow apps to analyze the poses, movements, and gestures of people, enabling a wide variety of potential features. Apple provides some examples, including a fitness app that could automatically track the exercise a user performs, a safety-training app that could help employees use correct ergonomics, and a media-editing app that could find photos or videos based on pose similarity.
Hand pose detection in particular promises to deliver a new form of interaction with apps. Apple's demonstration showed a person holding their thumb and index finger together and then being able to draw in an iPhone app without touching the display.
Additionally, apps could use the framework to overlay emoji or graphics on a user's hands that mirror the specific gesture, such as a peace sign.
Another example is a camera app that automatically triggers photo capture when it detects the user making a specific hand gesture in the air.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing. The algorithm can also experience difficulties if a person is close to edge of the screen or partially obstructed.
Similar functionality is already available through ARKit, but it is limited to augmented reality sessions and only works with the rear-facing camera on compatible iPhone and iPad models. With the updated Vision framework, developers have many more possibilities.
Apple's iPhone 17 Pro and iPhone 17 Pro Max models will feature a number of significant display, thermal, and battery improvements, according to new late-stage rumors.
According to the Weibo leaker known as "Instant Digital," the iPhone 17 Pro models will feature displays with higher brightness, making it more suitable for use in direct sunlight for prolonged periods. The iPhone 16 Pro and...
Tuesday September 2, 2025 1:50 pm PDT by Joe Rossignol
Just one week before Apple is expected to unveil the iPhone 17 series, an analyst has shared new price estimates for the devices.
Here are J.P. Morgan analyst Samik Chatterjee's price estimates for the iPhone 17 series in the United States, according to 9to5Mac:
Model
Starting Price
Model
Starting Price
Change
iPhone 16
$799
iPhone 17
...
Apple is expected to unveil the iPhone 17 series on Tuesday, September 9, and last-minute rumors about the devices continue to surface.
The latest info comes from a leaker known as Majin Bu, who has shared alleged images of Apple's Clear Case for the iPhone 17 Pro and Pro Max, or at least replicas.
Image Credit: @MajinBuOfficial
The images show three alleged changes compared to Apple's iP...
Monday September 1, 2025 4:35 am PDT by Tim Hardwick
Apple will launch its new iPhone 17 series this month, and the iPhone 17 Pro models are expected to get a new design for the rear casing and the camera area. But more significant changes to the lineup are not expected until next year, when the iPhone 18 models arrive.
If you're thinking of trading in your iPhone for this year's latest, consider the following features rumored to be coming to...
Thursday August 28, 2025 4:08 am PDT by Tim Hardwick
An iPhone 17 announcement is a dead cert for September 2025 – Apple has already sent out invites for an "Awe dropping" event on Tuesday, September 9 at the Apple Park campus in Cupertino, California. The timing follows Apple's trend of introducing new iPhone models annually in the fall.
At the event, Apple is expected to unveil its new-generation iPhone 17, an all-new ultra-thin iPhone 17...
Thursday September 4, 2025 7:38 am PDT by Tim Hardwick
We're only days away from Apple's "Awe dropping" fall event scheduled to take place on Tuesday, September 9 – and along with the new iPhone 17 series, we're going to get a new version of the Apple Watch Ultra for the first time since 2023.
By the time the Ultra 3 is unveiled, it will have been two years since the previous model arrived. The intervening period has left plenty of room for...
Honestly, this seems like the kinda stuff that'd make Apple AR compelling—being able to draw in midair means you’d also be able to navigate an interface in midair with just your hands.
Using AR/VR without bringing a controller everywhere seems analogous to what set the iPhone apart from other touchscreen phones in 2007; you didn’t need a stylus.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing.
I don't think the wizarding community is going to be too happy about this...
Biggest design overhaul since iOS 7 with Liquid Glass, plus new Apple Intelligence features and improvements to Messages, Phone, Safari, Shortcuts, and more. Developer beta available now ahead of public beta in July.
