Today at Apple’s 2021 WWDC event, the company unveiled the new iOS 15, iPadOS 15 operating systems. This year, Apple presented a large number of new features and improvements across both the main OS components as well as Apple’s core ecosystem apps. While we are just scratching the surface, we picked out a few highlight features that are looking forward to test later in the year once the new versions will be hitting consumers in their final versions.

Offline Siri

A large new upgrade to Siri isn’t exactly a functional one, but rather the way the voice assistant works under the hood. With iOS 15, Siri now becomes fully offline, with all voice queries being captured, and processed and interpreted by the Neural Engine inside of your iDevice SoC. Beyond greatly improving the security aspect of things, given that data never leaves your device for cloud processing queries, a large functional benefit of the new on-device processing is that it vastly reduces round-trip time and accelerates Siri’s response time. This means that Siri will now act or respond more immediately, rather taking a pause to think about every query, greatly improving the experience.

Live Text (OCR)

Another big new feature to iOS 15 is Live Text, which is essentially an OCR mechanism built right into the OS that will be able to seamlessly integrate into apps and extract text from pictures, either on the web – or in photos that you’ve captured on your device.

“Visual Look Up” is an object identification mechanism that works in your photos app to categorize and classify objects and persons – notably allowing you to also search your galleries now.

New Maps (for those lucky)

One of the more impressive showcases at this year’s WWDC was the new more detailed Maps app. The updated app and dataset now features far more datail for major cities, featuring even advanced 3D terrain representation of the scene. This also extends over to navigation as well, which now features advanced 3D modelling of roadways and bridges, as welll as information on a road's various lanes (turn, etc), simplifying complex intersections for example.

The negative part of this new announcement is that it’s limited to only a hand-full of US cities and London – with most of the rest of the world having to wait out for future updates – if they ever happen.

New FaceTime & Screen Sharing

Apple spent a lot of time around the new FaceTime and screen sharing features. Essentially Apple here is attempting to create their own virtual social ecosystem and build a conferencing integration around FaceTime. FaceTime itself received a large number of new features including spatial audio reproduction-based Group FaceTime calls in which callers’ audio is mimicked to be spatially spread out.

FaceTime calls now essentially are able to get Zoom-ified in terms of creation of links into calls – which now can be integrated and easily shared across applications. FaceTime now even works for any other non-Apple device by allowing users which have access to a link to join in via a web browser interface. And, given how many devices this expands Facetime to, it's a bit of a shame that Apple didn't go into more detail on how that will work.

SharePlay is a screen sharing and application experience synchronisation feature. It allows callers to share media with each other, such as playing back the same audio or movie content, and goes quite deep into app integration as Apple introduces as swathe of new APIs that allow sharing and essentially casting of content across devices and applications.

New Notificiation Summary

iOS’s notification system has lagged behind Android’s for quite some years now, and Apple today took a further step in trying to get a grip over the more traditional “linear” notification system that iOS had to deal with until now. “Notification Summary” is essentially a second layer filtering and grouping mechanism of your notifications, allowing you to prioritise notifications which are important to you and set aside more general clutter notifications for later. It’s said that the priority algorithm will learn based on your apps and contacts, and try to keep those notifications at the top for you to see.

New Safari UI

Safari has seen a large new UI shift on the larger form-factor macOS and iPadOS variant, but also the iOS variant has seen important UI changes such as a more full-screen browsing experience with a dynamic address and navigation bar that appears and disappears at the bottom of the screen, which is a nice ergonomics improvement given the size of devices nowadays. Under the hood, the introduction of full fledged browser extensions is sure to vastly augment the iOS Safari experience.

A larger Wallet

A new expanded Wallet can now go well beyond your Apple Pay uses or even transport card uses today. Apple will be partnering up with states and providers to enable integration of things such as hotel room keys, or even your actual government ID. The system will continue to use the secure enclave which enables Apple Pay.

Beta starting July

Apple will be making available the new versions of iOS for developers starting today, while the public beta will start in July with the usual planned final roll-out in the September period.

Related Reading:

POST A COMMENT

16 Comments

View All Comments

  • bernstein - Monday, June 7, 2021 - link

    offline siri: great, IF you can force it to not go online automatically. useless otherwise
    live text: just great.
    full web extensions: hell it's about time.
    new maps: google maps is just so much better. has 3d textured with satellit imagery...
    facetime : obviously you pick your friends & business contacts based on their phone manufacturers.
    what nothing more? nope not really... oh wait they didn't know what else to do, so they made a bad authy clone :-D
    Reply
  • bernstein - Monday, June 7, 2021 - link

    oh and they brought logitech flow to ipad/ios and made it work with all mice & keyboards. Reply
  • name99 - Monday, June 7, 2021 - link

    You say all that like it's a bad thing!

    Apple users have been complaining for years that the OS has too many papercuts, things that work -- but badly, or that don't work the way you expect they should, or are too slow/take too much memory/are buggy.

    We've had some releases that promised mainly to fix bugs, and that partially (but never fully) delivered. If this is another of those mostly bug-fixing+polishing releases, most of us in the Apple camp will be VERY HAPPY with that choice.
    Fix bugs, concentrate on all the low-level-stuff still required by the ARM64 transition and the closer merging of iOS+macOS, and leave the features for next year!
    Reply
  • ABR - Tuesday, June 8, 2021 - link

    Exactly. Reply
  • Samus - Friday, June 11, 2021 - link

    True, the only things I really care about here are the improved Maps and Safari interface (because currently both are inferior to alternatives) while my primary focus is on bug fixes and further optimizations. For one, they really need to bring back an option to revert the appointment interface to the iOS13-style because setting a time now is totally ridiculous. Reply
  • web2dot0 - Tuesday, June 8, 2021 - link

    So what you are saying is if Apple implements features you like already ... "it's lame".
    Improvements on existing features ... "so what"
    facetime : it works across phone manufacturers now .... did you read the OP?

    Hmmm ... you must really be fun at parties ... you know, the ones that complains about everything
    Reply
  • name99 - Monday, June 7, 2021 - link

    "Siri now becomes fully offline"
    That's not correct, is it?

    All that becomes "fully offline" is the AUDIO processing. The request in "digital form" may still need to be handled by a remote server. In principal the back-end logic (eg map "what's 3 pounds in kilograms" into a request be sent to Wolfram Alpha, map "when does Iron Man 4 open" to request to be sent to?Fandango?, etc) could happen on-device, but I did not get the impression that's what they're doing now.

    And of course doing the mapping locally restricts how rapidly they can iterate upgrading that back-end.
    Reply
  • skaurus - Tuesday, June 8, 2021 - link

    They said that only some commands work fully offline, like launching an application or starting a timer for example. Reply
  • eastcoast_pete - Monday, June 7, 2021 - link

    Would love to know if "Offline Siri" has the same problems online Siri has with accents or even minor speech impediments. Those issues aren't limited to Siri, Amazon's speech recognition sucks just as badly here. Reply
  • 29a - Tuesday, June 8, 2021 - link

    I live in an area with accents Siri doesnt like and it seems the more you use it the better it gets. Reply

Log in

Don't have an account? Sign up now