What caught our eye for developers and users
13 Sep 2016
With the release of iOS 10 a number of existing frameworks have been enhanced to provide better support for new devices and features. There are some changes to CoreImage, which now includes support for RAW images (on devices with A8 and A9 CPUs). Metal now supports hardware tessellation enabling the rendering of more detailed geometry. SceneKit includes Physically Based Rendering, which provides the means of creating realistic shading. Numerous changes have been made to AVFoundation and the camera capture pipeline, related to the addition of wide color gamut support and dual cameras to the newest iPhone.
In addition to those changes new frameworks have been introduced, which enable the newest features of iOS like Siri and Messages integration, more advanced notifications and many others. Here are our top 5 key new additions to iOS:
An integration with Siri is now available for apps in specific domains through SiriKit. This requires creating an extension using the Intents and Intents UI frameworks, which can receives an intent object from SiriKit, containing information about the user’s request and data related to it. The app extension should provide a response object. Optionally the extension could provide custom UI which could provide information from the app inside the Maps or Siri interfaces. Supported services which can be integrated with SiriKit are audio and video calling, messaging, sending and receiving payments, searching photos, booking a ride, managing workouts. Booking and reservations are supported by Maps as well as Siri, and they are handled in the same way, however for a custom UI, the Intents UI extension can handle the requests from both apps differently.
2. User Notification
The newly introduced User Notifications and User Notifications UI frameworks support the delivery and handling of local and remote notifications and the customisation of the notification appearance.
With the use of the former local notifications can be scheduled to be delivered on specific condition, e.g. location or time. This framework can be used to receive and modify notifications when they are delivered to the device. The UI frameworks can be used to define an app extension which customises the appearance of a notification and handles user actions associated with these notifications.
The CallKit framework facilitates the integration of Voice over IP apps with the Phone app. In that way the user can receive an incoming call on the lock screen from VoIP app. Another feature of the framework lets users manage contacts from VoIP apps in their Recents and Favourites views in the Phone app. Call blocking and caller identification could be achieved with new extensions introduced by CallKit.
4. Proactive Suggestions
In addition to the in-app search, introduced in iOS 9, iOS now enables app developers to share information with the system about the way an app is being used by the user, so that the system can suggest the app in places like keyboard QuickType suggestions, Maps, CarPlay, Siri etc. This enhanced integration is supported by Core Spotlight, the indexing framework which makes app content searchable, as well as other frameworks, like MapKit, UIKit and Media Player frameworks. The NSUserActivity object, used to restore application state and also to improve Spotlight search, now can be provided with location information, so that appropriate locations could be suggested throughout the usage of different apps. Another example would be the addition of semantic meaning of the expected content of a textfield, so that an appropriate keyboard proactive suggestions from other apps would be automatically chosen.
iOS introduces new app extension types:
- Call Directory - Introduces the ability to block numbers and identify callers
- Intents - Processes requests from Siri or Maps.
- Intents UI - Displays custom app content inside the Siri or Maps interface.
- Messages - Interact with the Messages app, sending texts, stickers, etc.
- Notification Content - Provides custom interface to your app’s notifications.
- Notification Service - Lets you modify the content of a remote notification before it is delivered to the user, or download content related to the extension.
- Sticker Pack - Sets of stickers to use in messages. DO NOT REQUIRE CODE!
Our personal favourite: the Messages re-vamp is especially cool, allowing you to play games inside the conversation, display and send full-screen animations, interact directly with selected apps, without having to leave the conversation. Here are the rest:
1. Lock Screen and Widgets
The Lock Screen in iOS 10 is completely redesigned to offer more advanced interaction with apps and widgets. One of the simplest, yet very useful changes is the fact that you have to press the home button to unlock your device. That together with the new ability to raise your phone to wake it gives you the chance to glance through your notifications, which would annoyingly disappear in the past versions, if you’ve set up a fingerprint. In the new version swipe to the right takes you to a customisable list of widgets which give you detailed information from the app, e.g. weather forecast or media playback info. Another change is the missing camera button on the bottom right hand corner of the lock screen. Instead you simply swipe to the left to access the camera without unlocking the phone to instantly start taking photos or videos. Notifications in iOS 10 are far more advanced, you can answer emails, add events to your calendar, check Maps destinations, read and answer iMessages. And you can do all of that without ever unlocking your device.
Siri is far more advanced in iOS 10. Integration is now available for third party apps with Siri, which means you can control your favourite apps with your voice. Sending someone a message through WeChat or playing Spotify can be done solely through Siri and does not require a touch anymore. Enormous advancements in machine learning enables Siri to give more intelligent suggestions while you are typing. For example it automatically suggests you share your location when someone asks you where you are, or contact information when you are talking about a friends phone number or e-mail address.
Photos in iOS benefit from complex machine learning algorithms as well. The app can now use facial recognition to create albums of your friends and family. The image search is even more powerful and allows for pretty abstract searches, using keywords. Memories is the new way of clustering photos together, based on location, date, people, topics. With a magazine-style interface, it can remind you of trips you’ve taken, events you participated in, people you’ve met. iOS 10 also automatically edits your photos and videos on a particular topic into a movie with a few EPIC-NESS options.
The Phone app now will support automatic voicemail transcription. A tighter integration with VoIP apps will be possible, which will let you take WhatsApp or Viber calls from the lock screen, just like a normal call. Contact information could include shortcuts to those messaging apps. And your recents and favourites will include VoIP calls/contacts as well.
There are a lot of changes coming to Messages on iOS 10. The keyboard identifies words which could be replaced with emojis and lets you replace them with a tap. The app also supports rich links including audio and video. There is a new Store especially for Messages, where you could get stickers, apps, even games, which you could play inside your Messages conversation. Or you can download a food delivery app, and place orders directly from inside Messages. You can send reactions to posts, choose among a few effects and even draw letters and send them as animations. Full screen animations like fireworks or balloons are displayed over the entire screen, and there are options for the way each message is displayed, e.g. “Invisible Ink”, which requires the message receiver to slide their finger over the message in order to reveal it.