FaceTime’s has not lived up to the potential it had. Apple’s FaceTime platform never reached a business audience because it was aimed at the consumer market. Apple uses Cisco’s Webex to communicate with business partners. Perhaps one day things will be different.
FaceTime now has video intelligence
The newly published Apple Patent (US patent number 20230109787) describes a variety of ways Apple could plan to make FaceTime a more useful communication tool for enterprise users – at least those using Apple’s system.
The patent, filed in September 2022 and published in April 23 was first noticed by Patently Apple. It describes a FaceTime that is capable of gesture recognition, referred to “Air Gestures” with a focus on collaboration.
Apple’s patent abstract explains that “the present disclosure generally relates embodiments of video communication interfaces for managing content shared during a session video communication,”
It is possible that the patent refers back to DeskView. It’s also possible that the patent hints at improvements to this popular feature.
What are the improvements described?
The improvement centers around gesture and imaging recognition. You can quickly skim through the document to find descriptions of these improvements in practice.
- A user can point to an object in the frame and the camera will focus on it.
- FaceTime’s background can be changed by placing your finger on your lip.
- By pinching your fingers together, or away from each other, you can zoom in or out.
- Pointing at an object will zoom in.
- FaceTime can now translate text into the language of the destination device.
- Apple also describes color management so that all participants see the same shades.
- FaceTime could be able to turn a drawing on a notepad into a digital sketch that all collaborators can then edit and review.
- A patent describes a feature that allows collaborators to see different virtual demonstrations in a loop. This will help them make better decisions by allowing them to really understand the examples.
- You can also share multiple views within a single conversation.
Some of these ideas are just next steps for some of the features Apple introduced more recently within FaceTime. These include Desk View, Continuity camera, and Freeform. When you have an iPhone and Mac together, DeskView allows you to show both your face and a view from above of your desk. Continuity Camera allows you to use your iPhone like a webcam.
What are the practical implications of these enhancements?
Apple wants to expand on these features. In Desk View, you could share a sketch on a piece of paper that is placed in the range of the camera. The camera will detect what you’re doing and display it in a format where others can zoom in, edit the image and even add text.
The camera on your Mac or iPhone will allow you to share the items in the meeting as well as launch presentations and show your desk from a top-down view.
It is the introduction of gesture controls that makes it more interesting. If FaceTime can recognize gestures in a collaboration session, then why not make similar gesture recognition features available within other applications? You are using a Mac or iPad, which has a built-in camera.
Apple glosses also over personalization in a few long paragraphs, mainly concerning the privacy of users. Apple seems to have a vision of FaceTime that includes the use of these systems for sharing health-related symptoms or information. I can imagine a doctor asking a patient where they hurt and the patient pointing at the painful area while the camera zooms.
Keep an AI with you
It’s also fascinating to see how these new FaceTime functions relate to Apple’s ongoing research on machine vision intelligence. Apple is working hard to improve accessibility and other features on its systems.
It makes sense that Apple would want to build systems that are reliable and capable of understanding the world. A car must be able to tell the difference between a road and a tree, just like a goggle wearer should know to not trip on the couch while exploring virtual worlds.
Apple should spend some time thinking of a feature that would automate the transcription of FaceTime conversations. This could include intelligently summing up the conversation and automatically populating things such as meeting requests, email messages, or documents.
What’s next?
Apple has the ability to implement the ideas described in the patent, and I believe that FaceTime enhancements will be based on these concepts.
If Apple allows them to access the APIs, it is possible that Webex and Zoom could offer similar features to those described in this patent.
Cisco’s Webex , for example , recently added support when running on an iPad for picture-in – picture and Stage Manager.
