Apple redefines AI: AI = Apple Intelligence
WWDC24: Apple has fulfilled all users' desires for AI.Author: Sukhoi, Jiazi Light Year
After months of waiting, the moment that Apple fans have been anticipating has finally arrived.
At 1 AM Beijing time on June 11, Apple's 2024 Worldwide Developers Conference (WWDC) kicked off, showcasing "incredible technology and creativity" at Apple Park in California this week.
Since its inception in 1983, WWDC has been a key platform for Apple to showcase significant product updates and communicate with the developer community. The highlight of this conference is "generative artificial intelligence," which occupied most of the time in the keynote speech.
Bingo card for WWDC 2024 created by software developer Dylan, source: X
At the conference, Apple officially launched a brand new artificial intelligence system called "Apple Intelligence," which will be integrated into iOS, iPadOS, macOS, and will expand to watchOS, tvOS, and visionOS in the future. At the same time, Apple confirmed its collaboration with OpenAI, combining OpenAI's technology to achieve collaborative processing between device and cloud, while emphasizing user privacy protection.
Almost all native applications of Apple's systems, such as camera, calendar, browser, and email, will receive AI enhancements.
Apple is one of the last tech giants to enter the field of artificial intelligence.
In May last year, Apple CEO Tim Cook pointed out that while the prospects for AI technology are broad, there are still many challenges. Over the past year, Wall Street investors have criticized Apple for lacking highlights in the AI field. During the earnings call in May this year, almost every analyst asked questions about AI, but Cook did not reveal much about future plans.
Cook stated that he did not want to disclose too much before the official announcement but confirmed that Apple is optimistic about the opportunities in generative AI and has invested $100 billion in research and development. Although Apple started late in AI, it still has a latecomer advantage due to its large user base and complete product ecosystem. OpenAI also wants to leverage Apple's latecomer advantage to seek greater growth opportunities.
In recent years, the iOS systems and other updates released at WWDC have often been criticized as "lackluster," with each year's WWDC being dubbed "the most boring one ever." Until 2024, as a latecomer in AI, Apple finally brought epic updates to its products.
If the iPhone led mobile users to abandon physical keyboards and fully transition to large-size multi-touch screens, then the updates released by Apple this time may officially lead the world's 4 billion smartphone users into the next round of transformation. AI phones have a new definition.
But Apple's ambitions may go beyond this. From the naming convention, it aims to reshape the entirety of AI.
Turning AI into Apple Intelligence.
Epic Updates from Apple
- Intelligent Leap with iOS 18
iOS 18 is undoubtedly the absolute focus of this conference.
This update not only achieves a "transformative" improvement in AI technology but also comprehensively optimizes user experience. In particular, by integrating large language models (LLMs), Siri's functionality has been significantly enhanced, becoming more intelligent and user-friendly. Users can edit photos, create personalized music playlists, and even write and modify documents through Siri using just voice commands, greatly enhancing Siri's practicality and convenience.
At the same time, iOS 18 introduces Rich Communication Services (RCS), enhancing the smoothness and interactivity of communication between iPhone and Android devices. Users can enjoy features such as read receipts, real-time input indicators, and emoji reactions when communicating across platforms.
In terms of interface optimization, iOS 18 has completely revamped the control center, introducing customizable layouts that allow users to freely arrange icons and tools according to their needs. This not only enhances operational efficiency but also improves the personalized experience. The optimization of the settings interface makes it easier for users to find and adjust the functions they need, while the search function has also been strengthened, making operations more intuitive and convenient.
In health monitoring features, iOS 18 has also expanded, adding more monitoring capabilities to help users better manage their health. Additionally, the newly added home screen customization tools and dark mode application options further enhance users' personalized experience and visual comfort.
Regarding the highly concerned privacy issues, Apple has enhanced privacy protection features in the latest operating system update, allowing users to lock individual applications, ensuring that even if the phone is accessed, these applications require separate unlocking to use. Apple also provides a hidden application folder that can be independently locked, further protecting user data security.
- Vision Pro Officially Enters the Chinese Market
Apple's recently released visionOS 2 operating system also brings many updates to the Vision Pro headset.
In this update, the real-time captioning feature can display real-time subtitles of conversations within the user's field of vision, making communication more convenient. At the same time, the layout of the home screen and application sorting has also been optimized for more intuitive operations.
In terms of applications, the Photos app has added a SharePlay feature, allowing users to share photos across different devices, and even convert 2D images into 3D spatial photos.
The resolution and width of the Mac virtual display have been improved, providing users with a broader and higher-resolution visual experience. The operating system has also simplified gesture controls, allowing users to flip through photos with gestures, such as opening their hands to click into the home screen or rotating their wrists to check the time and battery level.
visionOS 2 has particularly optimized developer tools, making it easier for developers to create spatial applications by introducing new frameworks and APIs, such as Tabletopkit and volumetric API. Apple has also collaborated with Blackmagic Design to improve immersive video content using its camera technology, further enhancing Vision Pro's spatial content creation capabilities.
Additionally, the newly added travel mode increases train support, making it more convenient for users to use Vision Pro on trains. Apple also announced that Vision Pro will launch in three Asian markets: China, Japan, and Singapore on June 28, with a starting price of 29,999 yuan in mainland China, limited to two units per person.
- iPad Gets a Calculator
The release of iPadOS 18 finally brings the calculator app to the iPad, a feature that has existed on the iPhone for 17 years but has been absent on the iPad.
iPadOS 18 introduces a new accessibility feature inspired by Vision Pro, allowing users to control the iPad using eye-tracking technology, a revolutionary hands-free operation feature. Apple has also enhanced multitasking capabilities on the iPad, particularly improving the functionality of Stage Manager, an area Apple has been researching for years.
Previously, there were expectations for unique features of the M4 iPad Pro or hopes to launch macOS directly from the iPad, but this year's update did not include such breakthroughs. However, iPadOS 18 benefits greatly from iOS 18, adding many practical features.
In this update, iPadOS 18 brings some eye-catching new designs and features, such as the integration of the new Apple Pencil with math memo functionality, allowing users to write mathematical expressions and formulas directly on the screen, which are automatically converted into numerical results. This feature not only supports handwritten graphic recognition but can also perform calculations based on the user's sketches, which is a huge surprise for math enthusiasts.
The new Apple M4 chip, first introduced in the iPad, is a chip manufactured using advanced 3-nanometer technology, featuring powerful CPU and GPU performance, greatly enhancing the display and AI processing capabilities of the iPad Pro.
iPadOS 18 also introduces new interaction methods, such as screen sharing and floating tab bars.
- macOS Sequoia
macOS 15 continues the tradition of naming after iconic California features, this time being "Sequoia." This version significantly improves user experience and work efficiency, introducing many AI features from iOS 18, including intelligent system applications and a redesigned system settings interface.
As Apple's longest-standing and most mature software platform, macOS typically adds fewer new features each year. However, this year seems different, especially in the AI field. It is equipped with newly launched Macs powered by M3 chips, which will fully utilize Apple's neural engine to bring a series of innovative AI features. In addition to AI, Apple has also updated the system settings application for Macs in response to user feedback.
macOS 15 now allows iPhone notifications to be displayed directly on the Mac, achieving efficient cross-device information processing. Additionally, this version introduces a dedicated password management application that can be used on the Mac and can also sync and manage passwords on Windows devices via iCloud for Windows, greatly enhancing security and convenience.
macOS 15 has also improved the smoothness of multitasking, such as automatically adjusting window sizes to make full use of screen space. These improvements ensure a high degree of consistency with iOS 18, allowing users to enjoy a unified and smooth operating experience on both iPhone and Mac.
Overall, most of the improvements in macOS 15 are expected to be aimed at maintaining consistency with the features of iOS 18 on iPhone, such as updates to system applications and the new version of Siri.
- Comprehensive Upgrades to Other Platforms
Apple has recently conducted comprehensive upgrades across multiple platforms, including iOS, macOS, watchOS 11, tvOS 18, and HomePod Software 18. Although detailed information has not yet been fully disclosed, these updates are expected to strengthen the integration of AI features, thereby enhancing the intelligence level of devices and user experience.
Apple encourages students worldwide to showcase their programming talents through the Swift Student Challenge. This year, 50 outstanding students will have the opportunity to visit Apple's headquarters and receive a one-year Apple Developer Program membership, which not only recognizes their skills but also serves as an incentive for them to continue advancing in the field of technological innovation.
Developers located in mainland China can now use local payment methods more conveniently to register and purchase Apple Developer Program memberships through the Apple Developer app. This will enable more developers to easily join Apple's development ecosystem.
Deeply Integrated with AI
At today's conference, Apple finally unveiled its highly anticipated new feature: deep integration of artificial intelligence into its various applications, with this new AI platform named "Apple Intelligence."
This is a significant advancement for Apple in the smart device field, which will fundamentally change the way users interact with their devices.
"Apple Intelligence" will be applied to Apple's iPhone, iPad, and Mac operating systems, providing a range of advanced tools, such as text and image generation, information retrieval, and analysis. These features will work across applications, greatly enhancing user convenience and personalized experience.
Apple CEO Cook stated that this is a "new journey," emphasizing, "This is an exciting beginning, and Apple Intelligence will change the way our products serve users."
It includes context-driven notifications that can identify and prioritize the most important information for users. Additionally, "Apple Intelligence" supports natural language understanding, allowing for the analysis of your content and commands across different applications, automatically refining emails, summarizing memos, and generating images, while supporting various styles such as sketches, illustrations, and animations.
In terms of privacy protection, Apple continues to make significant efforts. Most data processing is done locally on the device, ensuring the security and privacy of user data. For features that require cloud computing, Apple uses special Apple Silicon-based servers, and all data transfers are encrypted to ensure information security.
Notably, Apple has also introduced the "genmoji" custom emoji tool, allowing users to create unique emojis based on their needs, showcasing the personalized service capabilities of Apple Intelligence.
Craig Federighi, Apple's Senior Vice President of Software Engineering, stated at the event, "We have been working on this for a long time."
The collaboration with OpenAI will allow Apple users to access ChatGPT through Siri without incurring additional costs. Federighi mentioned that users will be able to use this AI feature through a beta version later this year, while some features will have to wait until 2025.
OpenAI CEO Sam Altman attended the event and posted on X that he is "very excited" to partner with Apple.
Screenshot of Sam Altman's social media
As early as May 26, Mark Gurman, a journalist from Bloomberg who has long tracked Apple, revealed that after two months of negotiations, Apple has reached an agreement with OpenAI to integrate the latter's chatbot ChatGPT into iOS 18, with the partnership expected to be officially announced at the June Worldwide Developers Conference (WWDC24).
However, some have criticized Apple's "quirky" move: the local AI feature requires at least 8GB of memory.
Currently, Apple has not clearly stated when all the features of the new Siri will be fully launched, but it has confirmed that the new intelligent assistant will support iPhone, iPad, and Mac devices. This improvement marks an important step for Apple in the field of "personal intelligence." Cook described this initiative as the company's "next step," marking new progress in providing smarter personal assistance.
This innovation may spark further discussions and research in the technology and AI fields, particularly regarding how to more effectively meet user needs when integrating hardware and intelligent services.
Shifting Focus from Car Manufacturing to AI
In February this year, Apple quietly made a decision to abandon its "mysterious" car project and fully pivot towards artificial intelligence.
Since 2014, there have been constant rumors about "Apple making cars." Ten years later, in a board meeting in January 2024, Apple lowered its autonomous driving goal from L4 to L2, and the launch time for the car was postponed from 2026 to 2028.
Currently, Silicon Valley is undergoing a major adjustment, with many large companies laying off employees while shifting resources towards developing generative AI.
Apple has also reorganized its team during this adjustment, integrating some software and cloud computing teams from the car project into the software department to focus on developing operating systems, while the autonomous driving and related hardware teams face adjustments or reductions. Meanwhile, Apple's AI team is shifting towards more machine learning and generative AI projects, with some personnel also being transferred to the Vision Pro team.
Dan Ives, an analyst at Wedbush, wrote in a report, "The AI strategy is a missing piece of Apple's growth puzzle." Developer conferences typically do not attract much investor attention, but this year is different. Ives believes this conference will be particularly noteworthy for investors, possibly the most important one for Apple in the past decade.
In fact, as early as 1985, Jobs foresaw the development potential of generative AI, a forward-thinking perspective that was ahead of its time.
With the evolution of technology, AI computing, initially driven by GPUs and cloud computing, is rapidly shifting to end devices. Unlike Microsoft, Google, and Amazon, Apple does not have a cloud computing business, but it possesses the most powerful end-device computing platform. Leveraging its large user base and developer network, Apple can create a unique personal AI device experience, presenting a significant market opportunity.
In the earnings call in May, Cook emphasized the transformative power of artificial intelligence and Apple's advantages in this new era to investors. He mentioned that Apple's integrated strategy—seamless integration of hardware, software, and services, along with industry-leading Apple chips and a focus on privacy—uniquely positions it to build a powerful local computing platform. By introducing neural network capabilities through its M series and A series chips, Apple has created a hybrid computing architecture that spans from the device to the cloud.
After all, companies like Qualcomm and Samsung are eyeing to disrupt its market position by strengthening end-device AI capabilities, and Apple must maintain its pace of innovation.
Generative AI still faces thorny issues such as regulation and privacy, and Apple's AI actions have been somewhat slow. However, this has given Apple more time to find solutions, and with its integrated advantages in chips, operating systems, applications, and product design, Apple may still maintain a lead in innovative product experiences.
Moving forward, user acceptance will be the first major test for Apple's AI.
