Microsoft's new term for AI PC, Copilot+PC

Microsoft's new term for AI PC, Copilot+PC

Last November, Microsoft renamed Bing Chat Copilot and began a series of AI feature integrations for its products. Now, this popular renaming campaign has also swept across AI PCs. Of course, the new product released this time is not just about hardware upgrades and renaming. Thanks to its powerful AI computing power and the Windows 11 24H2 update, Copilot+PC has brought a series of heavyweight AI features.
Microsoft has introduced the Recall feature for Windows, which combines screen area detection, optical character recognition, natural language processing, text encoding, and image encoding functions. Windows can generate a snapshot based recall timeline based on the user's operating history, helping users review their PC operating history. The backtracking function utilizes the user's personal semantic index, which is completely stored and built on the device side, without worrying about the risk of privacy leakage. Users can also delete snapshots from a certain time period, filter specific applications and websites, or pause the use of this function.
On the AI drawing tool Cocreator, with the help of NPU's computing power and powerful local small language models, creative works can be generated more quickly. At the same time, Microsoft has also introduced it into the built-in applications of drawing and photography systems, such as AI creation of works in drawing applications through the use of brushes and text descriptions. Users can generate high-resolution paintings in real-time from their simple strokes.
On Copilot+PC, Microsoft will introduce a global real-time subtitle feature, which can provide real-time and accurate subtitle translation for both video conferences and conversations in video files. However, currently this feature only supports translating 44 languages into English.
In the field of video games, Microsoft has also been trying to develop a new AI assistant feature. By integrating Copliot into the game, the AI assistant will automatically provide reasonable strategy guidance based on game data after receiving natural language voice questions. In the above AI functions, Microsoft emphasizes that data will be saved locally and will not be uploaded to the cloud, nor will it be used for language model training without authorization.
Although Microsoft's main focus this time is on the Surface series, OEM partners have also prepared corresponding Copilot+PC products, such as Acer's Swift 14 AI, Asus's Vivobook S 15, Dell's XPS 13, HP's OmniBook X, Lenovo's Yoga Slim 7x, and Samsung's Galaxy Book4 Edge.
The first batch of Copilot+PCs will include over 20 models, which are currently open for pre order and will be shipped gradually on June 18th this year. We believe that in June's Computex 2024, we can see more intelligence about new Copilot+PC devices.
The reason for the launch of Qualcomm Snapdragon X series: Copilot+PC has NPU computing power requirements
The major devices that launched Copilot+PC this time have undergone a significant change in hardware, especially in computing units. Taking Microsoft's Suface series as an example, compared to the Intel Core Ultra chips used in the previous generation, the newly released Surface has all been replaced by Qualcomm's Snapdragon X series chips, including Snapdragon ® X Plus and Snapdragon ® Two versions of X Elite.
Under Qualcomm's latest Hexagon NPU architecture, both chips have NPUs with computing power of up to 45TOPS. According to data provided by Qualcomm, its NPU performance is 2.6 times that of the Apple M3 chip and 5.4 times that of the Intel Core Ultra 7 155H. Although the total AI computing power of the Core Ultra 9 185H processor CPU, GPU, and NPU combined may be above 30TOPS, its NPU computing power is still only 11TOPS, and Microsoft's definition of Copilot+PC is a device with NPU computing power above 40TOPS.
It can be seen that in order to ensure that different computing units perform their respective duties without compromising performance due to mixed tasks and extending PC endurance, NPU computing power has also become an important indicator, which is why Microsoft wants to standardize it. But this also means that among the currently available mobile chips, only Qualcomm's Snapdragon X series chips meet the definition of Copilot+PC.
Microsoft also has special hardware requirements for Copilot+PC security. On all Copilot+PCs, Microsoft's Pluton security processor will be enabled by default. Pluton security processor is a chip to cloud security technology designed by Microsoft and integrated into the processor by its chip partners.
As for the next generation AI PC chips from Intel and AMD, Microsoft has stated that Copilot+PC based on these hardware will be launched later this year. According to current intelligence, both Intel's Lunar Lake and AMD's Strix Point will integrate an NPU with a computing power of around 45TOPS, thus meeting Microsoft's definition of Copilot+PC.

More native software support for Windows On Arm

However, for a PC, the new features of the operating system and hardware enhancements are only a part of the upgrade, and the most crucial aspect is the ecological support of the application software. On the Apple side, they have fully achieved the transition from x86 to Arm with the M-series chips, while on the Windows On Arm side, progress is slightly behind.
But at the Microsoft Build Developer Conference held in May last year, a series of applications such as Dropbox, WhatsApp, and GoodNotes announced native support for Windows On Arm. Over the past year, applications such as Chrome, Opera, Spotify, Zoom, Blender, etc. have also added native support for Arm. Nowadays, Windows On Arm is not likely to lose to macOS in the native software ecosystem.
With the release of Copilot+PC, another batch of applications have added native support for the Arm architecture. Compared to various Adobe software that only adapted to macOS's Arm hardware platform before, Adobe's adaptation progress to Windows On Arm has been quite slow. Not long ago, only Photoshop, Lightroom, Firefly, and Express were adapted. At this press conference, Adobe announced that it will provide a wider range of native adaptations for Windows On Arm, and soon vector drawing software Illustrator and video editing software Premiere Pro will also join the ranks of native applications.
With the help of NPU's AI computing power, hardware acceleration has also been introduced to its AI functionality in many native applications that support Windows On Arm. In Adobe's graphics processing software such as Photoshop and Lightroom, editing tools such as AI removal and AI virtualization have been implemented with hardware; In video editing applications, the magic mask function of DaVinci Resolve Studio has been hardware accelerated for NPU, and the clip also supports NPU accelerated AI background removal function.
Not only native applications, Prism translation runs x86 applications
In addition to the continuous increase in the number of native Arm software, Microsoft has also made a new round of upgrades for simulating/translating x86 applications on Arm chips. Just as Apple completed the transition from x86 to Arm with the help of Rosetta 2, Microsoft has also created a new translation layer for Windows On Arm, Prism.
Microsoft claims that this is not just a renaming of the original translation technology, but also the addition of more optimizations at the bottom level. With the update of Windows 11 24H2, users can experience a 10% to 20% performance improvement on the same Arm hardware. Even if users are using the older generation Qualcomm Arm chips, they can still enjoy this optimization after updating the system.
Prism does not require developers to make additional changes to the application code, thereby reducing adaptation workload. But ultimately, translation is still a temporary solution. To fully utilize the hardware performance of the processor, especially the performance of the NPU, native applications are still the first choice. Microsoft firmly believes that on the upcoming Copilot+PC, 87% of user time will be spent on native applications, and the remaining 13% will be where Prism comes into play.

AI PC is in its early stages

Based on Microsoft's release of Copilot+PC and its accompanying AI features, it can be seen that AI PCs are already in the right direction of development, and the focus on interactive and experiential features and applications is the real reason for users to switch to new devices. The AI functions that used to focus on video conferencing and document/image processing were more like independent AI applications and did not intersect with the system.
And AI functions such as backtracking connect various system interfaces with the goal of improving productivity and efficiency, which can better stimulate the motivation of users to switch machines. The hardware base that drives its implementation is the new generation of AI PC chips. In the era of AI PC, NPUs may occupy more design space on chips, and manufacturers such as Qualcomm, Intel, AMD, and MediaTek have to pay attention to the development of the next generation NPU architecture.