Pico4Ultra: ByteDance's "Standout" Work, the Gateway to the MR Era

Pico4Ultra: ByteDance's "Standout" Work, the Gateway to the MR EraLast week, the biggest news in the XR industry was undoubtedly Pico, ByteDance's subsidiary, announcing its new product plan. With ByteDance's strong backing, Pico's every move is closely watched, as Pico's stance largely reflects ByteDance's commitment to the XR field

Pico4Ultra: ByteDance's "Standout" Work, the Gateway to the MR Era

Last week, the biggest news in the XR industry was undoubtedly Pico, ByteDance's subsidiary, announcing its new product plan. With ByteDance's strong backing, Pico's every move is closely watched, as Pico's stance largely reflects ByteDance's commitment to the XR field. Especially against the backdrop of overseas tech giants like Google, Meta, and Microsoft significantly investing in XR, Chinese tech giants have been somewhat "calm" in recent years. Baidu, Alibaba, and Tencent have essentially abandoned their XR businesses, leaving only ByteDance holding its ground. Pico's last product launch was back in the fall of 2022 with the Pico4 series.

However, the "life cycle" of the Pico4 series may not be over yet. According to official information, Pico will hold a 2024 new product communication meeting on August 20 in Beijing, using the keywords "standout" and "entry." Currently, it appears that Pico's new product is the long-rumored Pico4S, which Lei Technology reported on back in March. At that time, we quoted Pico OS product manager Ma Jiesi, who said Pico's new product would not be released before the end of April, and it was expected to be released in the second half of the year. However, specifically in the domestic market, this new product will be called Pico4Ultra. Therefore, we will use "Pico4Ultra" throughout this article to refer to Pico's upcoming new product.

With the official announcement of August 20, more information about Pico4Ultra has started to emerge. Recently, we finally saw the "real face" of Pico4Ultra.

Pico4Ultra: Minor Adjustments in Appearance, Breakthroughs in Functionality

According to images provided by the well-known leakster @Onleaks, compared to the previously released Pico4 series, Pico4Ultra has not undergone major changes in overall design. The most direct change is the "camera" on the front of the headset. In addition to the original four VGA cameras at each corner, the Pico4 series also features a central RGB camera (for VST video see-through), while Pico4Ultra has been replaced with two symmetrical cameras, which also appear to be significantly larger.

Additionally, as previously leaked, the Pico4Ultra controllers feature a ringless design, making them more minimalist. This means less obstruction in daily use, and also signifies that Pico has made significant progress in its controller tracking algorithm, allowing it to remove the tracking ring.

However, the changes in Pico4Ultra are clearly not limited to this.

Pico4Ultra: A Crucial Step Towards the MR Era

Although the market performance of Apple VisionPro has disappointed many, it is undeniable that mixed reality (MR) remains the trend for the entire industry and is the focus of Pico4Ultra's key upgrades.

It should be noted that the Pico4 series was one of the first consumer-level headsets to support VST color passthrough. Through the central RGB camera, Pico users have experienced VST functionality since 2022.

To briefly explain, VST (VideoSee-Through), translates as "video see-through," works by capturing real-world space through external cameras and displaying it on the inner screen to simulate real space. However, strictly speaking, the Pico4 series can hardly be considered an MR product. The most prominent issue is the low quality of the passthrough, coupled with the inability of a single camera to "replicate" the three-dimensionality of the real world. Users primarily use VST intermittently, such as for short periods when drinking water, answering phone calls, or going to the bathroom, needing to leave the virtual world and return to the real world, thereby reducing the cost of everyday wear.

This is also one of the most anticipated changes in Pico4Ultra. From the leaked images, it's clear that the two cameras on the front of Pico4Ultra are significantly larger. Combined with continuous iterations in camera hardware and algorithms, we have reason to believe that Pico4Ultra's video passthrough quality should be significantly better than before.

On the other hand, as mentioned earlier, a single camera often results in a two-dimensional image without depth or three-dimensionality. However, just as humans form a sense of three-dimensionality through the visual differences between our two eyes, symmetrical dual cameras can also simulate the three-dimensionality of real space to some extent. From this perspective, the three-dimensionality of Pico4Ultra's video passthrough will also be significantly enhanced, resulting in a more realistic and natural spatial experience.

Based on better picture quality and three-dimensionality, Pico4Ultra may be Pico's first true MR headset, where users can interact with virtual content in a realistic physical environment.

Pico4Ultra: Comprehensive Hardware Upgrades

However, in terms of display quality, we still need to look at the screen, which is currently the most uncertain aspect of Pico4Ultra. On the Pico4/4Pro, Pico uses a 2.56-inch Fast-LCD screen with a binocular resolution of 4320x2160, reaching a maximum of 20.6 PPD (angular resolution). In Pico4Ultra, we probably won't see the expensive single-eye 4K MicroOLED screen. Lei Technology believes that it will likely still be a Fast-LCD screen, but the resolution might get a boost.

Furthermore, another question about Pico4Ultra is whether it will follow suit with the Pico4Pro and feature a camera on the inside of the headset for eye tracking, and support automatic interpupillary distance adjustment, which will impact the price and user experience of Pico4Ultra.

The upgrade of the processor is basically confirmed. According to leaks, Pico4Ultra will be equipped with the Snapdragon XR2 Gen 2, the same chip as the Meta Quest 3. Released at last year's Snapdragon Summit, this XR chip boasts a 2.5x improvement in GPU performance compared to its predecessor, and an energy efficiency increase of up to 50%. It also supports parallel processing of up to 10 sensor and camera data streams and a VST latency as low as 12ms. In other words, Pico4Ultra should offer a smoother experience in daily use and high-load games, and benefit from the advancements in image processing capabilities of the Snapdragon XR2 Gen 2, resulting in a significant improvement in video passthrough latency.

Moreover, compared to the default 8GB of RAM in the Quest 3, Pico4Ultra is rumored to be equipped with 12GB of RAM, which theoretically should offer better performance than the former.

Pico4Ultra: The Best Alternative to Apple VisionPro?

Overall, if the price remains within a reasonable range, Pico4Ultra could be the best alternative to Apple VisionPro in the foreseeable future. Furthermore, given Pico's (and ByteDance's) accumulated VR content and app ecosystem, it may even be better than Apple VisionPro in terms of entertainment.

Why Is MR the Inevitable Path for Headsets?

It's undeniable that in 2024, simply doing VR is not enough to captivate the attention of mass consumers or even tech enthusiasts. Apart from the upcoming Pico4Ultra, at the beginning of the year, Lei Technology saw many MR new products at CES, including new releases from Sony, NOLO, and other manufacturers. Additionally, in the past six months, there has been the 1X99 USD (final price undetermined) Play in Dreams MR, and Vivo's official announcement of entering the MR market.

Major XR manufacturers all agree on the promising future of MR, and this is not just due to the leadership of Apple VisionPro. In fact, MR experiences based on VST technology address several long-standing problems with past headset products, such as the temporary "interruptions" mentioned earlier, and the high spatial requirements of VR.

As we all know, VR has always had high spatial requirements, especially in applications or games involving movement. In typical home environments, safety zones are generally small, and users are in a completely virtual environment, raising concerns about hitting objects or people while gesturing. In contrast, with sufficiently good VST video passthrough, human eyes can observe the layout and changes in real space in real time, serving as a reliable basis for judgment during movement, which in turn expands the usage areas and scenarios of headset products.

From the user's perspective, this is also the core difference between Pico4Ultra and previous Pico products. The key change is that the MR experience enabled by VST allows users to interact with both the physical world and digital content simultaneously, while still being able to return to the highly immersive virtual environment at any time.

Although former Oculus CTO John Carmack disagrees with this direction, at least in the foreseeable future, we cannot completely sever ties with the real world. If so, then "face the reality." At least MR provides a viable direction.

Pico4Ultra: We'll See

Before experiencing it firsthand and finding out the price, Lei Technology cannot accurately evaluate Pico4Ultra. Regardless, this is a product we are highly anticipating, and one of the most anticipated heavyweight cutting-edge products in the consumer electronics industry in 2024.

Admittedly, Pico4Ultra may not have dramatic changes in hardware, but the series of upgrades solely focused on MR experience are sufficient to greatly change the daily user experience. In other words, if you're also