XR Developer News - Meta Connect 2024
So, Meta Connect has come and gone, and it was definitely an interesting one this year! The Orion AR glasses were clearly the highlight of the show, but there was plenty of interesting developer news as well. Quest 3 (which had been completely leaked in the months leading up to Connect) is officially out, and now we have to wait and see how well it will sell at $300, which will heavily influence what the market for XR development will look like over the next 12-24 months.
Below I've made a selection of the news from Connect, because not all of it is relevant to this newsletter. Some of it has been covered more than enough by other publications, and some of it doesn't offer a developer platform, like the current Ray-Ban Meta smart glasses.
If you want to check out the full thing, the first 12 minutes of the keynote cover Quest, the next section up to 33 minutes covers AI and the Ray-Ban Meta glasses, and from there up to 47 minutes it's Orion.
Meta Connect 2024 - Keynote
The developer keynote had a similar setup, with Quest and Horizon Worlds running the first 33 minutes of the presentation.
Meta Connect 2024 - Developer Keynote
For those wanting the full picture, the first 25 minutes of Mark Zuckerberg's Decoder interview with Alex Heath are worth watching as well, just like Alex Heath's first impressions video.
Orion
Let's get into the star of the show: Orion. Meta's explanation video and blog post are good primers to start with.
Andrew Bosworth's Orion explainer
Some facts on Orion: 70 degrees field of view, 98 grams, separate puck to power the compute which is connected to the glasses wirelessly (12 feet max range), a neural wristband for input, eye tracking, hand tracking, 2-3 hour battery life, roughly $10000 cost per unit. And... many years from being a consumer product. But, Meta is clearly convinced that it has 'line of sight' towards getting to that point. Oh, and you also pretty much can't get one if you're outside of Meta, it's primarily a Meta-internal developer kit.
What strikes me is how next level Orion seems compared to the Snap Spectacles 5: better weight, better look, better battery life, better field of view, to the point where Orion is 'I might actually wear that' while Spectacles is still substantially removed from that.
But that isn't a fair comparison to be honest. Spectacles is a 'what we could almost ship now, at a somewhat doable price' product, while Orion is a 'many years from now, at an unrealistic price' product, so it has very different constraints. Still, I wonder if Snap would be able to pull off Orion.
Orion doesn't run Horizon OS and nothing is known about the developer platform (I saw a rumour that the demos Meta used were built on the recently scrapped Meta Spark, but that obviously wouldn't be a long term approach), so from a developer point of view it's not really a factor yet. But it is good to know that this is the future we're heading towards, and it's slowly coming into view.
I am curious though whether I was the only one who had a brief flash of '$10K? Let's go! Where can I send the check?'...
Meta Quest 3S and related hardware
Surprising absolutely nobody, Meta announced Quest 3S will launch October 15th. Roughly speaking it has the optical stack and (crucially) $300 price of Quest 2, combined with the compute and mixed reality passthrough of Quest 3. UploadVR has a detailed comparison. How well it sells and potentially grows the XR market will be crucial for developers in 2025 and 2026. Meta is clearly hoping that it will see success similar to or greater than Quest 2, but we won't really have the first indications of that until after the holiday period at the end of the year.
Meta gave a more detailed overview of Quest 3 for developers in one of the deep dive sessions of Connect.
Quest 2 and Quest Pro are being retired, which makes sense and simplifies the line up considerably. No sign of headsets by partners yet though, so perhaps that's for next year.
An interesting addition to the Quest peripherals is a new Open Facial Interface, which makes the Quest 3 or 3S much more like the Quest Pro, in the sense that you're no longer fully locked into the headset. A pleasant addition for mixed reality use cases I expect.
Another peripheral that is now available is the Logitech MX Ink stylus, which should be interesting for creative use cases which require fine control. Roberto Coviello has a nice tutorial on working with the hardware and implementing the related SDK.
Software
There was plenty of Quest/Horizon software news as well of course. Definitely check out the two keynotes, as those contain the full overview, but a few things stood out:
- Next year Meta will open up access to the camera feed on Quest, similar to what Apple announced (in their case only for Enterprise apps). Details are scarce for now, but assume this will be at the very least be gated behind a permission dialog which users have to approve. It should unlock a lot of interesting innovation, so it will be really interesting to see what people manage to build with it.
- Meta is now seriously pushing towards getting more 2D Android apps and Progressive Web Apps (PWA) into the Horizon Store. Two deep dive sessions at Connect went into the topic, with one being a general overview and the other going specifically into the new Spatial SDK, which allows for adding 3D elements to Kotlin Android apps (similar to RealityKit and such in visionOS) and the new Meta Spatial Editor spatial scene composition tool (similar to Reality Composer Pro on the Apple side).
- A new version of Meta's Avatars is close to being launched, with a serious graphics update, although it's nowhere near Codec Avatars yet and still firmly in the cartoony look.
- Meta announced the Hyperscape app, which seems to be based around high quality, cloud streamed gaussian splats. Apparently it looks quite impressive, but it's only available in the US (likely because of the cloud streaming server infrastructure) so I haven't been able to test it here in Europe. Although local on-device support or SDKs for splats have appeared on some other platforms recently (e.g. Pico), no such thing was announced at Meta Connect.
Session videos
While I already included some of the deep dive session videos from Connect throughout the text above, there were a few more which are now available on YouTube. Have a browse to see if any of them might be interesting to you. They're all pretty short, so it never hurts to just do a quick double speed watch through.
- Momentum in Mixed Reality: a pretty broad session with some insights into what type of games and apps are succeeding in the Quest ecosystem.
- Get Started with Mixed Reality: an introductory session on developing mixed reality for Quest covering some of the basic tools you'll encounter as a developer.
- Paths and Pitfalls of Mixed Reality Design: a session on designing mixed reality Quest apps and games, based on insights from the Niantic team that worked on bringing Peridot to Quest.
- How we're improving the Meta Horizon developer experience: gives a quick run through of various (always welcome) improvements to developer workflow, tooling, etc.
- Making Meta Quest 3 Shine: Tools, Tips and Best Practices for Maximum Performance: a more tech-heavy session on pushing performance of apps and games to the maximum.
- Yeeps: A Meta Horizon Start Success Story: a mini-session on the Horizon Start program.
- Launch and Grow in the Meta Horizon Store: a deep dive into the latest details of the Meta Horizon Store and functionality it offers to anyone publishing an app or game there.
Related XR tidbits
- While strictly speaking not part of Meta Connect, it was shared shortly before and is pretty cool and strongly Meta Quest development related: the open source Godot game engine and editor can now be used directly on Quest, which is a pretty nifty trick, allowing for an interesting new approach to how to develop XR software. Might be worth giving it a try, even if Godot is not your engine of choice.
Godot Engine
- While we're at it, Godot recently shared a post on open standards body Khronos Group's blog covering its progress with OpenXR.
A bit about this newsletter
Each month I try to round up all the interesting developments in the XR developer landscape. New hardware and software releases, events, interesting tooling, etc. Feel free to reach out to me on LinkedIn, for instance if I missed anything which definitely should be in the monthly round up next time.
