XR Developer News - September 2024

Pieter Siekerman

21 Sep 2024 — 14 min read

Snap Spectacles 5

With Meta Connect coming up, I thought it wise to get all the September XR developer news leading up to it out of the way, as Meta will surely drown out everything else very soon (and provide ample material for a special edition of this newsletter).

Big news has obviously been Snap's new iteration of its Spectacles AR glasses, so we'll definitely get into that. However the biggest bombshell dropped just a day after I published the August edition: Meta is killing Meta Spark, its XR developer platform for Instagram and Facebook Messenger filters.

Plenty to talk about, so let's get into it!

Snap

First up let's dive into Snap. After a turbulent year with many layoffs, Snap combined its Partner Summit and Lens Fest into a single two-day event. The Partner Summit keynote on the first day contained all the major news, covering both advances in Lens Studio (skip to 32:20) and the announcement of Snap's new Spectacles (starts at 48:40).

The Lens Fest keynote on the second day consisted of a Q&A session with the CEO and CTO of Snap (starts at 3:30), followed by a deep dive into the hardware architecture of the new Spectacles 5 (at 37:50). Especially that second part is interesting to watch if you want to get an understanding of the difficult trade offs that go into hardware engineering AR glasses.

On the Lens Studio side, the news was mainly centered around new generative AI capabilities on top of those previously announced in June, such as Easy Lens to create full lenses based on just a prompt, and upcoming support for gaussian splats. Snap released v5.1 of Lens Studio during the event.

The big news was obviously the new Spectacles 5 hardware, the Snap OS running on it (with interesting features like multiplayer support and smartphone spectator mode) and the related support in Lens Studio, following Snap's Spectacles 4 from three years ago. It's good to see Snap still committed to pushing AR glasses hardware forward, even with the rough time the organization has gone through earlier this year.

Part of me is itching to play around with them, as they represent the cutting edge of what's possible in this form factor and should be a lot of fun to prototype with. But part of me realises that Spectacles 5 also clearly shows how long the road to consumer-ready AR glasses still is. They're heavy (226 grams), bulky, the field of view is limited (46 degrees), the battery life short (45 minutes) and the price high ($1200 per year rental). This really is 'just' a developer kit for a product which is still far off into the future. But still, I have to give Snap credits for continuing to push forward!

I do have to criticise Snap for some of the messaging during the launch. There was some heavy dunking on mixed reality headsets and their size, which of course is fair, but those are products actually shipping to a consumer market at volume. To position your own developer kit, which is many many years away from that consumer-ready point as 'the superior alternative' seems a bit misplaced to me.

The other point which felt a bit strange was the 'no developer tax' storyline (1:00:10 in the Partner Summit keynote). Not taking a cut of a non-existent market on a 0 user platform is a very easy and empty thing to say, especially as such promises can shift radically at any time in the future (e.g. Snap's ARES team/initiative which was cancelled only 6 months after its inception, or Meta ending Spark - which we'll get to a bit further down). It would have been better to just leave out that bit.

If you're curious to learn more, the most in depth analysis I've encountered was by Jason McDowall on The AR Show, Janko Roettgers in Lowpass and Alex Heath at The Verge. All definitely worth checking out!

Meta Spark

So, onwards to the giant in the room. And it made quite an impact this month, by announcing the end of Meta's Spark platform (FAQ) and all the filters that have been created with it by January 14th, 2025. That's the livelihood of a whole ecosystem of XR creators (see the comments here) suddenly shut down, with very little warning.

Meta clearly went for the 'ripping the bandaid off quickly' approach with a pretty terse 'we're shutting it down, thank you for the hard work' blog post and FAQ. Bosworth gave a tiny bit more context on the thinking behind the decision (they made the assessment that Spark wasn't the right technical foundation for building software for the head-worn devices which are Meta's primary long term focus in XR). Something approaching a 'we understand the impact on people, we're sorry, but had to make this choice' would have been nice.

For a while last year it seemed Spark had a rich future ahead of it as foundation of the Augments functionality on Quest which was announced at Meta Connect last year. While Andrew Bosworth shared the cancellation of those plans earlier in the year, a full shut down of Spark is definitely a much bigger deal.

For a while I've felt mobile AR has hit a ceiling, as the growth in software capabilities seems to have reached a plateau (the last really major innovation over the last few years was the launch of Visual Positioning System tech by Google and Niantic). Fundamentally we're running into the limits to what users are willing to do in XR through a rectangle in their hand, and how good you can make an XR experience in a smartphone form factor. Some use cases work, generally those that are low friction to enter and take a very short time (web AR through QR codes, social AR filters/lenses/effects), but breaking out of that I think will not happen with this hardware form factor. And I suppose Meta made a similar determination and decided it wasn't the future it is aiming for, not a market large enough to warrant further investment. A pretty cold calculus.

I wonder where the people who were seriously invested in the platform will go. One option would be migrating to the remaining mobile social AR platforms Snap and/or TikTok. At the very least the former still seems heavily invested, while you never know with ByteDance (judging by the heavy cuts to Pico's consumer ambitions this year). Perhaps web AR is an avenue to explore, but that lacks the natural distribution mechanism of a social network, making it a whole different ball game. Another option could be switching to Quest and embracing mixed reality headsets, but I wonder whether anyone recently burned so badly by Meta would happily jump on another Meta platform. Whichever way they go, let's hope they find more long term business stability than on Spark.

Meta Quest

While the Spark drama was unfolding, the other parts of Meta continued pushing out work at the same steady cadence they do every month. We'll see a lot more at Meta Connect in a few days (in advance Meta shared a recap of some recent news), so it's understandably not as long a list as in some other months.

Quest OS v69 was released (release notes) with a lot of nice quality of life improvements. MIXED and UploadVR have good overviews of the changes.
Meta introduced the concept of MR Motifs in a blog post, as a name for guidelines it will share for common recurring features developers might need, such as how to transition between VR and MR.

Some new analytics tools for Quest/Horizon Store apps and games are available.

XR Hardware

HTC announced the Vive Focus Vision, an upgrade of the 3 year old Vive Focus 3. The main new addition seems to be mixed reality capabilities through the colour cameras on the outside. You can use it in stand-along mode, but its main use case will likely be PC VR. At $1000, releasing on October 17th, I'm not sure what to make of it, although it seems to have a business focus. It's in no way competitive with Quest 3 at that price point, so seems to be primarily aimed at competing in the much smaller PC VR space. More details at The Verge, UploadVR and Road to VR.

Apple released visionOS 2. While containing no surprises, as it was fully covered during WWDC in June, various outlets have good roundups of the changes, which together are a good update to the Vision Pro. For developers some new sample projects are also available.

Pico released a major update of its SDKs with v3.0.0 to support the new headset and motion trackers it released recently and improved its Unreal Engine support.

French mixed reality headset builder Lynx seems to actually have survived its cash crunch and found new funding, after a long time of uncertainty. They're not sharing any details yet on the company they're working with, but clearly they're in a better place (for instance with the financial breathing room to refund those who don't want their R1 anymore). I'm really looking forward to seeing what they're working on for the Lynx-R2.

Microsoft's retreat from the XR space continues ever more, as it announced the shut down of Azure Remote Rendering a year from now.

XR Software

Niantic's 8th Wall jumped on the opportunity offered by Meta's shutdown of Spark, attempting to pull in creators, developers and designers in search of a new home, by significantly reducing the pricing of its licenses for launching commercial projects. Previously a massive $3000 per month, they are now at a 'mere' $700 per monthly. Still not cheap by a long run, but a very different calculus when you're considering it.
- I wonder though whether this isn't also a sign of the limits of the mobile AR market which I discussed in the Meta Spark section. I don't expect Niantic lowered its pricing because it wanted to (they've held onto their very high pricing for so long after all), so I'm curious whether it wasn't also forced to do so in part by reduced market demand for mobile AR.

Aside from changing pricing, Niantic made a big push into gaussian splatting. It launched version 4 of its Scaniverse app, which revolves fully around splats now, and added support for those splats to Niantic Studio, its recently introduced new tool for building XR experiences on the web. That same Niantic Studio update included several other improvements in working with traditional 3d models.

By the way, if you're interested in Gaussian Splatting, check out the new View Dependent Podcast for some expert takes on the technology and a recent interview with Gracia on XR AI Spotlight.

Niantic also released v3.8 of the Lightship ARDK with some bugfixes.
Unity announced a major change to its pricing model by deciding to cancel the per-install runtime fee it announced a year ago to much criticism and uproar. However, there are some caveats and questions, as it has been scrapped for gaming content, but it's unclear what the situation is for non-gaming content, which includes plenty of Unity-based XR apps. Also, the pricing of its per-seat licenses are going up significantly to compensate, especially for Enterprise licenses (up by 25%).

Having gotten that out of the way, Unity was ready to celebrate Unite 2024 in Barcelona. In the keynote and a related blog they announced that Unity 6 will be launching on October 17th. It was quite notable though that XR and any other non-gaming uses of Unity have taken a back seat in its messaging, as explicitly stated by the CEO (keynote at 30:22) and demonstrated by the merely passing mentions of anything XR, except for a short 1 minute segment in the Engine roadmap session (at 10:04). That's great for game developers, but what about non-game Unity developers? Hopefully the specific session dedicated to XR will turn out interesting once the recording becomes available.

Unity released some small updates to its visionOS support.
At Epic they keep making progress in adding support for visionOS, even with the strained relationship with Apple.
I missed this one last time, but there was a good overview post of the XR-related improvements in Godot 4.3.

TikTok released v4.6.0 of Effect House, which seems to be a pretty solid update. It adds the node-based Visual Effects Editor (which is now out of beta) and adds some other nice smaller improvements.
three.js r168 was released.
Wonderland Engine released v1.2.3.

Other XR tidbits

Open standards group Khronos Group shared some useful resources, such as an overview of and related info for all its sessions at SIGGRAPH 2024, covering such topics as glTF and OpenXR. It also shared a YouTube playlist with various sessions from OpenXR Live 2024, so if you're doing anything with OpenXR, definitely good to check out.

Roblox held its annual developer conference RDC 2024. While not really in the realm of XR, it's always worth keeping an eye on adjacent platforms. The most interesting bit I found to be the work they're doing on integrating generative AI in various aspects of the creator tooling. There's a section in the keynote worth having a look at (starts at 31:45, runs up to 45:00), which gives a good overview. A related interview in MIT Technology Review is also good to check out.

If you do anything with XR design and/or Gravity Sketch, it's worth checking out the sessions of Around Festival, which happened this month. While a primary topic is using XR tools for product design, there's often a lot of high quality general XR design talk as well.

LIV's CEO appeared on the XR AI Spotlight podcast, after last month's interview on the Being Virtual Show. Again an interesting interview to check out.

Upcoming XR events

September 25-26, 2024 - Meta Connect.
September 30th, 2024 - Niantic 8th Wall Create and Play Developer Challenge submission deadline ($10 prize pool).
September 30th - October 3rd, 2024 - Epic's Unreal Fest 2024 in Seattle.
October 15th, 2024 - Meta Quest Lifestyle App Accelerator submission deadline.
October 23rd, 2024 - XR Creators Connect #4, the next edition of the in-person XR meetup I co-organize in The Netherlands. This time in Eindhoven with a focus on XR on the web. Should be an interesting one again!
October 29-30, 2024 - AWE EU in Vienna, Austria.
December 4-6, 2024 - Immersive Tech Week in Rotterdam, The Netherlands, the big yearly XR conference in NL.
January 23-27, 2025 - MIT Reality Hack in Cambridge, MA.

Each month I try to round up all the interesting developments in the XR developer landscape. New hardware and software releases, events, interesting tooling, etc. Feel free to reach out to me on LinkedIn, for instance if I missed anything which definitely should be in this monthly round up next time.

XR Developer News - September 2024

Pieter Siekerman