The face of the future isn't just for the metaverse

Lumirithmic's realistic avatars are following the money

Hello there,

When a startup is featured here in PreSeed Now, it’s usually their first media exposure. For today’s startup, that is definitely not the case.

Although I started talking to them a few weeks ago, in the meantime there have been a handful of short news stories about them as well as a report about them on the BBC’s Click tech show last weekend. But this newsletter is a chance to dig into what they’re doing in more detail.

It’s well worth reading the article in full, as Lumirithmic’s co-founder and CEO has some really interesting thoughts about the value of the data they’re creating. As usual, paying PreSeed Now members get the full article.

– Martin

---

Lumirithmic's realistic avatars aren’t just for the metaverse

A face captured through Lumirithmic’s technology

No matter where you stand on the much-hyped metaverse, it’s clear that there’s going to be plenty of demand for digital avatars of humans in the next few years. 

Whether it’s for gaming, social VR, or enterprise applications, the use cases for accurate physical representations of ourselves are becoming greater just as the technology to create them is maturing.

Lumirithmic is an Imperial College London spinout that can create a realistic 3D version of your face with a rig of multiple iPads - and has recently cracked the ability to create a really good version with just a single smartphone.

CEO Gaurav Chawla explains that while Hollywood movies feature impressively lifelike CGI versions of humans, the rest of us have cartoon avatars - just think of Snap’s Bitmoji, Nintendo’s Miis, or Meta’s much-mocked Mark Zuckerberg avatar. 

Lumirithmic’s first stage was the multi-device rig, but generating an avatar with a single device is the real focus for the future of the tech.

It’s worth watching the BBC’s report for a closer look at how it works. If you’re in the UK, you can access it from 19’26” here.

“It's the world's first 3D selfie in some way, right? You scan yourself with a single phone, and get a 3D version of your face and can rotate it around, put it under different lighting,” says Chawla.

The startup is being publicly coy about exactly how they achieve the smartphone-sourced avatars as they have patents pending alongside a number already granted. But I have seen a video that gives a rough understanding of the process, which doesn’t require any particular model of smartphone. We agreed to keep the specifics off the record.

The video above is a desktop captured avatar. Below you can see an example of a face captured via the single mobile phone technique:

Face the potential

Lumirithmic is built on the work of its co-founder and CTO, Professor Abhijeet Ghosh, who previously developed facial screening technology that was used in the landmark 2009 CGI movie Avatar, and tech that found its way into Google’s Pixel smartphones.

Ghosh and Chawla came together to spin out Ghosh’s latest work at Imperial College into a commercial proposition.

“I said: how can we take this technology to a billion people? How can we make it accessible and cheap for everyone?,” says Chawla.

“My product vision was ‘let's not build any hardware. Let's make use of existing commodity hardware. Can we make use of tablets and phones and not build special rigs and special equipment and electronics and things like that. Because that will mean it won't scale; we won’t get to a billion people.’”

Gaurav Chawla and Abhijeet Ghosh

While gaming and VR leap to mind as obvious use cases for this kind of technology, the first vertical Lumirithmic is addressing commercially is skincare.

Chawla says current consumer-facing beauty tech uses computer vision to, for example, identify your lips so it can apply a simulation of lipstick upon them. By contrast, he says Lumirithmic’s product can identify skin hydration, oxygenation, melanin etc and then simulate the results of applying hydration cream for seven or 14 days.

To bring this type of approach to life, Chawla says the startup is running a pilot programme with L’Oreal to test in-store deployments and a mobile offering, with a view to a global rollout.

Lumirithmic is also exploring opportunities for its tech to allow gamers to insert themselves into the games they play. “It’s the merging of Hollywood quality effects with gaming… as if you were in your own Avengers movie where you are an Avenger”. Chawla also sees potential use cases in advertising and even in video conferencing, where you could present “a better version of you.”

What about Meta and the metaverse?

Some of these use cases sound speculative, but there’s little doubt that the technology has real potential. Interestingly, in describing Lumirithmic’s work Chawla hasn’t used the word ‘metaverse’ yet, so I ask him for his views on the term. 

He says that he’s focused on where the money is this year and next. Skincare and gaming are two mature verticals that consumers spend a lot of money on. While some games and other virtual experiences can be called ‘metaverses’, the Mark Zuckerberg vision of a vast metaverse of interlinked experiences is so far off that the large-scale commercial opportunities aren’t there yet as far as Chawla is concerned.

One company that is very much interested in the metaverse and realistic 3D avatars is of course Zuckerberg’s own Meta. It recently showed off the latest version of its lifelike, but yet to be publicly launched, Codec avatars. They bear more than little similarity to the high-quality output Lumithimic can create. Is Chawla worried about such a well-resourced rival working on similar tech?

“They've created something which looks like it's fit for purpose for their business,” he says. He notes that the demos Meta has shown off have baked-in lighting, so it’s unclear if the assets as they stand can react to changes in (virtual) light for additional realism.

“If the asset is only for the metaverse, maybe it works, but not if you want to do things like actual skin, or the kind of assets that integrate with standard gaming pipelines - that’s not what they’ve created.

“What we’ve created is a re-lightable avatar, fully integrated with existing technologies… it’s not made for the metaverse, it’s made for several applications.” 

From a technical point of view, it’s difficult to accurately assess how different Lumirithmic’s output is to what Meta has shown off as a work-in-progress, but it’s undeniable that Meta’s business focus is very different to Lumirithmic’s, and there’s room in the world for more than one source of realistic avatars.

“Other people might say they can do [avatars] with a phone, but can they do fully re-lightable avatars that can be taken into any digital environment?

“We're doing things with the more generic use case, which means we can spread across different industries and super-focus on faces. Faces are very hard to nail down because you need one tenth of a millimetre accuracy. If I get one millimetre off, I could be creating a very freaky version of you. My eyes would spot that’s not real.”

Future plans, investment, and the geopolitics of face data

Looking a few years from now, aside from generating revenue in all the verticals it addresses, Chawla hopes Lumirithmic can become “a new name people can trust on facial data security”.