Real time object classification isn't a fever dream, neural networks that classify objects can be run on very modest hardware today. The tricky part is making the glasses stylish and not having cords connecting them to your phone.

But the text is blurred and then you hear a little ding and renew your subscription to WorldFacts to see again.

Or what's truly terrifying is walking down High street and all the floating AR ads are generated and targeted just at you.

The same coffee shop-

"Come drink at Liberal Tears fuck dem commies am I right HU-RA"

"This is an LGBTQ-IE Coffee Haus, all beans are triple washed and ethically ground"

Cyberpunk dystopia here we come!

With ads or a subscription fee

[deleted]

my favorite use case to imagine is: you're walking down a city street looking at all the seemingly random people you're passing—but that guy on the left you've actually seen 36 times mostly every Thursday in this spot, and that woman in the right has been in the coffee shop back in your neighborhood with you 8 times this year

the social implications are weeiirddd, it could make big cities start to feel a lot smaller

It probably is up there.

Think of the number of problems that it has to solve.

Medical imaging is pretty straightforward. You’re beaming energy into something and then capturing the image. Humans are doing all of the recognition.

Microprocessors are very difficult but each thing is an evolution of the last.

AR glasses is probably the most difficult integration problem ever solved so far. It is taking a ton of technologies that have only every been used at much larger scales (cars, planes, stationary cameras), shrinking them down, then they’re trying to integrate that with a pair of glasses that are both comfortable and stylish and have a long enough battery to use, and are affordable to buy.

The object recognition is getting pretty decent, but right now phones struggle with this let alone something glasses sized. AND getting the device to do it on device vs the cloud for both privacy and latency. Then once you have object recognition, you have to then filter out which recognized objects are important to display. How does the device figure this out? Watching your eyes? Have you seen a video of human eyes moving? They’re not exactly stable, they jump around constantly and your brain stitches all the image’s together into something cohesive. What about when you want to use gestures to control things, like scrolling through menus on a virtual screen? How does it differentiate between that and when you’re touching an actual screen? So now you need cameras on the inside too, but also super advanced AI onboard to figure out gestures from random hand waving. You also need to build an entire suite of apps that work with this. Smartphones didn’t exactly start out with huge app stores and we didn’t have high expectations for them anyway. Now, our expectations are sky high. We want to replace our phones and laptops with these. So we need that experience but better.

Then on top of that, you need a transparent screen that doesn’t fuck with your vision both while you’re wearing the glasses and permanently that can show high resolution, high frame rate, variably transparent images, video, and text, that can both be stationary in space or motion smoothed depending on the users motion. For example, when you’re sitting and you have virtual computer monitors, you want those monitors to be fixed over a desk, always, even when you turn your head. But when you walk away, you want those monitors to stay there and not move. You want them to be there when you come back though. But you also want them to be able to pop up in the “right” place while you’re on the train so you can work on your commute. You’re facing sideways and the train is moving forward. The glasses need to figure out that motion and allow you to work or watch a movie without a ton of jittering, but it needs enough jittering to match the bounces of the train because otherwise you’ll get motion sickness. It also needs a different interface for while you’re walking and want to do things. It needs stationary menus always in the same place in your field of view (which, define that) but also things that move with the environment, some transparent and others completely opaque.

And it needs to do all of this with absolutely no lag. And not look dorky, and the battery needs to last all day, and it needs to work with different interfaces like keyboards, mice, fingers, eye tracking, figure out when it needs to ignore eye tracking and use the mouse, when it needs to combine the two. And it can’t just be something that only Jeff Bezos can afford.

I should have clarified consumer devices.

Though medical imagining, microprocessors, and datacenter networking are all a major part of AR glasses. For AR glasses to work well, they need cutting edge tech in each of those areas - Space travel, perhaps not.

> I don't work with hardware AR or VR but I am sure the problems are not that complex that you make it out to be. I think the limited processing power is what is limiting it. Display technology has matured enough due to smartphones that they should not be the problem.

AR can't use any existing displays in a consumer viable form, and the optics stack has to be invented mostly from scratch. Optics in particular are very difficult because light is so finnicky and difficult to deal with. Then you have to attain a wide field of view, without distortion, somehow produce pure black with 100% transparency, work dynamically at many focal lengths, with HDR in several tens of thousands of nits (even the world's best HDR TV doesn't go beyond 2000), on a all-day or decently long battery life in a pair of glasses without dissipating too much heat, while stabilizing overlayed content with high precision including high precision environment mapping.

And we haven't even gotten into the main input method for AR, which is likely a brain-computer interface (EMG), software complexity and UX design being much harder due to 3D being a much wider canvas for interactions than a 2D screen.