[D] Did YouTube just add upscaling?

Submitted by Avelina9X t3_10gxs5i in MachineLearning

So, these pictures below are taken from a 144p video on YouTube. You cannot tell me that these aren't CNN upscaling artefacts.

So this raises the question of.... how exactly is this implemented? What model are they using which is tiny enough to run on (i assume) WebGL2? Is it a CNN inside of GLSL shaders? Is it something else? CPU side or GPU side?

And also... how have I not seen a single other person pointing this out, anywhere on the internet. Believe me I looked. Ain't no one talking about this.

EDIT: UPDATE this is doing it in ALL videos in chrome now. It only works in Chrome, not in Discord or Edge, so its not GPU/Windows fuckery. But the strange thing is other friends testing this with the same version of Chrome ***DONT*** have this? And the even stranger thing is... this is running on Intel Integrated Graphics...

https://preview.redd.it/jnjwjzyag7da1.png?width=3240&format=png&auto=webp&v=enabled&s=6a91ab73189c746551f07206419a94c7069dd66c

https://preview.redd.it/6vzyx5f1g7da1.png?width=1182&format=png&auto=webp&v=enabled&s=a03a56e57a4d8b9498224b356b60c1451453195c

https://preview.redd.it/bo36ko40g7da1.png?width=365&format=png&auto=webp&v=enabled&s=cd09523cc574120d0f0e78d2c5b3319d150b1e09

https://preview.redd.it/16zpxwqyf7da1.png?width=333&format=png&auto=webp&v=enabled&s=079d10cb84a70b19cc8cc875f97ae89658fa9056

84

Comments

You must log in or register to comment.

IntelArtiGen t1_j55at5j wrote

I don't really see how and why they would do it. What's the video? You can check the codec they used with right click > "stats for nerds", the codec should say which algorithm they used to encode/decode the video. Using CNNs client-side for this task would probably be quite cpu/gpu intensive and I doubt they would do it (except perhaps if it's an experiment). And using CNNs server-side wouldn't make sense if it increases the size of data download.

It does look like CNN artifacts.

46

Avelina9X OP t1_j55h54h wrote

I think it's clientside. Which is why I mentioned perhaps its using a GLSL based CNN which is absolutely possible in WebGL2 and I've been experimenting with that sort of tech myself (not for upscaling, but just as a proof of concept CNN in WebGL).

15

aidv t1_j57r35e wrote

Then check the JS source code

9

currentscurrents t1_j573tug wrote

They announced upscaling support in Chrome at CES 2023.

>The new feature will work within the Chrome and Edge browsers, and also requires an Nvidia RTX 30-series or 40-series GPU to function. Nvidia didn't specify what exactly is required from those two GPU generations to get the new upscaling feature working, nor if there's any sort of performance impact, but at least this isn't a 40-series only feature.

Interesting though that it's working with your GTX 1660 Ti. Maybe Chrome is implementing a simpler upscaler as a fallback for older GPUs?

Check your chrome://flags for anything that looks related.

30

Avelina9X OP t1_j57b8cf wrote

Its not even running on my 1660 Ti. It's running on my integrated intel graphics. Dedicated graphics is completely idle during this. Aaaand theres nothing related in the Chrome Flags at all.

8

muchcharles t1_j5mbjtb wrote

> Dedicated graphics is completely idle during this.

Are you sure fixed function decoder/upscale stuff is reported in GPU utilization graphs?

1

Avelina9X OP t1_j5o47kr wrote

Not in Task Manager, but at least something would show up in GPU-Z like a clock increase over idle, memory bus usage, GPU utilisation, thermals, power draw, etc etc.

1

NotARedditUser3 t1_j55avp6 wrote

It's possible, if you have one of the very newest graphics cards, that it is your very own hardware doing this and not the website. Dunno

11

Avelina9X OP t1_j55hcfz wrote

It's a GTX 1660 Ti in a tablet laptop. No other video platform does this.

6

Avelina9X OP t1_j55kz89 wrote

Correction: Vimeo does this. It's only in Chrome. But other people also running 109.0.5414.75 (Official Build) (64-bit) (cohort: Stable) do not see this behaviour.

5

2Punx2Furious t1_j58f52a wrote

> other people also running 109.0.5414.75 (Official Build) (64-bit) (cohort: Stable) do not see this behaviour

Might be A/B testing for now.

1

LiquidDinosaurs69 t1_j558t5l wrote

Woah

10

Avelina9X OP t1_j558wqk wrote

Im not going crazy, right? Those are absolutely CNN upscaling artefacts.

9

f10101 t1_j55angk wrote

Not necessarily. This kind of thing will also happen if you chain upscaling, quantization, smoothing and sharpening techniques.

What's the video link?

23

Avelina9X OP t1_j55gyws wrote

Heres the video link https://www.youtube.com/watch?v=yPUGPLAfhTk

But if YouTube are doing A/B testing your hardware/account/IP/region might not be marked for rollout yet.

3

wintermute93 t1_j55igrj wrote

I'm definitely not seeing whatever you're seeing, if I set that video to 144p and full-screen it (1440p monitor) I get an unwatchable mess, not an unwatchable mess that's been upscaled and sharpened like your screenshots.

10

Avelina9X OP t1_j55l3ks wrote

What version of Chrome? What's your region? I'm in the UK, using a GTX 1660 Ti (but Chrome running on Intel Iris graphics) with chrome version 109.0.5414.75 (Official Build) (64-bit) (cohort: Stable)

2

wintermute93 t1_j55rhlf wrote

Same Chrome build as you, GTX 1080. I'm US but usually have my VPN set to somewhere in eastern Europe, no difference after turning it off. I see this.

3

Glum-Bookkeeper1836 t1_j57qhk9 wrote

It might be anything in your particular tech stack, not just YouTube. Very interesting though, about time too.

1

tomvorlostriddle t1_j559v2b wrote

Sure that this is not your hardware clientside?

6

Avelina9X OP t1_j55kvbx wrote

Okay. This is occuring in Chrome, but only Chrome (not Discord or Edge). It happens on YouTube and Vimeo. But this doesn't occur in others' Chromes even though we're on the same version 109.0.5414.75 (Official Build) (64-bit) (cohort: Stable)

0

F1ckReddit t1_j55jxbk wrote

I want to know why this was removed. I saved a screenshot of the post

6

Avelina9X OP t1_j55kcjo wrote

Yeah that's really weird. We're documenting google chrome silently adding upscaling. I think it's a really worthwhile discussion for the community to figure out what model its using as well as how they're implementing it in a cross platform, GPU agnostic way that is buttery smooth and doesn't use a tone of resources.

14

dojoteef t1_j56v9rx wrote

Reddit automatically removed it, likely due to editing the post. Don't know why, but that occasionally triggers their spam filter. I've approved the post again.

6

MisterManuscript t1_j58tvbk wrote

Could be from simple upscaling techniques (e.g. bilinear interpolation) combined with antialiasing. CNNs don't seem feasible given the compute power needed for the number of users using YouTube.

3

DigThatData t1_j58zwdc wrote

do you have images of "baseline" compression artifacts to compare this against?

2

yaosio t1_j58dycj wrote

Microsoft added AI upscaling to Xbox cloud streaming on Edge and it works really well. At least I think it's AI upscaling, it could be something like FSR. Either way it looks really good. If Microsoft can do it for lag sensitive gaming then Google can do it for regular videos.

1

ZenDragon t1_j58pumo wrote

Ok so I'm not losing my mind, I think I got this recently too.

1

SlickBlueML t1_j59buju wrote

Testing features with sub groups of users before fully rolling out is common practice at Google, I’d imagine that’s what’s going on. Cool stuff

1

I_will_delete_myself t1_j5b4ccq wrote

It doesn't make any sense to run neural network on the client side at all. Youtube takes a moment to process your video before it gets uploaded, which is probably when their deep learning algorithms get to work. After that you just save the frames and don't run the neural networks again.

This is a valid guess because it takes a lot longer to upload a video on Youtube in comparison to other platforms that do no checks at all.

1

aidv t1_j57rrqa wrote

I believe that this might be fully true. I’ll tell you why:

I don’t know how many times I’ve felt like the voice of speakers in videos have sounded like they are AI generated.

Like, voices of people that I subscribe to.

I am was convinced that they were doing some AI fuckery, and this post only pretty much confirms it.

It’s probably to save bandwidth and storage on site, so makes sense.

−2

yaosio t1_j58f7g6 wrote

That's just the way they talk. One popular youtuber does it so everybody does it. It's like radio voice or news anchor voice.

1

aidv t1_j58fczi wrote

No you don’t understand. When working with machine learning for long enough, you start to recognize what the artifacts and errors sound snd look like.

−2