Intel SVT-AV1

[rwill]

Originally Posted by Z2697

What is the mysterious SEI in the stream beginning?
This is not x265, right?

That is a compressed configuration record. Its the HEVC encoder from my employer.

[GeoffreyA]

Originally Posted by benwaggoner

For x265 you really don't need to reduce deblock like x264's --tune film does. Strong intra smoothing is generally fine too. SAO helps when otherwise ringing artifacts would occur, so it's bitrate dependent, but I expect would be net better at CRF 25.5. x265 is also somewhat more content dependent in tuning. For grainy content --rskip 0 can be a lot better. --aq-mode 4 can be better than the default 2, at least for 8-bit SDR content. 2 seems best for 10-bit HDR. --rd 4 is better for grainy UHD resolutions, but 6 is generally preferable for cleaner and lower resolutions (where the low-pass filter of downscaling improves the signal-to-noise ratio. Fine grain at 2160p can make the image almost all noise with little signal, which can freak out --rd 6).

I don't think you'll get 10-bit encoding in x264 without specifying High10, or in x265 without Main10. But perhaps the behavior changed without my noticing. MediaInfo reveals all.

Thanks for the tips. I'll keep them in mind when encoding x265. Here, I wanted to keep the command as simple and untweaked as possible, apart from SAO, to bring out more of a "default" x265. I set --rd to 4 because I remember 6 creating artefacts in another encode, but yes, some of these settings depend heavily on content.

Originally Posted by rwill

I was watching it in somewhat optimal viewing conditions and the blurry frames felt like someone slaps me upside the head each time. Cannot recommend. I am not so sure what is causing it because according to the command line its a CRF encode so quantization should be almost constant within a scene? I cannot work on AV1, otherwise I would investigate.

I think the problem is temporal or keyframe filtering, which can get even worse in libaom. SVT-AV1-PSY seems to have largely overcome it, but is, perhaps, still constrained by AV1's nature. VVC, through vvenc, comes out even worse in this regard. It's also disaster for many of these encoders, x264 included, in two-pass mode. x265's rate control was solid in two passes.

[Z2697]

Originally Posted by benwaggoner

I don't think you'll get 10-bit encoding in x264 without specifying High10, or in x265 without Main10. But perhaps the behavior changed without my noticing. MediaInfo reveals all.

With FFmpeg, that's the default behavior, or maybe the only behavior, as the bitdepth conversion code of x265 is not included in the "core encoding function" that FFmpeg wraps around, it will only try to keep the input pixel format, the profile is automatically determined based on that.
That's probably also how almost all of the encoders in FFmpeg works, when some input formats not supported by an encoder, the pixel format conversion is done by auto-insterted swscale filter.
So if the x265 library that linked to FFmpeg is compiled with 10bit support, Main10 profile will be used in this case. The profile parameter doesn't work, even if you specified Main10, but input 8bit, the result is Main profile.

[Z2697]

Originally Posted by rwill

That is a compressed configuration record. Its the HEVC encoder from my employer.

The quality is suprising. How's the performance? Is it software or hardware encoder?

[Z2697]

Originally Posted by GeoffreyA

Thanks for the tips. I'll keep them in mind when encoding x265. Here, I wanted to keep the command as simple and untweaked as possible, apart from SAO, to bring out more of a "default" x265. I set --rd to 4 because I remember 6 creating artefacts in another encode, but yes, some of these settings depend heavily on content.



I think the problem is temporal or keyframe filtering, which can get even worse in libaom. SVT-AV1-PSY seems to have largely overcome it, but is, perhaps, still constrained by AV1's nature. VVC, through vvenc, comes out even worse in this regard. It's also disaster for many of these encoders, x264 included, in two-pass mode. x265's rate control was solid in two passes.

I tried disable temporal filter with enable-tf=0 but the blurring (or more appropriately, sudden quality drop?) still occurs.

[rwill]

Originally Posted by Z2697

The quality is suprising. How's the performance? Is it software or hardware encoder?

Its a software encoder. Performance wise, well, the overall architecture goal was to not have one of the fastest software encoders but aims it to be one of the prettiest if one is willing to spend extra compute... but not really slower than x265 'veryslow'.

[GeoffreyA]

Originally Posted by Z2697

I tried disable temporal filter with enable-tf=0 but the blurring (or more appropriately, sudden quality drop?) still occurs.

It's puzzling. I tried mainline SVT-AV1 and libaom but it shows up in one form or another.

[juliobbv]

Originally Posted by GeoffreyA

It's a bitrate virus with that grain. I was surprised to see even x265 get knocked so hard. VVC takes the easy route of just denoising the whole thing, despite SAO being turned off. Come to think of it, I think there's a MT-something setting that disables the temporal denoising in vvenc. Should try that. As for SVT-AV1-PSY, there is certainly a wider range of settings to adjust these results. Simply changing from one tune to another, for instance, makes a visual difference. All in all, it's a big departure from libaom, though.

I tried your clip with SVT-AV1-HDR's new film grain tune (--tune 3): https://drive.google.com/file/d/1G5z...ew?usp=sharing

Visual quality is significantly better than the previous AV1 clip.

Edit: another version, with FGS `--film-grain 30`: https://drive.google.com/file/d/1ggp...ew?usp=sharing

[GeoffreyA]

Originally Posted by juliobbv

I tried your clip with SVT-AV1-HDR's new film grain tune (--tune 3): https://drive.google.com/file/d/1G5z...ew?usp=sharing

Visual quality is significantly better than the previous AV1 clip.

Edit: another version, with FGS `--film-grain 30`: https://drive.google.com/file/d/1ggp...ew?usp=sharing

Indeed, stepping through the clip frame by frame, the regular blurring and dropping of quality have been defeated.

Tune 3 is equivalent to setting these parameters: --tune 0 --enable-tf 0 --enable-restoration 0 --enable-cdef 0 --spy-rd 1 --psy-rd 4.00 (SDR), 6.00 (HDR)

Is temporal filtering, the loop-restoration filter, or the CDEF responsible for the artefacts we were seeing; or was it a combination of the three?

[BlueSwordM]

Originally Posted by GeoffreyA

Indeed, stepping through the clip frame by frame, the regular blurring and dropping of quality have been defeated.

Tune 3 is equivalent to setting these parameters: --tune 0 --enable-tf 0 --enable-restoration 0 --enable-cdef 0 --spy-rd 1 --psy-rd 4.00 (SDR), 6.00 (HDR)

Is temporal filtering, the loop-restoration filter, or the CDEF responsible for the artefacts we were seeing; or was it a combination of the three?

Nope. tune 0/3 have a noise detector that disables CDEF/restoration filters if the image is too noisy.

On noisy clips like the one julio linked, the noise detector works quite well at disabling those encoding features; you can test it by enabling/disabling these features with tune 0/3 on small isolated noisy clips and seeing that many of them are bit exact.

Anyway, the main reasons behind why Julio's encode looks much better are that Julio did some fancy HDR stuff regarding chroma and variance boost, and he used much more aggressive settings than what I gave you earlier:
https://github.com/juliobbv-p/svt-av...handle.c#L4641

I gave you

--psy-rd 0.8

.
He gave you

--psy-rd 6.0 --spy-rd 1

, so much stronger psy-rd influence and the highest level for spy-rd.

Very aggressive settings for this source.

[Boulder]

Originally Posted by BlueSwordM

Nope. tune 0/3 have a noise detector that disables CDEF/restoration filters if the image is too noisy.

On noisy clips like the one julio linked, the noise detector works quite well at disabling those encoding features; you can test it by enabling/disabling these features with tune 0/3 on small isolated noisy clips and seeing that many of them are bit exact.

Anyway, the main reasons behind why Julio's encode looks much better are that Julio did some fancy HDR stuff regarding chroma and variance boost, and he used much more aggressive settings than what I gave you earlier:
https://github.com/juliobbv-p/svt-av...handle.c#L4641

I gave you

--psy-rd 0.8

.
He gave you

--psy-rd 6.0 --spy-rd 1

, so much stronger psy-rd influence and the highest level for spy-rd.

Very aggressive settings for this source.

What puzzles me is that the changes were made into a separate fork and not in the same one by adding a new tune option for grain retention. As you've co-operated already earlier, are you planning to have one encoder with best of both worlds?

[juliobbv]

Originally Posted by GeoffreyA

Is temporal filtering, the loop-restoration filter, or the CDEF responsible for the artefacts we were seeing; or was it a combination of the three?

It can be boiled down to these factors:
- Significantly more aggressive psy-rd strength (4.00)
- Better psy-rd algo in general (modulate strength based on frame temporal layer index)
- Some minor tweaks to rdoq to keep chroma information better

[juliobbv]

Originally Posted by Boulder

What puzzles me is that the changes were made into a separate fork and not in the same one by adding a new tune option for grain retention. As you've co-operated already earlier, are you planning to have one encoder with best of both worlds?

It's not puzzling. As previously mentioned, -PSY is discontinued (and archived on GitHub), so I've been making improvements at my own pace on my personal repo. I have a full-time job, so expect future changes to be much more sporadic.

Think of -HDR as "here are my tweaks that I use to encode AV1 videos that I've found helpful". Development is more relaxed, and there are no expectations on future releases or bug-fixes. I'd still encourage people to try out -HDR because my changes have been obviously helpful though.

I do intend my changes to eventually find their way to mainline SVT-AV1, whether it's me who ports the changes, or another person. But again, a demanding full-time job will necessarily make such port process much slower.

[BlueSwordM]

Originally Posted by Boulder

What puzzles me is that the changes were made into a separate fork and not in the same one by adding a new tune option for grain retention. As you've co-operated already earlier, are you planning to have one encoder with best of both worlds?

Yes of course.

I do have a separate repo called svt-av1-psyex, where I now work as the main "svt-av1-psy" repo for features that are intended to be pushed to mainline.

I'm going to be pushing a few helpful updates before I force myself to work on the PR I submitted since I do have a bit more free time than the last few months.

As Julio helpfully stated, most of the current changes will be pushed to mainline, all fixed and tidied up neatly for mainstream domination.

[Boulder]

Thanks, both of you and let's hope you have the needed time to work on the current and future PRs

My question was prompted by the fact that the new "tune grain" feature looks like it could be incorporated to psyex as a new tune method, and the PQ variance boost curve as its own so the users can choose between SDR/HDR and clean/grainy settings without having to change the encoder binary

[juliobbv]

Originally Posted by BlueSwordM

Anyway, the main reasons behind why Julio's encode looks much better are that Julio did some fancy HDR stuff regarding chroma and variance boost...

BTW, this source is SDR, so the HDR Variance Boost and chroma stuff doesn't apply here. psy-rd is set to 4.00, which is the default for SDR content.

[juliobbv]

Originally Posted by Boulder

Thanks, both of you and let's hope you have the needed time to work on the current and future PRs

My question was prompted by the fact that the new "tune grain" feature looks like it could be incorporated to psyex as a new tune method, and the PQ variance boost curve as its own so the users can choose between SDR/HDR and clean/grainy settings without having to change the encoder binary

BTW, there will be cross-pollination between PSYEX and HDR, so you can expect features from one repo to be cherry-picked to the other.

[juliobbv]

Originally Posted by benwaggoner

There was a blog post link a few posts back. It sounds highly amicable, thoughtful, and gradual. Life stuff that made it hard to keep on spending many hours a week coding for free was part of it. And something that sounds something like "I'll be starting a new job working on psychovisual encoding so I can help migrate what I did but can't introduce new algorithms."

This is correct. We're still friends and everything, and we're keeping in touch (on Discord mostly). Gianni is currently working on a non-FOSS encoder, so his involvement on SVT-AV1 is now much more restricted regarding new feature development. I now have a full-time video encoding job, so I have to be more mindful with how I spend my free time.

Anyway, Blue and I now have each our own SVT-AV1-PSY fork. Blue's is -PSYEX, and mine is -HDR. We decided that separate repos made the most sense for us, but new features and improvements can be cherry-picked between them at any time. We just have different expectations on releases, rebase timing onto mainline, and bug-fixing priorities.

[GeoffreyA]

Originally Posted by BlueSwordM

Nope. tune 0/3 have a noise detector that disables CDEF/restoration filters if the image is too noisy.

On noisy clips like the one julio linked, the noise detector works quite well at disabling those encoding features; you can test it by enabling/disabling these features with tune 0/3 on small isolated noisy clips and seeing that many of them are bit exact.

Anyway, the main reasons behind why Julio's encode looks much better are that Julio did some fancy HDR stuff regarding chroma and variance boost, and he used much more aggressive settings than what I gave you earlier:
https://github.com/juliobbv-p/svt-av...handle.c#L4641

I gave you

--psy-rd 0.8

.
He gave you

--psy-rd 6.0 --spy-rd 1

, so much stronger psy-rd influence and the highest level for spy-rd.

Very aggressive settings for this source.

Originally Posted by juliobbv

It can be boiled down to these factors:
- Significantly more aggressive psy-rd strength (4.00)
- Better psy-rd algo in general (modulate strength based on frame temporal layer index)
- Some minor tweaks to rdoq to keep chroma information better

I appreciate the explanations. Yesterday, I tested this out, and found that disabling TF and the loop filters did not eliminate the temporal "flickering," squaring with your notes that psy-rd was key. Thanks for all the work, Blue and Julio. We hope that time could be found to merge these welcome improvements into mainline so that encoders could use one encoder and call it a day.

I do have a separate repo called svt-av1-psyex, where I now work as the main "svt-av1-psy" repo for features that are intended to be pushed to mainline.

Blue, going forward, should people target your repo for the "main" SVT-AV1-PSY branch, or stick with the archived 3.0.2 version?

[Leo 69]

Guys, is there some kind of zones parameter? How do I encode separate parts of videos with different settings with this codec like it's done with x265 for example? Or is av1an the only option for this?