Psy RDO: Official testing thread (version 0.6 out!)

[MfA]

Small moving objects as focal points in a scene seem pretty much worst case scenario for this method.

[Dark Shikari]

Originally Posted by MfA

Small moving objects as focal points in a scene seem pretty much worst case scenario for this method.

Which is somewhat odd, because the research actually shows that moving objects should be worse quality than they currently are (e.g. motion-based adaptive quantization).

[akupenguin]

Originally Posted by Dark Shikari

Which is somewhat odd, because the research actually shows that moving objects should be worse quality than they currently are (e.g. motion-based adaptive quantization).

Not if they're the focus. Then you track that one object and the rest of the frame counts as moving.

[Dark Shikari]

Originally Posted by akupenguin

Not if they're the focus. Then you track that one object and the rest of the frame counts as moving.

Hmm, true. Of course, its hard to make an algorithm that decides what is the "focus" of the scene...

[Dark Shikari]

New patch. Updates:

1. Ignores chroma. This speeds things up and didn't seem to change things visually at all. Even if we used chroma, it should probably be weighted a lot lower anyways.

2. Raises B-RDO threshold.

3. Now available as a commandline option (--rdcmp).

[jethro]

Nice discussion. The differences between what people consider to be better might be due to their displays (because i don't think our eyes differ this much). For example, in x264 encodes I don't see much ringing on my monitor but i definitely see blurring and ugly smearing of details (esp without VAQ). This new RD seems to make pictures more 'dirty' which is similar in effect to dct noise mentioned in xvid encodes. Of course, there are also preferences but I can't believe displays aren't a factor.

Originally Posted by Merlin7777

Truthfully, I rather prefer the second picture, honestly. The jaggies introduced along all of the edges are making my eyes bleed. If this can be fixed by lowering aq, then great! But frankly, I will take something over the jaggies. Sorry DS.

Yes, the jaggies come from VAQ. I noticed that default AQ 1.0 seems too strong for low resolutions as object edges get few bits. I often find myself using AQ 0.5-0.7 for something like 720x304. For HD res actually AQ 1.0 is fine and AQ 0.5 gets blocky.

EDIT: I'd like to add that I also prefer details over artifacts

[Dark Shikari]

Here's a really dramatic example of Psy RDO's effectiveness:

No Psy RDO

Psy RDO

[Atak_Snajpera]

Screenshots from frame 600

PSY ON


PSY OFF

[IgorC]

I think it also depends of resolution. My sample had dvd resolution while new psy rdo maybe more optimized for HD. Artifacts look different for HD and DVD. And of course it depends of display and other conditions of watching samples.

[Sagittaire]

Originally Posted by Dark Shikari

Edit: Oh, another possible issue is that FGO lowers B-frame quantizers and adjusts RD thresholds, which this doesn't (yet). Thus comparisons could potentially get misleading when trying to compare just the RD metric alone.

After size analyse I think that it's simply a different complexity analysis. For really grainy picture (with really fine grain) fgo mode will use really more bit than psy mode.

vs vs

PSY vs FGO vs Elecard

31 832 vs 64 267 vs 63 751 bits (all frame are Pframe)

And here FGO produce by far the best quality ...

[Razorholt]

Originally Posted by Sagittaire

After size analyse I think that it's simply a different complexity analysis. For really grainy picture (with really fine grain) fgo mode will use really more bit than psy mode.

vs vs

PSY vs FGO vs Elecard

31 832 vs 64 267 vs 63 751 bits

And here FGO produce by far the best quality ...

How about the original frame? How many bits?

[Adub]

I agree. FGO does seem to come out better here.

[Dark Shikari]

Originally Posted by Merlin7777

I agree. FGO does seem to come out better here.

Given that its frame size is double that of psy RDO, how am I not surprised?

If its a B-frame, you can compensate for that by setting pbratio lower... that's what FGO does anyways and its probably wrong to put anything into mainline x264 that changes settings like that when the user should be in control of it.

If it isn't, maybe you didn't use the same bitrate for the two clips...?

Also, note that here, the Elecard frame isn't even the same frame of the video...

[Sagittaire]

Originally Posted by Razorholt

How about the original frame? How many bits?

Source is FFV1 master ...

[Sagittaire]

Originally Posted by Dark Shikari

Given that its frame size is double that of psy RDO, how am I not surprised?

Yes not surprising for quality. The surprise come from decision. Here FGO seem make better choice than PSY mode. It's a really complexe frame with very fine but visible grain (grain on blue sky)

If its a B-frame, you can compensate for that by setting pbratio lower... that's what FGO does anyways and its probably wrong to put anything into mainline x264 that changes settings like that when the user should be in control of it.

Only Pframe here.

If it isn't, maybe you didn't use the same bitrate for the two clips...?

6900 Kbps for overall bitrate (more than 200 000 frames).

Also, note that here, the Elecard frame isn't even the same frame of the video...

yes because it's bframe for Elecard. Anyway all the Pframe for the encoded sequence are the same size with equivalent grain complexity for picture source. So the comparison is correct here.

[Razorholt]

Originally Posted by Sagittaire

Source is FFV1 master ...

Thanks Sagittaire. I just wanted to find out whether PSY was actually increasing the size of the video, as FGO does.

[Dark Shikari]

Originally Posted by Sagittaire

Yes not surprising for quality. The surprise come from decision. Here FGO seem make better choice than PSY mode. It's a really complexe frame with very fine but visible grain (grain on blue sky)

I'm proposing that FGO has absolutely nothing to do with the double-size frame and that there has been user error here, such as comparing B-frames with different pbratio or encoding with different bitrates. Especially since the clips you posted earlier were not even of the same video clip, I don't exactly trust that you've done everything right here...

[Sagittaire]

Originally Posted by Dark Shikari

I'm proposing that FGO has absolutely nothing to do with the double-size frame and that there has been user error here, such as comparing B-frames with different pbratio or encoding with different bitrates. Especially since the clips you posted earlier were not even of the same video clip, I don't exactly trust that you've done everything right here...

Well it's just fast size partition with mkvmerge without range selection. Anyway the overall bitrate for the complete encoding are exactly the same: 6.9 Mbps. Sample come from complete encoding (more than 200 000 frame). It's just local difference for this frame. For the Pframe 47 499 I have another size: 122 889 vs 130 734 vs 99 707, and for the 64 190 frame another size: 82 008 vs 44 114 vs 74 374.


Source is Casino Royal with really different grain level.

[ToS_Maverick]

about the frequencies and the quantization...

that sounds like Psy RDO works somehow like a custom matrix? I understand that a matrix uses fixed values, but it goes into the same direction, be more "kind" to the lower frequencies, to avoid blocking, preserving a good "look"...

[Sagittaire]

Originally Posted by Dark Shikari

I'm proposing that FGO has absolutely nothing to do with the double-size frame and that there has been user error here, such as comparing B-frames with different pbratio or encoding with different bitrates. Especially since the clips you posted earlier were not even of the same video clip, I don't exactly trust that you've done everything right here...

Well I find the problem ... it's a Rate Control problem between first and second pass. With the same final bitrate crf mode produce really better result.


2 pass mode with fast first pass:

x264.exe --threads 3 --thread-input --keyint 24 --min-keyint 2 --vbv-maxrate 20000 --vbv-bufsize 30000 --mvrange 511 --level 4.1 --bframe 3 --b-pyramid --bime --weightb --ref 1 --mixed-refs --direct auto --deblock -2:-2 --bitrate 6900 --pass 1 --stats "casino.log" --ipratio 1.10 --pbratio 1.10 --partitions "all" --8x8dct --me "hex" --subme 5 --trellis 0 --aq-mode 0 --aq-strength 0.00 --aud --nal-hrd --sar 1:1 --cqmfile Sagittaire.cfg --progress -o NUL Lossless.avs

And after I use always the same casino.log stat for all the encoding.

x264.exe --threads 3 --thread-input --keyint 24 --min-keyint 1 --vbv-maxrate 20000 --vbv-bufsize 30000 --mvrange 511 --level 4.1 --bframe 3 --b-pyramid --b-rdo --bime --weightb --ref 3 --mixed-refs --direct auto --deblock -2:-2 --bitrate 6900 --pass 2 --stats "casino.log" --ipratio 1.10 --pbratio 1.10 --partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --aq-mode 2 --aq-strength 1.00 --no-fast-pskip --no-dct-decimate --aud --nal-hrd --sar 1:1 --cqmfile Sagittaire.cfg --zone 201560,207442,b=0.33 --progress -o 1080p_3.264 Lossless.avs


Quality mode:

x264.exe --threads auto --thread-input --keyint 24 --min-keyint 1 --crf %E_BR% --vbv-maxrate %MAX_BR% --vbv-bufsize %BUF_BR% --mvrange 511 --level 4.1 --bframe 3 --b-pyramid --b-rdo --bime --weightb --ref 3 --mixed-refs --direct auto --deblock -2:-2 --ipratio 1.10 --pbratio 1.10 --partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --no-fast-pskip --no-dct-decimate --aud --nal-hrd --sar 1:1 --cqmfile Sagittaire.cfg --aq-strength 1.00 --aq-mode 2 --zone %CRE_FR%,%END_FR%,b=0.33 --progress -o 1080p_Q1.264 %E_SRC%


With 2 pass encoding and exactly the same setting than crf mode (and same final bitrate) I have completely different Rate Control (for example 31 832 bits vs 70 926 bits for frame 20 491). VAQ + PSY seem imply major modification in RC between first and second pass. It's perhaps necessary to use VAQ and PSY in first pass too ... ???