Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th December 2019, 23:43   #1  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 982
XviD 1.3.7 is out!

https://downloads.xvid.com/downloads...7-20191228.exe

XviD Codec 1.3.7 changelog:
  • xvidcore
  • Fix for a regression in initializing the Inter matrix with MPEG
  • Quantization
XviD Codec 1.3.6 changelog:
  • Fix for various, long-standing and potentially critical security vulnerabilities in the decoder (credit to OSS-Fuzz)
  • Always use .text sections in nasm code for macho target

Last edited by hajj_3; 11th January 2020 at 03:31.
hajj_3 is offline   Reply With Quote
Old 8th December 2019, 01:40   #2  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,098
I will compile and release the VfW stuff "as-soon-as-possible".

Too bad they have chosen an installer so stupid that cannot be unpacked -_-
filler56789 is offline   Reply With Quote
Old 20th December 2019, 23:27   #3  |  Link
Liisachan
李姗倩 Lǐ Shān Qin
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,230
Quote:
Originally Posted by filler56789 View Post
I will compile and release the VfW stuff "as-soon-as-possible".

Too bad they have chosen an installer so stupid that cannot be unpacked -_-
You mean it's a known problem that VfW doesn't work well with this installer? After I installed 1.3.6 using the above link, XviD.avi (both encoded by 1.3.5 and 1.3.6) becomes blurry and sometimes very blocky. So I re-installed 1.3.5, and the problem is gone.
Encoded with 1.3.6 (Bitstream Version 68) + Decoded by 1.3.5 (Bitstream ver 67) = seems fine (at least not blocky).
Liisachan is offline   Reply With Quote
Old 26th December 2019, 14:49   #4  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,098
@Liisachan:

I actually meant an installer so stupid that cannot be unpacked.

In the goode olde days the Xvid "bundles" contained only the core DLL, the VfW applet and the DirectShow decoder. Today the installer contains additional garbage and cannot be unpacked because the inventors of that stupid installer (Bitrock InstallBuilder) are really stupid.

But thanks for reporting the encoding (or decoding? ) issue in the newest Xvid, I wasn't aware of that. Probably this time I won't even try to compile it, since it's buggy

I stopped using Xvid years ago, for encoding to MPEG-4 ASP I will keep using DivX 6.9.2 "forever". But there are many people who still prefer Xvid, this is the reason why I used to build and release only what really matters to the Windows folks i.e., the core DLL, the VfW applet and xvid_encraw.exe.
filler56789 is offline   Reply With Quote
Old 28th December 2019, 16:45   #5  |  Link
blob2500
Registered User
 
Join Date: Sep 2007
Location: Italy
Posts: 9
Xvid 1.3.7

Quote:
Originally Posted by Liisachan View Post
You mean it's a known problem that VfW doesn't work well with this installer? After I installed 1.3.6 using the above link, XviD.avi (both encoded by 1.3.5 and 1.3.6) becomes blurry and sometimes very blocky. So I re-installed 1.3.5, and the problem is gone.
Encoded with 1.3.6 (Bitstream Version 68) + Decoded by 1.3.5 (Bitstream ver 67) = seems fine (at least not blocky).
Xvid 1.3.7 is out. Xvid bitstream version 69. It fix the issue.

https://downloads.xvid.com/downloads...7-20191228.exe

Quote:
Xvid-1.3.7-20191228 _Final Release_


This is Xvid 1.3.7 bugfix release. It replaces the previous 1.3.6 stable release.

Changes since 1.3.6:

* xvidcore
- Fix for a regression in initializing the Inter matrix with MPEG
Quantization

Changes since 1.3.5:

* xvidcore
- Fix for various, long-standing and potentially critical security
vulnerabilities in the decoder (credit to OSS-Fuzz)

- Always use .text sections in nasm code for macho target

https://www.xvid.com/

Last edited by blob2500; 28th December 2019 at 17:15.
blob2500 is offline   Reply With Quote
Old 31st December 2019, 04:33   #6  |  Link
Liisachan
李姗倩 Lǐ Shān Qin
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,230
Quote:
Originally Posted by filler56789 View Post
In the goode olde days the Xvid "bundles" contained only the core DLL, the VfW applet and the DirectShow decoder. Today the installer contains additional garbage and cannot be unpacked because the inventors of that stupid installer (Bitrock InstallBuilder) are really stupid.

But thanks for reporting the encoding (or decoding? ) issue in the newest Xvid, I wasn't aware of that. Probably this time I won't even try to compile it, since it's buggy

I stopped using Xvid years ago, for encoding to MPEG-4 ASP I will keep using DivX 6.9.2 "forever". But there are many people who still prefer Xvid, this is the reason why I used to build and release only what really matters to the Windows folks i.e., the core DLL, the VfW applet and xvid_encraw.exe.
Okay. While of course many people are now using newer codecs like x264, I personally like the simple VfW interface e.g. for subtitle typesetting, so AVI is the simplest choice, and I appreciate it that a few people like you are still maintaining those now old-fashioned things.

Quote:
Originally Posted by blob2500 View Post
Xvid 1.3.7 is out. Xvid bitstream version 69. It fix the issue.

https://downloads.xvid.com/downloads...7-20191228.exe
Thanks for addressing the problem quickly. I'll test that shortly. Was that a decoder-side problem? I.e. if someone encoded AVI using 1.3.6, they don't have to re-encode?

EDIT (2020-01-01)
The installer writes bitrock_installer*.log in %TEMP% (or %TMP%), and creates a lot of "rollbackBackupDirectory*" in Program Files\Xvid (at least on my Win7). Like filler56789 said it's not exactly a simple, minimalist installer, although it works fine for me.

Last edited by Liisachan; 2nd January 2020 at 18:05.
Liisachan is offline   Reply With Quote
Old 2nd January 2020, 12:45   #7  |  Link
blob2500
Registered User
 
Join Date: Sep 2007
Location: Italy
Posts: 9
Quote:
Originally Posted by Liisachan View Post
Thanks for addressing the problem quickly. I'll test that shortly. Was that a decoder-side problem? I.e. if someone encoded AVI using 1.3.6, they don't have to re-encode?
The problem was encoder-side (too), In MPEG matrix quantization internal settings.
( view here: http://websvn.xvid.org/cvs/viewvc.cg...rix.c?view=log )

If you encode with 1.3.6 and you set a MPEG matrix (custom or standard), the inter-matrix values in encoded file are wrongly changed. This reflects it to decoder-side too, because decoder use another matrix values, instead correct values to decode a file ( all MPEG-4 ASP files).

If you used H.263 quantization in your encodings with 1.3.6 I think re-encoding is not necessary. If you used MPEG... I think yes because quantization is wrong.
You can analyze your 1.3.6 encoded files with very well known Avinaptic free software, if you want see their conditions (matrix primarily)

Last edited by blob2500; 2nd January 2020 at 12:56.
blob2500 is offline   Reply With Quote
Old 2nd January 2020, 17:24   #8  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 1,163
Although seeing a new release of xvid in 2020 it's a bit weird, I thank you for this bugfix release.
I gotta say that I still use .avi and xvid when I release several version of a file once a week on a website I've been working on ever since 2006 in my spare time.
Back in 2006 there was SD only and I used to release SD xvid BT601 version muxed in .avi and I kept that release.
Nowadays I encode the very same file in SD BT601 SDR xvid, H.264 BT709 SDR HD, H.264 BT709 SDR FULL HD and H.265 BT2020 SDR 4K.
Since it's a fan-based thing from my early age when I was young and I kept it going "for fun", xvid is still totally fine.
FranceBB is offline   Reply With Quote
Old 3rd January 2020, 04:53   #9  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,906
Gimme One Good Reason...

Still using XviD VFW from time to time, my preferred version is a Celtic Druid build from 2008:
http://esby.free.fr/CelticDruid/mirr...ead.MTKVAQ.exe

I never liked Koepi's builds at that time. They had memory leaks and crashed DVD2SVCD when used together with the CCE SP encoder. Switching to Celtic Druid's builds solved it, and the latest of his builds even has the VAQ patch.

I use it under the latest stable 32-bit StaxRip version which uses VDubMod as the host application. My settings are for max compatibility with HW DVD players (Advanced Simple L5, H.263, VAQ ON, QP and GMC OFF, BVOPs ON at 2, Packed Bitstream OFF). With these settings I never had any problems, with a quantizer of 3 the quality was always pretty good.

Now this latest version 1.3.7 got me curious, I made a few tests using identical settings to my old setup. But I could not see any advantage. The resulting files were a little bigger, encoding time was a little slower, and visual quality was the same.

So why in the world should I switch to this latest version? Because newer is always better? Just give me one good reason...

Last edited by manolito; 3rd January 2020 at 04:57.
manolito is offline   Reply With Quote
Old 3rd January 2020, 05:28   #10  |  Link
estir
Registered User
 
Join Date: Dec 2019
Posts: 3
Why not make PSNR-HVS-M the default metric? Better quality for only slight performance penalty.
estir is offline   Reply With Quote
Old 3rd January 2020, 12:59   #11  |  Link
blob2500
Registered User
 
Join Date: Sep 2007
Location: Italy
Posts: 9
Quote:
Originally Posted by manolito View Post
My settings are for max compatibility with HW DVD players (Advanced Simple L5, H.263, VAQ ON, QP and GMC OFF, BVOPs ON at 2, Packed Bitstream OFF). With these settings I never had any problems, with a quantizer of 3 the quality was always pretty good.
AS@L5 is not good for max compatibility with all (and olds) standalone player, because it has VBV maxbitrate: 8.000Mbit/s; For max compatibility VBV bitrate must be 4.854 Mbit/s
For this there are "sub-profiles" of AS: (DivX)Home Theater and Xvid Home profiles (Xvid Home set to max consecutive b-frames = 1 or 2, and Quantization matrix set to MPEG standard/H.263 only); Celtic's and Jawor's builds have good and similar MTK profiles too.

With 2 B-vops: Packed bitstream must be OFF to play fine in most player, but must be ON to play fine in others...
Max compatibility is: 1 B-vops with packed bitstream ON.

--

I use Xvid (1.3.7 now) for my HD encodings/rips (source: usb of DVB-S2 TV decoder) for many years.
I use Xvid HD 720 profile set to compatibility with 720 HD profile of DivX: Max 2 B-vops, packed bitstream (is forced) and I set VAQ, no qpel, standard PAR etc...Custom Matrix and PB are not never a problem in HD DVD/BR player (at least in certified DivX, and in most multimedia players), and I use Sharktooth EQM V3LR matrix: quality is very good. It's a HVS full compliant matrix too
I don't use HVS PSNR metric: encoding is too much slow , relate to quality that produces.


Quote:
So why in the world should I switch to this latest version? Because newer is always better? Just give me one good reason...
IMHO 1.3.2, and newer versions, make best quality (slightly) in 2-pass encoding, in relation to the file size.

Last edited by blob2500; 3rd January 2020 at 13:54.
blob2500 is offline   Reply With Quote
Old 3rd January 2020, 19:47   #12  |  Link
kalehrl
Registered User
 
Join Date: Feb 2011
Posts: 295
Has anyone compiled the latest version of xvid encraw for use in, for example, MeGUI?
kalehrl is offline   Reply With Quote
Old 3rd January 2020, 20:32   #13  |  Link
estir
Registered User
 
Join Date: Dec 2019
Posts: 3
Isn't PSNR-HVS-M is pretty much mandatory with a custom HVS matrix and/or VAQ?
estir is offline   Reply With Quote
Old 3rd January 2020, 21:21   #14  |  Link
Liisachan
李姗倩 Lǐ Shān Qin
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,230
Quote:
Originally Posted by blob2500 View Post
The problem was encoder-side (too), In MPEG matrix quantization internal settings.
( view here: http://websvn.xvid.org/cvs/viewvc.cg...rix.c?view=log )

If you encode with 1.3.6 and you set a MPEG matrix (custom or standard), the inter-matrix values in encoded file are wrongly changed. This reflects it to decoder-side too, because decoder use another matrix values, instead correct values to decode a file ( all MPEG-4 ASP files).

If you used H.263 quantization in your encodings with 1.3.6 I think re-encoding is not necessary. If you used MPEG... I think yes because quantization is wrong.
You can analyze your 1.3.6 encoded files with very well known Avinaptic free software, if you want see their conditions (matrix primarily)
Thank you. I did use the standard MPEG matrix when I experienced the problem. So it was an accidental typo in source code, i.e. while changing elem>>1 to max(1,elem)>>1 to make sure the value is non-negative (?), it happened to become max(1,elem>>1). If I understand correctly, the value was 1 for elem<=1 when it should have been 0.

AVI/xvid is still handy and useful for me, as there are a few applications that use VfW (although one can load MP4/MKV e.g. via AVS/ffms2.dll, AVI is simpler, lighter, using less memory). I do still routinely use AVI/xvid e.g. when working internally like typesetting subs, and AVI/Lagarith; even when the final encoding will be in MKV or MP4 with a newer codec. I guess this is like MP3 is still commonly used for some purposes, even though there are Vorbis, AAC, etc.

@manolito
The name Celtic_Druid brings back memories... Many years ago our sever was mirroring Celtic_Druid binaries, though mainly ffdshow.

PS 1.3.5 vs 1.3.7 Quick Tests:
"MPEG" and "H.263" are tested separately. For each:
  • The .pass files are identical with 1.3.5 or with 1.3.7, except for the comment-header lines.
  • The final results of 2-pass encoding are bit-identical, except for the bit-stream version strings.
  • Outputs seem deterministic, i.e. if you encode twice with the exact same settings, then the final results are identical.
The test conditions are old-fashioned: a 640x480 anime episode (~25 mins) was compressed to ~200MiB, with Cartoon mode, Lumi mask, Qpel. VHQ mode=4, VHQ metric=1, VHQ for B=yes, Use chroma motion=yes.

Last edited by Liisachan; 4th January 2020 at 15:30. Reason: 1.3.5 vs 1.3.7 test
Liisachan is offline   Reply With Quote
Old 4th January 2020, 19:30   #15  |  Link
blob2500
Registered User
 
Join Date: Sep 2007
Location: Italy
Posts: 9
Quote:
Originally Posted by Liisachan View Post
PS 1.3.5 vs 1.3.7 Quick Tests:
"MPEG" and "H.263" are tested separately. For each:
  • The .pass files are identical with 1.3.5 or with 1.3.7, except for the comment-header lines.
  • The final results of 2-pass encoding are bit-identical, except for the bit-stream version strings.
  • Outputs seem deterministic, i.e. if you encode twice with the exact same settings, then the final results are identical.
I'm not sure 100%, but I think images of video are identical from 1.3.2 version (except perhaps solved bugs)
Last important differences are with 1.3.1 and previous; DCT/iDCT improvement in 1.3.2. "look" image is changed from this version; IMHO is better this new algorithm (better images quality); for someone is better quality of previous algorithm (1.2.x - 1.3.1).
http://websvn.xvid.org/cvs/viewvc.cg...dcore/src/dct/

Last edited by blob2500; 4th January 2020 at 20:04.
blob2500 is offline   Reply With Quote
Old 4th January 2020, 20:53   #16  |  Link
Liisachan
李姗倩 Lǐ Shān Qin
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,230
Quote:
Originally Posted by blob2500 View Post
I'm not sure 100%, but I think images of video are bit identical from 1.3.2 version (except perhaps solved bugs)
In a way, that's a good thing too, stable & reliable and...
  1. Even though the image quality is identical, updating from 1.3.5 is basically a good move for end-users, as potentially critical security issues have been fixed.
  2. However, since there was an accidental regression in 1.3.6, I was a bit paranoid, wanting to verify that 1.3.7 is okay again.
  3. Also, in that context, the following report by manolito was worrying; but so far I could not reproduce it:
Quote:
Originally Posted by manolito View Post
Now this latest version 1.3.7 got me curious, I made a few tests using identical settings to my old setup. But I could not see any advantage. The resulting files were a little bigger
Since 1.3.5 and 1.3.7 write the essentially same file with identical file size in my quick tests, and 1.3.7 is supposed to be more robust, potential vulns fixed, one has a good reason to update to 1.3.7 sooner or later, even if it's just a maintenance release without no encoding-quality changes [and as such there is no need to switch to 1.3.7 urgently].
Liisachan is offline   Reply With Quote
Old 4th January 2020, 21:53   #17  |  Link
blob2500
Registered User
 
Join Date: Sep 2007
Location: Italy
Posts: 9
Quote:
Originally Posted by Liisachan View Post
In a way, that's a good thing too, stable & reliable and...
  1. Even though the image quality is identical, updating from 1.3.5 is basically a good move for end-users, as potentially critical security issues have been fixed.
  2. However, since there was an accidental regression in 1.3.6, I was a bit paranoid, wanting to verify that 1.3.7 is okay again.
  3. Also, in that context, the following report by manolito was worrying; but so far I could not reproduce it:
Since 1.3.5 and 1.3.7 write the essentially same file with identical file size in my quick tests, and 1.3.7 is supposed to be more robust, potential vulns fixed, one has a good reason to update to 1.3.7 sooner or later, even if it's just a maintenance release without no encoding-quality changes [and as such there is no need to switch to 1.3.7 urgently].
I agree with you

About that report and question of @manolito:

"So why in the world should I switch to this latest version? Because newer is always better? Just give me one good reason.."


...in summary, I think: Yes produced file is little bigger than Celtic_Druid's Xvid build 1.2. -127 (or more stable final Jawor's Xvid 1.2.2), but 1.3.7 (or 1.3.2/1.3.5) make (slightly) better images; and it's not always true: depends on other variables too: type of video's content, and if encoding is 1 pass (fix quantizer or fix average bitrate) or 2 pass. IMHO 1.3.7 is better in 2 pass mode; 1.2.x is better in 1 pass mode (or first pass of 2-pass encoding) and sometimes vice-versa ... but, anyhow, let's talk about minimal differences... for this imho is better to use last, and more bug-free, version.
blob2500 is offline   Reply With Quote
Old 5th January 2020, 00:02   #18  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,906
Thanks everybody for the explanations...

Since I only use Xvid for conversions to SD and only in single-pass mode with a constant quantizer, I think that I will stick with the old Celtic Druid version. I have been using it since it came out, and it never failed for myself, plus I never got any complaints by friends who played it on their HW DVD players.

Another thing I like about the Celtic Druid version is that his installer does not install any bloatware. It does not even create a new folder under Program Files, this is my preferred behavior.
manolito is offline   Reply With Quote
Old 5th January 2020, 23:25   #19  |  Link
Liisachan
李姗倩 Lǐ Shān Qin
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,230
MiniConvert.exe false positive?

As of writing this (2020-01-05 UTC), my antivirus ClamWin sees malware in MiniConvert.exe coming with Xvid installers (I'm on Win7). This is likely to be a false positive, as only 2 out of 70 engines "detect" it (Virustotal, Jotti I don't really trust those sites, just for reference).

Quote:
Originally Posted by manolito View Post
Another thing I like about the Celtic Druid version is that his installer does not install any bloatware. It does not even create a new folder under Program Files, this is my preferred behavior.
To be fair, the current installer lets you install files in the folder you select (C:\Program Files\Xvid is the default, but not forced).
Possibly bad things about this installer include:
  • You can't open it as an archive and extract the actually installed files (as pointed out by filler56789)
  • When you update or re-install a different version of Xvid, the installer creates subfolder rollbackBackupDirectory, rollbackBackupDirectory1, rollbackBackupDirectory2, etc. and copy the old files there. The uninstaller doesn't clean up these old files.
  • It doesn't keep the old Xvid encoding settings. The settings will be initialized when you update (this may be necessary and safer in general, though).
  • It doesn't remember that the user doesn't want auto-update checking, even though it does save old files. You have to manually select "No" every time you install Xvid.
A good thing about this installer is:
  • It explicitly asks if the user wants auto-update checking. Nowadays, so many applications try to phone home without asking the user. Xvid at least doesn't do such a rude thing.
I couldn't reproduce your results because my tests are only against 1.3.5 (relatively new) and using different settings (2-pass, not 1-pass).
Liisachan is offline   Reply With Quote
Old 16th January 2020, 20:46   #20  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 709
Quote:
Originally Posted by Liisachan View Post
The test conditions are old-fashioned: a 640x480 anime episode (~25 mins) was compressed to ~200MiB, with Cartoon mode, Lumi mask, Qpel. VHQ mode=4, VHQ metric=1, VHQ for B=yes, Use chroma motion=yes.
Actually Cartoon mode don't work well on anime content, the same for lumi mask, vaq is better, that or no-aq.
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.