Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
18th June 2019, 08:04 | #1 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,904
|
Interlaced encoded as progressive and repeated fields
Hi,
I'm in troubles. The service who shot and edited a beach soccer match delivered a mess: the file has been encoded (by them) in H.264 60fps progressive, but it's actually interlaced. The problem is that fields are screwed up and I can't solve the problem by simply bobbing everything. By applying plain and brute bobbing, I notice that chroma goes outside the luma boundaries and it looks simply awful. TDeint fails to deinterlace it properly and creates many artifacts with still noticeable fields in the picture. Libav Deinterlace with linear blending gives a somewhat better result, however I noticed that there are still repeated frames left. Here's why I'm in trouble: air time is tomorrow and my target is 25i, so either 25fps progressive flagged as interlaced or 50fps progressive divided in fields to get a truly interlaced 25i. I've already contacted the service to send another file that is not so screwed up, however they won't be able to send it 'till tomorrow, therefore I gotta try to make something out of what I currently have. I have uploaded a few sample: a lossless uncompressed one (but beware that the source they sent was already compressed) of 10 and 20 seconds and a long sample that I re-encoded in H.265 10bit leaving everything as it was (fields as they were) so that you can play with it. Long Sample (lossy): https://we.tl/t-qaUrbdD8ns Short Sample 1 (lossless): https://we.tl/t-yuNOTvgnzG Short Sample 2 (lossless): https://we.tl/t-IUKdRvriLf Short Sample 3 (lossless): https://we.tl/t-dQdrJm6Mq0 Short Sample 4 (lossless): https://we.tl/t-bocAED6sJb Thank you in advance, I have faith in you, Frank. |
18th June 2019, 09:03 | #2 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
There is probably a better way. But this looks at least more or less ok. Or lets say it a start...
Code:
clip = core.lsmas.LWLibavSource(source=r"D:\Download\Sample_short.mkv") clip0 = clip.std.SeparateFields(0).std.SelectEvery( cycle=2, offsets=0) clip1 = clip.std.SeparateFields(1).std.SelectEvery( cycle=2, offsets=1) clipi = core.std.Interleave([clip0, clip1]).std.SelectEvery( cycle=2, offsets=0) clip = core.std.DoubleWeave(clipi, tff=True) clip = haf.QTGMC(clip, TR2=1, TFF=False).std.SelectEvery( cycle=2, offsets=0)
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 18th June 2019 at 09:12. |
18th June 2019, 09:42 | #3 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,904
|
Quote:
I'm not familiar with VapourSynth. Translated in Avisynth language would be: Code:
#Indexing LWLibavVideoSource("Sample.mkv") #Separate fields clip0=SeparateFields(0).SelectEvery( cycle=2, offsets=0) clip1=SeparateFields(1).SelectEvery( cycle=2, offsets=1) #Interleaving both fields Interleave(clip0, clip1) SelectEvery(cycle=2, offsets=0) #Weave DoubleWeave(tff=True) #Deinterlace QTGMC(TR2=1, TFF=False) SelectEvery(cycle=2, offsets=0) |
|
18th June 2019, 10:04 | #4 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
I think this should be the correct translation:
Code:
ffms2("D:\Download\Sample_short.mkv") clip0=AssumeBFF().SeparateFields().SelectEvery(2, 0) clip1=AssumeTFF().SeparateFields().SelectEvery(2, 1) #Interleaving both fields Interleave(clip0, clip1) SelectEvery(2, 0) #Weave AssumeTFF() DoubleWeave() #Deinterlace QTGMC() ## use a preset here! SelectEven() I also noticed that the chroma has blends in SeparateFields(). Not sure what a good fix would look like. And I think every 6th frame is a double frame!? But it doesnt seems to be a fixed pattern. EDIT: I tested just qtgmc alone again and now it doesn't look "jumpy" like the first time. I don't understand why... the code above is unnecessary and not entirely correct anyway. I will post new code soon.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 18th June 2019 at 11:02. |
18th June 2019, 11:56 | #5 | Link |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,496
|
What a mess. I can't see any pattern to the repeats.
Try this, though: Code:
tdecimate(mode = 1, cycler = 7, cycle = 12) Unfortunately tdecimate only works in 8 bit, and there will always be remaining dupes which result in a stutter, but I think it's as good as you're going to get unless I can work out the exact dupe pattern (it's so random I think it was done analytically). Also the chroma is completely blended so there's no hope of fixing that. Also tell them they suck at compositing! Last edited by wonkey_monkey; 18th June 2019 at 11:59. |
18th June 2019, 13:26 | #7 | Link | ||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
from:- https://forum.doom9.org/showthread.p...08#post1873808 Quote:
Duplicity/DropDeadGorgeous only 8 bit, but could convert to 8 bit for duplicity DBase creation, and hack DropDeadGorgeous script for 2nd pass with 10 bit and DropDeadGorgeous(). Code:
Function ZZ_AverageLuma(clip c, int "n") { ### Replace RT_AverageLuma() calls to ZZ_AverageLuma() EDIT: Simple Search/Replace in text editor. c # EDIT: Oops, added current_frame = Default(n,current_frame) return AverageLuma }
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 18th June 2019 at 13:51. |
||
18th June 2019, 13:50 | #8 | Link |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,496
|
No interpolation is needed, just precise removal of exact (barring compression differences) duplicates. But there might be some scenes with static graphics, so there needs to be a limit on how many frames are removed per second, at least on average.
It seems that, at most, there are three duplicate frames in a row (original correct frame, plus three duplicates). |
18th June 2019, 13:54 | #9 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Perhaps, but given the alternative of [EDIT: left over dupes/] jerks, might be an option.
Duplicity has MaxDupeLen (where if greater then is presumed static scene). EDIT: Duplicity does not have a 1 in n decimation mode [well, when creating Frame files, it does not actually decimate at all].
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 18th June 2019 at 14:05. |
18th June 2019, 16:52 | #10 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
You mentioned h.264 60fps progressive, but what was it? 4:2:2 8bit h264 ? 4:2:0 ?
The short sample is 8bit 4:2:2 ffv1, but long sample is 10bit 4:2:0 Why didn't you just upload the cut original ? - smaller filesize compared to ffv1 - less chance of introducing more errors a) maybe other issues like decoding, timecodes causing the problem, such as repeat fields. Maybe other decoding methods are better for the actual source ? Certain versions of ffms2 / lsmash/ ffmpeg are known to give problems , e.g. interlaced AVC transport streams b)possibly introducing other interlaced/progressive up/downsampling errors exacerbating the chroma ghosting . It probably doesn't matter here, but you should always upload the original , untouched as much as possible |
18th June 2019, 20:21 | #11 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,904
|
Quote:
For everyone: I've tried the code posted above and I encoded a few samples. I also tried various modes and parameters of tdecimate, chained tdecimate, single tdecimate, SRestore for dups only, ApparentFPS, Duplicity2 and I spent hours and hours on this flipping mess. I even tried to manually detect and trim out dups and then blend the missing frames to get a truly interlaced 25i (50fps) and although it took a lot of time, the result was promising. In order to try to get transition between missing frames smoother, I even tried linear interpolation but it was so inconstant that it was creating artifacts and it wasn't worth it. I called the producer and I showed them each and every result of the various encoding test, but he said that the "automatic solutions" were not good and although the manually decimated and blended one was promising, he recognised that it was too much effort to do it for a whole game of over 1h (I did it for several minutes but I was far from the end and my eyes were crossing already...). Anyway, the producer informed the playout that we're gonna miss the scheduled deadline and that he's gonna talk to the service. (I don't know about what contract we have with them; it's not my field, I'm an encoder). Alright, I would like to thank you all for the support as you tried to help me (well, "help us") and my boss is also thanking you as he knows about doom9 and he's totally in favour of open source and reciprocal help. Thank you all anyway. Alright, fellas, time to go home now and have dinner. What a day at work today has been... EDIT: They actually managed to send the real untouched masterfile in AppleProRes during the night and we encoded it straight away so we will be able to air in today, which also made me solve the "mystery" of what happened to the provided file: the original masterfile was an 25i truly interlaced (50fps) 4:2:2 yv16 10bit planar, however it has been edited by the guy who sent the file in a timeline that was set as 60fps progressive and that screwed up the fields. Then, he also brought it from 10bit to 8bit by truncation and it converted it from 4:2:2 planar to 4:2:0 planar while interlaced but treating it as progressive, which made the chroma expand from the fields to the frames... What a mess. Luckily the original untouched master has been provided just in time. Last edited by FranceBB; 19th June 2019 at 06:21. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|