Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > Advanced authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th November 2020, 04:14   #1  |  Link
HDVProjection
Registered User
 
Join Date: Jul 2012
Posts: 6
Stray Frame Detection

I'm posting in this sub-forum simply because Blu-ray is our main issue here.

We are authoring Blu-rays as emergency backups to our theatrical DCPs, and the most common issue we've found with them is stray frames: a frame from a scene 30 or xxx seconds earlier (or later) getting randomly placed in the middle of another scene on the Blu-ray. Does anyone know of software that can detect stray frames? It's frustrating to QC a Blu-ray, get 60 minutes into it and find one stray frame.

I tested Interra's BATON software, and it seems to be buckets of awesome, and it can definitely detect stray frames, but it's also ungodly expensive. I was hoping for something closer to the $0-to-free range. I'm thinking that maybe ffmpeg might have similar functionality in regards to this one issue? But also, does it need to check only the .m2ts file, or can it check "the blu-ray-as-a-whole"? Thanks.
HDVProjection is offline   Reply With Quote
Old 24th November 2020, 12:07   #2  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,009
Tell your workflow. It is better to get that straight in the first place instead of trying to fix that up afterwards.
Still you can rip your Blu-ray, decode that and diff the 2 streams, source and encode.
ShowTheDifference is an old one, and StainlessS has written nice scripts to achieve that in greater detail and ease of work.
Frame-accurate source filters are a must, you may use Atak_Snipera's tool for evaluation of frame-accuracy of a given filter.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're working on that issue. Synce invntoin uf lingöage..."
Emulgator is offline   Reply With Quote
Old 25th November 2020, 04:15   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,327
Quote:
ShowTheDifference
,
Not sure bout that, maybe SplitTheDiff(). http://forum.doom9.org/showpost.php?...8&postcount=15
However, sounds to me like HDVProjection has some kind of source filter problem, like frames are coming in with the occasional one out-of-order.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 25th November 2020, 19:20   #4  |  Link
HDVProjection
Registered User
 
Join Date: Jul 2012
Posts: 6
Quote:
Originally Posted by Emulgator View Post
Tell your workflow. It is better to get that straight in the first place instead of trying to fix that up afterwards.
Still you can rip your Blu-ray, decode that and diff the 2 streams, source and encode.
ShowTheDifference is an old one, and StainlessS has written nice scripts to achieve that in greater detail and ease of work.
Frame-accurate source filters are a must, you may use Atak_Snipera's tool for evaluation of frame-accuracy of a given filter.
I edit in Premiere CS6 (normally just adding a few bumpers to the front of a feature prores), then dynamic link to Encore. The only advice I get from the Adobe forums is to clear my media cache, which I do in both Premiere and Encore, but it seems to have maybe a 50/50 chance of producing better results.

Authoring doesn't take that long, so I almost don't mind redoing it as needed. The real time consuming part is the QC afterwards. Watching the same 90-minute film 3 or 4 times, just to spot one stray frame an hour into it is just soul crushing.

Given the above, do you still recommend any of the tools you suggested? Thanks.
HDVProjection is offline   Reply With Quote
Old 25th November 2020, 21:20   #5  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,611
Is the "stray frame" an inserted frame (framecount not same as source, possible sync issues), or replaced frame (framecount is the same) ?

Is dynamic link the problem ? Did you try encoding, then authoring ? Or does a "physically" exported file have the same issue ? I've done probably 100's of BD's using EncoreCS6 in the past, never an issue with an exported file method


What is the DCP package generated from ?

Some options

1) ffmpeg psnr . Test DCP MXF as "source" against BD M2TS. Generate per frame metrics. This assumes your DCP is ok. One issue is DCP is 12bit 444 xyz, and BD is 8bit 420 YUV AVC/MPEG2/VC1 . So instead of getting "infinity" for matched frames, you have to look for large dips in dB

1b) msu vqmt; the free version I think only supports 720p but that's more than enough for a "stray" frame detection. The nice thing about this is the visualization features and it plots your graphs, so it's easy to see which frames are mismatched. It accepts avs script input - you can use avisynth scripts to downcovert both MXF to match the BD version in bitdepth, colorspace, chroma subsampling, and to satisfy the 720p limit on the free version

2) avisynth writefileif . It will print out frame numbers in a text file that are detected according to criteria you set.

Write an expression that logs frames that have some difference from previous n-1 and next n+1 ( or more if you want n-2, n+2, or some average, but you probably don't want that if it's a single frame replacement) . You have to tweak the detection threshold values, because something like a strobe light scene, or explosion will have a very different next frame . You want something that does not generate too many false positives, but you'd want to err on the side of caution and have more false detections than to miss a real "stray" frame. ie. you'd prefer it more sensitive

It would look something like this for looking at the difference between the Y plane for n-1, n+1 . An replaced random frame from another scene should be higher for both n-1, n+1 . (You can add some fancy averaging of multiple frames, or some other logic operators if required). This example has both n-1, n+1 >20 (You might have to set it higher or lower, it's just an arbitrary number for discussion)

Code:
LWLibavVideoSource("BD.m2ts")
WriteFileIf (last,  "writefile.log",  "YDifferenceFromPrevious > 20  &&  YDifferenceToNext > 20", "current_frame", """ ":" """, "YDifferenceFromPrevious", """ ":" """, "YDifferenceToNext")

Last edited by poisondeathray; 25th November 2020 at 21:30.
poisondeathray is offline   Reply With Quote
Old 25th November 2020, 22:05   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,327
PDR script would probably do you, but I already did this.

Code:
# Requires AVS+, RT_Stats for outlog : Use maybe AvsMeter to scan clip and produce log, or VDub2 Video Analysis Pass
# Can show DebugView to see results in real time.

AviSource(".\Sample2.mkv.AVI")                      # Your clip, maybe some other source filter

####### DEMO Make Bad Frame # Comment out this block for real clip, this just to test if works
#Part_1 = Trim(0,-1000)             # 1000, frames
#Part_2 = Trim(1000,0)              # Remainder
#Bad_Frame = Trim(0,-1)             # Use FIRST Frame as Bad Frame
#Part_1 ++ Bad_Frame ++ Part_2      # Splice with BAD frame inserted at frame 1000 (length is now original length + 1)
####### END  DEMO Make Bad Frame ############

### User Config ###
LOG = ".\MyLog.Log"                                 # Log in current dir
DEBUG=False                                         # Output all numbers to DebugView (Google)
MyName = "HDVProjection: "                          # Name flagged in DebugView
THRESH = 20.0                                       # Prev<->Curr and Curr<->Next Must be higher than THRESH for BAD Detect

############
RT_FileDelete(LOG)

SSS="""
    n = current_frame                               # Rem current frame
    if(0 < n < FrameCount-1) {
        current_frame = n - 1                       # Point at Previous frame (n-1)
        Prev = YDifferenceToNext                    # Diff between Previous Frame and current frame, dif(n-1 , n)
        Alt  = YDifferenceToNext(Offset=2)          # Diff between Prev and Next Frame (either side of current_frame), dif(n-1 , n+1)
        current_frame = n                           # Restore current_frame
        Next = YDifferenceToNext                    # Diff between current frame and Next frame. dif(n , n+1)
        if(DEBUG) {
            RT_DebugF("%d] Prev=%f Next=%f Alt=%f",n,Prev,Next,Alt,name=myName)
        }
        if(Prev > THRESH && Next > THRESH && (Alt < Prev || Alt < Next)) {           # Flag current frame for inspection
            RT_WriteFile(LOG,"%d",n,Append=True)                                     # I dont offhand know how to use new AVS+ log funcs.
            RT_DebugF("%d] *** Prev=%f Next=%f Alt=%f",n,Prev,Next,Alt,name=myName)  # Output to DebugView even if DEBUG=False
        }
    }
    Return Last
"""

Return ScriptClip(SSS)
EDIT: Modified a bit, small functional difference.

EDIT: Outputs this to DebugView for demo BAD frame @ 1000
Code:
00002755	20:57:13	[4792] HDVProjection: 1000] Prev=68.198410 Next=68.360115 Alt=10.026074
Also outputs 1000, to MyLog.Log.

EDIT:
Code:
&& Alt < Prev && Alt < Next
Should cut out almost all false +ve's
Can set DebugView HighLight filter to ":" to only show our debugview messages.
(other programs/drivers/anti malware, may be sending stuff to debugVew window)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th November 2020 at 13:54.
StainlessS is online now   Reply With Quote
Old 25th November 2020, 22:06   #7  |  Link
VoodooFX
Registered User
 
Join Date: Sep 2008
Posts: 193
If bug is random then you can compare two encodes with:

Code:
Subtract(clip1,clip2)
WriteFileIf (last,  "log.log",  "LumaDifference(clip1, clip2)>0", "current_frame")
In your case this at least should show first discrepancy in the files.


EDIT:


Just random idea: Interleave good_clip with encode_clip then check where there is no duplicate frame.

Last edited by VoodooFX; 25th November 2020 at 22:24.
VoodooFX is offline   Reply With Quote
Old 25th November 2020, 22:28   #8  |  Link
HDVProjection
Registered User
 
Join Date: Jul 2012
Posts: 6
Quote:
Originally Posted by poisondeathray View Post
Is the "stray frame" an inserted frame (framecount not same as source, possible sync issues), or replaced frame (framecount is the same) ?

Is dynamic link the problem ? Did you try encoding, then authoring ? Or does a "physically" exported file have the same issue ? I've done probably 100's of BD's using EncoreCS6 in the past, never an issue with an exported file method


What is the DCP package generated from ?

Some options

1) ffmpeg psnr . Test DCP MXF as "source" against BD M2TS. Generate per frame metrics. This assumes your DCP is ok. One issue is DCP is 12bit 444 xyz, and BD is 8bit 420 YUV AVC/MPEG2/VC1 . So instead of getting "infinity" for matched frames, you have to look for large dips in dB

1b) msu vqmt; the free version I think only supports 720p but that's more than enough for a "stray" frame detection. The nice thing about this is the visualization features and it plots your graphs, so it's easy to see which frames are mismatched. It accepts avs script input - you can use avisynth scripts to downcovert both MXF to match the BD version in bitdepth, colorspace, chroma subsampling, and to satisfy the 720p limit on the free version

2) avisynth writefileif . It will print out frame numbers in a text file that are detected according to criteria you set.

Write an expression that logs frames that have some difference from previous n-1 and next n+1 ( or more if you want n-2, n+2, or some average, but you probably don't want that if it's a single frame replacement) . You have to tweak the detection threshold values, because something like a strobe light scene, or explosion will have a very different next frame . You want something that does not generate too many false positives, but you'd want to err on the side of caution and have more false detections than to miss a real "stray" frame. ie. you'd prefer it more sensitive

It would look something like this for looking at the difference between the Y plane for n-1, n+1 . An replaced random frame from another scene should be higher for both n-1, n+1 . (You can add some fancy averaging of multiple frames, or some other logic operators if required). This example has both n-1, n+1 >20 (You might have to set it higher or lower, it's just an arbitrary number for discussion)

Code:
LWLibavVideoSource("BD.m2ts")
WriteFileIf (last,  "writefile.log",  "YDifferenceFromPrevious > 20  &&  YDifferenceToNext > 20", "current_frame", """ ":" """, "YDifferenceFromPrevious", """ ":" """, "YDifferenceToNext")
Thanks for the response. I have in fact been re-encoding first to a high-def h264 beforehand, simply because CS6 has started displaying some importer difficulties with the prores files we've been getting (seems to be getting worse every year, I'm assuming because of codec changes and CS6 no longer being supported). Then I drop that mp4 into my CS6 timeline and straight to dynamic link --> Encore.

The DCPs never have these issues (whether I make them myself with OpenDCP or we shop it out). Would it be easier and make more sense just to use the mp4 I'm creating for the compare? That is technically the source file, after all.

Can I get just a bit more hand-holding? What's the syntax for the ffmpeg psnr? Let's say the source is "MyFile.mp4" and the blu-ray stream is "00000.m2ts". Also, do I need to put these two in the same folder or anything? Finally, can I turn off any audio checks, have it just focus on picture consistency? Thanks again.
HDVProjection is offline   Reply With Quote
Old 26th November 2020, 00:56   #9  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,611
Quote:
Originally Posted by HDVProjection View Post
Thanks for the response. I have in fact been re-encoding first to a high-def h264 beforehand, simply because CS6 has started displaying some importer difficulties with the prores files we've been getting (seems to be getting worse every year, I'm assuming because of codec changes and CS6 no longer being supported). Then I drop that mp4 into my CS6 timeline and straight to dynamic link --> Encore.
What you should be doing is encoding to a BD compatible elementary stream (no container), then importing that directly into Encore (or any authoring software) .


Quote:
The DCPs never have these issues (whether I make them myself with OpenDCP or we shop it out). Would it be easier and make more sense just to use the mp4 I'm creating for the compare? That is technically the source file, after all.
You're assuming the MP4 file is valid in the first place (it might not be) . Did you encode to a I-frame version ? That might reduce your chances of errors if you stick with your current process


Quote:
Can I get just a bit more hand-holding? What's the syntax for the ffmpeg psnr? Let's say the source is "MyFile.mp4" and the blu-ray stream is "00000.m2ts". Also, do I need to put these two in the same folder or anything? Finally, can I turn off any audio checks, have it just focus on picture consistency? Thanks again.

for ffmpeg, it's important to normalize the container timebase (e.g. mp4 vs. mkv vs. m2ts might have different timebases), or else you can be comparing different frames

You can explicitly specify the paths in the command line, or if the files are in the same directory, and you open the cmd at that directory, you can omit paths

eg. for a 23.976 BD (24000/1001),
if we assume "MyFile.mp4" is the source (let's assume it's ok or now...it might not be)

Code:
ffmpeg -r 24000/1001 -i "PATH\00000.m2ts" -r 24000/1001 -i "PATH\MyFile.mp4".mkv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr="stats_file=ffmpeg_psnr.log"" -f null -

The problem is you get a per frame reading for every frame. You can't just eyeball it and "ah-ha" tell that certain frames are bad. The PSNR should be highish for every frame ,but should drop considerably on a mismatched frame . There might be some ways in python or excel/spreadsheet to automatically "cull" the values . If you plot them you can easily see dips

Or try one of the avisynth methods above, they only print lines which meet the criteria, so the frame numbers are already "culled"
poisondeathray is offline   Reply With Quote
Old 26th November 2020, 04:37   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,611
I took a YT trailer, No Time To Die, and swapped up 2 frames for a test (frame 775 is 372 and frame 2314 is 368, just so you know), re-encoded it (not BD settings, just generic AVC, but it shouldn't matter)

Code:
a = LSmashVideoSource("NO TIME TO DIE - Trailer 2.mp4")
b = LSmashVideoSource("remap.mp4")

a
WriteFileIf (last,  "log.log",  "LumaDifference(a,b)>10", "current_frame", """ ":" """, "LumaDifference(a,b)")
LumaDifference(a,b)>0 would print if there are any difference, but every frame does and should have a difference because it's a lossy re-encode. This is the fulllog.log where >0 is set just to see what types of values you get for calibration purposes

Just eyeballing it, on average each frame differences were <1 . Frame 775 was 33.890984, frame 2314 was 35.824036, so the delta is large. Some "same" frames were as high as 1.something. So if you set LumaDifference >10 or something it should be accurate

The log is correct and printed the two correct frames. This is log.log

Code:
775:33.890984
2314:35.824036

It's fairly fast, this 2min 34sec clip took ~12 seconds, with CPU decoding of both clips on an older computer

To run an avs script, you can use avsmeter64 script.avs, or use a GUI like vdub file=> run video analysis pass

I included the video files, script , both logs in a zip if you want to play with it
http://www.mediafire.com/file/eayon5..._test.zip/file

BTW avisynth starts frame numbering at zero, but ffmpeg psnr starts at "one"

This method , like PSNR, or any other clip to clip comparison assumes your "source" file is accurate

To load a m2ts, you could use lsmash too, but using LWLibavVideoSource
Code:
LWLibavVideoSource("00000.m2ts")

Last edited by poisondeathray; 26th November 2020 at 04:46.
poisondeathray is offline   Reply With Quote
Old 26th November 2020, 13:47   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,327
Post #6 modified a bit , small functional difference

Code:
SSS="""
    n = current_frame                               # Rem current frame
    if(0 < n < FrameCount-1) {
        current_frame = n - 1                       # Point at Previous frame (n-1)
        Prev = YDifferenceToNext                    # Diff between Previous Frame and current frame, dif(n-1 , n)
        Alt  = YDifferenceToNext(Offset=2)          # Diff between Prev and Next Frame (either side of current_frame), dif(n-1 , n+1)
        current_frame = n                           # Restore current_frame
        Next = YDifferenceToNext                    # Diff between current frame and Next frame. dif(n , n+1)
        if(DEBUG) {
            RT_DebugF("%d] Prev=%f Next=%f Alt=%f",n,Prev,Next,Alt,name=myName)
        }
        if(Prev > THRESH && Next > THRESH && (Alt < Prev || Alt < Next)) {           # Flag current frame for inspection
            RT_WriteFile(LOG,"%d",n,Append=True)                                     # I dont offhand know how to use new AVS+ log funcs.
            RT_DebugF("%d] *** Prev=%f Next=%f Alt=%f",n,Prev,Next,Alt,name=myName)  # Output to DebugView even if DEBUG=False
        }
    }
    Return Last
"""
BLUE NO functional difference, DARKORCHID small logical difference. [Was:- Alt < Prev && Alt < Next]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th November 2020 at 14:15.
StainlessS is online now   Reply With Quote
Old 2nd December 2020, 22:36   #12  |  Link
HDVProjection
Registered User
 
Join Date: Jul 2012
Posts: 6
Thanks again to all. I'm not headed back to the office for a bit, but I'll try these options out then. Cheers.
HDVProjection is offline   Reply With Quote
Old 1st May 2021, 18:33   #13  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,554
Quote:
Originally Posted by HDVProjection View Post
I'm posting in this sub-forum simply because Blu-ray is our main issue here.

We are authoring Blu-rays as emergency backups to our theatrical DCPs, and the most common issue we've found with them is stray frames: a frame from a scene 30 or xxx seconds earlier (or later) getting randomly placed in the middle of another scene on the Blu-ray. Does anyone know of software that can detect stray frames? It's frustrating to QC a Blu-ray, get 60 minutes into it and find one stray frame.
Forget about detecting such a frames....

there is only 1 proper solution- fix your workflow. Such a random frames are unacceptable and you should never continue using your current encoder etc. Don't know a single decent tool which would ever introduce such a crap.

What is your solution then? Find a bad frame and what? Do another encode, check again and possibly find it in another place? This is nonsense- fix a reason for those bad frames instead of chasing them.

Last edited by kolak; 1st May 2021 at 18:35.
kolak is offline   Reply With Quote
Old 1st May 2021, 19:54   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,327
Good sound advice Kolak. [dont fix the result, fix the cause].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 16th July 2021, 00:38   #15  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,009
Late follow-up:
Had a few occurences of stray frames recently.
Win7U64 32GB, Vegas Pro 14, 20GB for scrubbing, 32bit precision, GPU activated.
FX chain using ProDAD Mercalli V4 beside other standard FX: Curves, etc.

Only if using Mercalli in GPU Mode:
Although all clips ("events" in Vegas speak) had been Mercalli-motion analysed through and through multiple times while cutting,
after lossless rendering just one stray frame occurred showing Mercalli's "Not Analysed Yet" spoiler.
Not at a clip border, just hidden in the middle.

Switching Mercalli to CPU helped: the stray frame went away.
GPU traffic guilty ? Processed Frame content not coming in time ?
One frame's worth input copied to output without processing ? Just guessing.

Besides of that, some cache management (here in Vegas) may also be responsible.
For a very complex project I had to close and reopen Vegas if Mercalli finds an already analysed event "Unanalysed",
but there the spoiler was applied across all frames of such event.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're working on that issue. Synce invntoin uf lingöage..."

Last edited by Emulgator; 16th July 2021 at 00:51.
Emulgator is offline   Reply With Quote
Old 19th July 2021, 16:52   #16  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,554
This is the proper way to solve it.
kolak is offline   Reply With Quote
Reply

Tags
stray frames

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:05.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.