Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > (HD) DVD & Blu-ray authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th February 2014, 21:04   #521  |  Link
videofan3d
Registered User
 
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
Quote:
Originally Posted by sef View Post




What am I doing wrong??
This is strange, this is how it should be used.
I ran your command without any issues

c:\Prj\IntelMedia\_bin\Win32\Release\FRIMDecode -i::mvc SRC_L.h264 SRC_R.h264 -sbs -o - | c:\Prj\IntelMedia\_bin\Win32\Release\FRIMEncode -sbs 2 -i - -o::mvc Z.h264 -w 1920 -h 1080 -f 23.976 -vbr 28000 40000
FRIM Encoder version 1.22 (build: Feb 2 2014)
- based on Intel(R) Media SDK

Media SDK impl HARDWARE - D3D9 (C:\Program Files\Intel\Media SDK\libmfx
hw32.dll)
Media SDK version 1.7
Memory type System

Input format YUV420

Output video AVC
Source picture:
Resolution 1920x1088
Crop X,Y,W,H 0,0,1920,1080
Destination picture:
Resolution 1920x1088
Crop X,Y,W,H 0,0,1920,1080
Frame rate 23.976
Bitrate control VBR
avg,maximum 28000,40000
GOP structure:
GOP length 24
I-/P-frame distance 4
IDR-frame interval 0
GOP type Opened
Num of slices 6
Target usage 4 (balanced)

Processing started
Frame number: 276
Processing finished in 3.43 seconds


Please try it with FRIMEncode -sw (or maybe also for FRIMDecode)
(I faced some issues when running in -hw mode, HW acceleration depends on driver version)

Last edited by videofan3d; 7th February 2014 at 21:40.
videofan3d is offline   Reply With Quote
Old 7th February 2014, 22:33   #522  |  Link
sef
Registered User
 
Join Date: Nov 2013
Location: Russia
Posts: 61
Quote:
Originally Posted by videofan3d View Post
Please try it with FRIMEncode -sw (or maybe also for FRIMDecode)
Problem solved, had to manually set the path to Intel Media SDK(2014).. Only -sw, with FRIMDecode and FRIMEncode simultaneously..
P.S. What is your video card?(HD 4000).. And, why -hw not working at all, with first example..

Last edited by sef; 7th February 2014 at 23:27.
sef is offline   Reply With Quote
Old 7th February 2014, 23:38   #523  |  Link
videofan3d
Registered User
 
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
Quote:
Originally Posted by sef View Post
Problem solved, had to manually set the path to Intel Media SDK(2014).. Only -sw, with FRIMDecode and FRIMEncode..
P.S. What is your video card?(HD 4000).. And, why -hw not working at all, with first example..
(I use Haswell CPU i7 4770K with integrated GPU HD 4600)

I noticed that there are sometimes some limitations (or bugs?) when using -hw mode.
In -hw mode Intel Media uses graphic driver and calls GPU accelerated routines. They are "bloody-faster", on decoding 8x(!), but with some limits.

I can only guess:
In -sw mode each process allocates resources (memory, thread pool, etc.) in regular PC memory, so there are almost no limitations.
But in -hw mode each process can use/share resources on the GPU - which are not endless, and probably depend on specific given GPU. Once exhausted (for example when running more processes/threads), each additional process fails. And number of available resources probably depends on specific GPU family.

My experience is to play with -hw and -sw, and try to find proper combination.
videofan3d is offline   Reply With Quote
Old 8th February 2014, 00:06   #524  |  Link
sef
Registered User
 
Join Date: Nov 2013
Location: Russia
Posts: 61
Quote:
Originally Posted by videofan3d View Post
And number of available resources probably depends on specific GPU family.
That's what I was expecting, thanks for the detailed explanation, soon I will replace hardware..
sef is offline   Reply With Quote
Old 8th February 2014, 15:36   #525  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by videofan3d View Post
Please send exact command which you are running + running conditions, I will try to simulate it.
The goal of running multiple instances is to speed up encoding. I've noticed that only a little over half of the available CPU usage was being used when doing a single instance encode. The multiple instances gives me about a 30% increase in encoding speed. But, as I said, only one of the instances will complete, and the other will get the error I posted. In the example here I am accepting a half-SBS source and reencoding to full MVC using FRIMEncode (to create a 3D BD disc). Frame serving is being provided by DGDecNV so I can get frame-accurate seeking. The two AVISYNTH scripts:
Code:
#Created by BD Rebuilder - v0.46.12 (beta)
LoadPlugin("C:\APPS\DGDecNV\DGDecodeNV.dll")
DGSource("D:\WORKING8\WORKFILES\VID_00000.DGI", fieldop=0).Trim(0,-91203)
Spline16Resize(3840,1080)
ConvertToYV12().AssumeFPS(24000,1001)
Code:
#Created by BD Rebuilder - v0.46.12 (beta)
LoadPlugin("C:\APPS\DGDecNV\DGDecodeNV.dll")
DGSource("D:\WORKING8\WORKFILES\VID_00000.DGI", fieldop=0).Trim(91203,0)
Spline16Resize(3840,1080)
ConvertToYV12().AssumeFPS(24000,1001)
Here are the command lines being used:
Code:
"D:\BD_Rebuilder\tools\FRIMEncode.exe" -avi -sbs 2 -i 
"D:\WORKING8\WORKFILES\VID_00000.1.avs" -viewoutput -o::mvc "D:\WORKING8\WORKFILES\VID_00000.1.264"
 "D:\WORKING8\WORKFILES\VID_00000.1.mvc" -w 3840 -h 1080 -f 23.976 -u 6 -cpbsize 3750 -l 6 -profile high -level 4.1 -vbr 
23376 35000 -gop 24 4 0 S -maxdpb 4
Code:
"D:\BD_Rebuilder\tools\FRIMEncode.exe" -avi -sbs 2 -i 
"D:\WORKING8\WORKFILES\VID_00000.2.avs" -viewoutput -o::mvc "D:\WORKING8\WORKFILES\VID_00000.2.264" 
"D:\WORKING8\WORKFILES\VID_00000.2.mvc" -w 3840 -h 1080 -f 23.976 -u 6 -cpbsize 3750 -l 6 -profile high -level 4.1 -vbr 
23376 35000 -gop 24 4 0 S -maxdpb 4
One of the two instances will end with the error I posted every time. If the source changes, the frame at which it fails changes -- but it has happened on every source I've tried. Unfortunately it can be several thousand frames into the encode, so repeating the issue always takes a lot of time.

If I do a single file with the same settings etc... it will complete with no issues.

It's not something critical -- as a single encode always works. The purpose of the experiment is just to save time when encoding.

[Edit] Note also, that either of the two specific command lines above will complete successfully if run as a single instance -- but one will fail if run concurrently.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net

Last edited by jdobbs; 8th February 2014 at 22:31.
jdobbs is offline   Reply With Quote
Old 9th February 2014, 09:42   #526  |  Link
videofan3d
Registered User
 
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
Quote:
Originally Posted by sef View Post
That's what I was expecting, thanks for the detailed explanation, soon I will replace hardware..
I just noticed that your HW system reports Media SDK Version 1.4 while on my side 1.7.
Maybe try also upgrade video driver - Intel HW library used to be part of it, and I realized it really matters what version you have.
videofan3d is offline   Reply With Quote
Old 9th February 2014, 09:46   #527  |  Link
videofan3d
Registered User
 
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
Quote:
Originally Posted by jdobbs View Post
The goal of running multiple instances is to speed up encoding. I've noticed that only a little over half of the available CPU usage was being used when doing a single instance encode. The multiple instances gives me about a 30% increase in encoding speed. But, as I said, only one of the instances will complete, and the other will get the error I posted. In the example here I am accepting a half-SBS source and reencoding to full MVC using FRIMEncode (to create a 3D BD disc). Frame serving is being provided by DGDecNV so I can get frame-accurate seeking. The two AVISYNTH scripts:
Code:
#Created by BD Rebuilder - v0.46.12 (beta)
LoadPlugin("C:\APPS\DGDecNV\DGDecodeNV.dll")
DGSource("D:\WORKING8\WORKFILES\VID_00000.DGI", fieldop=0).Trim(0,-91203)
Spline16Resize(3840,1080)
ConvertToYV12().AssumeFPS(24000,1001)
Code:
#Created by BD Rebuilder - v0.46.12 (beta)
LoadPlugin("C:\APPS\DGDecNV\DGDecodeNV.dll")
DGSource("D:\WORKING8\WORKFILES\VID_00000.DGI", fieldop=0).Trim(91203,0)
Spline16Resize(3840,1080)
ConvertToYV12().AssumeFPS(24000,1001)
Here are the command lines being used:
Code:
"D:\BD_Rebuilder\tools\FRIMEncode.exe" -avi -sbs 2 -i 
"D:\WORKING8\WORKFILES\VID_00000.1.avs" -viewoutput -o::mvc "D:\WORKING8\WORKFILES\VID_00000.1.264"
 "D:\WORKING8\WORKFILES\VID_00000.1.mvc" -w 3840 -h 1080 -f 23.976 -u 6 -cpbsize 3750 -l 6 -profile high -level 4.1 -vbr 
23376 35000 -gop 24 4 0 S -maxdpb 4
Code:
"D:\BD_Rebuilder\tools\FRIMEncode.exe" -avi -sbs 2 -i 
"D:\WORKING8\WORKFILES\VID_00000.2.avs" -viewoutput -o::mvc "D:\WORKING8\WORKFILES\VID_00000.2.264" 
"D:\WORKING8\WORKFILES\VID_00000.2.mvc" -w 3840 -h 1080 -f 23.976 -u 6 -cpbsize 3750 -l 6 -profile high -level 4.1 -vbr 
23376 35000 -gop 24 4 0 S -maxdpb 4
One of the two instances will end with the error I posted every time. If the source changes, the frame at which it fails changes -- but it has happened on every source I've tried. Unfortunately it can be several thousand frames into the encode, so repeating the issue always takes a lot of time.

If I do a single file with the same settings etc... it will complete with no issues.

It's not something critical -- as a single encode always works. The purpose of the experiment is just to save time when encoding.

[Edit] Note also, that either of the two specific command lines above will complete successfully if run as a single instance -- but one will fail if run concurrently.
I don't have DGDecNV (it is linked to NVidia CUDA (and for that reason requires some 3rd party SW license) card which I don't have).
I'll try to simulate it somehow differently ...
videofan3d is offline   Reply With Quote
Old 9th February 2014, 14:41   #528  |  Link
sef
Registered User
 
Join Date: Nov 2013
Location: Russia
Posts: 61
Quote:
Originally Posted by videofan3d View Post
Intel HW library used to be part of it..
I know, but unfortunately, for my card(HD 3000) already installed the latest drivers..
sef is offline   Reply With Quote
Old 9th February 2014, 15:23   #529  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by videofan3d View Post
I don't have DGDecNV (it is linked to NVidia CUDA (and for that reason requires some 3rd party SW license) card which I don't have).
I'll try to simulate it somehow differently ...
I'll run the same job with DirectshowSource() and post the results. I'll have to do the split differently, though, since I can't count on it's seeking.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net
jdobbs is offline   Reply With Quote
Old 9th February 2014, 21:08   #530  |  Link
HWK
Registered User
 
HWK's Avatar
 
Join Date: Feb 2009
Location: Toronto, Ontario, Canada
Posts: 1,059
Quote:
Originally Posted by videofan3d View Post
I don't have DGDecNV (it is linked to NVidia CUDA (and for that reason requires some 3rd party SW license) card which I don't have).
I'll try to simulate it somehow differently ...
Quote:
Originally Posted by jdobbs View Post
I'll run the same job with DirectshowSource() and post the results. I'll have to do the split differently, though, since I can't count on it's seeking.
I have Nvidia card and dgdecnv, if you want jdobbs I can run test as well and see if it is related specific configuration.
__________________
If you fail to plan; you plan to fail, would you not agree? Think about it.
HWK is offline   Reply With Quote
Old 10th February 2014, 00:13   #531  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by HWK View Post
I have Nvidia card and dgdecnv, if you want jdobbs I can run test as well and see if it is related specific configuration.
I'll send you a test version of BD-RB that does multiple instances. That way you can run a job and see what happens.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net
jdobbs is offline   Reply With Quote
Old 10th February 2014, 00:13   #532  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by jdobbs View Post
I'll run the same job with DirectshowSource() and post the results. I'll have to do the split differently, though, since I can't count on it's seeking.
Yeah, it choked with DirectshowSource() also.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net
jdobbs is offline   Reply With Quote
Old 10th February 2014, 08:01   #533  |  Link
HWK
Registered User
 
HWK's Avatar
 
Join Date: Feb 2009
Location: Toronto, Ontario, Canada
Posts: 1,059
Not sure, if this is any help or not. I finish encoding world war z with multiple instance of FRIMEncode (x2) and it finished without a problem, will do Pacific Rim next with two instance and balance setting in FRIM.

(Around 60% done)


(100% done)


First Instance
Quote:
FRIMDecode.exe -ts -i::mvc "E:\BD_RB\WORKFILES\00800.M2TS" "E:\BD_RB\WORKFILES\00800.M2TS" -o \\.\pipe\_L1.yuv -o \\.\pipe\_R1.yuv | FrimEncode.exe -i \\.\pipe\_L1.yuv -i \\.\pipe\_R1.yuv -viewoutput -o::mvc "E:\WORLD_WAR_Z_Part1.264" -o "E:\WORLD_WAR_Z_Part1.mvc" -w 1920 -h 1080 -f 23.976 -u 7 -cpbsize 3750 -profile high -level 4.1 -vbr 30000 60000 -gop 24 4 0 S -maxdpb 4
pause
Second Instance
Quote:
FRIMDecode.exe -ts -i::mvc "E:\BD_RB\WORKFILES\00810.M2TS" "E:\BD_RB\WORKFILES\00810.M2TS" -o \\.\pipe\_L2.yuv -o \\.\pipe\_R2.yuv | FrimEncode.exe -i \\.\pipe\_L2.yuv -i \\.\pipe\_R2.yuv -viewoutput -o::mvc "E:\WORLD_WAR_Z_Part2.264" -o "E:\WORLD_WAR_Z_Part2.mvc" -w 1920 -h 1080 -f 23.976 -u 7 -cpbsize 3750 -profile high -level 4.1 -vbr 30000 60000 -gop 24 4 0 S -maxdpb 4
pause
I split movie into two files 00800.m2ts and 00810.m2ts file and ran two instance simultaneous.
__________________
If you fail to plan; you plan to fail, would you not agree? Think about it.

Last edited by HWK; 10th February 2014 at 16:01.
HWK is offline   Reply With Quote
Old 10th February 2014, 12:13   #534  |  Link
videofan3d
Registered User
 
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
Quote:
Originally Posted by jdobbs View Post
Yeah, it choked with DirectshowSource() also.
I tried to reproduce it .... with mixed results ...

VID.1.avs:
Code:
L=DirectShowSource("VIDEO.m2ts").Trim(0,-60000)
StackHorizontal(L,L)
VID.2.avs:
Code:
L=DirectShowSource("VIDEO.m2ts").Trim(60000, 0)
StackHorizontal(L,L)
Both inputs were artifically made 2*1920 to simulate 3D SBS input.
Input video was ~130000 frames long.

Encode 1
Code:
FRIMEncode.exe -avi -sbs 2 -sw -i VID.1.avs -viewoutput -o::mvc VID.1.h264 VID.1.mvc -w 3840 -h 1080 -f 23.976 -u 6 -cpbsize 3750 -l 6 -profile high -level 4.1 -vbr 23376 35000 -gop 24 4 0 S -maxdpb 4
Encode 2
Code:
FRIMEncode.exe -avi -sbs 2 -sw -i VID.2.avs -viewoutput -o::mvc VID.2.h264 VID.2.mvc (...same params..)
i.e. I tried to set similar conditions as you have.
Running on i7 Haswell.

And my results:

When I run in -sw encoding mode, I faced the same (rather stochastic) failure on one process.

When I run in -hw encoding mode, both finished correctly!

As you commented, it is very time consuming to reproduce it.

.... at this moment I can only add it will be very difficult to investigate the root cause - in order even to try to fix it ...)

Last edited by videofan3d; 10th February 2014 at 12:17.
videofan3d is offline   Reply With Quote
Old 10th February 2014, 16:08   #535  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Did you get a significant speed-up with -hw set? Just wondering... I'm just going to disable multiple instance processing for FRIM in BD-RB, I think it's too unpredictable as to when it will or will not fail. While it was failing on every job before, I've since had a couple shorter jobs that completed successfully. The probability is that the issue is in the library somewhere anyway -- and that would have to be fixed by Intel anyway.

I think it's a lot more likely to happen when BD-RB starts the process and has to make frequent checks for process completion -- although I don't know why that would impact it. But even then it goes through thousands of frames (and a few hours) before it finally trips.

Thanks for the look.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net

Last edited by jdobbs; 10th February 2014 at 16:13.
jdobbs is offline   Reply With Quote
Old 10th February 2014, 16:16   #536  |  Link
HWK
Registered User
 
HWK's Avatar
 
Join Date: Feb 2009
Location: Toronto, Ontario, Canada
Posts: 1,059
Jdobbs, so you are dropping the idea of multi-process for time being. I manage to do something which is shown in post # 533 not sure if it is any good.

I am gone try later today with method similar to yours for dgdecnv.
__________________
If you fail to plan; you plan to fail, would you not agree? Think about it.

Last edited by HWK; 10th February 2014 at 16:18.
HWK is offline   Reply With Quote
Old 10th February 2014, 16:25   #537  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by HWK View Post
Jdobbs, so you are dropping the idea of multi-process for time being. I manage to do something which is shown in post # 533 not sure if it is any good.

I am gone try later today with method similar to yours for dgdecnv.
I'm not sure whether I'll drop it completely. I don't have a system that would support -hw -- so I'm not sure whether is has any issues at all. I would think, though, that -hw might be where it might benefit the most.

I may just leave it in and use an undocumented hidden option to enable it. Unfortunately when it happens, the encode process never ends and BD-RB just keeps waiting. Eventually you have hit the "Abort" button. Unless you have "show_encoder=1" set in the config file, you don't even know why it's hung.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net

Last edited by jdobbs; 10th February 2014 at 16:30.
jdobbs is offline   Reply With Quote
Old 10th February 2014, 16:28   #538  |  Link
HWK
Registered User
 
HWK's Avatar
 
Join Date: Feb 2009
Location: Toronto, Ontario, Canada
Posts: 1,059
Quote:
Originally Posted by jdobbs View Post
I'm not sure whether I'll drop it completely. I don't have a system that would support -hw -- so I'm not sure whether is has any issues at all. I would think, though, that -hw might be where it might benefit the most.

I'll probably just leave it in and use a hidden option (with warning) to enable it.
Hmm, I use software option and used two instance of frimdecode & frimencode and it worked without problem, three work as well but it slows down encoding with little or no benefit.

At least there is light at end of tunnel Anyways, like I said I will use script posted by you and see if it works at all for information purposes, if it works fine and if not good with me.
__________________
If you fail to plan; you plan to fail, would you not agree? Think about it.

Last edited by HWK; 10th February 2014 at 16:53.
HWK is offline   Reply With Quote
Old 10th February 2014, 16:35   #539  |  Link
jdobbs
Moderator
 
Join Date: Oct 2001
Posts: 20,973
Quote:
Originally Posted by HWK View Post
Hmm, I use software option and used two instance of frimdecode 7 frimencode and it worked without problem, three work as well but it slows down encoding with little or no benefit.

At least there is light at end of tunnel Anyways, like I said I will use script posted by you and see if it works at all for information purposes, if it works fine and if not good with me.
Oh, I'm confident you'll get it if you repeat my method. I get it every time on a long job. But sometimes it may be 50,000+ frames before it happens. On other jobs it may happen at 8,000. I also noticed you're using FRIMDecode in your test. I'm using AVISYNTH and the -avi option, which might make a difference. In my test version of BD-RB it is currently only enabled when converting SBS to MVC.

I see the same speed effect on my system when trying to run more than 2 instances. But that makes sense when there's no available processor bandwidth available. I would expect it to do better with -hw set.
__________________
Help with development of new apps: Donations.
Website: www.jdobbs.net

Last edited by jdobbs; 10th February 2014 at 16:42.
jdobbs is offline   Reply With Quote
Old 10th February 2014, 16:38   #540  |  Link
HWK
Registered User
 
HWK's Avatar
 
Join Date: Feb 2009
Location: Toronto, Ontario, Canada
Posts: 1,059
Quote:
Originally Posted by jdobbs View Post
Oh, I'm confident you'll get it. I get it every time on a long job. But sometimes it may be 50,000+ frames before it happens. On other jobs it may happen at 8,000.
Fair enough, but it doesn't hurt to try. On a side note I was reading Neuron2 has intention to develop plugin which mvc decode without need NVidia and is frame accurate or at least that is what I understand. Let me search for the link and will post it here, if I find it.

http://forum.doom9.org/showthread.ph...83#post1667083
__________________
If you fail to plan; you plan to fail, would you not agree? Think about it.

Last edited by HWK; 10th February 2014 at 16:41.
HWK is offline   Reply With Quote
Reply

Tags
encoders, mvc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.