Standalone Faster-Whisper-XXL - AI auto-transcription-translation

[StainlessS]

I think you are using an old version, current is r134.6.

Yeah, probably bout 5 weeks since update.

Not odd, Whisper models doesn't support transcription of multilingual audio. ...

Well whether it supports it or not, some (but not all) of it (Spanish) was translated to English.

[VoodooFX]

Compiled Linux and Mac OS X executables. Enjoy it.

[xarzu]

I got an error which, I assume, is due to the fact that my input is a 4 hour long video, or maybe I have too many windows open, or both.

[Emulgator]

I can only suggest to split long files into multiple jobs, it helps a great deal with everything whisper.
For sequences which are hard to guess: restrict a whisper job to only that part, and repeats or overlooked, bad guesses can be mended.

[VoodooFX]

Originally Posted by xarzu

I got an error which, I assume, is due to the fact that my input is a 4 hour long video, or maybe I have too many windows open, or both.

No, that's irrelevant.
From the screenshot you can see that it's out of the video memory.
How much of VRAM there is in your GPU, and what is the model of your GPU?

[VoodooFX]

Released Standalone Faster-Whisper-XXL with the additional features.
You want to use them on movies or noisy podcasts containing music & ect.

[Yosho]

I could use some help with the syntax if you wouldn't mind..

So far I've got C:\Users\Downloads\Faster-Whisper-XXL_r192.3.4_windows\Faster-Whisper-XXL\faster-whisper-xxl.exe "C:\Users\Downloads\blah.mkv" --language English -m large -d CPU -o "C:\Users\Downloads" -f srt --task transcribe

Additionally, is there a way to include this in subtitle edit by chance? Or would it need to be put in by the programmer themselves of the app?
Thanks for the help!

[VoodooFX]

Originally Posted by Yosho

I could use some help with the syntax if you wouldn't mind..

I don't mind.

Originally Posted by Yosho

Additionally, is there a way to include this in subtitle edit by chance? Or would it need to be put in by the programmer themselves of the app?

That you should ask the developer of Subtitle Edit.

[Yosho]

Originally Posted by VoodooFX

I don't mind.

Great, thanks! What am I missing from the commands I posted to run it to first download the _models and then do the transcription with the whisper?

So far I've got C:\Users\Downloads\Faster-Whisper-XXL_r192.3.4_windows\Faster-Whisper-XXL\faster-whisper-xxl.exe "C:\Users\Downloads\blah.mkv" --language English -m large -d CPU -o "C:\Users\Downloads" -f srt --task transcribe

Thank you for your help!

[VoodooFX]

Originally Posted by Yosho

What am I missing from the commands...

No idea what you are missing there, you tell me what you are missing... only the input is needed to run it.

I can tell you what I'm missing - I don't see you posting your problem, if you have any.

[Yosho]

Originally Posted by VoodooFX

No idea what you are missing there, you tell me what you are missing... only the input is needed to run it.

I can tell you what I'm missing - I don't see you posting your problem, if you have any.

I figured it out.
I now have subtitles. They are out of sync, and needing to be synced to the dialog.
Do you have something you regularly use for automatic subtitle syncing, or advice on what's best in your educated opinion to use for automatic subtitle syncing?

[VoodooFX]

Originally Posted by Yosho

I now have subtitles. They are out of sync, and needing to be synced to the dialog.

For me it produces mostly in sync results, can you share an audio sample where it's out of sync?

Originally Posted by Yosho

Do you have something you regularly use for automatic subtitle syncing, or advice on what's best in your educated opinion to use for automatic subtitle syncing?

There is no perfect automatic subtitle syncing.

[VoodooFX]

If someone doesn't have fast CUDA GPU, you can use one for free in Colab.

EDIT: Look at the first post.

[TR-9970X]

Thanks for directing me here:-

So I have been spending a LOT of time with the 6 episode near 5 hour's of Das Boot.

Have downloaded so many variants, I'm getting lost & confused.

So to cut a very long story short, I got 2 different sets, as one was 1080p. and the 720p had the good dubbed engish audio, but each episode was minutes shorter, so I had to overcome that issue, and did, the audio overdub is preety damn good

So once I trimmed all the episodes of the useless intro, outro & recaps, I then wanted to transcribe the dubbed english audio. to get matching subtitles. as the original audio & english subs are crap.

I tried Whisper @ Google Colab, and that did a pretty good job on the majority, but there a few really tricky bits that I want to get as correct as I can, and this one I've sent you is a little song, either in French or German. and I just can't quite get it.

I tried to set up WhisperX on my PC, but I got stuck at some error that I couldn't get any help on. so I gave up.

Using SE, the only model that came close to getting this clip, was const-me large. ALL the other's didn't find anything

I'd really like to be able to post my final compilation somewhere for anyone that wants a good "ENGLISH" copy of Das Boot, the massive near 5 hours of it.

Here's the file, I sent a video, so you can see what you're dealing with.

https://www.mediafire.com/file/s8kbo...e+song.7z/file

BTW, I figured out how to use waveforms, what a godsend that is

Cheers & thanks again.

[VoodooFX]

Originally Posted by TR-9970X

https://www.mediafire.com/file/s8kbo...e+song.7z/file

The command used:

-m large-v2 --vad_method pyannote_v3 -o source --standard --max_gap 1 --realign --ff_vocal_extract mb-roformer --multilingual true --batched

Subs:

1
00:00:01,076 --> 00:00:01,701
Exactly.

2
00:00:03,608 --> 00:00:04,080
Me too.

3
00:00:04,705 --> 00:00:09,019
Avec les marins s'attendent, les ponts
cassant leur aveugle.

4
00:00:09,419 --> 00:00:14,500
Moi, je sais bien ce qui leur manque à
tous ces mondes et leurs généraux.

5
00:00:14,859 --> 00:00:21,917
Allez, je m'appelle la reine, la reine du
pays de la Rochelle.

6
00:00:22,120 --> 00:00:29,492
Côle bleu, sentinelle, m'appelle... M
'appelle la reine de la Rochelle.

7
00:00:32,447 --> 00:00:36,042
Come on now, he's too young to play games
with you, Monique, huh?

Originally Posted by TR-9970X

I'd really like to be able to post my final compilation somewhere for anyone that wants a good "ENGLISH" copy of Das Boot

Why? There are retail English subs for it.

[TR-9970X]

Originally Posted by VoodooFX

The command used:

-m large-v2 --vad_method pyannote_v3 -o source --standard --max_gap 1 --realign --ff_vocal_extract mb-roformer --multilingual true --batched

Subs:

1
00:00:01,076 --> 00:00:01,701
Exactly.

2
00:00:03,608 --> 00:00:04,080
Me too.

3
00:00:04,705 --> 00:00:09,019
Avec les marins s'attendent, les ponts
cassant leur aveugle.

4
00:00:09,419 --> 00:00:14,500
Moi, je sais bien ce qui leur manque à
tous ces mondes et leurs généraux.

5
00:00:14,859 --> 00:00:21,917
Allez, je m'appelle la reine, la reine du
pays de la Rochelle.

6
00:00:22,120 --> 00:00:29,492
Côle bleu, sentinelle, m'appelle... M
'appelle la reine de la Rochelle.

7
00:00:32,447 --> 00:00:36,042
Come on now, he's too young to play games
with you, Monique, huh?

Why? There are retail English subs for it.

Of course I'm guessin' that's with your Pro version ??

Can it do translations??

I let Google translate, and now I should be able to add that.

I have searched far & wide, and all the English subtitles are crap, the dubbed english transcript is pretty much spot on.

And the 6 part Das Boot is pretty rare, as well.

So you've all but convinced me that your Pro version is pretty damn good.

So, not to be a pain, but how do I get it, how much is it going to cost me, and how to integrate it in SE.

Oh, and thanks for doing that little job for me so quickly

[VoodooFX]

Originally Posted by TR-9970X

Of course I'm guessin' that's with your Pro version ??

Can it do translations??

I let Google translate, and now I should be able to add that.

So, not to be a pain, but how do I get it, how much is it going to cost me, and how to integrate it in SE.

Yes.
Yes, it does translation to English.
Its translation is better than Google translate.
Look there: https://github.com/Purfview/whisper-...iscussions/456
Just copy it to the same folder where the regular version is in SE. (Delete the old files there, excluding the models)

Originally Posted by TR-9970X

I have searched far & wide, and all the English subtitles are crap, the dubbed english transcript is pretty much spot on.

The retail subs are for the original German audio, I guess you're comparing them to the dubbed audio, which is different from the original.

[TR-9970X]

Originally Posted by VoodooFX

Yes.
Yes, it does translation to English.
Its translation is better than Google translate.
Look there: https://github.com/Purfview/whisper-...iscussions/456
Just copy it to the same folder where the regular version is in SE. (Delete the old files there, excluding the models)




The retail subs are for the original German audio, I guess you're comparing them to the dubbed audio, which is different from the original.

OK, so if you're minimum "donation" is £50, that calculates to nearly AU$100...is that correct ??

From what I've seen, the english that comes with the video, isn't that good, and the subtitles don't match either.

The dubbed english was done by the actors that were in the movie, so it's pretty good.

[VoodooFX]

Originally Posted by TR-9970X

OK, so if you're minimum "donation" is £50, that calculates to nearly AU$100...is that correct ??

Yes, it should be around that.

[TR-9970X]

Originally Posted by VoodooFX

Yes, it should be around that.

OK, well, I'm going to have to give that some serious thought.

Regards.