Sync Old AAC audio track w/new H264 video with different sample rates?

kwanbis · 18th February 2021, 22:08

Hello everyone. It's been a while since I posted here. I have two video/audio files:

Code:

-------- OLD MP4 FILE WITH LOCAL LANGUAGE -------- 
Complete name                            : OLD.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/avc1)
File size                                : 48.3 MiB
Duration                                 : 22 min 16 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 303 kb/s
Encoded date                             : UTC 2009-12-07 19:43:41
Tagged date                              : UTC 2009-12-07 19:43:41

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings                          : CABAC / 8 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 8 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 22 min 16 s
Bit rate                                 : 268 kb/s
Maximum bit rate                         : 985 kb/s
Width                                    : 512 pixels
Height                                   : 384 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.045
Stream size                              : 42.7 MiB (88%)
Writing library                          : x264 core 56 svn-680
Encoding settings                        : cabac=1 / ref=5 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=6 / brdo=1 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=0 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / bime=1 / keyint=250 / keyint_min=25 / scenecut=40(pre) / rc=2pass / bitrate=268 / ratetol=1.0 / rceq='blurCplx^(1-qComp)' / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30
Encoded date                             : UTC 2009-12-07 19:43:41
Tagged date                              : UTC 2009-12-07 19:43:47
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC SBR PS
Format/Info                              : Advanced Audio Codec Low Complexity with Spectral Band Replication and Parametric Stereo
Commercial name                          : HE-AACv2
Format settings                          : Implicit / NBC
Codec ID                                 : mp4a-40-5
Duration                                 : 22 min 16 s
Bit rate mode                            : Variable
Bit rate                                 : 32.0 kb/s
Maximum bit rate                         : 35.7 kb/s
Channel(s)                               : 2 channels
Channel layout                           : C
Sampling rate                            : 44.1 kHz
Frame rate                               : 21.533 FPS (2048 SPF)
Compression mode                         : Lossy
Stream size                              : 5.10 MiB (11%)
Encoded date                             : UTC 2009-12-07 19:43:47
Tagged date                              : UTC 2009-12-07 19:43:47

-------- NEW MKV FILE WITH BETTER VIDEO --------
General
Complete name                            : NEW.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 394 MiB
Duration                                 : 25 min 2 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 200 kb/s
Encoded date                             : UTC 2019-10-18 03:10:59
Writing application                      : mkvmerge v10.0.0 ('To Drown In You') 32bit
Writing library                          : libebml v1.3.4 + libmatroska v1.4.5

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 5 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 5 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 25 min 2 s
Bit rate                                 : 1 749 kb/s
Width                                    : 1 440 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.047
Stream size                              : 313 MiB (80%)
Writing library                          : x264 core 152 r2851 ba24899
Encoding settings                        : cabac=1 / ref=5 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=4 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=0 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : FLAC
Format/Info                              : Free Lossless Audio Codec
Codec ID                                 : A_FLAC
Duration                                 : 25 min 2 s
Bit rate mode                            : Variable
Bit rate                                 : 448 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 11.719 FPS (4096 SPF)
Bit depth                                : 16 bits
Compression mode                         : Lossless
Stream size                              : 80.3 MiB (20%)
Title                                    : Main
Writing library                          : libFLAC 1.3.2 (UTC 2017-01-01)
Language                                 : Japanese
Default                                  : Yes
Forced                                   : No

I want to remux the OLD audio track from the MP4 into the MKV file.

Now, the old audio is delayed by 60 seconds, so I use 60.000 as the delay in MKVmerge and it works fine in the beginning, but then the audio gets de-synchronized with the video over time.

I understand it is because of the difference in sample rate, so I was wondering how can I fix the old audio so that I can include it in the NEW MKV file.

If it can be done with FFMPEG or some command-line tool, much better. I don't mind re-encoding the AAC.

Any pointers, tips, or ideas? Thanks!

EDIT: I keep looking around and it seems the problem is that the MP4 is 29.970fps and the MKV is 23.976fps, so I am currently searching for ways to make the AAC 23.976fps, although I am not sure that is the real problem.

tebasuna51 · 19th February 2021, 00:16

Nope, the problem is not the video fps, both are NTSC but they have different sequences.

Change the audio duration in a relation 29.970/23.976 = 5/4 = 1.25 is not possible.

kwanbis · 19th February 2021, 12:18

Quote:

Originally Posted by tebasuna51

Nope, the problem is not the video fps, both are NTSC but they have different sequences.

Change the audio duration in a relation 29.970/23.976 = 5/4 = 1.25 is not possible.

Sorry, I didn't understand your last phrase.

So apparently (and I am at work so I could only check in the morning briefly for 10 minutes during breakfast), the new version includes an additional frame here and there. So apparently, that is the problem. I need to do a more thorough check.

In any case, assuming that the videos had the same length and exact same frames, shouldn't I still re-sync the audio because of the FPS difference on the videos?

This is all totally new for me, so I am just wondering if before taking the time to edit the audio in Audacity or similar, don't I need to first re-sync to the new FPS or something like that?

Also, what tool is good to sync audio/video manually? I am not sure I have the time or the skills to do it, but in case I want to give it a try?

Thanks.

tebasuna51 · 19th February 2021, 15:13

Yes, you need to resync manually with Audacity inserting silences when the new version insert new video frames.

1) When the videos have the same frames and duration must have the same fps and don't need resync.

2) When the videos have the same duration but one at 23.976 and other 29.97 most the time the last one was created from the first adding a frame each four.
The audio must be in sync.

3) Only when videos have the same frames but different duration, because have different fps in values 23.976, 24 or 25, the audios was modified previously and you need recover the correct duration with a TimeStretch procedure.

kwanbis · 19th February 2021, 17:04

Quote:

Originally Posted by tebasuna51

Yes, you need to resync manually with Audacity inserting silences when the new version insert new video frames.

1) When the videos have the same frames and duration must have the same fps and don't need resync.

2) When the videos have the same duration but one at 23.976 and other 29.97 most the time the last one was created from the first adding a frame each four.
The audio must be in sync.

3) Only when videos have the same frames but different duration, because have different fps in values 23.976, 24 or 25, the audios was modified previously and you need recover the correct duration with a TimeStretch procedure.

Thanks, so in this case, I just need to add the silence even if the files have different frames (and different duration obviously)?

tebasuna51 · 19th February 2021, 19:10

You can insert silence, or the japanese audio if is without dialogs, in the added frames.

19th February 2021, 00:16	#2 \| Link
tebasuna51 Moderator Join Date: Feb 2005 Location: Spain Posts: 6,915	Nope, the problem is not the video fps, both are NTSC but they have different sequences. Change the audio duration in a relation 29.970/23.976 = 5/4 = 1.25 is not possible. __________________ BeHappy, AviSynth audio transcoder.

19th February 2021, 15:13	#4 \| Link
tebasuna51 Moderator Join Date: Feb 2005 Location: Spain Posts: 6,915	Yes, you need to resync manually with Audacity inserting silences when the new version insert new video frames. 1) When the videos have the same frames and duration must have the same fps and don't need resync. 2) When the videos have the same duration but one at 23.976 and other 29.97 most the time the last one was created from the first adding a frame each four. The audio must be in sync. 3) Only when videos have the same frames but different duration, because have different fps in values 23.976, 24 or 25, the audios was modified previously and you need recover the correct duration with a TimeStretch procedure. __________________ BeHappy, AviSynth audio transcoder.

19th February 2021, 19:10	#6 \| Link
tebasuna51 Moderator Join Date: Feb 2005 Location: Spain Posts: 6,915	You can insert silence, or the japanese audio if is without dialogs, in the added frames. __________________ BeHappy, AviSynth audio transcoder.