[mkver]

The second audio track (track #3) starts at 5ms, the first audio track (track #2) at 65ms. The first video keyframe has a timestamp of 1571ms, the lowest video frame has a timestamp of 1511ms (this file uses open GOP). If you extracted both track #2 and track #1 (the video) and muxed it back with mkvmerge, mkvmerge would give the lowest video frame a timestamp of 0ms and the first video keyframe a timestamp of 60ms. This is a difference of 1511ms; in order to keep AV sync, you would have to offset the audio by 1511ms, too; extracting audio to elementary streams makes it loose its initial delay (here 5ms and 65ms) which already amounts to subtracting 5ms resp. 65ms from the audio tracks. So you would have to subtract a further 1511ms - 65ms from track #3. But this is only true if you actually extracted both audio and video and remuxed the elementary tracks -- your comment seems as if you didn't extract the video at all.
Btw: Why extract the tracks to elementary tracks at all? If you have transmission errors (and therefore missing packets), you will loose A/V sync no matter whether the initial offsets were right.