Timestamp offset stability fix for muxed audiovideo mp4 #7436

robwalch · 2025-07-29T03:25:00Z

This PR will...

Fix resetting of initPTS when muxed audiovideo sample start times are inconsistent

Why is this Pull Request needed?

When the audio sample start time offsets from playlist times differ between segments or parts, this can cause small changes in initPTS. These changes may cause issues with appending and fragment/part finding.

trackId is set in stream-controller this.initPTS[frag.cc] so that the trackId is maintained between variant switches. This prevents initPTS track and timescale from changing on switch, especially if audio doesn't always start before video in segments or parts.
pasthrough-remuxer initSegment result remains consistent with initPTS unless isInvalidInitPts returns true. This prevents the stream-controller from resetting initPTS when initSegment.initPTS changes slightly.
initPTS with a timescale of 1 is no longer created when baseOffsetSamples are used to gather initPTS and timescale.

Are there any points in the code the reviewer needs to double check?

Verify past fixes are still valid:

Resolves issues:

Fixes #7431

Checklist

changes have been done against master branch, and PR does not conflict
new unit / functional tests have been added (whenever applicable)
API or design changes are documented in API.md

hongjun-bae

It seems better to include the trackId when assigning initPTS. The following issue has been resolved.

… inconsistent Fixes #7431

hongjun-bae · 2025-07-30T01:22:57Z

Here is the problem scenario along with key logs from version 1.4.10 (where the issue does not occur) and version 1.6.6 (where the issue does occur). It looks like more detailed logging needs to be added!

Problem Scenario:

Part 0 and 1 of segment 348813 are successfully loaded.
loadingParts is set to false.
A full segment load is attempted again for the same segment 348813.
A new transmux session is created → buffer discontinuity occurs.
The shouldLoadParts() method re-enabled part loading without checking whether the parts were already loaded. This caused:
Unnecessary state transitions
Redundant loading attempts for the same fragment
contiguous flag in the transmuxer being set to false, leading to a new session creation.

Comparison Log:

1.4.10

inject.js:32 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 348292 p: -1 level: 2 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: false
        timeOffset: 3594
        initSegmentChange: true
11:11:48.640 inject.js:32 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 348292 p: -1 level: 4 id: 1
        discontinuity: false
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: false
        timeOffset: 3596
        initSegmentChange: true
11:11:53.852 inject.js:32 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 348296 p: 0 level: 4 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: false
        timeOffset: 3598
        initSegmentChange: true

1.6.6

    inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348410 part: 0 level: 4 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: false
        timeOffset: 3596
        initSegmentChange: true
11:15:49.333 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348413 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3602
        initSegmentChange: false
11:15:51.044 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348415 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3606
        initSegmentChange: false
11:15:53.187 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348416 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3608.010645833332
        initSegmentChange: false
11:15:54.878 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348417 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3610.0159791666083
        initSegmentChange: false
11:15:56.981 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348418 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3612
        initSegmentChange: false
11:15:59.069 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348419 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3614.005333333276
        initSegmentChange: false
11:16:01.131 inject.js:32 [log] > [transmuxer-interface]: Starting new transmux session for main sn: 348420 level: 4 id: 1
        discontinuity: false
        trackSwitch: false
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 3616.010645833332
        initSegmentChange: false

…back likely due to a gap

robwalch · 2025-07-30T04:49:45Z

@brodiddev Thanks for the problem break down.

I've identified a few points where logging or warning can be improved with 2934cf0

The shouldLoadParts() method re-enabled part loading without checking whether the parts were already loaded.

These changes also prevent part loading for falling back to fragment loading in these cases.

It looks like more detailed logging needs to be added!

I'm not changing the "Starting new transmux session" log message. The only other available context that may be worth adding is the playlist time (so that you can see it going back from earlier messages). The time of the previous session is no longer available - you must read other log messages to gather where streaming was prior to the re-anchoring of the stream.

Revised Problem Scenario:

Part 0 and 1 of segment 348813 are successfully loaded.
The buffer is not filled to match the playlist (Two parts only make ~1 second of buffer):
[stream-controller]: Buffered main sn: 395954 part: 1 of level 2 (part:[27-28] > buffer:[26.000-27.089])
No unloaded part is found belonging to fragment at 27.089 → loadingParts is set to false.
A full segment load is attempted again for the same segment 348813.
A new transmux session is created

…e offset

robwalch added this to the 1.6.8 milestone Jul 29, 2025

robwalch marked this pull request as ready for review July 29, 2025 03:25

hongjun-bae approved these changes Jul 29, 2025

View reviewed changes

robwalch mentioned this pull request Jul 29, 2025

Audio start timestamp offset in fMP4 stream is ignored #7432

Closed

5 tasks

Fix resetting of initPTS when muxed audiovideo sample start times are…

2934cf0

… inconsistent Fixes #7431

robwalch force-pushed the bugfix/timestamp-offset-stability-initpts branch from 011e509 to 2934cf0 Compare July 29, 2025 23:24

robwalch added this to HLS.js Release Planning and Backlog Jul 29, 2025

github-project-automation bot moved this to Top priorities in HLS.js Release Planning and Backlog Jul 29, 2025

Continue loading parts even with gaps and warn when frag selects fall…

65dbb20

…back likely due to a gap

robwalch added 2 commits July 30, 2025 21:07

Update log message

95552f1

Either set initPTS based on parsed initSegment offset or maintain sam…

e59b7ae

…e offset

robwalch merged commit 8d1053c into master Jul 31, 2025
16 checks passed

robwalch deleted the bugfix/timestamp-offset-stability-initpts branch July 31, 2025 07:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Timestamp offset stability fix for muxed audiovideo mp4 #7436

Timestamp offset stability fix for muxed audiovideo mp4 #7436

robwalch commented Jul 29, 2025 •

edited

Loading

Uh oh!

hongjun-bae left a comment •

edited

Loading

Uh oh!

hongjun-bae commented Jul 30, 2025

Uh oh!

robwalch commented Jul 30, 2025

Uh oh!

Uh oh!

Uh oh!

Timestamp offset stability fix for muxed audiovideo mp4 #7436

Timestamp offset stability fix for muxed audiovideo mp4 #7436

Conversation

robwalch commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR will...

Why is this Pull Request needed?

Are there any points in the code the reviewer needs to double check?

Resolves issues:

Checklist

Uh oh!

hongjun-bae left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hongjun-bae commented Jul 30, 2025

Problem Scenario:

Comparison Log:

1.4.10

1.6.6

Uh oh!

robwalch commented Jul 30, 2025

Revised Problem Scenario:

Uh oh!

Uh oh!

Uh oh!

robwalch commented Jul 29, 2025 •

edited

Loading

hongjun-bae left a comment •

edited

Loading