Skip to content

[video_player] Add audio track metadata support (bitrate, sample rate, channels, codec) #9782

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

nateshmbhat
Copy link

@nateshmbhat nateshmbhat commented Aug 11, 2025

Description

This PR adds comprehensive audio track metadata support to the video_player package, enabling developers to access detailed information about available audio tracks including bitrate, sample rate, channel count, and codec information.

Changes Made

  • Enhanced VideoAudioTrack model with new metadata fields: bitrate, sampleRate, channelCount, codec
  • Added qualityDescription getter that formats metadata into human-readable strings (e.g., "128kbps • Stereo • AAC")
  • Updated platform implementations:
    • Android: Real metadata extraction from ExoPlayer Format objects
    • iOS: Metadata extraction from AVFoundation with HLS stream simulation
  • Updated Pigeon interfaces for both Android and iOS platforms
  • Added comprehensive test coverage with 96 tests passing
  • Updated example app to display quality information in the UI

Related Issues :

#59437 - Add audio track metadata support to video_player

Testing

  • Added unit tests for all new functionality
  • Tested with HLS streams on both platforms
  • Verified backward compatibility

Breaking Changes

None - all changes are additive and backward compatible.

Pre-Review Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] page, which explains my responsibilities.
  • I read and followed the [relevant style guides] and ran [the auto-formatter].
  • I signed the [CLA].
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I [linked to at least one issue that this PR fixes] in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the [pub versioning philosophy], or I have commented below to indicate which [version change exemption] this PR falls under[^1].
  • I updated CHANGELOG.md to add a description of the change, [following repository CHANGELOG style], or I have commented below to indicate which [CHANGELOG exemption] this PR falls under[^1].
  • I updated/added any relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or I have commented below to indicate which [test exemption] this PR falls under[^1].
  • All existing and new tests are passing.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds valuable support for audio track metadata, including bitrate, sample rate, and codec information. The changes are well-structured across the different packages, with good test coverage for the new functionality. The new VideoAudioTrack model and qualityDescription getter in the platform interface are great additions.

I have two main points of feedback. First, the new example page uses a video that doesn't showcase the multi-track feature well; I've suggested a more suitable one. Second, and more importantly, the iOS/macOS implementation for HLS streams appears to fabricate metadata to match Android's behavior. This could be misleading, and I've recommended providing only the actual data available from the platform.

Comment on lines 29 to 30
const String videoUrl =
'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4';

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example video URL points to BigBuckBunny.mp4, which is unlikely to have multiple audio tracks. This makes the demo less effective at showcasing the new audio track selection feature. Consider using a test stream that is known to have multiple audio tracks, such as an HLS stream, to make the example more helpful for developers.

For example, Apple provides test streams that could be used here:

    const String videoUrl =
        'https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8';

@nateshmbhat
Copy link
Author

since in my code i've used the packages of video_player from the same branch's code instead of published ones using "path" option, i think some of these checks are failing. What should i do in this case ? until its published, the code would have to use "path" option right ?

@stuartmorgan-g
Copy link
Contributor

Thanks for the contribution! A few high-level comments before this can be reviewed:

since in my code i've used the packages of video_player from the same branch's code instead of published ones using "path" option, i think some of these checks are failing. What should i do in this case ?

Please see the instructions in our contributor docs.

  • Added comprehensive test coverage with 96 tests passing

Please check that you've uploaded all of your changes; the PR as it is here does not have comprehensive test coverage. The only testing here appears to be of the Dart data class, with no testing of any of the new platform logic.

  • Added qualityDescription getter that formats metadata into human-readable strings (e.g., "128kbps • Stereo • AAC")

This is out of scope for the plugin, and should be removed. It's up to plugin clients how/if they want to format any data.

Use Cases

  • Adaptive streaming: Select tracks based on network conditions
  • Quality preferences: Let users choose between high/low quality audio

This PR does not appear to have any support for these use cases. Is the PR description inaccurate, or is the PR still a work in progress?

@stuartmorgan-g
Copy link
Contributor

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for audio track metadata, including bitrate, sample rate, channel count, and codec information. The changes are well-implemented across the platform interface, Android (ExoPlayer), and iOS (AVFoundation) layers. The addition of the VideoAudioTrack model is clean, and the new getAudioTracks method is a valuable feature. The accompanying tests are thorough, and the example application is a great way to demonstrate the new functionality. I've identified a couple of minor issues in the example app code that could be improved for robustness and correctness, but overall this is an excellent contribution.

Comment on lines 361 to 369
onPressed: () async {
if (_controller.value.isInitialized) {
final audioTracks = await _controller.getAudioTracks();
setState(() {
_audioTracks = audioTracks;
_isLoadingTracks = false;
});
}
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The _isLoadingTracks flag is never set to true, which means the CircularProgressIndicator in the button's icon is never shown. To provide visual feedback to the user while tracks are being fetched, you should set _isLoadingTracks to true before the asynchronous call and to false after it completes. Using a try...finally block is recommended to ensure the loading state is correctly reset even if an error occurs.

                  onPressed: () async {
                    if (!_controller.value.isInitialized) {
                      return;
                    }
                    setState(() {
                      _isLoadingTracks = true;
                    });
                    try {
                      final List<VideoAudioTrack> audioTracks =
                          await _controller.getAudioTracks();
                      if (mounted) {
                        setState(() {
                          _audioTracks = audioTracks;
                        });
                      }
                    } finally {
                      if (mounted) {
                        setState(() {
                          _isLoadingTracks = false;
                        });
                      }
                    }
                  },

Comment on lines 45 to 64
Future<void> _getAudioTracks() async {
if (_controller == null) return;

setState(() {
_isLoading = true;
});

try {
final tracks = await _controller!.getAudioTracks();
setState(() {
_audioTracks = tracks;
_isLoading = false;
});
} catch (e) {
debugPrint('Error getting audio tracks: $e');
setState(() {
_isLoading = false;
});
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The _getAudioTracks method can be simplified to improve readability and ensure setState is only called once after the asynchronous operation completes. This also makes it easier to handle the widget's lifecycle by checking mounted once before updating the state.

  Future<void> _getAudioTracks() async {
    if (_controller == null) {
      return;
    }

    setState(() {
      _isLoading = true;
    });

    List<VideoAudioTrack> tracks = <VideoAudioTrack>[];
    try {
      tracks = await _controller!.getAudioTracks();
    } catch (e) {
      debugPrint('Error getting audio tracks: $e');
    }

    if (!mounted) {
      return;
    }

    setState(() {
      _audioTracks = tracks;
      _isLoading = false;
    });
  }

@nateshmbhat
Copy link
Author

nateshmbhat commented Aug 11, 2025

@stuartmorgan-g

  1. Removed qualityDescription formatting from library.
  2. removed the use cases section in PR as the PR is currently just to fetch the audio tracks
  3. went through Changing Federated plugin section doc and updated pubspec with overrides as changed by the script tool.

@nateshmbhat
Copy link
Author

@stuartmorgan-g have addressed your high level concerns. anything else that i need to do ?

@stuartmorgan-g stuartmorgan-g removed the request for review from ditman August 12, 2025 10:59
@stuartmorgan-g stuartmorgan-g added triage-ios Should be looked at in iOS triage triage-android Should be looked at in Android triage labels Aug 12, 2025
@stuartmorgan-g
Copy link
Contributor

stuartmorgan-g commented Aug 12, 2025

@stuartmorgan-g have addressed your high level concerns. anything else that i need to do ?

There are some CI failures that still need to be addressed, but those don't need to block review. I won't be the primary reviewer here, but some things I can flag from a quick skim while this is waiting for a full review:

  • This will need more testing. Looking at iOS for instance, there are almost 400 lines of tests of this trivial passthrough statement, but no testing at all of the ~300 lines of complex branching Obj-C code. The native Android code is much simpler, but also entirely untested. Future changes to this package could completely break this feature, up to the point of crashing the app on any attempt to get track info, without any tests failing. The PR will need integration tests and/or native unit tests. (I would expect integration tests to be straightforward here since there's no UI interaction involved.)

  • It's not clear that the iOS code is ready for review. This code has comments describing logic that doesn't exist, and unused variable declarations that appear to be placeholders.

  • The iOS code has comments saying that HLS needs to be handled differently and then appears to switch critical logic on an unreliable guess based on the URL; it's not clear why that guesswork is necessary vs actually inspecting the data you are looking for to see if it's there. If that's actually required, the code will need to explain why.

  • The iOS code currently does all of the complex adaptation between the platform interface and the underlying native objects in Objective-C. I would strongly recommend doing most of that work in Dart instead:

    • It will be easier to unit test.
    • It will likely be easier to iterate on review feedback, since it looks like you are more familiar with Dart than Objective-C.
    • This package in particular is moving toward doing as much as possible in Dart rather than native code.

    You could define two Pigeon data classes that would closely mirror the underlying data from the two codepaths, make them both inherit from an empty sealed base class, and then return one of those from the Pigeon getAudioTracks depending on the case, then do all the transforms, switching, etc. to make it match the platform interface format in Dart.


// Check if this is an HLS stream
if ([urlString containsString:@".m3u8"] ||
[urlString containsString:@"application/x-mpegURL"]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what circumstances would a MIME type appear in a URL?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p: video_player platform-android platform-ios platform-macos triage-android Should be looked at in Android triage triage-ios Should be looked at in iOS triage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants