Skip to content

transcribe: add speaker diarization test #12410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 20, 2025
Merged

Conversation

sannya-singal
Copy link
Contributor

@sannya-singal sannya-singal commented Mar 19, 2025

Motivation

Add a test to Localstack Transcribe with an enabled speaker partitioning/diarization. Speaker partitioning labels the speech from individual speakers in the media file.

This can be achieved by enabling ShowSpeakerLabels in the transcription settings and specifying the MaxSpeakerLabels which is the maximum number of speakers we want to partition in the media file.

Changes

This PR:

  • adds two AWS validated tests (currently being skipped from Localstack) for the speaker diarization.
  • adds a manually generated media file tests/aws/files/multi-speaker.wav with 2 speakers.

@sannya-singal sannya-singal added the semver: patch Non-breaking changes which can be included in patch releases label Mar 19, 2025
@sannya-singal sannya-singal added this to the 4.3 milestone Mar 19, 2025
@sannya-singal sannya-singal self-assigned this Mar 19, 2025
@sannya-singal sannya-singal marked this pull request as ready for review March 19, 2025 10:44
Copy link

github-actions bot commented Mar 19, 2025

LocalStack Community integration with Pro

    2 files  ± 0      2 suites  ±0   1h 50m 25s ⏱️ + 1m 38s
4 304 tests +11  3 983 ✅ +11  321 💤 ±0  0 ❌ ±0 
4 306 runs  +11  3 983 ✅ +11  323 💤 ±0  0 ❌ ±0 

Results for commit 74b5289. ± Comparison against base commit 5d2e542.

♻️ This comment has been updated with latest results.

Copy link
Member

@viren-nadkarni viren-nadkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt this test will be useful. Without the actual models used by AWS Transcribe, we will not be able to reproduce the same timing and other numerical values. It's the same reason we can't AWS validate transcription tests. That said, it doesn't hurt to have this in as a reference when diarisation is being added.

@sannya-singal
Copy link
Contributor Author

Thanks for the review 🙌 I have slightly different viewpoint here, although I agree that we won't be able to reproduce the same timing and other numerical values without the actual models being used by AWS Transcribe, but in my opinion this test could be really helpful in kicking off the speaker diarization implementation in future, where we could atleast know the currently missing attributes and functionality considering the logged AWS responses, also making it easier to understand and implement the feature of atleast having speaker_label and the associated transcripts in the response and having other mocked values.

This could be later modified according to the then implementation and skip the snapshot of certain functionality.

Copy link
Member

@viren-nadkarni viren-nadkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments. Looks good to merge 👍

@sannya-singal sannya-singal merged commit a9e6262 into master Mar 20, 2025
31 checks passed
@sannya-singal sannya-singal deleted the speaker-diarization branch March 20, 2025 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants