System Info

transformers.js: 3.7.1

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

Hi,
I've been digging into the audio preprocessing in transformers.js and noticed an issue:

There are currently no unit tests for the audio_utils module in the JS implementation.

The output of spectrogram function in transformers.js does not match the expected results from the Python tests in test_audio_utils.py.

This can lead to inconsistencies when running models like Whisper in the browser. It would be great to align the JS behavior with the Python reference and introduce unit tests for validation.

import { spectrogram } from "@huggingface/transformers"
const waveform = new Float32Array(40);
waveform[9] = 1.0
const spectrogramResult = await spectrogram(
	waveform,
	window_function(12, "hann", {frame_length:16}),
	16, //frame_length
	4, //hop_length,
	{
		power: 1,
		center:true,
		pad_mode:"reflect",
		onesided:true,
	}
)
console.log(`expected: 0.0, 0.0669873, 0.9330127, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0`)
console.log(`result: ${spectrogramResult.data}`)
// throws transformers.js:34606 Uncaught Error: Length of the window (12) must equal frame_length (16)

After adding padding to resolve the exception, the output values still do not match these in python.

Let me know if you'd like help reproducing the issue or drafting tests. Thanks!

Reproduction

see description