Skip to content

[safetensors] parameters count based on quantization config #1673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 6, 2025
Merged

[safetensors] parameters count based on quantization config #1673

merged 1 commit into from
Aug 6, 2025

Conversation

mishig25
Copy link
Collaborator

@mishig25 mishig25 commented Aug 6, 2025

Enhance safetensors metadata parsing to support new data types and quantization configurations

  • Added support for sub-byte data types: F4, F6_E2M3, F6_E3M2, E8M0, FP4, and UE8.
  • Implemented fetching and parsing of model configuration for quantization information.
  • Updated parameter counting functions to consider quantization effects on tensor parameters.
  • Added tests to validate the handling of new data types and parameter counting for large models.

@mishig25 mishig25 requested a review from coyotte508 as a code owner August 6, 2025 08:41
@mishig25 mishig25 requested review from julien-c and SunMarc August 6, 2025 08:42
@mishig25 mishig25 merged commit db10397 into main Aug 6, 2025
4 of 5 checks passed
@mishig25 mishig25 deleted the st_quant_config branch August 6, 2025 08:50
SunMarc
SunMarc reviewed Aug 6, 2025
View reviewed changes
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, thanks for taking into account quantization for parameters count !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunMarc SunMarc SunMarc left review comments

@coyotte508 coyotte508 Awaiting requested review from coyotte508 coyotte508 is a code owner

@julien-c julien-c Awaiting requested review from julien-c

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mishig25 @SunMarc