Skip to content

Add Timestep shift, SGM Uniform and Simple scheduler and support for NitroFusion #675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

rmatif
Copy link

@rmatif rmatif commented May 9, 2025

This PR adds support for the timestep-shift technique required for inference with NitroFusion models and Diff-Instruct* and other one-step models. It also adds support for two schedulers: SGM Uniform and Simple, because the existing scheduler, for a mysterious reason, fails at step 2 calculation and produces an output similar to step 1.

NitroFusion is one of the best models for single-step inference, making it useful for inference on compute-constrained devices like mobile phones or CPUs.

Example command:

./bin/sd -m nitrosd-realism_f16.gguf -v -p "cute cat" --cfg-scale 1 --steps 1 --timestep-shift 250 -H 1024 -W 1024 --seed 2024 --schedule sgm_uniform

Step NitroSD-Realism (Timestep-shift 250) NitroSD-Vibrant (Timestep-shift 500) Diff-Instruct* (Timestep-shift 400)
Step 1 1step 1step-vibrant 1step-diff-instructstar
Step 2 2step 2step-vibrant Not well supported
Step 3 3step 3step-vibrant Not well supported
Step 4 4step 4step-vibrant Not well supported

The recommended timestep-shift values by the authors are 250 for NitroSD-Realism and 500 for NitroSD-Vibrant and 400 for Diff-Instruct*.

I created GGUF versions of NitroFusion that already include the fixed SDXL VAE, available for download here.

The authors mentioned it's possible to extract LoRA weights from these models and apply them to other checkpoints. I’ll try to do that in the future.

EDIT : Just add Diff-Instruct* GGUF
References:

Timestep-shift implementation: node.py
SGM Uniform: sd_schedulers
Simple: samplers.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant