Skip to content

Add CosXL support #683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add CosXL support #683

wants to merge 7 commits into from

Conversation

stduhpf
Copy link
Contributor

@stduhpf stduhpf commented May 19, 2025

Fixes #630

ComfyUI output This PR's output (same settings)
image output

@stduhpf stduhpf force-pushed the cosxl branch 2 times, most recently from 9b0c158 to a6bbebf Compare May 19, 2025 12:58
@stduhpf
Copy link
Contributor Author

stduhpf commented May 20, 2025

Hmm it looks like the --schedule argument has no effect, maybe this can be related to those artifacts...

@stduhpf
Copy link
Contributor Author

stduhpf commented May 20, 2025

That doesn't fix the other issue. Though it seems like when using AYS or Gits, the oversaturation is not as bad as with the default exponential schedule. (Discrete schedule is not working at all, maybe this is expected)

@stduhpf
Copy link
Contributor Author

stduhpf commented May 23, 2025

Looks like adding a simple negative prompt helps a lot to mitigate the overasturation issue, but it still doesn't look quite right:

output

(ComfyUI doesn't need a negative prompt to generate good images with cosXL, so something fishy is going on here)

Copy link
Contributor Author

@stduhpf stduhpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting force_zero_embeddings to false seems to also work for some reason.

@stduhpf stduhpf marked this pull request as ready for review May 23, 2025 11:19
@stduhpf stduhpf changed the title Add CosXL support (WIP, help needed) Add CosXL support May 23, 2025
@stduhpf stduhpf mentioned this pull request May 23, 2025
3 tasks
stduhpf added a commit to stduhpf/stable-diffusion.cpp that referenced this pull request May 26, 2025
cosxl: smol cleanup

CosXL: fix schedule choice

Rename EDMVDenoiser

Avoid inf for EDMVDenoiser + discrete schedule

make parametrization flags public

Fix CosXL with empty negative prompts

Instruct-p2p support

support 2 conditionings cfg

Do not re-encode the exact same image twice

pix2pix: fixes for 2-cfg

Fix pix2pix latent inputs + improve inpainting a bit + fix naming

prepare for other pix2pix-like models

Support sdxl ip2p

CoxXL edit: fix reference image embeddings

Support 2-cond cfg properly in cli

fix typo in help
stduhpf added a commit to stduhpf/stable-diffusion.cpp that referenced this pull request May 26, 2025
cosxl: smol cleanup

CosXL: fix schedule choice

Rename EDMVDenoiser

Avoid inf for EDMVDenoiser + discrete schedule

make parametrization flags public

Fix CosXL with empty negative prompts
stduhpf added a commit to stduhpf/stable-diffusion.cpp that referenced this pull request May 26, 2025
cosxl: smol cleanup

CosXL: fix schedule choice

Rename EDMVDenoiser

Avoid inf for EDMVDenoiser + discrete schedule

make parametrization flags public

Fix CosXL with empty negative prompts

Instruct-p2p support

support 2 conditionings cfg

Do not re-encode the exact same image twice

pix2pix: fixes for 2-cfg

Fix pix2pix latent inputs + improve inpainting a bit + fix naming

prepare for other pix2pix-like models

Support sdxl ip2p

CoxXL edit: fix reference image embeddings

Support 2-cond cfg properly in cli

fix typo in help

Support masks for ip2p models
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for CosXL/CosXL Edit
1 participant