Instruct-Pix2pix support #679
Conversation
|
Awesome! Could you please take a look at cosxl-edit as well? It acts as an ip2p, if I understood correctly. I think we're just missing the EDM VPred schedule |
I may take a look at it later. |
1e25a9b to
75af1bd
Compare
|
For some reason, the "image CFG" (controlled by |
|
Ah I think I found the issue. By default, the model samples the VAE distribution, bit pix2pix expects the mean of the distribution. |
|
I'm pretty sure it's working properly now. I think inpaint might be slightly improved too, especially when strength is set to <1 and with higher CFG. |
thse ones might also be interesting, and they may be even easier to implement: Edit: The SDXL one was pretty easy. Now, I can't figure out how to easily convert sd3.x models from diffusers format to the original format, so I cant test if it would work... |
|
CosXL edit is now working properly. |
cosxl: smol cleanup CosXL: fix schedule choice Rename EDMVDenoiser Avoid inf for EDMVDenoiser + discrete schedule make parametrization flags public Fix CosXL with empty negative prompts Instruct-p2p support support 2 conditionings cfg Do not re-encode the exact same image twice pix2pix: fixes for 2-cfg Fix pix2pix latent inputs + improve inpainting a bit + fix naming prepare for other pix2pix-like models Support sdxl ip2p CoxXL edit: fix reference image embeddings Support 2-cond cfg properly in cli fix typo in help
cosxl: smol cleanup CosXL: fix schedule choice Rename EDMVDenoiser Avoid inf for EDMVDenoiser + discrete schedule make parametrization flags public Fix CosXL with empty negative prompts
cosxl: smol cleanup CosXL: fix schedule choice Rename EDMVDenoiser Avoid inf for EDMVDenoiser + discrete schedule make parametrization flags public Fix CosXL with empty negative prompts Instruct-p2p support support 2 conditionings cfg Do not re-encode the exact same image twice pix2pix: fixes for 2-cfg Fix pix2pix latent inputs + improve inpainting a bit + fix naming prepare for other pix2pix-like models Support sdxl ip2p CoxXL edit: fix reference image embeddings Support 2-cond cfg properly in cli fix typo in help Support masks for ip2p models
ref: #61
sd.exe -M img2img --model instruct-pix2pix-00-22000.safetensors -p "turn him into a cyborg" --color --strength 1 -i .\example.jpg --steps 50 --cfg-scale 7.5 --img-cfg-scale 1.2 --sampling-method euler_asd.exe -M img2img --model instruct-pix2pix-00-22000.safetensors -p "Make it a cat" --strength 1 -i input.png --steps 100 --cfg-scale 7.5 --img-cfg-scale1.5 --sampling-method euler_a --schedule karrasTODOs:
(rebased on top of #683 for CosXL edit support)