Skip to content

Impossible to use LoRa #684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
DNDEBUG opened this issue May 19, 2025 · 6 comments
Open

Impossible to use LoRa #684

DNDEBUG opened this issue May 19, 2025 · 6 comments

Comments

@DNDEBUG
Copy link

DNDEBUG commented May 19, 2025 •
edited
Loading

I am using the Vulkan backend because ROCm isn't an option for gfx1103
This program is very good because i don't have to deal with the python bs
but so far i only have been able to make a single LoRa model work once, i have not been able to make it work again
its completely inconsistent, its not related to size or model, it happens to SD 1.5 and SDXL
from 10 MB models to 300 MB models
i don't believe its ram because i also tried some big checkpoint models, i tried using CPU mode to only use ram i tried tweaking all the settings but LoRa's simply don't work
i believe i am doing it right <lora:model_name:0.6>
but it just crashes, some times it gives a reason, some times it doesn't
so i don't really know what i need to do to get LoRa's working

update - its definitely not ram

so, i decided to start from 0 and made some discoveries
i was able to use lora even on SDXL models with a basic run
so it was probably some other options causing the crash
and with all my tests i found out that this program is extremely finicky
got similar results on both SD1.5 and SDXL and turbo models
sampling method can cause a crash (this one makes sense)
steps can cause a crash (tested on a turbo model with default 8 steps then 20 steps, for some reason it didn't crash with 20 steps)
lora combination, if the lora models can work together i will work perfectly, else everything crashes
for example a SD 1.5 model, some sd 1.5 models will always crash, some will work but alone, some will work with other models
so far, prompt size has not resulted in a crash, i tried a very massive prompt and it worked perfectly well
@wbruna
Copy link

wbruna commented May 20, 2025

Could you give a few more specific examples of those crashes? Preferably with the complete command line, so they can be reproduced?

@DNDEBUG
Copy link
Author

DNDEBUG commented May 20, 2025

Could you give a few more specific examples of those crashes? Preferably with the complete command line, so they can be reproduced?

it isn't clear for me now, so i will do some more tests to get a better definition of the types of crash
there are different types and some times they happen, some times they don't

@DNDEBUG
Copy link
Author

DNDEBUG commented May 21, 2025 •
edited
Loading

this one happened after loading a loRa
it works without the loRa

./sd -m "models/checkpoint/dreamshaperXL_v21TurboDPMSDE.safetensors" -p "Tall Goth girl, light skin, sunglasses, red lipstick, temple background, sword, vivid lights, high quality, realistic, realism, sfw, rating:safe, lora:PlayStation_1_SDXL:0.8" --lora-model-dir "models/lora" -n "blurry, blur, nude, naked, nsfw, lens distortion, bikini, text, watermark" --vae-tiling --vae-on-cpu --schedule karras -H 768 -W 768 --steps 8 -b 1 --seed -1

stable-diffusion.cpp/ggml/src/ggml.c:5764: GGML_ASSERT(cgraph->n_nodes < cgraph->size) failed
./sd(+0x1ca786) [0x5ffa95d45786]
./sd(+0x1cd66a) [0x5ffa95d4866a]
./sd(+0x1cd8df) [0x5ffa95d488df]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1cd7b7) [0x5ffa95d487b7]
./sd(+0x1d2203) [0x5ffa95d4d203]
./sd(+0x2a5b83) [0x5ffa95e20b83]
./sd(+0x2aabdd) [0x5ffa95e25bdd]
./sd(+0x285cf6) [0x5ffa95e00cf6]
./sd(+0x295fa3) [0x5ffa95e10fa3]
./sd(+0x2779c2) [0x5ffa95df29c2]
./sd(+0x27a862) [0x5ffa95df5862]
./sd(+0x61bd4) [0x5ffa95bdcbd4]
/usr/lib/libc.so.6(+0x2752e) [0x7bb283e3952e]
/usr/lib/libc.so.6(__libc_start_main+0x8a) [0x7bb283e395ea]
./sd(+0x6b8a5) [0x5ffa95be68a5]

@DNDEBUG
Copy link
Author

DNDEBUG commented May 21, 2025 •
edited
Loading

one consistent thing is this cgraph->n_nodes < cgraph->size thing
almost never related to memory
some times it doesn't really say what causes the crash

SD 1.5 seems very stable as long as i don't mess much with sampling method
the consistent thing with SDXL is that i can't use LoRa with it, changing sampling method changes nothing

@vargad
Copy link

vargad commented May 22, 2025 •
edited
Loading

Indeed, none of the SDXL/LoRA combinations, I tried, worked with stablediffusion.cpp. I can confirm however that this PR fixes the issue: https://github.com/leejet/stable-diffusion.cpp/pull/658/files
Note that the number should be higher for some LoRAs. I wonder how much performance hit would be to make it a runtime choice, instead of a define macro.

@DNDEBUG
Copy link
Author

DNDEBUG commented May 22, 2025

Yes, there should be an option for the user to set it during runtime instead of being hard coded

@wbruna wbruna mentioned this issue May 28, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wbruna @vargad @DNDEBUG