Z Image lora training is solved! A new Ztuner trainer soon!
Finally, the day we have all been waiting for has arrived. On X we got the answer:
https://x.com/bdsqlsz/status/2019349964602982494
The problem was that adam8bit performs very poorly, and even AdamW and earlier it was found by a user "None9527", but now we have the answer: it is "prodigy_adv + Stochastic rounding". This optimizer will get the job done and not only this.
Soon we will get a new trainer called "Ztuner".
And as of now OneTrainer exposes Prodigy_Adv as an optimizer option and explicitly lists Stochastic Rounding as a toggleable feature for BF16/FP16 training.
Hopefully we will get this implementation soon in other trainers too.
https://redd.it/1qwj4hu
@rStableDiffusion
Finally, the day we have all been waiting for has arrived. On X we got the answer:
https://x.com/bdsqlsz/status/2019349964602982494
The problem was that adam8bit performs very poorly, and even AdamW and earlier it was found by a user "None9527", but now we have the answer: it is "prodigy_adv + Stochastic rounding". This optimizer will get the job done and not only this.
Soon we will get a new trainer called "Ztuner".
And as of now OneTrainer exposes Prodigy_Adv as an optimizer option and explicitly lists Stochastic Rounding as a toggleable feature for BF16/FP16 training.
Hopefully we will get this implementation soon in other trainers too.
https://redd.it/1qwj4hu
@rStableDiffusion
X (formerly Twitter)
青龍聖者 (@bdsqlsz) on X
Through ablation experiments and collaboration with the official team, the training problem was finally solved.
Recommended configuration now: prodigy_adv + Stochastic rounding
It has been confirmed that adam8bit performs very poorly, and adamw seems to do…
Recommended configuration now: prodigy_adv + Stochastic rounding
It has been confirmed that adam8bit performs very poorly, and adamw seems to do…
Ace step 1.5 instrument only = garbage ?
Is it me or does everyone else have the same problem ? i really just want calm southing piano music and everything i get is like dubstep .... any advices ?
https://redd.it/1qwe940
@rStableDiffusion
Is it me or does everyone else have the same problem ? i really just want calm southing piano music and everything i get is like dubstep .... any advices ?
https://redd.it/1qwe940
@rStableDiffusion
Reddit
From the StableDiffusion community on Reddit
Explore this post and more from the StableDiffusion community