Creating data I couldn't find when I was researching: Pro 6000, 5090, 4090, 5060 benchmarks

Both when I was upgrading from my 4090 to my 5090 and from my 5090 to my RTX Pro 6000, I couldn't find solid data of how Stable Diffusion would perform. So I decided to fix that as best I could with some benchmarks. Perhaps it will help you.

I'm also SUPER interested if someone has a RTX Pro 6000 Max-Q version, to compare it and add it to the data. The benchmark workflows are mostly based around the ComfyUI default workflows for ease of re-production, with a few tiny changes. Will link below.

Testing methodology was to run once to pre-cache everything (so I'm testing the cards more directly and not the PCIE lanes or hard drive speed), then run three times and take the average. Total runtime is pulled from ComfyUI queue (so includes things like image writing, etc, and is a little more true to life for your day to day generations), it/s is pulled from console reporting. I also monitored GPU usage and power draw to ensure cards were not getting bottlenecked.

https://preview.redd.it/p7n8gpz5i17g1.png?width=1341&format=png&auto=webp&s=46c58aac5f862826001d882a6fd7077b8cf47c40

https://preview.redd.it/p2e7otbgl17g1.png?width=949&format=png&auto=webp&s=4ece8d0b9db467b77abc9d68679fb1d521ac3568

Some interesting observations here:

\- The Pro 6000 can be significantly (1.5x) faster than a 5090

\- Overall a 5090 seems to be around 30% faster than a 4090

\- In terms of total power used per generation, the RTX Pro 6000 is by far the most power efficient.

I also wanted to see what power level I should run my cards at. Almost everything I read says "Turn down your power to 90/80/50%! It's almost the same speed and you use half the power!"

https://preview.redd.it/vjdu878aj17g1.png?width=925&format=png&auto=webp&s=cb1069bc86ec7b85abd4bdd7e1e46d17c46fdadc

https://preview.redd.it/u2wdsxebj17g1.png?width=954&format=png&auto=webp&s=54d8cf06ab378f0d940b3d0b60717f8270f2dee1

This appears not to be true. For both the pro and consumer card, I'm seeing a nearly linear loss in performance as you turn down the power.

Fun fact: At about 300 watts, the Pro 6000 is nearly as fast as the 5090 at 600W.

And finally, was curious about fp16 vs fp8, especially when I started running into ComfyUI offloading the model on the 5060. This needs to be explored more thoroughly, but here's my data for now:

https://preview.redd.it/0cdgw1i9k17g1.png?width=1074&format=png&auto=webp&s=776679497a671c4de3243150b4d826b6853d85b4

In my very limited experimentation, switching from fp16 to fp8 on a Pro 6000 was only a 4% speed increase. Switching on the 5060 Ti and allowing the model to run on the card only came in at 14% faster, which surprised me a little. I think the new Comfy architecture must be doing a really good job with offload management.


Benchmark workflows download (mostly the default ComfyUI workflows, with any changes noted on the spreadsheet):

http://dl.dropboxusercontent.com/scl/fi/iw9chh2nsnv9oh5imjm4g/SD\_Benchmarks.zip?rlkey=qdzy6hdpfm50d5v6jtspzythl&st=fkzgzmnr&dl=0

https://redd.it/1plwzwg
@rStableDiffusion

PromptCraft(Prompt-Forge) is available on github ! ENJOY !

https://redd.it/1plycql
@rStableDiffusion

To be very clear: as good as it is, Z-Image is NOT multi-modal or auto-regressive, there is NO difference whatsoever in how it uses Qwen relative to how other models use T5 / Mistral / etc. It DOES NOT "think" about your prompt and it never will. It is a standard diffusion model in all ways.

A lot of people seem extremely confused about this and appear to be convinced that Z-Image is something it isn't and never will be (the somewhat misleadingly worded, perhaps intentionally but perhaps not, blurbs on various parts of the Z-Image HuggingFace being mostly to blame).

TLDR it loads Qwen the SAME way that any other model loads any other text encoder, it's purely processing with absolutely none of the typical Qwen chat format personality being "alive". This is why for example it also cannot refuse prompts that Qwen certainly otherwise would if you had it loaded in a conventional chat context on Ollama or in LMStudio.

https://redd.it/1pm5vw0
@rStableDiffusion

It turns out that weight size matters quite a lot with Kandinsky 5

fp8

bf16

Sorry for the boring video, I initially set out to do some basics with CFG on the Pro 5s T2V model, and someone asked which quant I was using, so I did this comparison while I was at it. This is same seed/settings, the only difference here is fp8 vs bf16. I'm used to most models having small accuracy issues, but this is practically a whole different result, so I thought I'd pass this along here.

Workflow: https://pastebin.com/daZdYLAv

edit: Crap! I uploaded the wrong video for bf16, this is the proper one:

proper bf16



https://redd.it/1pm4y7t
@rStableDiffusion

Lots of fun with Z-Image Turbo

https://redd.it/1pm7qqn
@rStableDiffusion

Simplest method increase the variation in z-image turbo

from https://www.bilibili.com/video/BV1Z7m2BVEH2/

Add a new K-sampler at the front of the original K-sampler The scheduler uses ddim_uniform, running only one step, with the rest remaining unchanged.

https://preview.redd.it/i7b9dajcd47g1.png?width=1688&format=png&auto=webp&s=8555bc28187e53edf922a1baaf7014b694415708

same prompt for 15 fig test

https://redd.it/1pm82hf
@rStableDiffusion

REALISTIC - WHERE IS WALDO? USING FLUX (test)
https://redd.it/1pm95c1
@rStableDiffusion

Release v1.0 - Minimalist ComfyUI Gradio extension

https://redd.it/1pmaa6c
@rStableDiffusion