[WIP] VRAM Suite — Declarative VRAM Management Layer for PyTorch / ComfyUI

I've been developing an experimental runtime-level framework called VRAM Suite — a declarative meta-layer designed to predict and orchestrate GPU memory behavior during graph execution.

The project started as an internal debugging tool and gradually evolved into a minimal architecture for VRAM state modeling, fragmentation analysis, and predictive release scheduling.


---

Core Concept

Instead of profiling memory usage after the fact, VRAM Suite introduces a predictive orchestration layer that manages VRAM pressure before out-of-memory conditions occur.

It uses an abstract resource denoscriptor (.vramcard) and a runtime guard to coordinate allocation bursts across independent workflow nodes.

‐--

Architecture Overview

.vramcard
A JSON-based denoscriptor that defines the VRAM state at each workflow phase (reserved, allocated, released, predicted_peak).

VRAM Reader
Collects live telemetry from the CUDA allocator (total, reserved, active, fragmented).
Lightweight and independent of PyTorch internals.

VRAM Guard
Implements a phase-based memory orchestration model.
Tracks allocation patterns between nodes and predicts release windows using lag between alloc_peak and release_time.

Workflow Profiler (WIP)
Integrates with ComfyUI node graphs to visualize per-node VRAM utilization and allocation overlap.


---

Technical Notes

Runtime: PyTorch ≥ 2.10 (CUDA 13.0)

Environment: Ubuntu 24.04 (WSL2)

Error margin of VRAM prediction: ~3%

No modification of CUDACachingAllocator

Designed for ComfyUI custom node interface



---

Motivation

Current ComfyUI pipelines fail under complex chaining (VAE → LoRA → Refiner) due to unpredictable fragmentation.
Allocator caching helps persistence, but not orchestration.
VRAM Suite models the temporal structure of allocations, providing a deterministic headroom window for each node execution.


---

Roadmap

Public repository and documentation release: within a week

Initial tests will include:

sample .vramcard schema

early Guard telemetry logs

ComfyUI integration preview




---

TL;DR

> VRAM Suite introduces a declarative and predictive layer for VRAM management in PyTorch / ComfyUI.
The first MVP is functional, with open testing planned in the coming week.

https://redd.it/1oouthy
@rStableDiffusion

I'm making a turn back to older models for a reason.

I guess this sub mostly knows me for my startup "Mann-E" which was and is focused on image generation. I personally enjoy making and modifying models and not joking, I love doing this stuff.

Honestly the whole beast of a startup I own now has been started as my hobby of modifying and fine-tuning models in my spare time. But nowadays, models get so big that there is no difference between Qwen Image or Nano Banana, for utilizing both - as long as you don't have a big enough GPU - you may need a cloud based solution or an API which is not really "open source" anymore.

So I just took a U-turn to SDXL, but I just want to make it a "personal project" of mine now. Not a startup, but a personal project with some new concepts and ideas.

Firstly, I am thinking of using Gemma (maybe 1b or even 270m) as the text encoder of the model. I know there was a gemma based model so it makes it easier to utilize it (maybe even 12B or 27B for bigger GPU's and the purpose of multilinguality).

Second, I am thinking of that we always had image editing abilities in this game of open models right? Why not having it again? I mean that it might not be nano banana, but it obviously will be a cool local product for med/low vram people who want to experiment with these models.

P.S: Also considering FLUX models, but I think quantized versions of FLUX won't have the good results of SDXL, specially the directory of artists most of SD based models (1.5 and XL) could recognize.

https://redd.it/1oozl0j
@rStableDiffusion

Considering a beefy upgrade. How much would WAN and VACE benefit from 96 GB VRAM?

Considering buying the RTX Pro 6000 with 96 GB VRAM to increase resolution and frame range in WAN. I also train models, but will mostly use it for high-end video diffusion and VFX projects. I have heard that WAN struggles with quality above 720p, but in my experience, 1-second test clips rendered in 1080p look fine. I have had good results at 1408 by 768 for about 121 frames, but hit OEM errors when going any higher on my current RTX 4090 24 GB.

I would love to hear any real-world experiences regarding maximum resolution and frame ranges with 96 GB VRAM before upgrading.

https://redd.it/1op2gqd
@rStableDiffusion

Masking and Scheduling LoRA
https://blog.comfy.org/p/masking-and-scheduling-lora-and-model-weights

https://redd.it/1op5sdw
@rStableDiffusion

I still find flux Kontext much better for image restauration once you get the intuition on prompting and preparing the images. Qwen edit ruins and changes way too much.

https://redd.it/1op7wv0
@rStableDiffusion

Messing with WAN 2.2 text-to-image

https://redd.it/1opd9y4
@rStableDiffusion

Qwen trained model wild examples both Realistic and Fantastic, Full step by step tutorial published, train with as low as 6 GB GPUs, Qwen can do amazing ultra complex prompts + emotions very well - Images generated with SwarmUI with our ultra easy to use presets - 1-Click to use

https://redd.it/1opivzh
@rStableDiffusion