Add fast_sampler.py with optimized sampling and VAE decoding, enhance PreviewImage #8136
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
May 15, 2025 06:54
… PreviewImage This commit introduces `fast_sampler.py`, a new module designed to enhance the performance of sampling and VAE decoding in ComfyUI. It replaces or augments functionality previously handled in `model_management.py`, providing better VRAM management, FP16 support, and tiled decoding for low-memory scenarios. Additionally, it improves the `PreviewImage` node in `nodes.py` for faster and more efficient preview generation. These changes improve efficiency, stability, and usability, particularly for GPU-based workflows. **Key Changes:** - Implemented `fast_ksampler` for optimized sampling with improved memory management, FP16 support via `torch.amp.autocast`, and `channels_last` memory format for better GPU performance. - Added `fast_vae_decode` for efficient VAE decoding, incorporating FP16 support, `channels_last`, and selective VRAM clearing to prevent out-of-memory errors. - Introduced `fast_vae_tiled_decode` for tiled VAE decoding, enabling processing of large latents on GPUs with limited VRAM by using configurable tile sizes and overlaps. - Added profiling and debugging utilities (`profile_section`, `profile_cuda_sync`) to track execution times and VRAM usage when `--profile` or `--debug` flags are enabled. - Improved VRAM management with `clear_vram`, ensuring sufficient free memory before loading models or VAE, with configurable thresholds and minimum free memory requirements. - Implemented `is_fp16_safe` to check GPU compatibility for FP16 operations, disabling them on unsupported hardware (e.g., GTX 1660/Turing). - Optimized tensor transfers with `optimized_transfer` and `optimized_conditioning` for synchronous device placement and dtype casting. - Enhanced model preloading with `preload_model`, which unloads VAE before loading U-Net to conserve VRAM and checks for already-loaded VAE to avoid redundant transfers. - Integrated `cudnn.benchmark` for for tests, disabled by default. - VRAM should now be managed efficiently. - Updated `PreviewImage` node in `nodes.py` to support adaptive resizing of preview images to a maximum dimension of ~512 pixels while preserving aspect ratio, using `Image.LANCZOS` for quality. Increased `compress_level` from 1 to 4 for faster PNG compression, optimizing preview generation. **Impact:** - Significantly reduces VRAM usage during sampling and VAE decoding, making ComfyUI more stable on GPUs with limited memory. - Improves performance for large-scale image generation through tiled decoding and FP16 optimizations. - Enhances debugging capabilities with detailed profiling and logging, aiding development and optimization. **Dependencies:** - Relies on `nodes.py` for integration with `KSampler`, `VAEDecode`, `VAEDecodeTiled`, and `PreviewImage` nodes. - Assumes compatibility with existing `ModelPatcher` functionality for model patching (e.g., in `LoraLoader`). **Notes:** - Users should enable `--profile` or `--debug` flags to access detailed performance logs. - FP16 support requires compatible GPU hardware (compute capability ≥ 8 or > 7). - Tiled decoding parameters (`tile_size`, `overlap`, etc.) may need tuning for specific workflows. - Preview images are now smaller and faster to generate, but users can adjust `max_size` in `PreviewImage` if higher resolution previews are needed. This is a foundational change to improve ComfyUI's performance and scalability, particularly for resource-constrained environments. Thanks to Grok @ xAI for help.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add fast_sampler.py with 8000 optimized sampling and VAE decoding, enhance PreviewImage
This update introduces
fast_sampler.py, a new module designed to enhance the performance of sampling and VAE decoding in ComfyUI. It replaces or augments functionality previously handled inmodel_management.py, providing better VRAM management, FP16 support, and tiled decoding for low-memory scenarios. Additionally, it improves thePreviewImagenode innodes.pyfor faster and more efficient preview generation. These changes improve efficiency, stability, and usability, particularly for GPU-based workflows.Key Changes:
fast_ksamplerfor optimized sampling with improved memory management, FP16 support viatorch.amp.autocast, andchannels_lastmemory format for better GPU performance.fast_vae_decodefor efficient VAE decoding, incorporating FP16 support,channels_last, and selective VRAM clearing to prevent out-of-memory errors.fast_vae_tiled_decodefor tiled VAE decoding, enabling processing of large latents on GPUs with limited VRAM by using configurable tile sizes and overlaps.profile_section,profile_cuda_sync) to track execution times and VRAM usage when--profileor--debugflags are enabled.clear_vram, ensuring sufficient free memory before loading models or VAE, with configurable thresholds and minimum free memory requirements.is_fp16_safeto check GPU compatibility for FP16 operations, disabling them on unsupported hardware (e.g., GTX 1660/Turing).optimized_transferandoptimized_conditioningfor synchronous device placement and dtype casting.preload_model, which unloads VAE before loading U-Net to conserve VRAM and checks for already-loaded VAE to avoid redundant transfers.cudnn.benchmarkfor for tests, disabled by default.PreviewImagenode innodes.pyto support adaptive resizing of preview images to a maximum dimension of ~512 pixels while preserving aspect ratio, usingImage.LANCZOSfor quality. Increasedcompress_levelfrom 1 to 4 for faster PNG compression, optimizing preview generation.Impact:
Dependencies:
nodes.pyfor integration withKSampler,VAEDecode,VAEDecodeTiled, andPreviewImagenodes.ModelPatcherfunctionality for model patching (e.g., inLoraLoader).Notes:
--profileor--debugflags to access detailed performance logs.tile_size,overlap, etc.) may need tuning for specific workflows.max_sizeinPreviewImageif higher resolution previews are needed.This is a foundational change to improve ComfyUI's performance and scalability, particularly for resource-constrained environments.
Thanks to Grok @ xAI for help.