Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLEASE QUANTIZE MODELS (FP8 ) :) #7

Open
lijackcoder opened this issue Dec 23, 2024 · 2 comments
Open

PLEASE QUANTIZE MODELS (FP8 ) :) #7

lijackcoder opened this issue Dec 23, 2024 · 2 comments

Comments

@lijackcoder
Copy link

It would be be great if you could create fp8 versions of the models :) thanks for lower vram or faster generation

@zhuang2002
Copy link
Collaborator

Thank you so much for your attention and support. To help reduce memory overhead, since ColorFlow does not rely on text conditions, I would kindly suggest avoiding the CFG strategy and instead using empty text for inference. Additionally, you could pre-save the T5 model’s output for empty text inputs, which would allow you to skip loading the T5 model entirely during inference. I’ve had quite a lot on my plate recently, but I’ll make sure to update the code with these optimizations as soon as I can. Thank you for your understanding and patience!

@nitinmukesh
Copy link

Looking forward to memory optimizations when you get time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants