obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,596 Commits 711 Branches 93 Tags
d230b9812835a8ce0942ff1ca569983f535eaef7
Commit Graph

4596 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
glide-the d230b98128 fix: CogVideox train dataset _preprocess_data crop video (#9574) 
* Removed int8 to float32 conversion (`* 2.0 - 1.0`) from `train_transforms` as it caused image overexposure.

Added `_resize_for_rectangle_crop` function to enable video cropping functionality. The cropping mode can be configured via `video_reshape_mode`, supporting options: ['center', 'random', 'none'].

* The number 127.5 may experience precision loss during division operations.

* wandb request pil image Type

* Resizing bug

* del jupyter

* make style

* Update examples/cogvideo/README.md

* make style

---------

Co-authored-by: --unset <--unset>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul 83b656d5b2 Update distributed_inference.md to include transformer.device_map (#9553) 
* Update distributed_inference.md to include `transformer.device_map`

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Eliseu Silva 252d0ed510 Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576) (#9587) 
* Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576)
 2024-12-23 13:02:16 +05:30
Yijun Lee 79798698fb refac: docstrings in import_utils.py (#9583) 
* refac: docstrings in import_utils.py

* Update import_utils.py
 2024-12-23 13:02:16 +05:30
captainzz c746126c37 fix vae dtype when accelerate config using --mixed_precision="fp16" (#9601) 
* fix vae dtype when accelerate config using --mixed_precision="fp16"

* Add param for upcast vae
 2024-12-23 13:02:16 +05:30
Sayak Paul 12fec91937 [Chore] add a note on the versions in Flux LoRA integration tests (#9598) 
add a note on the versions.
 2024-12-23 13:02:16 +05:30
Clem 4528d4ad91 fix xlabs FLUX lora conversion typo (#9581) 
* fix startswith syntax in xlabs lora conversion

* Trigger CI

https://github.com/huggingface/diffusers/pull/9581#issuecomment-2395530360
 2024-12-23 13:02:16 +05:30
YiYi Xu 1800f5ae89 [sd3] make sure height and size are divisible by 16 (#9573) 
* check size

* up
 2024-12-23 13:02:16 +05:30
Xiangchendong bdeff1ec64 fix cogvideox autoencoder decode (#9569) 
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Darren Hsu d3f668a45d Support bfloat16 for Upsample2D (#9480) 
* Support bfloat16 for Upsample2D

* Add test and use is_torch_version

* Resolve comments and add decorator

* Simplify require_torch_version_greater_equal decorator

* Run make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
JuanCarlosPi 91360def68 Add PAG support to StableDiffusionControlNetPAGInpaintPipeline (#8875) 
* Add pag to controlnet inpainting pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 4548843ba7 Add beta sigmas to other schedulers and update docs (#9538) 2024-12-23 13:02:16 +05:30
Sayak Paul 35d704ca70 [LoRA] support Kohya Flux LoRAs that have text encoders as well (#9542) 
* support kohya flux loras that have tes.
 2024-12-23 13:02:16 +05:30
Sayak Paul df90ea984a [chore] fix: retain memory utility. (#9543) 
* fix: retain memory utility.

* fix

* quality

* free_memory.
 2024-12-23 13:02:16 +05:30
Anand Kumar 91d4b38e0a [train_instruct_pix2pix.py]Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9316) 
Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan bb43d9933c [refactor] remove conv_cache from CogVideoX VAE (#9524) 
* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 2a38836db8 [Core] fix variant-identification. (#9253) 
* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 9405b1a2e4 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-12-23 13:02:16 +05:30
PromeAI 19399207ae [examples] add train flux-controlnet scripts in example. (#9324) 
* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 33a77c6a99 [LoRA] make set_adapters() method more robust. (#9535) 
* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan f018acdb11 [bug] Precedence of operations in VAE should be slicing -> tiling (#9342) 
* bugfix: precedence of operations should be slicing -> tiling

* fix typo

* fix another typo

* deprecate current implementation of tiled_encode and use new impl

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Álvaro Somoza 5cef1c58bf [Tests] Fix ChatGLMTokenizer (#9536) 
fix
 2024-12-23 13:02:16 +05:30
YiYi Xu 143fa19bbf flux controlnet fix (control_modes batch & others) (#9507) 
* flux controlnet mode to take into account batch size

* incorporate yiyixuxu's suggestions (cleaner logic) as well as clean up control mode handling for multi case

* fix

* fix use_guidance when controlnet is a multi and does not have config

---------

Co-authored-by: Christopher Beckham <christopher.j.beckham@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky a4baaae02b [Schedulers] Add beta sigmas / beta noise schedule (#9509) 
Add beta sigmas / beta noise schedule
 2024-12-23 13:02:16 +05:30
v2ray 6f9f26f8f8 [Doc] Improved level of clarity for latents_to_rgb. (#9529) 
Fixed latents_to_rgb doc.

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
YiYi Xu 4ed3249840 update get_parameter_dtype (#9526) 
* up

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Steven Liu b142eea26d [docs] Model sharding (#9521) 
* flux shard

* feedback
 2024-12-23 13:02:16 +05:30
Lee Penkman c45940d23b Update community_projects.md (#9266) 2024-12-23 13:02:16 +05:30
Sayak Paul d8244b52a9 [Community Pipeline] Batched implementation of Flux with CFG (#9513) 
* batched implementation of flux cfg.

* style.

* readme

* remove comments.
 2024-12-23 13:02:16 +05:30
hlky 0e22145b93 Add exponential sigmas to other schedulers and update docs (#9518) 2024-12-23 13:02:16 +05:30
YiYi Xu 30dd52ce90 a few fix for SingleFile tests (#9522) 
* update sd15 repo

* update more
 2024-12-23 13:02:16 +05:30
Sayak Paul eec17ced14 [CI] allow faster downloads from the Hub in CI. (#9478) 
* allow faster downloads from the Hub in CI.

* HF_HUB_ENABLE_HF_TRANSFER: 1

* empty

* empty

* remove ENV HF_HUB_ENABLE_HF_TRANSFER=1.

* empty
 2024-12-23 13:02:16 +05:30
LukeLin 775f163ebc [Doc] Fix path and and also import imageio (#9506) 
* Fix bug

* import imageio
 2024-12-23 13:02:16 +05:30
captainzz c4e77f1ee1 fix bugs for sd3 controlnet training (#9489) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 74c74629b3 Add Noise Schedule/Schedule Type to Schedulers Overview documentation (#9504) 
* Add Noise Schedule/Schedule Type to Schedulers Overview docs

* Update docs/source/en/api/schedulers/overview.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Seongbin Lim 078286aa85 Allow DDPMPipeline half precision (#9222) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 7d92452bf9 [Schedulers] Add exponential sigmas / exponential noise schedule (#9499) 
* exponential sigmas

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
pibbo88 715a6c99a2 Fix the bug of sd3 controlnet training when using gradient checkpointing. (#9498) 
Fix the bug of sd3 controlnet training when using gradient_checkpointing. Refer to issue #9496
 2024-12-23 13:02:16 +05:30
M Saqlain 298963abbe [Tests] Reduce the model size in the lumina test (#8985) 
* Reduced model size for lumina-tests

* Handled failing tests
 2024-12-23 13:02:16 +05:30
Sayak Paul 8fb47719db [Cog] some minor fixes and nits (#9466) 
* fix positional arguments in check_inputs().

* add video and latetns to check_inputs().

* prep latents_in_channels.

* quality

* multiple fixes.

* fix
 2024-12-23 13:02:16 +05:30
Sayak Paul 53786f5eab [CI] fix nightly model tests (#9483) 
* check if default attn procs fix it.

* print

* print

* replace

* style./

* replace revision with variant.

* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.

* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.

* fix
 2024-12-23 13:02:16 +05:30
Aryan 91aa68b360 [refactor] LoRA tests (#9481) 
* refactor scheduler class usage

* reorder to make tests more readable

* remove pipeline specific checks and skip tests directly

* rewrite denoiser conditions cleaner

* bump tolerance for cog test
 2024-12-23 13:02:16 +05:30
Vladimir Mandic e1ea942b42 Several fixes to Flux ControlNet pipelines (#9472) 
* fix flux controlnet pipelines

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan fd15a9d7bd [training] CogVideoX Lora (#9302) 
* cogvideox lora training draft

* update

* update

* update

* update

* update

* make fix-copies

* update

* update

* apply suggestions from review

* apply suggestions from reveiw

* fix typo

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix lora alpha

* use correct lora scaling for final test pipeline

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* apply suggestions from review; prodigy optimizer

YiYi Xu <yixu310@gmail.com>

* add tests

* make style

* add README

* update

* update

* make style

* fix

* update

* add test skeleton

* revert lora utils changes

* add cleaner modifications to lora testing utils

* update lora tests

* deepspeed stuff

* add requirements.txt

* deepspeed refactor

* add lora stuff to img2vid pipeline to fix tests

* fight tests

* add co-authors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

Co-Authored-By: zR <2448370773@qq.com>

* fight lora runner tests

* import Dummy optim and scheduler only wheh required

* update docs

* add coauthors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

* remove option to train text encoder

Co-Authored-By: bghira <bghira@users.github.com>

* update tests

* fight more tests

* update

* fix vid2vid

* fix typo

* remove lora tests; todo in follow-up PR

* undo img2vid changes

* remove text encoder related changes in lora loader mixin

* Revert "remove text encoder related changes in lora loader mixin"

This reverts commit f8a8444487.

* update

* round 1 of fighting tests

* round 2 of fighting tests

* fix copied from comment

* fix typo in lora test

* update styling

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: zR <2448370773@qq.com>
Co-authored-by: Fu-Yun Wang <1697256461@qq.com>
Co-authored-by: bghira <bghira@users.github.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 240f753324 [Flux] add lora integration tests. (#9353) 
* add lora integration tests.

* internal note

* add a skip marker.
 2024-12-23 13:02:16 +05:30
Anatoly Belikov bef6cfa785 adapt masked im2im pipeline for SDXL (#7790) 
* adapt masked im2im pipeline for SDXL

* usage for masked im2im stable diffusion XL pipeline

* style

* style

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan cb3a0defb9 set max_shard_size to None for pipeline save_pretrained (#9447) 
* update default max_shard_size

* add None check to fix tests

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan 0494cbd01a Remove CogVideoX mentions from single file docs; Test updates (#9444) 
* remove mentions from single file

* update tests

* update
 2024-12-23 13:02:16 +05:30
Subho Ghosh 4c08316cf7 Feature flux controlnet img2img and inpaint pipeline (#9408) 
* Implemented FLUX controlnet support to Img2Img pipeline
 2024-12-23 13:02:16 +05:30
Linoy Tsaban 502191f3a6 [Flux with CFG] add flux pipeline with cfg support (#9445) 
* true_cfg

* add check negative prompt/embeds inputs

* move to community pipelines

* move to community pipelines

* revert true cfg changes to the orig pipline

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30