obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,607 Commits 711 Branches 93 Tags
1c038d8104bc9d4eeefbc1f197c40ccc5e2baef2
Commit Graph

4607 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
M Saqlain 1c038d8104 Add Differential Diffusion to Kolors (#9423) 
* Added diff diff support for kolors img2img

* Fized relative imports

* Fized relative imports

* Added diff diff support for Kolors

* Fized import issues

* Added map

* Fized import issues

* Fixed naming issues

* Added diffdiff support for Kolors img2img pipeline

* Removed example docstrings

* Added map input

* Updated latents

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Updated `original_with_noise`

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Improved code quality

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
GSSun 67c0df279d fix IsADirectoryError when running the training code for sd3_dreambooth_lora_16gb.ipynb (#9634) 
Add files via upload

fix IsADirectoryError when running the training code
 2024-12-23 13:02:16 +05:30
Subho Ghosh 827dd12311 flux controlnet control_guidance_start and control_guidance_end implement (#9571) 
* flux controlnet control_guidance_start and control_guidance_end implement

* minor fix - added docstrings, consistent controlnet scale flux and SD3
 2024-12-23 13:02:16 +05:30
Sayak Paul 3a508bd89a [LoRA] fix dora test to catch the warning properly. (#9627) 
fix dora test.
 2024-12-23 13:02:16 +05:30
Pakkapon Phongthawee 168a50a892 make controlnet support interrupt (#9620) 
* make controlnet support interrupt

* remove white space in controlnet interrupt
 2024-12-23 13:02:16 +05:30
SahilCarterr 1d278abe60 add PAG support for SD Img2Img (#9463) 
* added pag to sd img2img pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul c8d7ed6966 [LoRA] allow loras to be loaded with low_cpu_mem_usage. (#9510) 
* allow loras to be loaded with low_cpu_mem_usage.

* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687

* low_cpu_mem_usage.

* fix-copies

* fix-copies again

* tests

* _LOW_CPU_MEM_USAGE_DEFAULT_LORA

* _peft_version default.

* version checks.

* version check.

* version check.

* version check.

* require peft 0.13.1.

* explicitly specify low_cpu_mem_usage=False.

* docs.

* transformers version 4.45.2.

* update

* fix

* empty

* better name initialize_dummy_state_dict.

* doc todos.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* style

* fix-copies

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Yijun Lee 6a5366cd56 refac/pipeline_output (#9582) 2024-12-23 13:02:16 +05:30
sanaka 752f5c215d Fix the bug that joint_attention_kwargs is not passed to the FLUX's transformer attention processors (#9517) 
* Update transformer_flux.py
 2024-12-23 13:02:16 +05:30
v2ray f17a9d2075 Fixed noise_pred_text referenced before assignment. (#9537) 
* Fixed local variable noise_pred_text referenced before assignment when using PAG with guidance scale and guidance rescale at the same time.

* Fixed style.

* Made returning text pred noise an argument.
 2024-12-23 13:02:16 +05:30
Sayak Paul 504535e8ef [LoRA] Handle DoRA better (#9547) 
* handle dora.

* print test

* debug

* fix

* fix-copies

* update logits

* add warning in the test.

* make is_dora check consistent.

* fix-copies
 2024-12-23 13:02:16 +05:30
glide-the d230b98128 fix: CogVideox train dataset _preprocess_data crop video (#9574) 
* Removed int8 to float32 conversion (`* 2.0 - 1.0`) from `train_transforms` as it caused image overexposure.

Added `_resize_for_rectangle_crop` function to enable video cropping functionality. The cropping mode can be configured via `video_reshape_mode`, supporting options: ['center', 'random', 'none'].

* The number 127.5 may experience precision loss during division operations.

* wandb request pil image Type

* Resizing bug

* del jupyter

* make style

* Update examples/cogvideo/README.md

* make style

---------

Co-authored-by: --unset <--unset>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul 83b656d5b2 Update distributed_inference.md to include transformer.device_map (#9553) 
* Update distributed_inference.md to include `transformer.device_map`

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Eliseu Silva 252d0ed510 Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576) (#9587) 
* Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576)
 2024-12-23 13:02:16 +05:30
Yijun Lee 79798698fb refac: docstrings in import_utils.py (#9583) 
* refac: docstrings in import_utils.py

* Update import_utils.py
 2024-12-23 13:02:16 +05:30
captainzz c746126c37 fix vae dtype when accelerate config using --mixed_precision="fp16" (#9601) 
* fix vae dtype when accelerate config using --mixed_precision="fp16"

* Add param for upcast vae
 2024-12-23 13:02:16 +05:30
Sayak Paul 12fec91937 [Chore] add a note on the versions in Flux LoRA integration tests (#9598) 
add a note on the versions.
 2024-12-23 13:02:16 +05:30
Clem 4528d4ad91 fix xlabs FLUX lora conversion typo (#9581) 
* fix startswith syntax in xlabs lora conversion

* Trigger CI

https://github.com/huggingface/diffusers/pull/9581#issuecomment-2395530360
 2024-12-23 13:02:16 +05:30
YiYi Xu 1800f5ae89 [sd3] make sure height and size are divisible by 16 (#9573) 
* check size

* up
 2024-12-23 13:02:16 +05:30
Xiangchendong bdeff1ec64 fix cogvideox autoencoder decode (#9569) 
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Darren Hsu d3f668a45d Support bfloat16 for Upsample2D (#9480) 
* Support bfloat16 for Upsample2D

* Add test and use is_torch_version

* Resolve comments and add decorator

* Simplify require_torch_version_greater_equal decorator

* Run make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
JuanCarlosPi 91360def68 Add PAG support to StableDiffusionControlNetPAGInpaintPipeline (#8875) 
* Add pag to controlnet inpainting pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 4548843ba7 Add beta sigmas to other schedulers and update docs (#9538) 2024-12-23 13:02:16 +05:30
Sayak Paul 35d704ca70 [LoRA] support Kohya Flux LoRAs that have text encoders as well (#9542) 
* support kohya flux loras that have tes.
 2024-12-23 13:02:16 +05:30
Sayak Paul df90ea984a [chore] fix: retain memory utility. (#9543) 
* fix: retain memory utility.

* fix

* quality

* free_memory.
 2024-12-23 13:02:16 +05:30
Anand Kumar 91d4b38e0a [train_instruct_pix2pix.py]Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9316) 
Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan bb43d9933c [refactor] remove conv_cache from CogVideoX VAE (#9524) 
* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 2a38836db8 [Core] fix variant-identification. (#9253) 
* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 9405b1a2e4 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-12-23 13:02:16 +05:30
PromeAI 19399207ae [examples] add train flux-controlnet scripts in example. (#9324) 
* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 33a77c6a99 [LoRA] make set_adapters() method more robust. (#9535) 
* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan f018acdb11 [bug] Precedence of operations in VAE should be slicing -> tiling (#9342) 
* bugfix: precedence of operations should be slicing -> tiling

* fix typo

* fix another typo

* deprecate current implementation of tiled_encode and use new impl

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Álvaro Somoza 5cef1c58bf [Tests] Fix ChatGLMTokenizer (#9536) 
fix
 2024-12-23 13:02:16 +05:30
YiYi Xu 143fa19bbf flux controlnet fix (control_modes batch & others) (#9507) 
* flux controlnet mode to take into account batch size

* incorporate yiyixuxu's suggestions (cleaner logic) as well as clean up control mode handling for multi case

* fix

* fix use_guidance when controlnet is a multi and does not have config

---------

Co-authored-by: Christopher Beckham <christopher.j.beckham@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky a4baaae02b [Schedulers] Add beta sigmas / beta noise schedule (#9509) 
Add beta sigmas / beta noise schedule
 2024-12-23 13:02:16 +05:30
v2ray 6f9f26f8f8 [Doc] Improved level of clarity for latents_to_rgb. (#9529) 
Fixed latents_to_rgb doc.

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
YiYi Xu 4ed3249840 update get_parameter_dtype (#9526) 
* up

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Steven Liu b142eea26d [docs] Model sharding (#9521) 
* flux shard

* feedback
 2024-12-23 13:02:16 +05:30
Lee Penkman c45940d23b Update community_projects.md (#9266) 2024-12-23 13:02:16 +05:30
Sayak Paul d8244b52a9 [Community Pipeline] Batched implementation of Flux with CFG (#9513) 
* batched implementation of flux cfg.

* style.

* readme

* remove comments.
 2024-12-23 13:02:16 +05:30
hlky 0e22145b93 Add exponential sigmas to other schedulers and update docs (#9518) 2024-12-23 13:02:16 +05:30
YiYi Xu 30dd52ce90 a few fix for SingleFile tests (#9522) 
* update sd15 repo

* update more
 2024-12-23 13:02:16 +05:30
Sayak Paul eec17ced14 [CI] allow faster downloads from the Hub in CI. (#9478) 
* allow faster downloads from the Hub in CI.

* HF_HUB_ENABLE_HF_TRANSFER: 1

* empty

* empty

* remove ENV HF_HUB_ENABLE_HF_TRANSFER=1.

* empty
 2024-12-23 13:02:16 +05:30
LukeLin 775f163ebc [Doc] Fix path and and also import imageio (#9506) 
* Fix bug

* import imageio
 2024-12-23 13:02:16 +05:30
captainzz c4e77f1ee1 fix bugs for sd3 controlnet training (#9489) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 74c74629b3 Add Noise Schedule/Schedule Type to Schedulers Overview documentation (#9504) 
* Add Noise Schedule/Schedule Type to Schedulers Overview docs

* Update docs/source/en/api/schedulers/overview.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Seongbin Lim 078286aa85 Allow DDPMPipeline half precision (#9222) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 7d92452bf9 [Schedulers] Add exponential sigmas / exponential noise schedule (#9499) 
* exponential sigmas

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
pibbo88 715a6c99a2 Fix the bug of sd3 controlnet training when using gradient checkpointing. (#9498) 
Fix the bug of sd3 controlnet training when using gradient_checkpointing. Refer to issue #9496
 2024-12-23 13:02:16 +05:30
M Saqlain 298963abbe [Tests] Reduce the model size in the lumina test (#8985) 
* Reduced model size for lumina-tests

* Handled failing tests
 2024-12-23 13:02:16 +05:30