obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,570 Commits 711 Branches 93 Tags
08080a85c741b2d5ef2feb6582b4920008710c8a
Commit Graph

3570 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Linoy Tsaban 08080a85c7 [bug fix] advanced dreambooth lora sdxl - fixes bugs described in #6486 (#6599) 
* fixes bugs:
1. redundant retraction
2. param clone
3. stopping optimization of text encoder params

* param upscaling

* style
 2024-12-23 13:02:06 +05:30
Patrick von Platen a366a8dcdb Update pr_test_peft_backend.yml to use 1 process for testing (#6613) 2024-12-23 13:02:06 +05:30
Aryan V S f3e47f4c70 FreeInit (#6315) 
* freeinit

* update freeinit implementation based on review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* fix

* another fix

* refactor

* fix timesteps missing bug

* apply suggestions from review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* add test for freeinit

* apply suggestions from review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* refactor

* fix test

* fix tensor not on same device

* update

* remove return_intermediate_results

* fix broken freeinit test

* update animatediff docs

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:06 +05:30
Steve Rhoades 5f3585a1a8 Fixes torch.compile() compatible training (#6589) 
resolve conflicts
 2024-12-23 13:02:06 +05:30
Bhavay Malhotra 1ac35a457e Update installation.md (#6438) 
* Update installation.md

* Update installation.md

* Update installation.md
 2024-12-23 13:02:06 +05:30
Celestial Phineas 9ff776cff6 [Fix] Multiple image conditionings in a single batch for StableDiffusionControlNetPipeline (#6334) 
* [Fix] Multiple image conditionings in a single batch for `StableDiffusionControlNetPipeline`.

* Refactor `check_inputs` in `StableDiffusionControlNetPipeline` to avoid redundant codes.

* Make the behavior of MultiControlNetModel to be the same to the original ControlNetModel

* Keep the code change minimum for nested list support

* Add fast test `test_inference_nested_image_input`

* Remove redundant check for nested image condition in `check_inputs`

Remove `len(image) == len(prompt)` check out of `check_image()`

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Better `ValueError` message for incompatible nested image list size

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Fix syntax error in `check_inputs`

* Remove warning message for multi-ControlNets with multiple prompts

* Fix a typo in test_controlnet.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Add test case for multiple prompts, single image conditioning in `StableDiffusionMultiControlNetPipelineFastTests`

* Improved `ValueError` message for nested `controlnet_conditioning_scale`

* Documenting the behavior of image list as `StableDiffusionControlNetPipeline` input

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:06 +05:30
Sayak Paul 9017578d88 remove omegaconf from the residues 👋 (#6600) 
remove omegaconf 👋
 2024-12-23 13:02:06 +05:30
JuanCarlosPi c9ffa6cd90 Change in ip-adapter docs. CLIPVisionModelWithProjection should be im… (#6597) 
Change in ip-adapter docs. CLIPVisionModelWithProjection should be imported from transformers, not diffusers
 2024-12-23 13:02:06 +05:30
Yondon Fu f6eb3d1a41 [SVD] Return np.ndarray when output_type="np" (#6507) 
[SVD] Fix output_type="np"
 2024-12-23 13:02:06 +05:30
Steve Rhoades b74ce41004 Fixes training resuming: Advanced Dreambooth LoRa Training (#6566) 
* Fixes #6418 Advanced Dreambooth LoRa Training

* change order of import to fix nit

* fix nit, use cast_training_params

* remove torch.compile fix, will move to a new PR

* remove unnecessary import
 2024-12-23 13:02:06 +05:30
Charchit Sharma 9144ed9335 Use of Posix to better support Windows compatibility in testing_utils (#6587) 
* changes in utils

* removed loc
 2024-12-23 13:02:06 +05:30
Charchit Sharma 12a87c273c change to posix for better Windows support for lora loaders (#6590) 
* posix lora

* changes and style fix
 2024-12-23 13:02:05 +05:30
Fabio Rigano 88d23edabe Fix path generation in IP Adapter (#6564) 
* Fix path generation on Windows

* Update set_default_attn_processors

* Use pathlib

* Fix quality

* Fix copy

* Revert changes in set_default_attn_processors

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
YiYi Xu 4075c49087 update slow test for SDXL k-diffusion pipeline (#6588) 
update expected slice
 2024-12-23 13:02:05 +05:30
Dong f8b5cfd0a1 🐛 fix ip-adapter controlnet img2img missing code (#6528) 
* 🐛 fix ip-adapter controlnet img2img missing code

* 📝 edit test

* 📝 edit test

* 📝 run make style and quality

* 🎨 remove slow tests
 2024-12-23 13:02:05 +05:30
SangKim 766a1d2802 Enable image resizing to adjust its height and width in StableDiffusionXLInstructPix2PixPipeline (#6581) 
* Enable image resizing to adjust its height and width in StableDiffusionXLInstructPix2PixPipeline

* Ensure that validation is performed at every 'validation_step', not at every step
 2024-12-23 13:02:05 +05:30
Aryan V S 4e8b2b3b71 [training] fix training resuming problem for fp16 (SD LoRA DreamBooth) (#6554) 
* fix training resume

* update

* update
 2024-12-23 13:02:05 +05:30
Patrick von Platen 3654809062 [Docs] Fix controlnet indent (#6578) 2024-12-23 13:02:05 +05:30
Fabio Rigano fb5bc6539b [Community Pipeline] IPAdapter FaceID (#6276) 
* Add support for IPAdapter FaceID

* Add docs

* Move subfolder to kwargs

* Fix quality

* Fix image encoder loading

* Fix loading + add test

* Move to community folder

* Fix style

* Revert constant update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
Aryan V S a4f13da91e [AnimateDiff+Controlnet] Fix multicontrolnet support (#6551) 
* fix multicontrolnet support

* update README with multicontrolnet example
 2024-12-23 13:02:05 +05:30
Sayak Paul ea85a1140a [OmegaConf] replace it with yaml (#6488) 
* remove omegaconf from convert_from_ckpt.

* remove from single_file.

* change to string based ubscription.

* style

* okay

* fix: vae_param

* no . indexing.

* style

* style

* turn getattrs into explicit if/else

* style

* propagate changes to ldm_uncond.

* propagate to gligen

* propagate to if.

* fix: quotes.

* propagate to audioldm.

* propagate to audioldm2

* propagate to musicldm.

* propagate to vq_diffusion

* propagate to zero123.

* remove omegaconf from diffusers codebase.
 2024-12-23 13:02:05 +05:30
Haofan Wang 05cddb9407 Fix a bug of flip in SDXL training script (#6547) 
* Update train_text_to_image_sdxl.py

* Update train_text_to_image_lora_sdxl.py
 2024-12-23 13:02:05 +05:30
Charchit Sharma af863c203a Make T2I Adapter SDXL Training Script torch.compile compatible (#6577) 
update for t2i_adapter
 2024-12-23 13:02:05 +05:30
dependabot[bot] 8f47dc1050 Bump jinja2 from 3.1.2 to 3.1.3 in /examples/research_projects/realfill (#6539) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-23 13:02:05 +05:30
Charchit Sharma 73712e2962 Make InstructPix2Pix SDXL Training Script torch.compile compatible (#6576) 
* changes for pix2pix_sdxl

* style fix
 2024-12-23 13:02:05 +05:30
Charchit Sharma acbb060fed Make InstructPix2Pix Training Script torch.compile compatible (#6558) 
* added torch.compile for pix2pix

* required changes
 2024-12-23 13:02:05 +05:30
Vinh H. Pham 1fbf1f6d2e Make text-to-image SDXL LoRA Training Script torch.compile compatible (#6556) 
make compile compatible
 2024-12-23 13:02:05 +05:30
Vinh H. Pham 53c73a4f14 Make text-to-image SD LoRA Training Script torch.compile compatible (#6555) 
make compile compatible
 2024-12-23 13:02:05 +05:30
gzguevara 975a00e268 SDXL text-to-image torch compatible (#6550) 
* torch compatible

* code quality fix

* ruff style

* ruff format
 2024-12-23 13:02:05 +05:30
Sayak Paul 7dbaf28563 [Training Utils] create a utility for casting the lora params during training. (#6553) 
create a utility for casting the lora params during training.
 2024-12-23 13:02:05 +05:30
Sayak Paul deaa152c86 [Training] fix training resuming problem when using FP16 (SDXL LoRA DreamBooth) (#6514) 
* fix: training resume from fp16.

* add: comment

* remove residue from another branch.

* remove more residues.

* thanks to Younes; no hacks.

* style.

* clean things a bit and modularize _set_state_dict_into_text_encoder

* add comment about the fix detailed.
 2024-12-23 13:02:05 +05:30
Vinh H. Pham 35c89defcd Make Dreambooth SD Training Script torch.compile compatible (#6532) 
* support compile

* make style

* move unwrap_model inside function

* change unwrap call

* run make style

* Update examples/dreambooth/train_dreambooth.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Revert "Update examples/dreambooth/train_dreambooth.py"

This reverts commit 70ab09732e.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
gzguevara 5d2fad2c67 SD text-to-image torch compile compatible (#6519) 
* added unwrapper

* fiz typo
 2024-12-23 13:02:05 +05:30
Suvaditya Mukherjee 5d685849ce Make ControlNet SD Training Script torch.compile compatible (#6525) 
* update: make controlnet script torch compile compatible

Signed-off-by: Suvaditya Mukherjee <suvadityamuk@gmail.com>

* update: correct earlier mistakes for compilation

Signed-off-by: Suvaditya Mukherjee <suvadityamuk@gmail.com>

* update: fix code style issues

Signed-off-by: Suvaditya Mukherjee <suvadityamuk@gmail.com>

---------

Signed-off-by: Suvaditya Mukherjee <suvadityamuk@gmail.com>
 2024-12-23 13:02:05 +05:30
Charchit Sharma 097f789ec7 Make ControlNet SDXL Training Script torch.compile compatible (#6526) 
* make torch.compile compatible

* fix quality
 2024-12-23 13:02:05 +05:30
Vinh H. Pham 8ae8108458 Make Dreambooth SD LoRA Training Script torch.compile compatible (#6534) 
support compile
 2024-12-23 13:02:05 +05:30
Radamés Ajna bca9ff76fc [DPO Training] pass tracker name as argument (#6542) 
pass tracker name as argumentw
 2024-12-23 13:02:05 +05:30
Yassine El Boudouri fe2236ee67 Remove conversion to RGB (#6479) 
* Remove conversion to RGB

* Add a Conversion Function

* Add type hint for convert_method

* Update src/diffusers/utils/loading_utils.py

Update docstring

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update docstring

* Optimize imports

* Optimize imports (2)

* Reformat code

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
Sayak Paul 7dc2d15219 [SVD] support generators that are created on GPU (#6484) 
* debug generator

* fix?

* fix?

* fix

* remove print.

* revert none check
 2024-12-23 13:02:05 +05:30
Aryan V S 27780993ca [Community] StyleAligned Pipeline (#6489) 
* add stylealigned sdxl pipeline

* bugfix

* update docs

* remove einops dependency

* update README

* update example docstring
 2024-12-23 13:02:05 +05:30
Sayak Paul b68464751f [Training] make checkpointing compatible when using torch.compile (part II) (#6511) 
make checkpointing compatible when using torch.compile.
 2024-12-23 13:02:05 +05:30
jquintanilla4 16799edf09 .load_ip_adapter in StableDiffusionXLAdapterPipeline (#6246) 
* Added testing notebook and .load_ip_adapter to XLAdapterPipeline

* Added annotations

* deleted testing notebook

* Update src/diffusers/pipelines/t2i_adapter/pipeline_stable_diffusion_xl_adapter.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* code clean up

* Add feature_extractor and image_encoder to components

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:05 +05:30
dg845 5debde1e17 Fix bug in LCM Distillation Scripts when args.unet_time_cond_proj_dim is used (#6523) 
* Fix bug where unet's time_cond_proj_dim is not set correctly if using args.unet_time_cond_proj_dim.

* make style
 2024-12-23 13:02:05 +05:30
Steven Liu 05322fb98d [docs] mask_blur and padding_mask_crop (#6498) 
new inpaint features
 2024-12-23 13:02:05 +05:30
Steven Liu 1a0673574e [docs] Callbacks (#6471) 
edits
 2024-12-23 13:02:05 +05:30
antoine-scenario 2d0b3e6ce4 Add IP-Adapter to StableDiffusionXLControlNetImg2ImgPipeline (#6293) 
* add IP-Adapter to StableDiffusionXLControlNetImg2ImgPipeline

Update src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl_img2img.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

fix tests

* fix failing test

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
YiYi Xu abcfcea5e2 [Refactor] splitingResnetBlock2D into multiple blocks (#6166) 
---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
YiYi Xu b1b6a90be3 add StableDiffusionXLKDiffusionPipeline (#6447) 
---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2024-12-23 13:02:05 +05:30
Rahul Raman 5c8115c765 example: Train Instruct pix2 pix with lora implementation (#6469) 
* base template file - train_instruct_pix2pix.py

* additional import and parser argument requried for lora

* finetune only instructpix2pix model -- no need to include these layers

* inject lora layers

* freeze unet model -- only lora layers are trained

* training modifications to train only lora parameters

* store only lora parameters

* move train script to research project

* run quality and style code checks

* move train script to a new folder

* add README

* update README

* update references in README

---------

Co-authored-by: Rahul Raman <rahulraman@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:05 +05:30
Steven Liu db484b5494 [docs] Stable video diffusion (#6472) 
svd
 2024-12-23 13:02:05 +05:30