obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,627 Commits 711 Branches 93 Tags
453ababb3febc1d28ec572003b132ee7a07308bf
Commit Graph

4627 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wony617 453ababb3f [docs] refactoring docstrings in community/hd_painter.py (#9593) 
* [docs] refactoring docstrings in community/hd_painter.py

* Update examples/community/hd_painter.py

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* make style

---------

Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:17 +05:30
Jiwook Han 3e941e92a1 [doc] Fix some docstrings in src/diffusers/training_utils.py (#9606) 
* refac: docstrings in training_utils.py

* fix: manual edits

* run make style

* add docstring at cast_training_params

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Charchit Sharma 1f3bc5fce8 refactor image_processor.py file (#9608) 
* refactor image_processor file

* changes as requested

* +1 edits

* quality fix

* indent issue

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
0x名無し 1c7ea5aa29 Dreambooth lora flux bug 3dtensor to 2dtensor (#9653) 
* fixed issue #9350, Tensor is deprecated

* ran make style
 2024-12-23 13:02:17 +05:30
Sayak Paul c3c6a55bcb [Chore] fix import of EntryNotFoundError. (#9676) 
fix import of EntryNotFoundError.
 2024-12-23 13:02:17 +05:30
Robin bb20f59d83 [Fix] when run load pretain with local_files_only, local variable 'cached_folder' referenced before assignment (#9376) 
Fix local variable 'cached_folder' referenced before assignment in hub_utils.py

Fix when use `local_files_only=True` with `subfolder`, local variable 'cached_folder' referenced before assignment issue.

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky d6b2dcfa11 Slight performance improvement to Euler, EDMEuler, FlowMatchHeun, KDPM2Ancestral (#9616) 
* Slight performance improvement to Euler

* Slight performance improvement to EDMEuler

* Slight performance improvement to FlowMatchHeun

* Slight performance improvement to KDPM2Ancestral

* Update KDPM2AncestralDiscreteSchedulerTest

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky 0ed01ee10a Refactor SchedulerOutput and add pred_original_sample in DPMSolverSDE, Heun, KDPM2Ancestral and KDPM2 (#9650) 
Refactor SchedulerOutput and add pred_original_sample

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky 48f795f991 Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel (#9651) 
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky 5844c7c9d4 Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel (#9652) 
Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky be464ed2af Add pred_original_sample to if not return_dict path (#9649) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
SahilCarterr 3fdfa1d828 Added Lora Support to SD3 Img2Img Pipeline (#9659) 
* add lora
 2024-12-23 13:02:17 +05:30
Tolga Cangöz bc2cbfba5d [Community Pipeline] Add 🪆Matryoshka Diffusion Models (#9157) 2024-12-23 13:02:17 +05:30
Leo Jiang 76d9796f25 Improve the performance and suitable for NPU computing (#9642) 
* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU computing

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Yuxuan.Zhang 4ea9e828b1 CogView3Plus DiT (#9570) 
* merge 9588

* max_shard_size="5GB" for colab running

* conversion script updates; modeling test; refactor transformer

* make fix-copies

* Update convert_cogview3_to_diffusers.py

* initial pipeline draft

* make style

* fight bugs 🐛🪳

* add example

* add tests; refactor

* make style

* make fix-copies

* add co-author

YiYi Xu <yixu310@gmail.com>

* remove files

* add docs

* add co-author

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* fight docs

* address reviews

* make style

* make model work

* remove qkv fusion

* remove qkv fusion tets

* address review comments

* fix make fix-copies error

* remove None and TODO

* for FP16(draft)

* make style

* remove dynamic cfg

* remove pooled_projection_dim as a parameter

* fix tests

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Ryan Lin 0e332644bc Flux - soft inpainting via differential diffusion (#9268) 
* Flux - soft inpainting via differential diffusion

* .

* track changes to FluxInpaintPipeline

* make mask arrangement simplier

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Co-authored-by: asomoza <somoza.alvaro@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul 6b5d5218cb [Tests] increase transformers version in test_low_cpu_mem_usage_with_loading (#9662) 
increase transformers version in test_low_cpu_mem_usage_with_loading
 2024-12-23 13:02:17 +05:30
Jinzhe Pan cb671beb5c [docs] Fix xDiT doc image damage (#9655) 
* docs: fix xDiT doc image damage

* doc: move xdit images to hf dataset

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul 15bdb8a09f [CI] replace ubuntu version to 22.04. (#9656) 
replace ubuntu version to 22.04.
 2024-12-23 13:02:17 +05:30
hlky ca7a9bbf03 FluxMultiControlNetModel (#9647) 2024-12-23 13:02:17 +05:30
M Saqlain 1c038d8104 Add Differential Diffusion to Kolors (#9423) 
* Added diff diff support for kolors img2img

* Fized relative imports

* Fized relative imports

* Added diff diff support for Kolors

* Fized import issues

* Added map

* Fized import issues

* Fixed naming issues

* Added diffdiff support for Kolors img2img pipeline

* Removed example docstrings

* Added map input

* Updated latents

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Updated `original_with_noise`

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Improved code quality

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
GSSun 67c0df279d fix IsADirectoryError when running the training code for sd3_dreambooth_lora_16gb.ipynb (#9634) 
Add files via upload

fix IsADirectoryError when running the training code
 2024-12-23 13:02:16 +05:30
Subho Ghosh 827dd12311 flux controlnet control_guidance_start and control_guidance_end implement (#9571) 
* flux controlnet control_guidance_start and control_guidance_end implement

* minor fix - added docstrings, consistent controlnet scale flux and SD3
 2024-12-23 13:02:16 +05:30
Sayak Paul 3a508bd89a [LoRA] fix dora test to catch the warning properly. (#9627) 
fix dora test.
 2024-12-23 13:02:16 +05:30
Pakkapon Phongthawee 168a50a892 make controlnet support interrupt (#9620) 
* make controlnet support interrupt

* remove white space in controlnet interrupt
 2024-12-23 13:02:16 +05:30
SahilCarterr 1d278abe60 add PAG support for SD Img2Img (#9463) 
* added pag to sd img2img pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul c8d7ed6966 [LoRA] allow loras to be loaded with low_cpu_mem_usage. (#9510) 
* allow loras to be loaded with low_cpu_mem_usage.

* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687

* low_cpu_mem_usage.

* fix-copies

* fix-copies again

* tests

* _LOW_CPU_MEM_USAGE_DEFAULT_LORA

* _peft_version default.

* version checks.

* version check.

* version check.

* version check.

* require peft 0.13.1.

* explicitly specify low_cpu_mem_usage=False.

* docs.

* transformers version 4.45.2.

* update

* fix

* empty

* better name initialize_dummy_state_dict.

* doc todos.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* style

* fix-copies

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Yijun Lee 6a5366cd56 refac/pipeline_output (#9582) 2024-12-23 13:02:16 +05:30
sanaka 752f5c215d Fix the bug that joint_attention_kwargs is not passed to the FLUX's transformer attention processors (#9517) 
* Update transformer_flux.py
 2024-12-23 13:02:16 +05:30
v2ray f17a9d2075 Fixed noise_pred_text referenced before assignment. (#9537) 
* Fixed local variable noise_pred_text referenced before assignment when using PAG with guidance scale and guidance rescale at the same time.

* Fixed style.

* Made returning text pred noise an argument.
 2024-12-23 13:02:16 +05:30
Sayak Paul 504535e8ef [LoRA] Handle DoRA better (#9547) 
* handle dora.

* print test

* debug

* fix

* fix-copies

* update logits

* add warning in the test.

* make is_dora check consistent.

* fix-copies
 2024-12-23 13:02:16 +05:30
glide-the d230b98128 fix: CogVideox train dataset _preprocess_data crop video (#9574) 
* Removed int8 to float32 conversion (`* 2.0 - 1.0`) from `train_transforms` as it caused image overexposure.

Added `_resize_for_rectangle_crop` function to enable video cropping functionality. The cropping mode can be configured via `video_reshape_mode`, supporting options: ['center', 'random', 'none'].

* The number 127.5 may experience precision loss during division operations.

* wandb request pil image Type

* Resizing bug

* del jupyter

* make style

* Update examples/cogvideo/README.md

* make style

---------

Co-authored-by: --unset <--unset>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul 83b656d5b2 Update distributed_inference.md to include transformer.device_map (#9553) 
* Update distributed_inference.md to include `transformer.device_map`

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Eliseu Silva 252d0ed510 Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576) (#9587) 
* Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576)
 2024-12-23 13:02:16 +05:30
Yijun Lee 79798698fb refac: docstrings in import_utils.py (#9583) 
* refac: docstrings in import_utils.py

* Update import_utils.py
 2024-12-23 13:02:16 +05:30
captainzz c746126c37 fix vae dtype when accelerate config using --mixed_precision="fp16" (#9601) 
* fix vae dtype when accelerate config using --mixed_precision="fp16"

* Add param for upcast vae
 2024-12-23 13:02:16 +05:30
Sayak Paul 12fec91937 [Chore] add a note on the versions in Flux LoRA integration tests (#9598) 
add a note on the versions.
 2024-12-23 13:02:16 +05:30
Clem 4528d4ad91 fix xlabs FLUX lora conversion typo (#9581) 
* fix startswith syntax in xlabs lora conversion

* Trigger CI

https://github.com/huggingface/diffusers/pull/9581#issuecomment-2395530360
 2024-12-23 13:02:16 +05:30
YiYi Xu 1800f5ae89 [sd3] make sure height and size are divisible by 16 (#9573) 
* check size

* up
 2024-12-23 13:02:16 +05:30
Xiangchendong bdeff1ec64 fix cogvideox autoencoder decode (#9569) 
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Darren Hsu d3f668a45d Support bfloat16 for Upsample2D (#9480) 
* Support bfloat16 for Upsample2D

* Add test and use is_torch_version

* Resolve comments and add decorator

* Simplify require_torch_version_greater_equal decorator

* Run make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
JuanCarlosPi 91360def68 Add PAG support to StableDiffusionControlNetPAGInpaintPipeline (#8875) 
* Add pag to controlnet inpainting pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 4548843ba7 Add beta sigmas to other schedulers and update docs (#9538) 2024-12-23 13:02:16 +05:30
Sayak Paul 35d704ca70 [LoRA] support Kohya Flux LoRAs that have text encoders as well (#9542) 
* support kohya flux loras that have tes.
 2024-12-23 13:02:16 +05:30
Sayak Paul df90ea984a [chore] fix: retain memory utility. (#9543) 
* fix: retain memory utility.

* fix

* quality

* free_memory.
 2024-12-23 13:02:16 +05:30
Anand Kumar 91d4b38e0a [train_instruct_pix2pix.py]Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9316) 
Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan bb43d9933c [refactor] remove conv_cache from CogVideoX VAE (#9524) 
* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 2a38836db8 [Core] fix variant-identification. (#9253) 
* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 9405b1a2e4 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-12-23 13:02:16 +05:30
PromeAI 19399207ae [examples] add train flux-controlnet scripts in example. (#9324) 
* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
 2024-12-23 13:02:16 +05:30