ec6bfdf069
Add prompt scheduling callback to community scripts ( #9718 )
2024-12-23 13:02:17 +05:30
a3e79f6e35
Fix bug in Textual Inversion Unloading ( #9304 )
...
* Update textual_inversion.py
* add unload test
* add comment
* fix style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
da0b9e1274
[advanced flux lora script] minor updates to readme ( #9705 )
...
* fix arg naming
* fix arg naming
* fix arg naming
* fix arg naming
2024-12-23 13:02:17 +05:30
afba632dec
[refactor] DiffusionPipeline.download ( #9557 )
...
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:17 +05:30
65fe4b98d9
[Flux] Add advanced training script + support textual inversion inference ( #9434 )
...
* add ostris trainer to README & add cache latents of vae
* add ostris trainer to README & add cache latents of vae
* style
* readme
* add test for latent caching
* add ostris noise scheduler
https://github.com/ostris/ai-toolkit/blob/9ee1ef2a0a2a9a02b92d114a95f21312e5906e54/toolkit/samplers/custom_flowmatch_sampler.py#L95
* style
* fix import
* style
* fix tests
* style
* --change upcasting of transformer?
* update readme according to main
* add pivotal tuning for CLIP
* fix imports, encode_prompt call,add TextualInversionLoaderMixin to FluxPipeline for inference
* TextualInversionLoaderMixin support for FluxPipeline for inference
* move changes to advanced flux script, revert canonical
* add latent caching to canonical script
* revert changes to canonical script to keep it separate from https://github.com/huggingface/diffusers/pull/9160
* revert changes to canonical script to keep it separate from https://github.com/huggingface/diffusers/pull/9160
* style
* remove redundant line and change code block placement to align with logic
* add initializer_token arg
* add transformer frac for range support from pure textual inversion to the orig pivotal tuning
* support pure textual inversion - wip
* adjustments to support pure textual inversion and transformer optimization in only part of the epochs
* fix logic when using initializer token
* fix pure_textual_inversion_condition
* fix ti/pivotal loading of last validation run
* remove embeddings loading for ti in final training run (to avoid adding huggingface hub dependency)
* support pivotal for t5
* adapt pivotal for T5 encoder
* adapt pivotal for T5 encoder and support in flux pipeline
* t5 pivotal support + support fo pivotal for clip only or both
* fix param chaining
* fix param chaining
* README first draft
* readme
* readme
* readme
* style
* fix import
* style
* add fix from https://github.com/huggingface/diffusers/pull/9419
* add to readme, change function names
* te lr changes
* readme
* change concept tokens logic
* fix indices
* change arg name
* style
* dummy test
* revert dummy test
* reorder pivoting
* add warning in case the token abstraction is not the instance prompt
* experimental - wip - specific block training
* fix documentation and token abstraction processing
* remove transformer block specification feature (for now)
* style
* fix copies
* fix indexing issue when --initializer_concept has different amounts
* add if TextualInversionLoaderMixin to all flux pipelines
* style
* fix import
* fix imports
* address review comments - remove necessary prints & comments, use pin_memory=True, use free_memory utils, unify warning and prints
* style
* logger info fix
* make lora target modules configurable and change the default
* make lora target modules configurable and change the default
* style
* make lora target modules configurable and change the default, add notes to readme
* style
* add tests
* style
* fix repo id
* add updated requirements for advanced flux
* fix indices of t5 pivotal tuning embeddings
* fix path in test
* remove `pin_memory`
* fix filename of embedding
* fix filename of embedding
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
f15d9921f1
[tests] fix name and unskip CogI2V integration test ( #9683 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
6796ef0c4f
[core] improve VAE encode/decode framewise batching ( #9684 )
...
* update
* apply suggestions from review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
5f88292f99
[pipeline] CogVideoX-Fun Control ( #9671 )
...
* cogvideox-fun control
* make style
* make fix-copies
* karras schedulers
* Update src/diffusers/pipelines/cogvideo/pipeline_cogvideox_fun_control.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* apply suggestions from review
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
6e02cd7241
[peft] simple update when unscale ( #9689 )
...
Update peft_utils.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
85dde53e30
[SD3 dreambooth-lora training] small updates + bug fixes ( #9682 )
...
* add latent caching + smol updates
* update license
* replace with free_memory
* add --upcast_before_saving to allow saving transformer weights in lower precision
* fix models to accumulate
* fix mixed precision issue as proposed in https://github.com/huggingface/diffusers/pull/9565
* smol update to readme
* style
* fix caching latents
* style
* add tests for latent caching
* style
* fix latent caching
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
674437a39f
[LoRA] log a warning when there are missing keys in the LoRA loading. ( #9622 )
...
* log a warning when there are missing keys in the LoRA loading.
* handle missing keys and unexpected keys better.
* add tests
* fix-copies.
* updates
* tests
* concat warning.
* Add Differential Diffusion to Kolors (#9423 )
* Added diff diff support for kolors img2img
* Fized relative imports
* Fized relative imports
* Added diff diff support for Kolors
* Fized import issues
* Added map
* Fized import issues
* Fixed naming issues
* Added diffdiff support for Kolors img2img pipeline
* Removed example docstrings
* Added map input
* Updated latents
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Updated `original_with_noise`
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Improved code quality
---------
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* FluxMultiControlNetModel (#9647 )
* tests
* Update src/diffusers/loaders/lora_pipeline.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix
---------
Co-authored-by: M Saqlain <118016760+saqlain2204@users.noreply.github.com >
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
64139fadb4
Resolves [BUG] 'GatheredParameters' object is not callable ( #9614 )
...
* gatherparams bug
* calling context lib object
* fix
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:17 +05:30
1b7000c836
Docs: CogVideoX ( #9578 )
...
* CogVideoX docs
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
dd83a81d65
[authored by @Anghellia) Add support of Xlabs Controlnets #9638 ( #9687 )
...
* Add support of Xlabs Controlnets
---------
Co-authored-by: Anzhella Pankratova <son0shad@gmail.com >
2024-12-23 13:02:17 +05:30
48c973cbbb
[training] CogVideoX-I2V LoRA ( #9482 )
...
* update
* update
* update
* update
* update
* add coauthor
Co-Authored-By: yuan-shenghai <963658029@qq.com >
* add coauthor
Co-Authored-By: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com >
* update
Co-Authored-By: yuan-shenghai <963658029@qq.com >
* update
---------
Co-authored-by: yuan-shenghai <963658029@qq.com >
Co-authored-by: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com >
2024-12-23 13:02:17 +05:30
1b5be33a1d
Fix some documentation in ./src/diffusers/models/adapter.py ( #9591 )
...
* Fix some documentation in ./src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* run make style
* make style & fix
* make style : 0.1.5 version ruff
* revert changes to examples
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:17 +05:30
b92255af3a
[docs] refactoring docstrings in models/embeddings_flax.py ( #9592 )
...
* [docs] refactoring docstrings in `models/embeddings_flax.py`
* Update src/diffusers/models/embeddings_flax.py
* make style
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:17 +05:30
453ababb3f
[docs] refactoring docstrings in community/hd_painter.py ( #9593 )
...
* [docs] refactoring docstrings in community/hd_painter.py
* Update examples/community/hd_painter.py
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* make style
---------
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:17 +05:30
3e941e92a1
[doc] Fix some docstrings in src/diffusers/training_utils.py ( #9606 )
...
* refac: docstrings in training_utils.py
* fix: manual edits
* run make style
* add docstring at cast_training_params
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
1f3bc5fce8
refactor image_processor.py file ( #9608 )
...
* refactor image_processor file
* changes as requested
* +1 edits
* quality fix
* indent issue
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
1c7ea5aa29
Dreambooth lora flux bug 3dtensor to 2dtensor ( #9653 )
...
* fixed issue #9350 , Tensor is deprecated
* ran make style
2024-12-23 13:02:17 +05:30
c3c6a55bcb
[Chore] fix import of EntryNotFoundError. ( #9676 )
...
fix import of EntryNotFoundError.
2024-12-23 13:02:17 +05:30
bb20f59d83
[Fix] when run load pretain with local_files_only, local variable 'cached_folder' referenced before assignment ( #9376 )
...
Fix local variable 'cached_folder' referenced before assignment in hub_utils.py
Fix when use `local_files_only=True` with `subfolder`, local variable 'cached_folder' referenced before assignment issue.
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
d6b2dcfa11
Slight performance improvement to Euler, EDMEuler, FlowMatchHeun, KDPM2Ancestral ( #9616 )
...
* Slight performance improvement to Euler
* Slight performance improvement to EDMEuler
* Slight performance improvement to FlowMatchHeun
* Slight performance improvement to KDPM2Ancestral
* Update KDPM2AncestralDiscreteSchedulerTest
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
0ed01ee10a
Refactor SchedulerOutput and add pred_original_sample in DPMSolverSDE, Heun, KDPM2Ancestral and KDPM2 ( #9650 )
...
Refactor SchedulerOutput and add pred_original_sample
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
48f795f991
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel ( #9651 )
...
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
5844c7c9d4
Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel ( #9652 )
...
Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
be464ed2af
Add pred_original_sample to if not return_dict path ( #9649 )
...
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
3fdfa1d828
Added Lora Support to SD3 Img2Img Pipeline ( #9659 )
...
* add lora
2024-12-23 13:02:17 +05:30
bc2cbfba5d
[Community Pipeline] Add 🪆 Matryoshka Diffusion Models ( #9157 )
2024-12-23 13:02:17 +05:30
76d9796f25
Improve the performance and suitable for NPU computing ( #9642 )
...
* Improve the performance and suitable for NPU
* Improve the performance and suitable for NPU computing
* Improve the performance and suitable for NPU
* Improve the performance and suitable for NPU
* Improve the performance and suitable for NPU
* Improve the performance and suitable for NPU
---------
Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
4ea9e828b1
CogView3Plus DiT ( #9570 )
...
* merge 9588
* max_shard_size="5GB" for colab running
* conversion script updates; modeling test; refactor transformer
* make fix-copies
* Update convert_cogview3_to_diffusers.py
* initial pipeline draft
* make style
* fight bugs 🐛 🪳
* add example
* add tests; refactor
* make style
* make fix-copies
* add co-author
YiYi Xu <yixu310@gmail.com >
* remove files
* add docs
* add co-author
Co-Authored-By: YiYi Xu <yixu310@gmail.com >
* fight docs
* address reviews
* make style
* make model work
* remove qkv fusion
* remove qkv fusion tets
* address review comments
* fix make fix-copies error
* remove None and TODO
* for FP16(draft)
* make style
* remove dynamic cfg
* remove pooled_projection_dim as a parameter
* fix tests
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:17 +05:30
0e332644bc
Flux - soft inpainting via differential diffusion ( #9268 )
...
* Flux - soft inpainting via differential diffusion
* .
* track changes to FluxInpaintPipeline
* make mask arrangement simplier
* make style
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
Co-authored-by: asomoza <somoza.alvaro@gmail.com >
2024-12-23 13:02:17 +05:30
6b5d5218cb
[Tests] increase transformers version in test_low_cpu_mem_usage_with_loading ( #9662 )
...
increase transformers version in test_low_cpu_mem_usage_with_loading
2024-12-23 13:02:17 +05:30
cb671beb5c
[docs] Fix xDiT doc image damage ( #9655 )
...
* docs: fix xDiT doc image damage
* doc: move xdit images to hf dataset
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
15bdb8a09f
[CI] replace ubuntu version to 22.04. ( #9656 )
...
replace ubuntu version to 22.04.
2024-12-23 13:02:17 +05:30
ca7a9bbf03
FluxMultiControlNetModel ( #9647 )
2024-12-23 13:02:17 +05:30
1c038d8104
Add Differential Diffusion to Kolors ( #9423 )
...
* Added diff diff support for kolors img2img
* Fized relative imports
* Fized relative imports
* Added diff diff support for Kolors
* Fized import issues
* Added map
* Fized import issues
* Fixed naming issues
* Added diffdiff support for Kolors img2img pipeline
* Removed example docstrings
* Added map input
* Updated latents
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Updated `original_with_noise`
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Improved code quality
---------
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
2024-12-23 13:02:16 +05:30
67c0df279d
fix IsADirectoryError when running the training code for sd3_dreambooth_lora_16gb.ipynb ( #9634 )
...
Add files via upload
fix IsADirectoryError when running the training code
2024-12-23 13:02:16 +05:30
827dd12311
flux controlnet control_guidance_start and control_guidance_end implement ( #9571 )
...
* flux controlnet control_guidance_start and control_guidance_end implement
* minor fix - added docstrings, consistent controlnet scale flux and SD3
2024-12-23 13:02:16 +05:30
3a508bd89a
[LoRA] fix dora test to catch the warning properly. ( #9627 )
...
fix dora test.
2024-12-23 13:02:16 +05:30
168a50a892
make controlnet support interrupt ( #9620 )
...
* make controlnet support interrupt
* remove white space in controlnet interrupt
2024-12-23 13:02:16 +05:30
1d278abe60
add PAG support for SD Img2Img ( #9463 )
...
* added pag to sd img2img pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:16 +05:30
c8d7ed6966
[LoRA] allow loras to be loaded with low_cpu_mem_usage. ( #9510 )
...
* allow loras to be loaded with low_cpu_mem_usage.
* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687
* low_cpu_mem_usage.
* fix-copies
* fix-copies again
* tests
* _LOW_CPU_MEM_USAGE_DEFAULT_LORA
* _peft_version default.
* version checks.
* version check.
* version check.
* version check.
* require peft 0.13.1.
* explicitly specify low_cpu_mem_usage=False.
* docs.
* transformers version 4.45.2.
* update
* fix
* empty
* better name initialize_dummy_state_dict.
* doc todos.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* style
* fix-copies
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:16 +05:30
6a5366cd56
refac/pipeline_output ( #9582 )
2024-12-23 13:02:16 +05:30
752f5c215d
Fix the bug that joint_attention_kwargs is not passed to the FLUX's transformer attention processors ( #9517 )
...
* Update transformer_flux.py
2024-12-23 13:02:16 +05:30
f17a9d2075
Fixed noise_pred_text referenced before assignment. ( #9537 )
...
* Fixed local variable noise_pred_text referenced before assignment when using PAG with guidance scale and guidance rescale at the same time.
* Fixed style.
* Made returning text pred noise an argument.
2024-12-23 13:02:16 +05:30
504535e8ef
[LoRA] Handle DoRA better ( #9547 )
...
* handle dora.
* print test
* debug
* fix
* fix-copies
* update logits
* add warning in the test.
* make is_dora check consistent.
* fix-copies
2024-12-23 13:02:16 +05:30
d230b98128
fix: CogVideox train dataset _preprocess_data crop video ( #9574 )
...
* Removed int8 to float32 conversion (`* 2.0 - 1.0`) from `train_transforms` as it caused image overexposure.
Added `_resize_for_rectangle_crop` function to enable video cropping functionality. The cropping mode can be configured via `video_reshape_mode`, supporting options: ['center', 'random', 'none'].
* The number 127.5 may experience precision loss during division operations.
* wandb request pil image Type
* Resizing bug
* del jupyter
* make style
* Update examples/cogvideo/README.md
* make style
---------
Co-authored-by: --unset <--unset>
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:16 +05:30
83b656d5b2
Update distributed_inference.md to include transformer.device_map ( #9553 )
...
* Update distributed_inference.md to include `transformer.device_map`
* Update docs/source/en/training/distributed_inference.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:16 +05:30
hlky