20f2d062d6
Remove negative_* from SDXL callback ( #10203 )
...
* Remove `negative_*` from SDXL callback
* Change example and add XL version
2024-12-23 13:02:19 +05:30
5fe30d94ff
Ci update tpu ( #10197 )
...
* Update nightly_tests.yml for TPU CI
* Update push_tests.yml
2024-12-23 13:02:19 +05:30
0bd5bc0cfb
[core] LTX Video ( #10021 )
...
* transformer
* make style & make fix-copies
* transformer
* add transformer tests
* 80% vae
* make style
* make fix-copies
* fix
* undo cogvideox changes
* update
* update
* match vae
* add docs
* t2v pipeline working; scheduler needs to be checked
* docs
* add pipeline test
* update
* update
* make fix-copies
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* update
* copy t2v to i2v pipeline
* update
* apply review suggestions
* update
* make style
* remove framewise encoding/decoding
* pack/unpack latents
* image2video
* update
* make fix-copies
* update
* update
* rope scale fix
* debug layerwise code
* remove debug
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* propagate precision changes to i2v pipeline
* remove downcast
* address review comments
* fix comment
* address review comments
* [Single File] LTX support for loading original weights (#10135 )
* from original file mixin for ltx
* undo config mapping fn changes
* update
* add single file to pipelines
* update docs
* Update src/diffusers/models/autoencoders/autoencoder_kl_ltx.py
* Update src/diffusers/models/autoencoders/autoencoder_kl_ltx.py
* rename classes based on ltx review
* point to original repository for inference
* make style
* resolve conflicts correctly
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:19 +05:30
dd8da2d1e8
[WIP][Training] Flux Control LoRA training script ( #10130 )
...
* update
* add
* update
* add control-lora conversion script; make flux loader handle norms; fix rank calculation assumption
* control lora updates
* remove copied-from
* create separate pipelines for flux control
* make fix-copies
* update docs
* add tests
* fix
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* remove control lora changes
* apply suggestions from review
* Revert "remove control lora changes"
This reverts commit 73cfc519c9contact.aryanvs@gmail.com >
* fix-copies
* updates for tests
* fix
* alpha_pattern.
* add a test for varied lora ranks and alphas.
* revert changes in num_channels_latents = self.transformer.config.in_channels // 8
* revert moe
* add a sanity check on unexpected keys when loading norm layers.
* contro lora.
* fixes
* fixes
* fixes
* tests
* reviewer feedback
* fix
* proper peft version for lora_bias
* fix-copies
* updates
* updates
* updates
* remove debug code
* update docs
* integration tests
* nis
* fuse and unload.
* fix
* add slices.
* more updates.
* button up readme
* train()
* add full fine-tuning version.
* fixes
* Apply suggestions from code review
Co-authored-by: Aryan <aryan@huggingface.co >
* set_grads_to_none remove.
* readme
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com >
2024-12-23 13:02:19 +05:30
2886506d31
[CI] merge peft pr workflow into the main pr workflow. ( #10042 )
...
* merge peft pr workflow into the main pr workflow.
* remove latest
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:19 +05:30
85766759a7
[LoRA] add a test to ensure set_adapters() and attn kwargs outs match ( #10110 )
...
* add a test to ensure set_adapters() and attn kwargs outs match
* remove print
* fix
* Apply suggestions from code review
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
* assertFalse.
---------
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
2024-12-23 13:02:19 +05:30
495cfdadde
Add support for XFormers in SD3 ( #8583 )
...
* Add support for XFormers in SD3
* sd3 xformers test
* sd3 xformers quality
* sd3 xformers update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:19 +05:30
f5d7e319c0
fix min-snr implementation ( #8466 )
...
* fix min-snr implementation
https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66
* Update train_dreambooth.py
fix variable name mse_loss_weights
* fix divisor
* make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:19 +05:30
bcabac2914
Add ControlNetUnion ( #10131 )
...
* ControlNetUnion model
2024-12-23 13:02:18 +05:30
93dfaec4a9
[Single File] Add single file support for AutoencoderDC ( #10183 )
...
* update
* update
* update
2024-12-23 13:02:18 +05:30
d1411b305f
Added Error when len(gligen_images ) is not equal to len(gligen_phrases) in StableDiffusionGLIGENTextImagePipeline ( #10176 )
...
* added check value error
* fix style
2024-12-23 13:02:18 +05:30
d38bca7765
Fix Nonetype attribute error when loading multiple Flux loras ( #10182 )
...
Fix Nonetype attribute error
2024-12-23 13:02:18 +05:30
d6bc975868
[community pipeline rf-inversion] - fix example in doc ( #10179 )
...
* fix example in doc
* remove redundancies
* change param
2024-12-23 13:02:18 +05:30
a7ab84ea16
Add PAG Support for Stable Diffusion Inpaint Pipeline ( #9386 )
...
* using sd inpaint pipeline and sdxl pag inpaint pipeline to add changes
* using sd inpaint pipeline and sdxl pag inpaint pipeline to add changes
* finished the call function
* added auto pipeline
* merging diffusers
* ready to test
* ready to test
* added copied from and removed unnecessary tests
* make style changes
* doc changes
* updating example doc string
* style fix
* init
* adding imports
* quality
* Update src/diffusers/pipelines/pag/pipeline_pag_sd_inpaint.py
* make
* Update tests/pipelines/pag/test_pag_sd_inpaint.py
* slice and size
* slice
---------
Co-authored-by: Darshil Jariwala <darshiljariwala@Darshils-MacBook-Air.local >
Co-authored-by: Darshil Jariwala <jariwala.darshil2002@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
7c4bfa64b1
Flux Control LoRA ( #9999 )
...
* update
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
9b031cc829
Use torch in get_3d_rotary_pos_embed/_allegro ( #10161 )
...
Use torch in get_3d_rotary_pos_embed/_allegro
2024-12-23 13:02:18 +05:30
641eebf784
Improve post-processing performance ( #10170 )
...
* Use multiplication instead of division
* Add fast path when denormalizing all or none of the images
2024-12-23 13:02:18 +05:30
42159b7c6a
[community pipeline] Add RF-inversion Flux pipeline ( #9816 )
...
* initial commit
* update denoising loop
* fix scheduling
* style
* fix import
* fixes
* fixes
* style
* fixes
* change invert
* change denoising & check inputs
* shape & timesteps fixes
* timesteps fixes
* style
* remove redundancies
* small changes
* update documentation a bit
* update documentation a bit
* update documentation a bit
* style
* change strength param, remove redundancies
* style
* forward ode loop change
* add inversion progress bar
* fix image_seq_len
* revert to strength but == 1 by default.
* style
* add "copied from..." comments
* credit authors
* make style
* return inversion outputs without self-assigning
* adjust denoising loop to generate regular images if inverted latents are not provided
* adjust denoising loop to generate regular images if inverted latents are not provided
* fix import
* comment
* remove redundant line
* modify comment on ti
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/community/pipeline_flux_rf_inversion.py
Co-authored-by: hlky <hlky@hlky.ac >
* fix syntax error
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
6816ee2406
[Single file] Support revision argument when loading single file config ( #10168 )
...
update
2024-12-23 13:02:18 +05:30
3271d389bc
support sd3.5 for controlnet example ( #9860 )
...
* support sd3.5 in controlnet
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:18 +05:30
24a8d2d872
Update ptxla training ( #9864 )
...
* update ptxla example
---------
Co-authored-by: Juan Acevedo <jfacevedo@google.com >
Co-authored-by: Pei Zhang <zpcore@gmail.com >
Co-authored-by: Pei Zhang <piz@google.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Pei Zhang <pei@Peis-MacBook-Pro.local >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
e179be2ca3
[LoRA] depcrecate save_attn_procs(). ( #10126 )
...
depcrecate save_attn_procs().
2024-12-23 13:02:18 +05:30
1feae4160e
fixed a dtype bfloat16 bug in torch_utils.py ( #10125 )
...
* fixed a dtype bfloat16 bug in torch_utils.py
when generating 1024*1024 image with bfloat16 dtype, there is an exception:
File "/opt/conda/lib/python3.10/site-packages/diffusers/utils/torch_utils.py", line 107, in fourier_filter
x_freq = fftn(x, dim=(-2, -1))
RuntimeError: Unsupported dtype BFloat16
* remove whitespace in torch_utils.py
* Update src/diffusers/utils/torch_utils.py
* Update torch_utils.py
---------
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
1d9eb2c1d9
[DC-AE] Add the official Deep Compression Autoencoder code(32x,64x,128x compression ratio); ( #9708 )
...
* first add a script for DC-AE;
* DC-AE init
* replace triton with custom implementation
* 1. rename file and remove un-used codes;
* no longer rely on omegaconf and dataclass
* replace custom activation with diffuers activation
* remove dc_ae attention in attention_processor.py
* iinherit from ModelMixin
* inherit from ConfigMixin
* dc-ae reduce to one file
* update downsample and upsample
* clean code
* support DecoderOutput
* remove get_same_padding and val2tuple
* remove autocast and some assert
* update ResBlock
* remove contents within super().__init__
* Update src/diffusers/models/autoencoders/dc_ae.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* remove opsequential
* update other blocks to support the removal of build_norm
* remove build encoder/decoder project in/out
* remove inheritance of RMSNorm2d from LayerNorm
* remove reset_parameters for RMSNorm2d
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* remove device and dtype in RMSNorm2d __init__
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/models/autoencoders/dc_ae.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/models/autoencoders/dc_ae.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/models/autoencoders/dc_ae.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* remove op_list & build_block
* remove build_stage_main
* change file name to autoencoder_dc
* move LiteMLA to attention.py
* align with other vae decode output;
* add DC-AE into init files;
* update
* make quality && make style;
* quick push before dgx disappears again
* update
* make style
* update
* update
* fix
* refactor
* refactor
* refactor
* update
* possibly change to nn.Linear
* refactor
* make fix-copies
* replace vae with ae
* replace get_block_from_block_type to get_block
* replace downsample_block_type from Conv to conv for consistency
* add scaling factors
* incorporate changes for all checkpoints
* make style
* move mla to attention processor file; split qkv conv to linears
* refactor
* add tests
* from original file loader
* add docs
* add standard autoencoder methods
* combine attention processor
* fix tests
* update
* minor fix
* minor fix
* minor fix & in/out shortcut rename
* minor fix
* make style
* fix paper link
* update docs
* update single file loading
* make style
* remove single file loading support; todo for DN6
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* add abstract
---------
Co-authored-by: Junyu Chen <chenjydl2003@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: chenjy2003 <70215701+chenjy2003@users.noreply.github.com >
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:18 +05:30
d54c70bcf6
[community] Load Models from Sources like Civitai into Existing Pipelines ( #9986 )
...
* Added example of model search.
* Combine processing into one file
* Add parameters for base model
* Bug Fixes
* bug fix
* Create README.md
* Update search_for_civitai_and_HF.py
* Create requirements.txt
* bug fix
* Update README.md
* bug fix
* Correction of typos
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* apply the changes
* Replace search_for_civitai_and_HF.py with pipeline_easy.py
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update examples/model_search/README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
* Organize the table of parameters
* Update README.md
* Update README.md
* Update README.md
* make style
* Fixing the style of pipeline
* Fix pipeline style
* fix
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:18 +05:30
b913ca1575
Remove duplicate checks for len(generator) != batch_size when generator is a list ( #10134 )
...
remove duplicate checks
2024-12-23 13:02:18 +05:30
d0b901017a
[Guide] Quantize your Diffusion Models with bnb ( #10012 )
...
* chore: initial draft
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* chore: link in place
* chore: review suggestions
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* chore: review suggestions
* Update docs/source/en/quantization/bitsandbytes.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* review suggestions
* chore: review suggestions
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* adding same changes to 4 bit section
* review suggestions
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:18 +05:30
b49984081e
[FIX] Bug in FluxPosEmbed ( #10115 )
...
* Fix get_1d_rotary_pos_embed in embedding.py
* Update embeddings.py
---------
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
9fcf353c80
[Tests] xfail incompatible SD configs. ( #10127 )
...
* xfail incompatible SD configs.
* fix
2024-12-23 13:02:18 +05:30
863dcc3b37
[Tests] fix condition argument in xfail. ( #10099 )
...
* fix condition argument in xfail.
* revert init changes.
2024-12-23 13:02:18 +05:30
6bf8b2bf97
Use torch.device instead of current device index for BnB quantizer ( #10069 )
...
* update
* apply review suggestion
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
98413880d8
[docs] load_lora_adapter ( #10119 )
...
* load_lora_adapter
* save
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
8fb2b6b5da
pass attn mask arg for flux ( #10122 )
2024-12-23 13:02:18 +05:30
053a48dd43
Fix a bug for SD35 control net training and improve control net block index ( #10065 )
...
* wip
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
ba0f59b47a
Fix a bug in the state dict judgment in ip_adapter.py. ( #10095 )
...
* fix a judging state dict bug in ip_adapter.py
* make
---------
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
8dff843847
[Flux Redux] add prompt & multiple image input ( #10056 )
...
* add multiple prompts to flux redux
---------
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
488b2a62e7
Add sigmas to pipelines using FlowMatch ( #10116 )
2024-12-23 13:02:18 +05:30
48d9c0908b
Fix Broken Links in ReadMe ( #10117 )
...
Update broken links in ReadME.
2024-12-23 13:02:18 +05:30
57acfe7180
[bitsandbytes] allow directly CUDA placements of pipelines loaded with bnb components ( #9840 )
...
* allow device placement when using bnb quantization.
* warning.
* tests
* fixes
* docs.
* require accelerate version.
* remove print.
* revert to()
* tests
* fixes
* fix: missing AutoencoderKL lora adapter (#9807 )
* fix: missing AutoencoderKL lora adapter
* fix
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fixes
* fix condition test
* updates
* updates
* remove is_offloaded.
* fixes
* better
* empty
---------
Co-authored-by: Emmanuel Benazera <emmanuel.benazera@jolibrain.com >
2024-12-23 13:02:18 +05:30
f80510b47a
Fix pipeline_stable_audio formating ( #10114 )
2024-12-23 13:02:18 +05:30
de77e2236d
add torch_xla support in pipeline_stable_audio.py ( #10109 )
...
Update pipeline_stable_audio.py
2024-12-23 13:02:18 +05:30
c71bdb70e5
[tests] refactor vae tests ( #9808 )
...
* add: autoencoderkl tests
* autoencodertiny.
* fix
* asymmetric autoencoder.
* more
* integration tests for stable audio decoder.
* consistency decoder vae tests
* remove grad check from consistency decoder.
* cog
* bye test_models_vae.py
* fix
* fix
* remove allegro
* fixes
* fixes
* fixes
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:18 +05:30
c33fd8b5ef
Use parameters + buffers when deciding upscale_dtype ( #9882 )
...
Sometimes, the decoder might lack parameters and only buffers (e.g., this happens when we manually need to convert all the parameters to buffers — e.g. to avoid packing fp16 and fp32 parameters with FSDP)
2024-12-23 13:02:18 +05:30
e50b36f759
Test skip_guidance_layers in SD3 pipeline ( #10102 )
...
* Test `skip_guidance_layers` in pipelines
* Move to test_pipeline_stable_diffusion_3
2024-12-23 13:02:18 +05:30
2f68e52db5
Fix multi-prompt inference ( #10103 )
...
* Fix multi-prompt inference
Fix generation of multiple images with multiple prompts, e.g len(prompts)>1, num_images_per_prompt>1
* make
* fix copies
---------
Co-authored-by: Nikita Balabin <nikita@mxl.ru >
2024-12-23 13:02:18 +05:30
69acaeec4b
[Bug fix] "previous_timestep()" in DDPM scheduling compatible with "trailing" and "linspace" options ( #9384 )
...
* Update scheduling_ddpm.py
* fix copies
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
94ee4c1f0a
Avoid compiling a progress bar. ( #10098 )
...
* Avoid creating a progress bar when it is disabled.
This is useful when exporting a pipeline, and allows a compiler to avoid trying to compile away tqdm.
* Prevent the PyTorch compiler from compiling progress bars.
* Update pipeline_utils.py
2024-12-23 13:02:18 +05:30
70557b92f2
update ( #7067 )
...
* add data_dir parameter to load_dataset
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
af7efa0454
DPM++ third order fixes ( #9104 )
...
* Fix wrong output on 3n-1 steps count
* Add sde handling to 3 order
* make
* copies
---------
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-23 13:02:18 +05:30
a9f8f2ea87
Fix Broken Link in Optimization Docs ( #10105 )
...
Update broken link.
2024-12-23 13:02:18 +05:30
hlky