fb17ba3b9d
[SANA LoRA] sana lora training tests and misc. ( #10296 )
...
* sana lora training tests and misc.
* remove push to hub
* Update examples/dreambooth/train_dreambooth_lora_sana.py
Co-authored-by: Aryan <aryan@huggingface.co >
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:19 +05:30
99885f3c55
[chore] Update README_sana.md to update the default model ( #10285 )
...
Update README_sana.md to update the default model
2024-12-23 13:02:19 +05:30
e896d718da
[chore] fix: reamde -> readme ( #10276 )
...
fix: reamde -> readme
2024-12-23 13:02:19 +05:30
c87d460777
[LoRA] feat: lora support for SANA. ( #10234 )
...
* feat: lora support for SANA.
* make fix-copies
* rename test class.
* attention_kwargs -> cross_attention_kwargs.
* Revert "attention_kwargs -> cross_attention_kwargs."
This reverts commit 23433bf9bcaryan@huggingface.co >
* style
* docs for attention_kwargs.
* remove lora_scale from pag pipeline.
* copy fix
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:19 +05:30
f5d7e319c0
fix min-snr implementation ( #8466 )
...
* fix min-snr implementation
https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66
* Update train_dreambooth.py
fix variable name mse_loss_weights
* fix divisor
* make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:19 +05:30
cd1d9e9610
Add prompt about wandb in examples/dreambooth/readme. ( #10014 )
...
Add files via upload
2024-12-23 13:02:18 +05:30
fbcf84d489
[SD3 dreambooth lora] smol fix to checkpoint saving ( #9993 )
...
* smol change to fix checkpoint saving & resuming (as done in train_dreambooth_sd3.py)
* style
* modify comment to explain reasoning behind hidden size check
2024-12-23 13:02:18 +05:30
b97b4c1444
[advanced flux training] bug fix + reduce memory cost as in #9829 ( #9838 )
...
* memory improvement as done here: https://github.com/huggingface/diffusers/pull/9829
* fix bug
* fix bug
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
dd3e554b42
[FIX] Fix TypeError in DreamBooth SDXL when use_dora is False ( #9879 )
...
* fix use_dora
* fix style and quality
* fix use_dora with peft version
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:18 +05:30
221d6dbeba
Updated _encode_prompt_with_clip and encode_prompt in train_dreamboth_sd3 ( #9800 )
...
* updated encode prompt and clip encod prompt
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
1a70054007
Reduce Memory Cost in Flux Training ( #9829 )
...
* Improve NPU performance
* Improve NPU performance
* Improve NPU performance
* Improve NPU performance
* [bugfix] bugfix for npu free memory
* [bugfix] bugfix for npu free memory
* [bugfix] bugfix for npu free memory
* Reduce memory cost for flux training process
---------
Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
2a9727bdd1
Handling mixed precision for dreambooth flux lora training ( #9565 )
...
Handling mixed precision and add unwarp
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-12-23 13:02:17 +05:30
ad754e6182
NPU Adaption for FLUX ( #9751 )
...
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
* NPU implementation for FLUX
---------
Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com >
2024-12-23 13:02:17 +05:30
2094e7a2b5
[training] use the lr when using 8bit adam. ( #9796 )
...
* use the lr when using 8bit adam.
* remove lr as we pack it in params_to_optimize.
---------
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-12-23 13:02:17 +05:30
5905401d1e
[flux dreambooth lora training] make LoRA target modules configurable + small bug fix ( #9646 )
...
* make lora target modules configurable and change the default
* style
* make lora target modules configurable and change the default
* fix bug when using prodigy and training te
* fix mixed precision training as proposed in https://github.com/huggingface/diffusers/pull/9565 for full dreambooth as well
* add test and notes
* style
* address sayaks comments
* style
* fix test
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
ba31a14700
[SD 3.5 Dreambooth LoRA] support configurable training block & layers ( #9762 )
...
* configurable layers
* configurable layers
* update README
* style
* add test
* style
* add layer test, update readme, add nargs
* readme
* test style
* remove print, change nargs
* test arg change
* style
* revert nargs 2/2
* address sayaks comments
* style
* address sayaks comments
2024-12-23 13:02:17 +05:30
dd6de12e1f
[Fix] remove setting lr for T5 text encoder when using prodigy in flux dreambooth lora script ( #9473 )
...
* fix: removed setting of text encoder lr for T5 as it's not being tuned
* fix: removed setting of text encoder lr for T5 as it's not being tuned
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-12-23 13:02:17 +05:30
969fa9f668
[refactor] enhance readability of flux related pipelines ( #9711 )
...
* flux pipline: readability enhancement.
2024-12-23 13:02:17 +05:30
003676e961
[SD3-5 dreambooth lora] update model cards ( #9749 )
...
* improve readme
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
23cf712349
post-release 0.31.0 ( #9742 )
...
* post-release
* style
2024-12-23 13:02:17 +05:30
85dde53e30
[SD3 dreambooth-lora training] small updates + bug fixes ( #9682 )
...
* add latent caching + smol updates
* update license
* replace with free_memory
* add --upcast_before_saving to allow saving transformer weights in lower precision
* fix models to accumulate
* fix mixed precision issue as proposed in https://github.com/huggingface/diffusers/pull/9565
* smol update to readme
* style
* fix caching latents
* style
* add tests for latent caching
* style
* fix latent caching
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:17 +05:30
1c7ea5aa29
Dreambooth lora flux bug 3dtensor to 2dtensor ( #9653 )
...
* fixed issue #9350 , Tensor is deprecated
* ran make style
2024-12-23 13:02:17 +05:30
df90ea984a
[chore] fix: retain memory utility. ( #9543 )
...
* fix: retain memory utility.
* fix
* quality
* free_memory.
2024-12-23 13:02:16 +05:30
753f0edff2
[docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8 ( #9428 )
...
* [docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8
Updated documentation as runwayml/stable-diffusion-v1-5 has been removed from Huggingface.
* Update docs/source/en/using-diffusers/inpaint.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Replace with stable-diffusion-v1-5/stable-diffusion-v1-5
* Update inpaint.md
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:16 +05:30
b8f01f5568
[Flux Dreambooth lora] add latent caching ( #9160 )
...
* add ostris trainer to README & add cache latents of vae
* add ostris trainer to README & add cache latents of vae
* style
* readme
* add test for latent caching
* add ostris noise scheduler
https://github.com/ostris/ai-toolkit/blob/9ee1ef2a0a2a9a02b92d114a95f21312e5906e54/toolkit/samplers/custom_flowmatch_sampler.py#L95
* style
* fix import
* style
* fix tests
* style
* --change upcasting of transformer?
* update readme according to main
* keep only latent caching
* add configurable param for final saving of trained layers- --upcast_before_saving
* style
* Update examples/dreambooth/README_flux.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/dreambooth/README_flux.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* use clear_objs_and_retain_memory from utilities
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:16 +05:30
64dd1c2f2a
Fix the issue on sd3 dreambooth w./w.t. lora training ( #9419 )
...
* Fix dtype error
* [bugfix] Fixed the issue on sd3 dreambooth training
* [bugfix] Fixed the issue on sd3 dreambooth training
---------
Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:16 +05:30
6f39b2bf0d
[Tests] fix some fast gpu tests. ( #9379 )
...
fix some fast gpu tests.
2024-12-23 13:02:16 +05:30
8e5940ea62
improve README for flux dreambooth lora ( #9290 )
...
* improve readme
* improve readme
* improve readme
* improve readme
2024-12-23 13:02:16 +05:30
32f85ed014
chore: add a cleaning utility to be useful during training. ( #9240 )
2024-12-23 13:02:16 +05:30
18a3c35aac
[Dreambooth flux] bug fix for dreambooth script (align with dreambooth lora) ( #9257 )
...
* fix shape
* fix prompt encoding
* style
* fix device
* add comment
2024-12-23 13:02:15 +05:30
8615d0fbff
fix(sd3): fix deletion of text_encoders etc ( #8951 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:15 +05:30
259b12a077
[Docs] Fix CPU offloading usage ( #9207 )
...
* chore: Fix cpu offloading usage
* Trim trailing white space
* docs: update Kolors model link in kolors.md
2024-12-23 13:02:15 +05:30
275e523ef5
post release 0.30.0 ( #9173 )
...
* post release
* fix quality
2024-12-23 13:02:15 +05:30
5fd5487f18
[Flux Dreambooth LoRA] - te bug fixes & updates ( #9139 )
...
* add requirements + fix link to bghira's guide
* text ecnoder training fixes
* text encoder training fixes
* text encoder training fixes
* text encoder training fixes
* style
* add tests
* fix encode_prompt call
* style
* unpack_latents test
* fix lora saving
* remove default val for max_sequenece_length in encode_prompt
* remove default val for max_sequenece_length in encode_prompt
* style
* testing
* style
* testing
* testing
* style
* fix sizing issue
* style
* revert scaling
* style
* style
* scaling test
* style
* scaling test
* remove model pred operation left from pre-conditioning
* remove model pred operation left from pre-conditioning
* fix trainable params
* remove te2 from casting
* transformer to accelerator
* remove prints
* empty commit
2024-12-23 13:02:15 +05:30
6e9c6a298b
[Flux] Dreambooth LoRA training scripts ( #9086 )
...
* initial commit - dreambooth for flux
* update transformer to be FluxTransformer2DModel
* update training loop and validation inference
* fix sd3->flux docs
* add guidance handling, not sure if it makes sense(?)
* inital dreambooth lora commit
* fix text_ids in compute_text_embeddings
* fix imports of static methods
* fix pipeline loading in readme, remove auto1111 docs for now
* fix pipeline loading in readme, remove auto1111 docs for now, remove some irrelevant text_encoder_3 refs
* Update examples/dreambooth/train_dreambooth_flux.py
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
* fix te2 loading and remove te2 refs from text encoder training
* fix tokenizer_2 initialization
* remove text_encoder training refs from lora script (for now)
* try with vae in bfloat16, fix model hook save
* fix tokenization
* fix static imports
* fix CLIP import
* remove text_encoder training refs (for now) from lora script
* fix minor bug in encode_prompt, add guidance def in lora script, ...
* fix unpack_latents args
* fix license in readme
* add "none" to weighting_scheme options for uniform sampling
* style
* adapt model saving - remove text encoder refs
* adapt model loading - remove text encoder refs
* initial commit for readme
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fix vae casting
* remove precondition_outputs
* readme
* readme
* style
* readme
* readme
* update weighting scheme default & docs
* style
* add text_encoder training to lora script, change vae_scale_factor value in both
* style
* text encoder training fixes
* style
* update readme
* minor fixes
* fix te params
* fix te params
---------
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:15 +05:30
f35bdb6a03
fix train_dreambooth_lora_sd3.py loading hook ( #9107 )
2024-12-23 13:02:15 +05:30
c6ac793955
Errata: Fix typos & \s+$ ( #9008 )
...
* Fix typos
* chore: Fix typos
* chore: Update README.md for promptdiffusion example
* Trim trailing white spaces
* Fix a typo
* update number
* chore: update number
* Trim trailing white space
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:14 +05:30
6d11129c5a
[Chore] add LoraLoaderMixin to the inits ( #8981 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
* loraloadermixin.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
a754d9071e
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8976 )
...
Revert "[LoRA] introduce LoraBaseMixin to promote reusability. (#8774 )"
This reverts commit 527430d0a4
2024-12-23 13:02:14 +05:30
82b37a4cc3
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8774 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
1b5d74a9e3
[Training] SD3 training fixes ( #8917 )
...
* SD3 training fixes
Co-authored-by: bghira <59658056+bghira@users.noreply.github.com >
* rewrite noise addition part to respect the eqn.
* styler
* Update examples/dreambooth/README_sd3.md
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com >
---------
Co-authored-by: bghira <59658056+bghira@users.noreply.github.com >
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com >
2024-12-23 13:02:14 +05:30
67b04c891d
Improve model card for push_to_hub trainers ( #8697 )
...
* Improve trainer model cards
* Update train_dreambooth_sd3.py
* Update train_dreambooth_lora_sd3.py
* add link to adapters loading doc
* Update train_dreambooth_lora_sd3.py
---------
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-12-23 13:02:14 +05:30
f5f9f9c083
[Tests] add test suite for SD3 DreamBooth ( #8650 )
...
* add a test suite for SD3 DreamBooth
* lora suite
* style
* add checkpointing tests for LoRA
* add test to cover train_text_encoder.
2024-12-23 13:02:13 +05:30
4d12d764f0
[SD3 LoRA Training] Fix errors when not training text encoders ( #8743 )
...
* fix
* fix things.
Co-authored-by: Linoy Tsaban <linoy.tsaban@gmail.com >
* remove patch
* apply suggestions
---------
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Linoy Tsaban <linoy.tsaban@gmail.com >
2024-12-23 13:02:13 +05:30
f8c8daa0ae
[Sd3 Dreambooth LoRA] Add text encoder training for the clip encoders ( #8630 )
...
* add clip text-encoder training
* no dora
* text encoder traing fixes
* text encoder traing fixes
* text encoder training fixes
* text encoder training fixes
* text encoder training fixes
* text encoder training fixes
* add text_encoder layers to save_lora
* style
* fix imports
* style
* fix text encoder
* review changes
* review changes
* review changes
* minor change
* add lora tag
* style
* add readme notes
* add tests for clip encoders
* style
* typo
* fixes
* style
* Update tests/lora/test_lora_layers_sd3.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/dreambooth/README_sd3.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* minor readme change
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
6f922c2a91
Fix redundant pipe init in sd3 lora ( #8680 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
f6172748c6
Errata - Fix typos and improve style ( #8571 )
...
* Fix typos
* Fix typos & up style
* chore: Update numbers
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
2c56360222
Errata - Trim trailing white space in the whole repo ( #8575 )
...
* Trim all the trailing white space in the whole repo
* Remove unnecessary empty places
* make style && make quality
* Trim trailing white space
* trim
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
c20d0ec72d
Errata - Fix typos & improve contributing page ( #8572 )
...
* Fix typos & improve contributing page
* `make style && make quality`
* fix typos
* Fix typo
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
492900099c
Update train_dreambooth_lora_sd3.py ( #8600 )
...
* Update train_dreambooth_lora_sd3.py
* Update train_dreambooth_lora_sd3.py
* Update train_dreambooth_sd3.py
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:13 +05:30
Sayak Paul