obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,583 Commits 711 Branches 93 Tags
df90ea984ad523fd9fdda6eddd345098bcb879dc
Commit Graph

4583 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sayak Paul df90ea984a [chore] fix: retain memory utility. (#9543) 
* fix: retain memory utility.

* fix

* quality

* free_memory.
 2024-12-23 13:02:16 +05:30
Anand Kumar 91d4b38e0a [train_instruct_pix2pix.py]Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9316) 
Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan bb43d9933c [refactor] remove conv_cache from CogVideoX VAE (#9524) 
* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 2a38836db8 [Core] fix variant-identification. (#9253) 
* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 9405b1a2e4 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-12-23 13:02:16 +05:30
PromeAI 19399207ae [examples] add train flux-controlnet scripts in example. (#9324) 
* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 33a77c6a99 [LoRA] make set_adapters() method more robust. (#9535) 
* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan f018acdb11 [bug] Precedence of operations in VAE should be slicing -> tiling (#9342) 
* bugfix: precedence of operations should be slicing -> tiling

* fix typo

* fix another typo

* deprecate current implementation of tiled_encode and use new impl

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Álvaro Somoza 5cef1c58bf [Tests] Fix ChatGLMTokenizer (#9536) 
fix
 2024-12-23 13:02:16 +05:30
YiYi Xu 143fa19bbf flux controlnet fix (control_modes batch & others) (#9507) 
* flux controlnet mode to take into account batch size

* incorporate yiyixuxu's suggestions (cleaner logic) as well as clean up control mode handling for multi case

* fix

* fix use_guidance when controlnet is a multi and does not have config

---------

Co-authored-by: Christopher Beckham <christopher.j.beckham@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky a4baaae02b [Schedulers] Add beta sigmas / beta noise schedule (#9509) 
Add beta sigmas / beta noise schedule
 2024-12-23 13:02:16 +05:30
v2ray 6f9f26f8f8 [Doc] Improved level of clarity for latents_to_rgb. (#9529) 
Fixed latents_to_rgb doc.

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
YiYi Xu 4ed3249840 update get_parameter_dtype (#9526) 
* up

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Steven Liu b142eea26d [docs] Model sharding (#9521) 
* flux shard

* feedback
 2024-12-23 13:02:16 +05:30
Lee Penkman c45940d23b Update community_projects.md (#9266) 2024-12-23 13:02:16 +05:30
Sayak Paul d8244b52a9 [Community Pipeline] Batched implementation of Flux with CFG (#9513) 
* batched implementation of flux cfg.

* style.

* readme

* remove comments.
 2024-12-23 13:02:16 +05:30
hlky 0e22145b93 Add exponential sigmas to other schedulers and update docs (#9518) 2024-12-23 13:02:16 +05:30
YiYi Xu 30dd52ce90 a few fix for SingleFile tests (#9522) 
* update sd15 repo

* update more
 2024-12-23 13:02:16 +05:30
Sayak Paul eec17ced14 [CI] allow faster downloads from the Hub in CI. (#9478) 
* allow faster downloads from the Hub in CI.

* HF_HUB_ENABLE_HF_TRANSFER: 1

* empty

* empty

* remove ENV HF_HUB_ENABLE_HF_TRANSFER=1.

* empty
 2024-12-23 13:02:16 +05:30
LukeLin 775f163ebc [Doc] Fix path and and also import imageio (#9506) 
* Fix bug

* import imageio
 2024-12-23 13:02:16 +05:30
captainzz c4e77f1ee1 fix bugs for sd3 controlnet training (#9489) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 74c74629b3 Add Noise Schedule/Schedule Type to Schedulers Overview documentation (#9504) 
* Add Noise Schedule/Schedule Type to Schedulers Overview docs

* Update docs/source/en/api/schedulers/overview.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Seongbin Lim 078286aa85 Allow DDPMPipeline half precision (#9222) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky 7d92452bf9 [Schedulers] Add exponential sigmas / exponential noise schedule (#9499) 
* exponential sigmas

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
pibbo88 715a6c99a2 Fix the bug of sd3 controlnet training when using gradient checkpointing. (#9498) 
Fix the bug of sd3 controlnet training when using gradient_checkpointing. Refer to issue #9496
 2024-12-23 13:02:16 +05:30
M Saqlain 298963abbe [Tests] Reduce the model size in the lumina test (#8985) 
* Reduced model size for lumina-tests

* Handled failing tests
 2024-12-23 13:02:16 +05:30
Sayak Paul 8fb47719db [Cog] some minor fixes and nits (#9466) 
* fix positional arguments in check_inputs().

* add video and latetns to check_inputs().

* prep latents_in_channels.

* quality

* multiple fixes.

* fix
 2024-12-23 13:02:16 +05:30
Sayak Paul 53786f5eab [CI] fix nightly model tests (#9483) 
* check if default attn procs fix it.

* print

* print

* replace

* style./

* replace revision with variant.

* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.

* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.

* fix
 2024-12-23 13:02:16 +05:30
Aryan 91aa68b360 [refactor] LoRA tests (#9481) 
* refactor scheduler class usage

* reorder to make tests more readable

* remove pipeline specific checks and skip tests directly

* rewrite denoiser conditions cleaner

* bump tolerance for cog test
 2024-12-23 13:02:16 +05:30
Vladimir Mandic e1ea942b42 Several fixes to Flux ControlNet pipelines (#9472) 
* fix flux controlnet pipelines

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan fd15a9d7bd [training] CogVideoX Lora (#9302) 
* cogvideox lora training draft

* update

* update

* update

* update

* update

* make fix-copies

* update

* update

* apply suggestions from review

* apply suggestions from reveiw

* fix typo

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix lora alpha

* use correct lora scaling for final test pipeline

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* apply suggestions from review; prodigy optimizer

YiYi Xu <yixu310@gmail.com>

* add tests

* make style

* add README

* update

* update

* make style

* fix

* update

* add test skeleton

* revert lora utils changes

* add cleaner modifications to lora testing utils

* update lora tests

* deepspeed stuff

* add requirements.txt

* deepspeed refactor

* add lora stuff to img2vid pipeline to fix tests

* fight tests

* add co-authors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

Co-Authored-By: zR <2448370773@qq.com>

* fight lora runner tests

* import Dummy optim and scheduler only wheh required

* update docs

* add coauthors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

* remove option to train text encoder

Co-Authored-By: bghira <bghira@users.github.com>

* update tests

* fight more tests

* update

* fix vid2vid

* fix typo

* remove lora tests; todo in follow-up PR

* undo img2vid changes

* remove text encoder related changes in lora loader mixin

* Revert "remove text encoder related changes in lora loader mixin"

This reverts commit f8a8444487.

* update

* round 1 of fighting tests

* round 2 of fighting tests

* fix copied from comment

* fix typo in lora test

* update styling

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: zR <2448370773@qq.com>
Co-authored-by: Fu-Yun Wang <1697256461@qq.com>
Co-authored-by: bghira <bghira@users.github.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 240f753324 [Flux] add lora integration tests. (#9353) 
* add lora integration tests.

* internal note

* add a skip marker.
 2024-12-23 13:02:16 +05:30
Anatoly Belikov bef6cfa785 adapt masked im2im pipeline for SDXL (#7790) 
* adapt masked im2im pipeline for SDXL

* usage for masked im2im stable diffusion XL pipeline

* style

* style

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan cb3a0defb9 set max_shard_size to None for pipeline save_pretrained (#9447) 
* update default max_shard_size

* add None check to fix tests

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan 0494cbd01a Remove CogVideoX mentions from single file docs; Test updates (#9444) 
* remove mentions from single file

* update tests

* update
 2024-12-23 13:02:16 +05:30
Subho Ghosh 4c08316cf7 Feature flux controlnet img2img and inpaint pipeline (#9408) 
* Implemented FLUX controlnet support to Img2Img pipeline
 2024-12-23 13:02:16 +05:30
Linoy Tsaban 502191f3a6 [Flux with CFG] add flux pipeline with cfg support (#9445) 
* true_cfg

* add check negative prompt/embeds inputs

* move to community pipelines

* move to community pipelines

* revert true cfg changes to the orig pipline

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 043517716f [CI] updates to the CI report naming, and accelerate installation (#9429) 
* chore: id accordingly to avoid duplicates.

* update properly.

* updates

* updates

* empty

* updates

* changing order helps?
 2024-12-23 13:02:16 +05:30
suzukimain 753f0edff2 [docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8 (#9428) 
* [docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8

Updated documentation as runwayml/stable-diffusion-v1-5 has been removed from Huggingface.

* Update docs/source/en/using-diffusers/inpaint.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Replace with stable-diffusion-v1-5/stable-diffusion-v1-5

* Update inpaint.md

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Yuxuan.Zhang 75682c97cf CogVideoX-5b-I2V support (#9418) 
* draft Init

* draft

* vae encode image

* make style

* image latents preparation

* remove image encoder from conversion script

* fix minor bugs

* make pipeline work

* make style

* remove debug prints

* fix imports

* update example

* make fix-copies

* add fast tests

* fix import

* update vae

* update docs

* update image link

* apply suggestions from review

* apply suggestions from review

* add slow test

* make use of learned positional embeddings

* apply suggestions from review

* doc change

* Update convert_cogvideox_to_diffusers.py

* make style

* final changes

* make style

* fix tests

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul 091ad8e647 [CI] make runner_type restricted. (#9441) 
make runner_type restricted.
 2024-12-23 13:02:16 +05:30
Aryan c2d23fbac6 Allow max shard size to be specified when saving pipeline (#9440) 
allow max shard size to be specified when saving pipeline
 2024-12-23 13:02:16 +05:30
Linoy Tsaban b8f01f5568 [Flux Dreambooth lora] add latent caching (#9160) 
* add ostris trainer to README & add cache latents of vae

* add ostris trainer to README & add cache latents of vae

* style

* readme

* add test for latent caching

* add ostris noise scheduler
https://github.com/ostris/ai-toolkit/blob/9ee1ef2a0a2a9a02b92d114a95f21312e5906e54/toolkit/samplers/custom_flowmatch_sampler.py#L95

* style

* fix import

* style

* fix tests

* style

* --change upcasting of transformer?

* update readme according to main

* keep only latent caching

* add configurable param for final saving of trained layers- --upcast_before_saving

* style

* Update examples/dreambooth/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/dreambooth/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* use clear_objs_and_retain_memory from utilities

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Leo Jiang 64dd1c2f2a Fix the issue on sd3 dreambooth w./w.t. lora training (#9419) 
* Fix dtype error

* [bugfix] Fixed the issue on sd3 dreambooth training

* [bugfix] Fixed the issue on sd3 dreambooth training

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Benjamin Bossan a76767b684 MAINT Permission for GH token in stale.yml (#9427) 
* MAINT Permission for GH token in stale.yml

See https://github.com/huggingface/peft/pull/2061 for the equivalent PR
in PEFT.

This restores the functionality of the stale bot after permissions for
the token have been limited. The action still shows errors for PEFT but
the bot appears to work fine.

* Also add write permissions for PRs

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul 457ff5dbf8 [LoRA] fix adapter movement when using DoRA. (#9411) 
fix adapter movement when using DoRA.
 2024-12-23 13:02:16 +05:30
Dhruv Nair 4bc1d700cc [CI] Nightly Test Updates (#9380) 
* update

* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul ca1692c2c4 [CI] update artifact uploader version (#9426) 
update artifact uploader version
 2024-12-23 13:02:16 +05:30
Juan Acevedo 04abe73b30 Ptxla sd training (#9381) 
* enable pxla training of stable diffusion 2.x models.

* run linter/style and run pipeline test for stable diffusion and fix issues.

* update xla libraries

* fix read me newline.

* move files to research folder.

* update per comments.

* rename readme.

---------

Co-authored-by: Juan Acevedo <jfacevedo@google.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan 5e6901d361 [docs] AnimateDiff FreeNoise (#9414) 
* update docs

* apply suggestions from review

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* apply suggestions from review

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30