d1e20be664
make style
2023-08-30 14:13:14 +02:00
af3854d6ad
sketch inpaint from a1111 for non-inpaint models ( #4824 )
...
* Create masked_stable_diffusion_img2img.py
* add MaskedIm2ImPipeline to readme
* Update README.md
2023-08-30 09:51:28 +02:00
9f1936d2fc
Fix Unfuse Lora ( #4833 )
...
* Fix Unfuse Lora
* add tests
* Fix more
* Fix more
* Fix all
* make style
* make style
2023-08-30 09:32:25 +05:30
fbca2e0a7a
Add loading ckpt from file for SDXL controlNet ( #4683 )
...
* Add load ckpt from file for ControlNet SDXL
* Reformat code
* Resort imports
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-30 09:00:53 +05:30
3768d4d77c
[Core] refactor encode_prompt ( #4617 )
...
* refactoring of encode_prompt()
* better handling of device.
* fix: device determination
* fix: device determination 2
* handle num_images_per_prompt
* revert changes in loaders.py and give birth to encode_prompt().
* minor refactoring for encode_prompt()/
* make backward compatible.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix: concatenation of the neg and pos embeddings.
* incorporate encode_prompt() in test_stable_diffusion.py
* turn it into big PR.
* make it bigger
* gligen fixes.
* more fixes to fligen
* _encode_prompt -> encode_prompt in tests
* first batch
* second batch
* fix blasphemous mistake
* fix
* fix: hopefully for the final time.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-30 08:57:26 +05:30
8ccb619416
VaeImageProcessor: Allow image resizing also for torch and numpy inputs ( #4832 )
...
Co-authored-by: Nikhil Gajendrakumar <nikhilkatte@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-29 22:45:05 +02:00
0699ac62f0
fix typo ( #4822 )
2023-08-29 20:54:36 +02:00
a76f2ad538
make style
2023-08-29 09:25:09 +02:00
7200daa412
Support saving multiple t2i adapter models under one checkpoint ( #4798 )
...
* adding save and load for MultiAdapter, adding test
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Adding changes from review test_stable_diffusion_adapter
* import sorting fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-29 09:24:40 +02:00
3eeaf4e041
Fix convert_original_stable_diffusion_to_diffusers script ( #4817 )
...
Fix stable diffusion conversion script
2023-08-29 09:14:45 +02:00
c583f3b452
Fuse loras ( #4473 )
...
* Fuse loras
* initial implementation.
* add slow test one.
* styling
* add: test for checking efficiency
* print
* position
* place model offload correctly
* style
* style.
* unfuse test.
* final checks
* remove warning test
* remove warnings altogether
* debugging
* tighten up tests.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debuging
* debugging
* debugging
* debugging
* suit up the generator initialization a bit.
* remove print
* update assertion.
* debugging
* remove print.
* fix: assertions.
* style
* can generator be a problem?
* generator
* correct tests.
* support text encoder lora fusion.
* tighten up tests.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-29 09:14:24 +02:00
12358b986f
add models for T2I-Adapter-XL ( #4696 )
...
* T2I-Adapter-XL
* update
* update
* add pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify modeling_text_unet
* fix styling.
* fix: copies.
* adapter settings
* new test case
* new test case
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* revert prints.
* new test case
* remove print
* org test case
* add test_pipeline
* styling.
* fix copies.
* modify test parameter
* style.
* add adapter-xl doc
* double quotes in docs
* Fix potential type mismatch
* style.
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
2023-08-29 10:34:07 +05:30
5eeedd9e33
add StableDiffusionXLControlNetImg2ImgPipeline ( #4592 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 08:16:27 -10:00
a971c598b5
fix auto_pipeline: pass kwargs to load_config ( #4793 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 07:42:16 -10:00
934d439a42
fix bug in StableDiffusionXLControlNetPipeline when use guess_mode ( #4799 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 06:51:17 -10:00
e3f3672f46
Fix Disentangle ONNX and non-ONNX pipeline ( #4656 )
...
* initial commit to fix inheritance issue
* clean up sd onnx upscale
* clean up
2023-08-28 21:14:49 +05:30
87ae330056
[Examples] Save SDXL LoRA weights with chosen precision ( #4791 )
...
* Increase min accelerate ver to avoid OOM when mixed precision
* Rm re-instantiation of VAE
* Rm casting to float32
* Del unused models and free GPU
* Fix style
2023-08-28 13:57:40 +05:30
1b46c66132
make style
2023-08-28 07:17:21 +00:00
031358988b
Fix save_path bug in textual inversion training script ( #4710 )
...
* Update textual_inversion.py
fixed safe_path bug in textual inversion training
* Update test_examples.py
update test_textual_inversion for updating saved file's name
* Update textual_inversion.py
fixed some formatting issues
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 09:17:08 +02:00
fd35689f25
[WIP] Add Fabric ( #4201 )
...
* empty PR
* init
* changes
* starting with the pipeline
* stable diff
* prev
* more things, getting started
* more functions
* makeing it more readable
* almost done testing
* var changes
* testing
* device
* device support
* maybe
* device malfunctions
* new new
* register
* testing
* exec does not work
* float
* change info
* change of architecture
* might work
* testing with colab
* more attn atuff
* stupid additions
* documenting and testing
* writing tests
* more docs
* tests and docs
* remove test
* empty PR
* init
* changes
* starting with the pipeline
* stable diff
* prev
* more things, getting started
* more functions
* makeing it more readable
* almost done testing
* var changes
* testing
* device
* device support
* maybe
* device malfunctions
* new new
* register
* testing
* exec does not work
* float
* change info
* change of architecture
* might work
* testing with colab
* more attn atuff
* stupid additions
* documenting and testing
* writing tests
* more docs
* tests and docs
* remove test
* change cross attention
* revert back
* tests
* reverting back to orig
* changes
* test passing
* pipeline changes
* before quality
* quality checks pass
* remove print statements
* doc fixes
* __init__ error something
* update docs, working on dim
* working on encoding
* doc fix
* more fixes
* no more dependent on 512*512
* update docs
* fixes
* test passing
* remove comment
* fixes and migration
* simpler tests
* doc changes
* green CI
* changes
* more docs
* changes
* new images
* to community examples
* selete
* more fixes
* changes
* fix
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-28 09:10:55 +02:00
e8c9069d6f
Update loaders.py ( #4805 )
...
* Update loaders.py
Solves an error sometimes thrown while iterating over state_dict.keys() caused by using the .pop() method within the loop.
* Update loaders.py
2023-08-28 11:23:25 +05:30
766aa50f70
[LoRA Attn Processors] Refactor LoRA Attn Processors ( #4765 )
...
* [LoRA Attn] Refactor LoRA attn
* correct for network alphas
* fix more
* fix more tests
* fix more tests
* Move below
* Finish
* better version
* correct serialization format
* fix
* fix more
* fix more
* fix more
* Apply suggestions from code review
* Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
* deprecation
* relax atol for slow test slighly
* Finish tests
* make style
* make style
2023-08-28 10:38:09 +05:30
c4d2823601
[SDXL Lora] Fix last ben sdxl lora ( #4797 )
...
* Fix last ben sdxl lora
* Correct typo
* make style
2023-08-26 23:31:56 +02:00
4f8853e481
[Torch compile] Fix torch compile for controlnet ( #4795 )
...
Fix torch compile for controlnete
2023-08-26 22:30:02 +02:00
fed88195e3
[docs] Fix syntax for compel ( #4794 )
...
* fix syntax
* update image
2023-08-26 11:33:10 -07:00
0de35e4a52
[Tests] Tighten up LoRA loading relaxation ( #4787 )
...
* debugging
* better logic for filtering.
* Update src/diffusers/loaders.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-26 15:01:16 +05:30
0d81e543a2
Unet fix ( #4769 )
...
* Optional images variable train_custom_diffusion.py
* Fixed train_custom_diffusion.py
* Revert accidental changes to unet_2d_condition.py
* "Format code with black"
2023-08-26 11:01:24 +02:00
3be0ff9056
[Core] Support negative conditions in SDXL ( #4774 )
...
* add: support negative conditions.
* fix: key
* add: tests
* address PR feedback.
* add documentation
* add img2img support.
* add inpainting support.
* ad controlnet support
* Apply suggestions from code review
* modify wording in the doc.
2023-08-26 09:13:44 +05:30
2764db3194
[SDXL] Add docs about forcing passed embeddings to be 0 ( #4783 )
...
* make style
* make style
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* make style
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-25 20:52:45 +02:00
048d901993
make style
2023-08-25 18:51:03 +00:00
cb432c4ebc
Allow passing a checkpoint state_dict to convert_from_ckpt (instead of just a string path) ( #4653 )
...
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 20:50:39 +02:00
b7b1a30bc4
refactor prepare_mask_and_masked_image with VaeImageProcessor ( #4444 )
...
* refactor image processor for mask
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-25 08:18:48 -10:00
7e5587a5ac
instance_prompt->class_prompt ( #4784 )
2023-08-25 20:06:55 +02:00
dc8da1d449
Fixed broken link of CLIP doc in evaluation doc ( #4760 )
2023-08-25 20:04:50 +02:00
3dd540171d
fix bug of progress bar in clip guided images mixing ( #4729 )
2023-08-25 18:54:03 +02:00
b3b2d30cd8
fix a bug in from_pretrained when load optional components ( #4745 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 06:25:48 -10:00
3bba44d74e
[WIP ] Proposal to address precision issues in CI ( #4775 )
...
* proposal for flaky tests
* clean up
2023-08-25 19:12:09 +05:30
b1290d3fb8
Convert MusicLDM ( #4579 )
...
* from audioldm
* fix vae
* move to new pipeline
* copied from audioldm
* remove redundant control flow
* iterate
* fix docstring
* finish pipeline
* tests: from audioldm2
* iterate
* finish fast tests
* finish slow integration tests
* add docs
* remove dtype test
* update toctree
* "copied from" in conversion (where possible)
* Update docs/source/en/api/pipelines/musicldm.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix docstring
* make nightly
* style
* fix dtype test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 13:31:00 +01:00
29a11c2a94
[AudioLDM 2] Pipeline fixes ( #4738 )
...
* fix docs
* fix unet docs
* use image output for latents
* fix hub checkpoints
* fix pipeline example
* update example
* return_dict = False
* revert image pipeline output
* revert doc changes
* remove dtype test
* make style
* remove docstring updates
* remove unet docstring update
* Empty commit to re-trigger CI
* fix cpu offload
* fix dtype test
* add offload test
2023-08-25 11:38:10 +01:00
cdacd8f1dd
Torch device ( #4755 )
2023-08-25 11:13:32 +02:00
470d51c8ed
improve setup.py ( #4748 )
2023-08-25 13:44:20 +05:30
d6141205cd
fix sdxl_lwp empty neg_prompt error issue ( #4743 )
...
* fix sdxl_lwp empty neg_prompt error issue
* fix sdxl_lwp empty neg_prompt error issue, update code format
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-25 09:55:56 +05:30
4447547eda
[Examples] fix sdxl dreambooth lora checkpointing. ( #4749 )
...
* fix sdxl dreambooth lora checkpointing.
* style
2023-08-25 09:50:02 +05:30
5222294748
[LoRA] relax lora loading logic ( #4610 )
...
* relax lora loading logic.
* cater to the other cases too.
* fix: variable name
* bring the chaos down.
* check
* deal with checkpointed files.
* Apply suggestions from code review
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
* style
---------
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-08-25 09:35:51 +05:30
c25c46137d
[Examples] Add madebyollin VAE to SDXL LoRA example, along with an explanation ( #4762 )
...
Add madebyollin VAE to LoRA example, along with an explenation
2023-08-25 09:34:32 +05:30
3105c710ba
[fix] multi t2i adapter set total_downscale_factor ( #4621 )
...
* [fix] multi t2i adapter set total_downscale_factor
* move image checks into check inputs
* remove copied from
2023-08-24 12:01:23 -07:00
58f5f748f4
[Tests] Fix paint by example ( #4761 )
...
* [Tests] Fix paint by example
* Update src/diffusers/pipelines/paint_by_example/image_encoder.py
2023-08-24 16:03:10 +02:00
4f05058bb7
Clean up flaky behaviour on Slow CUDA Pytorch Push Tests ( #4759 )
...
use max diff to compare model outputs
2023-08-24 18:58:02 +05:30
5d4413001b
make style
2023-08-24 10:19:47 +00:00
863e741614
Bugfix for SDXL model loading in low ram system. ( #4628 )
...
Update convert_from_ckpt.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-24 12:19:16 +02:00
Patrick von Platen