5fd42e5d61
Add SDXL refiner only tests ( #5041 )
...
* add refiner only tests
* make style
2023-09-15 12:58:03 +05:30
e70cb1243f
[WIP] adding Kandinsky training scripts ( #4890 )
...
* Add files via upload
Co-authored-by: Shahmatov Arseniy <62886550+cene555@users.noreply.github.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-09-14 06:58:20 -10:00
fe4837a96e
add step_index and clear noise_sampler at begining of each loop ( #5024 )
...
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-14 06:48:35 -10:00
342c5c02c0
[Release 0.21] Bump version ( #5018 )
...
* [Release 0.21] Bump version
* fix & remove
* fix more
* fix all, upload
2023-09-14 18:28:57 +02:00
169fc4add5
Add Prompt2Prompt pipeline ( #4563 )
...
* Initial commit P2P
* Replaced CrossAttention, added test skeleton
* bug fixes
* Updated docstring
* Removed unused function
* Created tests
* improved tests
- made fast inference tests faster
- corrected image shape assertions
* Corrected expected output shape in tests
* small fix: test inputs
* Update tests
- used conditional unet2d
- set expected image slices
- edit_kwargs are now not popped, so pipe can be run multiple times
* Fixed bug in int tests
* Fixed tests
* Linting
* Create prompt2prompt.md
* Added to docs toc
* Ran make fix-copies
* Fixed code blocks in docs
* Using same interface as StableDiffusionPipeline
* Fixed small test bug
* Added all options SDPipeline.__call_ has
* Fixed docstring; made __call__ like in SD
* Linting
* Added test for multiple prompts
* Improved docs
* Incorporated feedback
* Reverted formatting on unrelated files
* Moved prompt2prompt to community
- Moved prompt2prompt pipeline from main to community
- Deleted tests
- Moved documentation to community and shorted it
* Update src/diffusers/utils/dummy_torch_and_transformers_objects.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-14 16:39:59 +02:00
566bdf4c44
Allow disabling from_pretrained tqdm ( #5007 )
2023-09-14 16:36:19 +02:00
0eb715d715
[LoRA] Make optional arguments explicit ( #5038 )
...
make optional arguments explciit
2023-09-14 15:58:54 +02:00
3aa641289c
[Import] Add missing settings / Correct some dummy imports ( #5036 )
...
* [Import] Add missing settings
* up
* up
* up
2023-09-14 12:42:54 +02:00
ef29b24fda
allow loading of sd models from safetensors without online lookups using local config files ( #5019 )
...
finish config_files implementation
2023-09-14 12:30:15 +02:00
8dc93ad3e4
[Import] Don't force transformers to be installed ( #5035 )
...
* [Import] Don't force transformers to be installed
* make style
2023-09-14 11:42:10 +02:00
e2033d2dff
Fix model offload bug when key isn't present ( #5030 )
...
* fix model offload bug when key isn't present
* make style
2023-09-14 11:02:06 +02:00
19edca82f1
[docs] Create clearer optimization sections ( #4870 )
...
* refactor
* update general optim sections
* update more sections
* few more updates
* benchmark code
2023-09-13 15:21:15 -07:00
b954c22a44
Fix broken link in docs ( #5015 )
...
fix broken link
2023-09-13 15:40:25 +02:00
77373c5eb1
[Wuerstchen] fix compel usage ( #4999 )
...
* fix compel usage
* minor changes in documentation
* fix tests
* fix more
* fix more
* typos
* fix tests
* formatting
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-13 14:54:59 +02:00
0ea51627f1
[Core] Fix dtype in InstructPix2Pix SDXL while computing image_latents ( #5013 )
...
* check out dtypes.
* check out dtypes.
* check out dtypes.
* check out dtypes.
* check out dtypes.
* check out dtypes.
* check out dtypes.
* potential fix
* check out dtypes.
* check out dtypes.
* working?
2023-09-13 10:50:24 +01:00
6d6a08f1f1
[Flax->PT] Fix flaky testing ( #5011 )
...
fix flaky flax class name
2023-09-13 11:29:13 +02:00
34bfe98eaf
Gligen Text to Image fix ( #5010 )
...
* fix gligen clip import issue
* fix dtype issue with gligen text to image pipeline
* make fix copies
2023-09-13 10:23:59 +01:00
b47f5115da
[Lora] fix lora fuse unfuse ( #5003 )
...
* fix lora fuse unfuse
* add same changes to loaders.py
* add test
---------
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com >
2023-09-13 11:21:04 +02:00
324aef6d14
[SDXL] Add LoRA to all pipelines ( #4896 )
...
* [SDXL] Add LoRA to all pipelines
* fix all
* fix all
* fix all
* fix more docs
* make style
2023-09-13 11:05:20 +02:00
8009272f48
[Tests and Docs] Add a test on serializing pipelines with components containing fused LoRA modules ( #4962 )
...
* add: test to ensure pipelines can be saved with fused lora modules.
* add docs about serialization with fused lora.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Empty-Commit
* Update docs/source/en/training/lora.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-13 10:01:37 +01:00
1037287e2b
examples fix t2i training ( #5001 )
...
* examples fix t2i training
* make style
2023-09-12 23:52:41 +02:00
6ea95b7a90
Fix PR template ( #4984 )
...
fix template
2023-09-12 19:36:38 +02:00
0e0db625d0
Fix safety checker seq offload ( #4998 )
...
* fix safety checker
* fix safety checker
* fix safety checker
2023-09-12 18:56:35 +02:00
1f948109b8
[docs] Fix DiffusionPipeline.enable_sequential_cpu_offload docstring ( #4952 )
...
* Fix an unmatched backtick and make description more general for DiffusionPipeline.enable_sequential_cpu_offload.
* make style
* _exclude_from_cpu_offload -> self._exclude_from_cpu_offload
* make style
* apply suggestions from review
* make style
2023-09-12 08:58:47 -07:00
37cb819df5
[Lora] Speed up lora loading ( #4994 )
...
* speed up lora loading
* Apply suggestions from code review
* up
* up
* Fix more
* Correct more
* Apply suggestions from code review
* up
* Fix more
* Fix more -
* up
* up
2023-09-12 17:51:15 +02:00
f64d52dbca
fix custom diffusion tests ( #4996 )
2023-09-12 17:50:47 +02:00
4d897aaff5
fix image variation slow test ( #4995 )
...
fix image variation tests
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-12 17:45:47 +02:00
b1105269b7
make style
2023-09-12 14:55:27 +00:00
5d28d2217f
[Wuerstchen] fix combined pipeline's num_images_per_prompt ( #4989 )
...
* fix encode_prompt
* added prompt_embeds and negative_prompt_embeds
* prompt_embeds for the prior only
2023-09-12 16:55:13 +02:00
73bf620dec
fix E721 Do not compare types, use isinstance() ( #4992 )
2023-09-12 16:52:25 +02:00
c806f2fad6
remove extra gligen in import ( #4987 )
2023-09-12 18:35:29 +05:30
18b7264bd0
[Utils] Correct custom init sort ( #4967 )
...
* [Utils] Correct custom init sort
* [Utils] Correct custom init sort
* [Utils] Correct custom init sort
* add type checking
* fix custom init sort
* fix test
* fix tests
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-09-12 11:05:53 +02:00
d82157b3ce
[Bug Fix] Should pass the dtype instead of torch_dtype ( #4917 )
...
.
2023-09-11 19:45:58 +02:00
93579650f8
Refactor model offload ( #4514 )
...
* [Draft] Refactor model offload
* [Draft] Refactor model offload
* Apply suggestions from code review
* cpu offlaod updates
* remove model cpu offload from individual pipelines
* add hook to offload models to cpu
* clean up
* model offload
* add model cpu offload string
* make style
* clean up
* fixes for offload issues
* fix tests issues
* resolve merge conflicts
* update src/diffusers/pipelines/pipeline_utils.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* make style
* Update src/diffusers/pipelines/latent_diffusion/pipeline_latent_diffusion.py
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-09-11 19:39:26 +02:00
16a056a7b5
Wuerstchen fixes ( #4942 )
...
* fix arguments and make example code work
* change arguments in combined test
* Add default timesteps
* style
* fixed test
* fix broken test
* formatting
* fix docstrings
* fix num_images_per_prompt
* fix doc styles
* please dont change this
* fix tests
* rename to DEFAULT_STAGE_C_TIMESTEPS
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
2023-09-11 15:47:53 +02:00
6c6a246461
Update README.md ( #4973 )
...
Add monthly pip installs
2023-09-11 15:45:19 +02:00
6bbee1048b
Make sure Flax pipelines can be loaded into PyTorch ( #4971 )
...
* Make sure Flax pipelines can be loaded into PyTorch
* add test
* Update src/diffusers/pipelines/pipeline_utils.py
2023-09-11 12:03:49 +02:00
2c60f7d14e
[Core] Remove TF import checks ( #4968 )
...
[TF] Remove tf
2023-09-11 11:22:40 +02:00
b6e0b016ce
Lazy Import for Diffusers ( #4829 )
...
* initial commit
* move modules to import struct
* add dummy objects and _LazyModule
* add lazy import to schedulers
* clean up unused imports
* lazy import on models module
* lazy import for schedulers module
* add lazy import to pipelines module
* lazy import altdiffusion
* lazy import audio diffusion
* lazy import audioldm
* lazy import consistency model
* lazy import controlnet
* lazy import dance diffusion ddim ddpm
* lazy import deepfloyd
* lazy import kandinksy
* lazy imports
* lazy import semantic diffusion
* lazy imports
* lazy import stable diffusion
* move sd output to its own module
* clean up
* lazy import t2iadapter
* lazy import unclip
* lazy import versatile and vq diffsuion
* lazy import vq diffusion
* helper to fetch objects from modules
* lazy import sdxl
* lazy import txt2vid
* lazy import stochastic karras
* fix model imports
* fix bug
* lazy import
* clean up
* clean up
* fixes for tests
* fixes for tests
* clean up
* remove import of torch_utils from utils module
* clean up
* clean up
* fix mistake import statement
* dedicated modules for exporting and loading
* remove testing utils from utils module
* fixes from merge conflicts
* Update src/diffusers/pipelines/kandinsky2_2/__init__.py
* fix docs
* fix alt diffusion copied from
* fix check dummies
* fix more docs
* remove accelerate import from utils module
* add type checking
* make style
* fix check dummies
* remove torch import from xformers check
* clean up error message
* fixes after upstream merges
* dummy objects fix
* fix tests
* remove unused module import
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-11 09:56:22 +02:00
88735249da
[Docs] fix: minor formatting in the Würstchen docs ( #4965 )
...
fix: minor formatting in the docs
2023-09-11 09:12:53 +02:00
4191ddee11
Revert revert and install accelerate main ( #4963 )
...
* Revert "Temp Revert "[Core] better support offloading when side loading is enabled… (#4927 )"
This reverts commit 2ab170499e
2023-09-11 08:49:46 +02:00
2ab170499e
Temp Revert "[Core] better support offloading when side loading is enabled… ( #4927 )
...
Revert "[Core] better support offloading when side loading is enabled. (#4855 )"
This reverts commit e4b8e7928b
2023-09-08 19:54:59 -07:00
914c513ee0
[Docs] add t2i adapter entry to overview of training scripts. ( #4946 )
...
add t2i adapter entry to overview of training scripts.
2023-09-09 06:52:11 +05:30
d73e6ad050
guard save model hooks to only execute on main process ( #4929 )
2023-09-08 10:30:06 -07:00
d0cf681a1f
[Tests] add: tests for t2i adapter training. ( #4947 )
...
add: tests for t2i adapter training.
2023-09-08 19:45:39 +05:30
dfec61f4b3
[examples] T2IAdapter training script ( #4934 )
...
* add t2i_example script
* remove in channels logic
* remove comments
* remove use_euler arg
* add requirements
* only use canny example
* use datasets
* comments
* make log_validation consistent with other scripts
* add readme
* fix title in readme
* update check_min_version
* change a few minor things.
* add doc entry
* add: test for t2i adapter training
* remove use_auth_token
* fix: logged info.
* remove tests for now.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-08 10:03:02 +05:30
0ec7a02b6a
[StableDiffusionXLAdapterPipeline] allow negative micro conds ( #4941 )
...
* allow negative micro conds in t2i pipeline
* Empty-Commit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-08 07:59:42 +05:30
626284f8d1
[StableDiffusionXLAdapterPipeline] add adapter_conditioning_factor ( #4937 )
...
add adapter_conditioning_factor
2023-09-07 19:05:28 +02:00
9800cc5ece
[InstructPix2Pix] Fix pipeline implementation and add docs ( #4844 )
...
* initial evident fixes.
* instructpix2pix fixes.
* add: entry to doc.
* address PR feedback.
* make fix-copies
2023-09-07 15:34:19 +05:30
541bb6ee63
Würstchen model ( #3849 )
...
* initial
* initial
* added initial convert script for paella vqmodel
* initial wuerstchen pipeline
* add LayerNorm2d
* added modules
* fix typo
* use model_v2
* embed clip caption amd negative_caption
* fixed name of var
* initial modules in one place
* WuerstchenPriorPipeline
* inital shape
* initial denoising prior loop
* fix output
* add WuerstchenPriorPipeline to __init__.py
* use the noise ratio in the Prior
* try to save pipeline
* save_pretrained working
* Few additions
* add _execution_device
* shape is int
* fix batch size
* fix shape of ratio
* fix shape of ratio
* fix output dataclass
* tests folder
* fix formatting
* fix float16 + started with generator
* Update pipeline_wuerstchen.py
* removed vqgan code
* add WuerstchenGeneratorPipeline
* fix WuerstchenGeneratorPipeline
* fix docstrings
* fix imports
* convert generator pipeline
* fix convert
* Work on Generator Pipeline. WIP
* Pipeline works with our diffuzz code
* apply scale factor
* removed vqgan.py
* use cosine schedule
* redo the denoising loop
* Update src/diffusers/models/resnet.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* use torch.lerp
* use warp-diffusion org
* clip_sample=False,
* some refactoring
* use model_v3_stage_c
* c_cond size
* use clip-bigG
* allow stage b clip to be None
* add dummy
* würstchen scheduler
* minor changes
* set clip=None in the pipeline
* fix attention mask
* add attention_masks to text_encoder
* make fix-copies
* add back clip
* add text_encoder
* gen_text_encoder and tokenizer
* fix import
* updated pipeline test
* undo changes to pipeline test
* nip
* fix typo
* fix output name
* set guidance_scale=0 and remove diffuze
* fix doc strings
* make style
* nip
* removed unused
* initial docs
* rename
* toc
* cleanup
* remvoe test script
* fix-copies
* fix multi images
* remove dup
* remove unused modules
* undo changes for debugging
* no new line
* remove dup conversion script
* fix doc string
* cleanup
* pass default args
* dup permute
* fix some tests
* fix prepare_latents
* move Prior class to modules
* offload only the text encoder and vqgan
* fix resolution calculation for prior
* nip
* removed testing script
* fix shape
* fix argument to set_timesteps
* do not change .gitignore
* fix resolution calculations + readme
* resolution calculation fix + readme
* small fixes
* Add combined pipeline
* rename generator -> decoder
* Update .gitignore
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* removed efficient_net
* create combined WuerstchenPipeline
* make arguments consistent with VQ model
* fix var names
* no need to return text_encoder_hidden_states
* add latent_dim_scale to config
* split model into its own file
* add WuerschenPipeline to docs
* remove unused latent_size
* register latent_dim_scale
* update script
* update docstring
* use Attention preprocessor
* concat with normed input
* fix-copies
* add docs
* fix test
* fix style
* add to cpu_offloaded_model
* updated type
* remove 1-line func
* updated type
* initial decoder test
* formatting
* formatting
* fix autodoc link
* num_inference_steps is int
* remove comments
* fix example in docs
* Update src/diffusers/pipelines/wuerstchen/diffnext.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* rename layernorm to WuerstchenLayerNorm
* rename DiffNext to WuerstchenDiffNeXt
* added comment about MixingResidualBlock
* move paella vq-vae to pipelines' folder
* initial decoder test
* increased test_float16_inference expected diff
* self_attn is always true
* more passing decoder tests
* batch image_embeds
* fix failing tests
* set the correct dtype
* relax inference test
* update prior
* added combined pipeline test
* faster test
* faster test
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix issues from review
* update wuerstchen.md + change generator name
* resolve issues
* fix copied from usage and add back batch_size
* fix API
* fix arguments
* fix combined test
* Added timesteps argument + fixes
* Update tests/pipelines/test_pipelines_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/wuerstchen/test_wuerstchen_prior.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
* up
* Fix more
* failing tests
* up
* up
* correct naming
* correct docs
* correct docs
* fix test params
* correct docs
* fix classifier free guidance
* fix classifier free guidance
* fix more
* fix all
* make tests faster
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Dominic Rampas <61938694+dome272@users.noreply.github.com >
2023-09-06 16:15:51 +02:00
Dhruv Nair