985cd2f174
[Core] introduce videoprocessor. ( #7776 )
...
* introduce videoprocessor.
* fix quality
* address yiyi's feedback
* fix preprocess_video call.
* video_processor -> image_processor
* fix
* fix more.
* quality
* image_processor -> video_processor
* support List[List[PIL.Image.Image]]
* change to video_processor.
* documentation
* Apply suggestions from code review
* changes
* remove print.
* refactor video processor (part # 7776) (#7861 )
* update
* update remove deprecate
* Update src/diffusers/video_processor.py
* update
* Apply suggestions from code review
* deprecate list of 5d for video and list of 4d for image + apply other feedbacks
* up
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* add doc.
* tensor2vid -> postprocess_video.
* refactor preprocess with preprocess_video
* set default values.
* empty commit
* more refactoring of prepare_latents in animatediff vid2vid
* checking documentation
* remove documentation for now.
* fix animatediff sdxl
* fix test failure [part of video processor PR] (#7905 )
up
* remove preceed_with_frames.
* doc
* fix
* fix
* remove video input as a single-frame video.
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
0d6df07890
add missing image processors to the docs ( #7910 )
...
add missing processors.
2024-12-23 13:02:11 +05:30
1bed93f358
upgrade to python 3.10 in the Dockerfiles ( #7893 )
...
* upgrade to python 3.10
* fix
* try https://askubuntu.com/questions/1459694/can-not-find-python3-10-after-apt-get-installation
* fix
* up
* yes
* okay
* up
* up
* up
* up
* up
* check
* okay
* up
* i[
* fix
2024-12-23 13:02:11 +05:30
0404c72b15
[scheduler] support custom timesteps and sigmas ( #7817 )
...
* support custom sigmas and timesteps, dpm euler
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:11 +05:30
2274fc4615
fix _optional_components in StableCascadeCombinedPipeline ( #7894 )
...
* fix
* up
2024-12-23 13:02:11 +05:30
7563ad6faa
Set max parallel jobs on slow test runners ( #7878 )
...
* set max parallel
* update
* update
* update
2024-12-23 13:02:11 +05:30
582071f3e9
[Tests] fix things after #7013 ( #7899 )
...
* debugging
* save the resulting image
* check if order reversing works.
* checking values.
* up
* okay
* checking
* fix
* remove print
2024-12-23 13:02:11 +05:30
e931befdbf
[Refactor] Better align from_single_file logic with from_pretrained ( #7496 )
...
* refactor unet single file loading a bit.
* retrieve the unet from create_diffusers_unet_model_from_ldm
* update
* update
* updae
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* tests
* update
* update
* update
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/loaders/single_file.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
8bc357be91
Fix several imports ( #7712 )
...
Fix imports
2024-12-23 13:02:11 +05:30
01d03e3020
Remove dead code and fix f-string issue ( #7720 )
...
* Remove dead code
* PylancereportGeneralTypeIssues: Strings nested within an f-string cannot use the same quote character as the f-string prior to Python 3.12.
* Remove dead code
2024-12-23 13:02:11 +05:30
174b93a5eb
Allow users to save SDXL LoRA weights for only one text encoder ( #7607 )
...
SDXL LoRA weights for text encoders should be decoupled on save
The method checks if at least one of unet, text_encoder and
text_encoder_2 lora weights are passed, which was not reflected in the
implentation.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
79a48eb53b
fix offload test ( #7868 )
...
fix
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:11 +05:30
f6b690301d
[Pipeline] AnimateDiff SDXL ( #6721 )
...
* update conversion script to handle motion adapter sdxl checkpoint
* add animatediff xl
* handle addition_embed_type
* fix output
* update
* add imports
* make fix-copies
* add decode latents
* update docstrings
* add animatediff sdxl to docs
* remove unnecessary lines
* update example
* add test
* revert conv_in conv_out kernel param
* remove unused param addition_embed_type_num_heads
* latest IPAdapter impl
* make fix-copies
* fix return
* add IPAdapterTesterMixin to tests
* fix return
* revert based on suggestion
* add freeinit
* fix test_to_dtype test
* use StableDiffusionMixin instead of different helper methods
* fix progress bar iterations
* apply suggestions from review
* hardcode flip_sin_to_cos and freq_shift
* make fix-copies
* fix ip adapter implementation
* fix last failing test
* make style
* Update docs/source/en/api/pipelines/animatediff.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* remove todo
* fix doc-builder errors
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:11 +05:30
23a5119e93
Check shape and remove deprecated APIs in scheduling_ddpm_flax.py ( #7703 )
...
`model_output.shape` may only have rank 1.
There are warnings related to use of random keys.
```
tests/schedulers/test_scheduler_flax.py: 13 warnings
/Users/phillypham/diffusers/src/diffusers/schedulers/scheduling_ddpm_flax.py:268: FutureWarning: normal accepts a single key, but was given a key array of shape (1, 2) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
noise = jax.random.normal(split_key, shape=model_output.shape, dtype=self.dtype)
tests/schedulers/test_scheduler_flax.py::FlaxDDPMSchedulerTest::test_betas
/Users/phillypham/virtualenv/diffusers/lib/python3.9/site-packages/jax/_src/random.py:731: FutureWarning: uniform accepts a single key, but was given a key array of shape (1,) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
u = uniform(key, shape, dtype, lo, hi) # type: ignore[arg-type]
```
2024-12-23 13:02:11 +05:30
c8a489399c
Fix image upcasting ( #7858 )
...
Fix image's upcasting before `vae.encode()` when using `fp16`
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
ee1b41e1fe
Modification on the PAG community pipeline (re) ( #7876 )
...
* edited_pag_implementation
* update
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
18aedaa230
7879 - adjust documentation to use naruto dataset, since pokemon is now gated ( #7880 )
...
* 7879 - adjust documentation to use naruto dataset, since pokemon is now gated
* replace references to pokemon in docs
* more references to pokemon replaced
* Japanese translation update
---------
Co-authored-by: bghira <bghira@users.github.com >
2024-12-23 13:02:11 +05:30
a6b6658dee
Fix for "no lora weight found module" with some loras ( #7875 )
...
* return layer weight if not found
* better system and test
* key example and typo
2024-12-23 13:02:11 +05:30
f9c78fc6f8
[docs] Distilled inference ( #7834 )
...
* combine
* edits
2024-12-23 13:02:11 +05:30
8d4f8cb2de
Ci - change cache folder ( #7867 )
2024-12-23 13:02:11 +05:30
e2d7831b8f
[docs] LCM ( #7829 )
...
* lcm
* lcm lora
* fix
* fix hfoption
* edits
2024-12-23 13:02:11 +05:30
93d9d2f9bd
Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed. ( #7816 )
...
* Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed.
* fix check code quality
* Decouple the NPU flash attention and make it an independent module.
* add doc and unit tests for npu flash attention.
---------
Co-authored-by: mhh001 <mahonghao1@huawei.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
18a5b2d98f
Remove installing python again in container ( #7852 )
...
update
2024-12-23 13:02:11 +05:30
4a008a7b30
Respect resume_download deprecation ( #7843 )
...
* Deprecate resume_download
* align docstring with transformers
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
4f985aa84d
[Tests] reduce the model size in the blipdiffusion fast test ( #7849 )
...
reducing model size
2024-12-23 13:02:11 +05:30
ec38dacf45
Update deps for pipe test fetcher ( #7838 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
585f1f7982
Change GPU Runners ( #7840 )
...
* Move to new GPU Runners for slow tests
* Move to new GPU Runners for nightly tests
2024-12-23 13:02:11 +05:30
5daae497e2
move to new runners ( #7839 )
2024-12-23 13:02:11 +05:30
683cf0cc1f
Update CI cache ( #7832 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
341c3e590e
Update download diff format tests ( #7831 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
affaa1832a
Fix hanging pipeline fetching ( #7837 )
...
update
2024-12-23 13:02:11 +05:30
f47aecb8a4
Fix key error for dictionary with randomized order in convert_ldm_unet_checkpoint ( #7680 )
...
fix key error for different order
Co-authored-by: yunseong <yunseong.cho@superlabs.us >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:11 +05:30
de4272e93e
[Tests] reduce the model size in the audioldm fast test ( #7833 )
...
chore: initial size reduction of models
2024-12-23 13:02:11 +05:30
2694cf960b
[ip-adapter] fix ip-adapter for StableDiffusionInstructPix2PixPipeline ( #7820 )
...
update prepare_ip_adapter_ for pix2pix
2024-12-23 13:02:11 +05:30
e560f7a6b8
update the logic of is_sequential_cpu_offload ( #7788 )
...
* up
* add comment to the tests + fix dit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
87b40ac14c
[Tests] fix: device map tests for models ( #7825 )
...
* fix: device module tests
* remove patch file
* Empty-Commit
2024-12-23 13:02:11 +05:30
c0410e4d14
Fix for pipeline slow test fetcher ( #7824 )
...
* update
* update
2024-12-23 13:02:11 +05:30
18f67e82d8
[docs] Community pipelines ( #7819 )
...
* community pipelines
* feedback
* consolidate
2024-12-23 13:02:11 +05:30
99b7ee9fd8
Fix CPU offload in docstring ( #7827 )
...
Fix cpu offload
2024-12-23 13:02:11 +05:30
eccd1d3701
SSH Runner Workflow Update ( #7822 )
...
* add debug workflow
* update
2024-12-23 13:02:11 +05:30
0a81020e53
Add B-Lora training option to the advanced dreambooth lora script ( #7741 )
...
* add blora
* add blora
* add blora
* add blora
* little changes
* little changes
* remove redundancies
* fixes
* add B LoRA to readme
* style
* inference
* defaults + path to loras+ generation
* minor changes
* style
* minor changes
* minor changes
* blora arg
* added --lora_unet_blocks
* style
* Update examples/advanced_diffusion_training/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* add commit hash to B-LoRA repo cloneing
* change inference, remove cloning
* change inference, remove cloning
add section about configureable unet blocks
* change inference, remove cloning
add section about configureable unet blocks
* Apply suggestions from code review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
970d780dbb
[Core] introduce _no_split_modules to ModelMixin ( #6396 )
...
* introduce _no_split_modules.
* unnecessary spaces.
* remove unnecessary kwargs and style
* fix: accelerate imports.
* change to _determine_device_map
* add the blocks that have residual connections.
* add: CrossAttnUpBlock2D
* add: testin
* style
* line-spaces
* quality
* add disk offload test without safetensors.
* checking disk offloading percentages.
* change model split
* add: utility for checking multi-gpu requirement.
* model parallelism test
* splits.
* splits.
* splits
* splits.
* splits.
* splits.
* offload folder to test_disk_offload_with_safetensors
* add _no_split_modules
* fix-copies
2024-12-23 13:02:11 +05:30
6561e9d5e7
[Tests] reduce the model size in the amused fast test ( #7804 )
...
* chore: reducing model sizes
* chore: shrinks further
* chore: shrinks further
* chore: shrinking model for img2img pipeline
* chore: reducing size of model for inpaint pipeline
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
2ba3356cbe
[Tests] reduce the model size in the ddpm fast test ( #7797 )
...
* chore: reducing unet size for faster tests
* review suggestions
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
44c27540f5
[Tests] reduce the model size in the ddim fast test ( #7803 )
...
chore: reducing model size for ddim fast pipeline
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
b3b3d961a7
Set main_input_name in StableDiffusionSafetyChecker to "clip_input" ( #7500 )
...
FlaxStableDiffusionSafetyChecker sets main_input_name to "clip_input".
This makes StableDiffusionSafetyChecker consistent.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
a59468b741
Added get_velocity function to EulerDiscreteScheduler. ( #7733 )
...
* Added get_velocity function to EulerDiscreteScheduler.
* Fix white space on blank lines
* Added copied from statement
* back to the original.
---------
Co-authored-by: Ruining Li <ruining@robots.ox.ac.uk >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:11 +05:30
277a06c705
Adding TextualInversionLoaderMixin for the controlnet_inpaint_sd_xl pipeline ( #7288 )
...
* added TextualInversionMixIn to controlnet_inpaint_sd_xl pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:11 +05:30
eaf01bc16a
Safetensor loading in AnimateDiff conversion scripts ( #7764 )
...
* update
* update
2024-12-23 13:02:11 +05:30
32a1059458
[Pipeline] Fix error of SVD pipeline when num_videos_per_prompt > 1 ( #7786 )
...
swap the order for do_classifier_free_guidance concat with repeat
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:11 +05:30
Sayak Paul