9dc84448ac
[Examples] InstructPix2Pix instruct training script ( #2478 )
...
* add: initial implementation of the pix2pix instruct training script.
* shorten cli arg.
* fix: main process check.
* fix: dataset column names.
* simplify tokenization.
* proper placement of null conditions.
* apply styling.
* remove debugging message for conditioning do.
* complete license.
* add: requirements.tzt
* wandb column name order.
* fix: augmentation.
* change: dataset_id.
* fix: convert_to_np() call.
* fix: reshaping.
* fix: final ema copy.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* address PR comments.
* add: readme details.
* config fix.
* downgrade version.
* reduce image width in the readme.
* note on hyperparameters during generation.
* add: output images.
* update readme.
* minor edits to readme.
* debugging statement.
* explicitly placement of the pipeline.
* bump minimum diffusers version.
* fix: device attribute error.
* weight dtype.
* debugging.
* add dtype inform.
* add seoarate te and vae.
* add: explicit casting/
* remove casting.
* up.
* up 2.
* up 3.
* autocast.
* disable mixed-precision in the final inference.
* debugging information.
* autocasting.
* add: instructpix2pix training section to the docs.
* Empty-Commit
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-23 10:15:01 +05:30
c681ad1af2
add: section on multiple controlnets. ( #2762 )
...
* add: section on multiple controlnets.
Co-authored-by: William Berman <WLBberman@gmail.com >
* fix: docs.
* fix: docs.
---------
Co-authored-by: William Berman <WLBberman@gmail.com >
2023-03-23 09:55:25 +05:30
e0d8c9ef83
Support for Offset Noise in examples ( #2753 )
...
* add noise offset
* make style
2023-03-23 09:36:17 +05:30
92e1164e2e
mps: remove warmup passes (#2771 )
...
* Remove warmup passes in mps tests.
* Update mps docs: no warmup pass in PyTorch 2
* Update imports.
2023-03-22 19:29:27 +01:00
ca1a22296d
[MS Text To Video] Add first text to video ( #2738 )
...
* [MS Text To Video} Add first text to video
* upload
* make first model example
* match unet3d params
* make sure weights are correcctly converted
* improve
* forward pass works, but diff result
* make forward work
* fix more
* finish
* refactor video output class.
* feat: add support for a video export utility.
* fix: opencv availability check.
* run make fix-copies.
* add: docs for the model components.
* add: standalone pipeline doc.
* edit docstring of the pipeline.
* add: right path to TransformerTempModel
* add: first set of tests.
* complete fast tests for text to video.
* fix bug
* up
* three fast tests failing.
* add: note on slow tests
* make work with all schedulers
* apply styling.
* add slow tests
* change file name
* update
* more correction
* more fixes
* finish
* up
* Apply suggestions from code review
* up
* finish
* make copies
* fix pipeline tests
* fix more tests
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* apply suggestions
* up
* revert
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-03-22 18:39:33 +01:00
7fe88613fa
[docs] Clarify purpose of reproducibility docs ( #2756 )
...
* clarify purpose of repro docs
* apply feedback
2023-03-21 17:35:21 -07:00
a39d42b91d
[docs] update torch 2 benchmark ( #2764 )
...
* Update benchmark for A100, 3090, 3090 Ti, 4090.
* Link to PyTorch blog.
* Update install instructions.
2023-03-21 17:41:13 +00:00
ca1e40726e
stable diffusion depth batching fix ( #2757 )
2023-03-21 10:18:44 -07:00
b33bd91fae
Add option to set dtype in pipeline.to() method ( #2317 )
...
add test_to_dtype to check pipe.to(fp16)
2023-03-21 15:21:23 +01:00
1fcf279d74
Fix mps tests on torch 2.0 ( #2766 )
2023-03-21 15:19:31 +01:00
58bcf46a8f
Add guidance start/end parameters to StableDiffusionControlNetImg2ImgPipeline ( #2731 )
...
* Add guidance start/end parameters to community controlnet img2img pipeline
* Fix formats
2023-03-21 14:38:43 +01:00
0042efd015
[1929]: Add CLIP guidance for Img2Img stable diffusion pipeline ( #2723 )
...
* [Img2Img]: Copyover img2img pipeline
* [Img2Img]: img2img pipeline
* [Img2Img]: img2img pipeline
* [Img2Img]: img2img pipeline
---------
Co-authored-by: njindal <njindal@adobe.com >
2023-03-21 13:53:00 +01:00
f024e00398
Fix typos ( #2715 )
...
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-21 13:45:04 +01:00
2120b4eee3
Improve Contribution Doc ( #2043 )
...
* first refactor
* more text
* improve
* finish
* up
* up
* up
* up
* finish
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* up
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* finished
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* finished
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-03-21 13:41:29 +01:00
c10d6854c0
Update numbers for Habana Gaudi in documentation ( #2734 )
...
Update numbers for Habana Gaudi in doc
2023-03-21 11:59:28 +01:00
73bdad08a1
add: controlnet entry to training section in the docs. ( #2677 )
...
* add: controlnet entry to training section in the docs.
* formatting.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* wrap in a tip block.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-03-21 07:23:24 +05:30
ba87c1607c
Update text_inversion.mdx ( #2751 )
...
Fix typos
2023-03-20 13:20:50 -07:00
afe59a920e
Update philosophy.mdx ( #2752 )
...
Fix typos
2023-03-20 13:19:43 -07:00
25ed7cb08b
Update dreambooth.mdx ( #2742 )
...
Fix typos
2023-03-20 17:40:56 +00:00
af86b0ccac
Update fp16.mdx ( #2746 )
...
Fix typos
2023-03-20 17:39:55 +00:00
a9f28b687c
Update torch2.0.mdx ( #2748 )
...
Fix typos
2023-03-20 17:39:04 +00:00
d91dc57d8a
Update mps.mdx ( #2749 )
...
Fix typos
2023-03-20 17:33:23 +00:00
fdcff560d0
Fix more slow tests
2023-03-18 19:41:38 +00:00
ec2c1bc95f
Update README.md
2023-03-18 19:39:24 +01:00
9ecd924859
[Tests] Correct PT2 ( #2724 )
...
* [Tests] Correct PT2
* correct more
* move versatile to nightly
* up
* up
* again
* Apply suggestions from code review
2023-03-18 18:38:04 +01:00
116f70cbf8
Enabling gradient checkpointing for VAE ( #2536 )
...
* updated black format
* update black format
* make style format
* updated line endings
* update code formatting
* Update examples/research_projects/onnxruntime/text_to_image/train_text_to_image.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/vae.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/vae.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* added vae gradient checkpointing test
* make style
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Will Berman <wlbberman@gmail.com >
2023-03-17 14:59:38 -07:00
a16957159e
[docs] Update ONNX doc to use optimum ( #2702 )
...
* minor edits to onnx and openvino docs.
* Apply suggestions from code review
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com >
---------
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com >
2023-03-17 18:17:42 +01:00
f4bbcb29c0
fix image link in inpaint doc ( #2693 )
...
fix link
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-16 19:35:27 -10:00
a41850a21d
Improve deprecation error message when using cross_attention import ( #2710 )
...
Improve error message
2023-03-17 00:17:53 +01:00
a4b2c2f150
train_unconditional save restore unet parameters ( #2706 )
2023-03-16 16:15:56 -07:00
77e0ea8048
[docs] Add safety checker to ethical guidelines ( #2699 )
...
add safety checker
2023-03-16 09:39:39 -07:00
d9227cf788
Adding use_safetensors argument to give more control to users ( #2123 )
...
* Adding `use_safetensors` argument to give more control to users
about which weights they use.
* Doc style.
* Rebased (not functional).
* Rebased and functional with tests.
* Style.
* Apply suggestions from code review
* Style.
* Addressing comments.
* Update tests/test_pipelines.py
Co-authored-by: Will Berman <wlbberman@gmail.com >
* Black ???
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Will Berman <wlbberman@gmail.com >
2023-03-16 15:57:43 +01:00
e828232780
Rename attention ( #2691 )
...
* rename file
* rename attention
* fix more
* rename more
* up
* more deprecation imports
* fixes
2023-03-16 00:35:54 +01:00
588e50bc57
[docs] Reorganize table of contents ( #2671 )
...
* reorg toc
* reorg toc some more
* remove duplicate config
2023-03-15 16:28:18 -07:00
a72d14fc8d
[docs] Create better navigation on index ( #2658 )
...
* create updated nav for index
* fix header
* apply feedback
2023-03-15 11:58:04 -07:00
1c2c594e3d
[docs] Add overviews to each section ( #2657 )
...
* add overviews to each section
* fix typo in toctree
* apply feedbacks
2023-03-15 11:57:32 -07:00
e52cd55615
Add image_processor ( #2617 )
...
* add image_processor
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-03-15 07:55:49 -10:00
c0b4d72095
Update unconditional_image_generation.mdx ( #2686 )
...
Fix typos
2023-03-15 18:19:57 +01:00
78afb84436
Update controlling_generation.mdx ( #2690 )
...
Fix typos
2023-03-15 18:18:41 +01:00
91570b2fda
Update conditional_image_generation.mdx ( #2687 )
...
Fix typos
2023-03-15 18:16:32 +01:00
3584f6b345
Update img2img.mdx ( #2688 )
...
Fix typos
2023-03-15 18:15:59 +01:00
b4bb5345cd
Update kerascv.mdx ( #2685 )
...
Fix typos
2023-03-15 18:15:51 +01:00
e71f73d8df
Update custom_pipeline_overview.mdx ( #2684 )
...
Fix typos
2023-03-15 18:14:37 +01:00
cf4227cd1e
T5Attention support for cross-attention ( #2654 )
...
* fix AttnProcessor2_0
Fix use of AttnProcessor2_0 for cross attention with mask
* added scale_qk and out_bias flags
* fixed for xformers
* check if it has scale argument
* Update cross_attention.py
* check torch version
* fix sliced attn
* style
* set scale
* fix test
* fixed addedKV processor
* revert back AttnProcessor2_0
* if missing if
* fix inner_dim
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-15 18:04:05 +01:00
9d1341d69b
Update Dockerfile CUDA ( #2682 )
...
* Update Dockerfile CUDA
* Apply suggestions from code review
2023-03-15 18:02:56 +01:00
4553c29d92
[Tests] fix: slow serialization test ( #2678 )
...
fix: slow serialization tests
2023-03-15 22:30:21 +05:30
c9477bf8a8
[Docs] Adds a documentation page for evaluating diffusion models ( #2516 )
...
* add a documentation page for evaluating diffuion models.
* fix: checkpoint link.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com >
* formatting fixes.
* formatting fixes.
* link to partiprompts dataset on hub.
* reflect on Pedro's comments.
Co-authored-by: Pedro <pedro@huggingface.co >
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* reflect on Pedro's comments.
Co-authored-by: Pedro <pedro@huggingface.co >
* update mention of FID.
* Apply suggestions from code review
Co-authored-by: Will Berman <wlbberman@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* minor nit.
* finish edges and add colab notebook.
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* run formatting.
* additional feedback.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com >
Co-authored-by: Pedro <pedro@huggingface.co >
Co-authored-by: Will Berman <wlbberman@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2023-03-15 17:05:01 +05:30
79eb3d07d0
Controlnet training ( #2545 )
...
* Controlnet training code initial commit
Works with circle dataset: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md
* Script for adding a controlnet to existing model
* Fix control image transform
Control image should be in 0..1 range.
* Add license header and remove more unused configs
* controlnet training readme
* Allow nonlocal model in add_controlnet.py
* Formatting
* Remove unused code
* Code quality
* Initialize controlnet in training script
* Formatting
* Address review comments
* doc style
* explicit constructor args and submodule names
* hub dataset
NOTE - not tested
* empty prompts
* add conditioning image
* rename
* remove instance data dir
* image_transforms -> -1,1 . conditioning_image_transformers -> 0, 1
* nits
* remove local rank config
I think this isn't necessary in any of our training scripts
* validation images
* proportion_empty_prompts typo
* weight copying to controlnet bug
* call log validation fix
* fix
* gitignore wandb
* fix progress bar and resume from checkpoint iteration
* initial step fix
* log multiple images
* fix
* fixes
* tracker project name configurable
* misc
* add controlnet requirements.txt
* update docs
* image labels
* small fixes
* log validation using existing models for pipeline
* fix for deepspeed saving
* memory usage docs
* Update examples/controlnet/train_controlnet.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/train_controlnet.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* remove extra is main process check
* link to dataset in intro paragraph
* remove unnecessary paragraph
* note on deepspeed
* Update examples/controlnet/README.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* assert -> value error
* weights and biases note
* move images out of git
* remove .gitignore
---------
Co-authored-by: William Berman <WLBberman@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-14 20:16:30 -07:00
279f744ce5
controlnet integration tests num_inference_steps=3 ( #2672 )
2023-03-14 14:42:32 -07:00
ee71d9d03d
Add support for different model prediction types in DDIMInverseScheduler ( #2619 )
...
* Add support for different model prediction types in DDIMInverseScheduler
Resolve alpha_prod_t_prev index issue for final step of inversion
* Fix old bug introduced when prediction type is "sample"
* Add support for sample clipping for numerical stability and deprecate old kwarg
* Detach sample, alphas, betas
Derive predicted noise from model output before dist. regularization
Style cleanup
* Log loss for debugging
* Revert "Log loss for debugging"
This reverts commit 76ea9c856fpatrick.v.platen@gmail.com >
2023-03-14 21:25:12 +01:00
Sayak Paul