fda1531d8a
Fixing implementation of ControlNet-XS ( #6772 )
...
* CheckIn - created DownSubBlocks
* Added extra channels, implemented subblock fwd
* Fixed connection sizes
* checkin
* Removed iter, next in forward
* Models for SD21 & SDXL run through
* Added back pipelines, cleared up connections
* Cleaned up connection creation
* added debug logs
* updated logs
* logs: added input loading
* Update umer_debug_logger.py
* log: Loading hint
* Update umer_debug_logger.py
* added logs
* Changed debug logging
* debug: added more logs
* Fixed num_norm_groups
* Debug: Logging all of SDXL input
* Update umer_debug_logger.py
* debug: updated logs
* checkim
* Readded tests
* Removed debug logs
* Fixed Slow Tests
* Added value ckecks | Updated model_cpu_offload_seq
* accelerate-offloading works ; fast tests work
* Made unet & addon explicit in controlnet
* Updated slow tests
* Added dtype/device to ControlNetXS
* Filled in test model paths
* Added image_encoder/feature_extractor to XL pipe
* Fixed fast tests
* Added comments and docstrings
* Fixed copies
* Added docs ; Updates slow tests
* Moved changes to UNetMidBlock2DCrossAttn
* tiny cleanups
* Removed stray prints
* Removed ip adapters + freeU
- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components
* Fixed test_save_load_float16
* Make style, quality, fix-copies
* Changed loading/saving API for ControlNetXS
- Changed loading/saving API for ControlNetXS
- other small fixes
* Removed ControlNet-XS from research examples
* Make style, quality, fix-copies
* Small fixes
- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints
* checkin May 11 '23
* CheckIn Mar 12 '24
* Fixed tests for SD
* Added tests for UNetControlNetXSModel
* Fixed SDXL tests
* cleanup
* Delete Pipfile
* CheckIn Mar 20
Started replacing sub blocks by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`
* check-in Mar 23
* checkin 24 Mar
* Created init for UNetCnxs and CnxsAddon
* CheckIn
* Made from_modules, from_unet and no_control work
* make style,quality,fix-copies & small changes
* Fixed freezing
* Added gradient ckpt'ing; fixed tests
* Fix slow tests(+compile) ; clear naming confusion
* Don't create UNet in init ; removed class_emb
* Incorporated review feedback
- Deleted get_base_pipeline / get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU
* Make style, quality, fix-copies
* Implemented review feedback
* Removed compatibility check for vae/ctrl embedding
* make style, quality, fix-copies
* Delete Pipfile
* Integrated review feedback
- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`
* Reduced size of test tensors
For this, added `norm_num_groups` as parameter everywhere
* Renamed cnxs-`Addon` to cnxs-`Adapter`
- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up
* Fixed save_pretrained/from_pretrained bug
* Removed redundant code
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-04-16 21:56:20 +05:30
2f23437618
Add (Scheduled) Pseudo-Huber Loss training scripts to research projects ( #7527 )
...
* add scheduled pseudo-huber loss training scripts
See #7488
* add reduction modes to huber loss
* [DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.
pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/c6495def1fbbaf2a0233110d50f976ed61620e83
* [DB Lora] add option for smooth l1 (huber / delta)
Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/dd22958caa56e4db885324f76188c13bdf504569
* [DB Lora] unify huber scheduling
Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/19a834c3ab448614e8887b07f2bb4e0aaabf0805
* [DB Lora] add snr huber scheduler
Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/47fb1a68547e76f33cd54a3da8d2c35b9489c56e
* fixup examples link
* use snr schedule by default in DB
* update all huber scripts with snr
* code quality
* huber: make style && make quality
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-13 07:26:08 +05:30
8e963d1c2a
7529 do not disable autocast for cuda devices ( #7530 )
...
* 7529 do not disable autocast for cuda devices
* Remove typecasting error check for non-mps platforms, as a correct autocast implementation makes it a non-issue
* add autocast fix to other training examples
* disable native_amp for dreambooth (sdxl)
* disable native_amp for pix2pix (sdxl)
* remove tests from remaining files
* disable native_amp on huggingface accelerator for every training example that uses it
* convert more usages of autocast to nullcontext, make style fixes
* make style fixes
* style.
* Empty-Commit
---------
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-02 20:15:06 +05:30
a51b6cc86a
[Docs] Fix typos ( #7451 )
...
* Fix typos
* Fix typos
* Fix typos
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-25 11:48:02 -07:00
e29f16cfaa
[Research Projects] ORPO diffusion for alignment ( #7423 )
...
* barebones orpo
* remove reference model.
* full implementation
* change default of beta_orpo
* add a training command.
* fix: dataloading issues.
* interpreting the formulation.
* revert styling
* add: wds full blown version
* fix: per_gpu_batch_siz
* start debuggin
* debugging
* remove print
* fix
* remove filter keys.
* turn on non-blocking calls.
* device_placement
* let's see.
* add bigger training run command
* reinitialize generator for fair repro
* add: detailed readme and requirements
---------
Co-authored-by: Sayak Paul <sayakpaul@Sayaks-MacBook-Pro-2.local >
2024-03-25 08:37:41 +05:30
e25e525fde
[LoRA test suite] refactor the test suite and cleanse it ( #7316 )
...
* cleanse and refactor lora testing suite.
* more cleanup.
* make check_if_lora_correctly_set a utility function
* fix: typo
* retrigger ci
* style
2024-03-20 17:13:52 +05:30
5d848ec07c
[Tests] Update a deprecated parameter in test files and fix several typos ( #7277 )
...
* Add properties and `IPAdapterTesterMixin` tests for `StableDiffusionPanoramaPipeline`
* Fix variable name typo and update comments
* Update deprecated `output_type="numpy"` to "np" in test files
* Discard changes to src/diffusers/pipelines/stable_diffusion_panorama/pipeline_stable_diffusion_panorama.py
* Update test_stable_diffusion_panorama.py
* Update numbers in README.md
* Update get_guidance_scale_embedding method to use timesteps instead of w
* Update number of checkpoints in README.md
* Add type hints and fix var name
* Fix PyTorch's convention for inplace functions
* Fix a typo
* Revert "Fix PyTorch's convention for inplace functions"
This reverts commit 74350cf65b
2024-03-14 12:17:35 -07:00
4fbd310fd2
[Chore] switch to logger.warning ( #7289 )
...
switch to logger.warning
2024-03-13 06:56:43 +05:30
eb942b866a
SDXL Turbo support and example launch ( #6473 )
...
* support and example launch for sdxl turbo
* White space fixes
* Trailing whitespace character
* ruff format
* fix guidance_scale and steps for turbo mode
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Radames Ajna <radamajna@gmail.com >
2024-03-06 11:51:01 +05:30
6246c70d21
[Community] PromptDiffusion Pipeline ( #6752 )
...
* Create promptdiffusioncontrolnet.py
* Update __init__.py
Added PromptDiffusionControlNetModel
* Update __init__.py
Added PromptDiffusionControlNetModel
* Update promptdiffusioncontrolnet.py
* Create pipeline_prompt_diffusion.py
Added Prompt Diffusion pipeline.
* Create convert_original_promptdiffusion_to_diffusers.py
* Update convert_from_ckpt.py
Added download_promptdiffusion_from_original_ckpt, convert_promptdiffusion_checkpoint
* Update promptdiffusioncontrolnet.py
* Update pipeline_prompt_diffusion.py
* Update README.md
* Update pipeline_prompt_diffusion.py
* Delete src/diffusers/models/promptdiffusioncontrolnet.py
* Update __init__.py
* Update __init__.py
* Delete scripts/convert_original_promptdiffusion_to_diffusers.py
* Update convert_from_ckpt.py
* Update README.md
* Delete examples/community/pipeline_prompt_diffusion.py
* Create README.md
* Create promptdiffusioncontrolnet.py
* Create convert_original_promptdiffusion_to_diffusers.py
* Create pipeline_prompt_diffusion.py
* Update README.md
* Update pipeline_prompt_diffusion.py
* Update README.md
* Update pipeline_prompt_diffusion.py
* Update convert_original_promptdiffusion_to_diffusers.py
* Update promptdiffusioncontrolnet.py
* Update README.md
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-05 09:06:02 +05:30
8da360aa12
[training scripts] add tags of diffusers-training ( #7206 )
...
* add tags for diffusers training
* add tags for diffusers training
* add tags for diffusers training
* add tags for diffusers training
* add tags for diffusers training
* add tags for diffusers training
* add dora tags for drambooth lora scripts
* style
2024-03-04 22:17:25 +05:30
fa633ed6de
refactor: move model helper function in pipeline to a mixin class ( #6571 )
...
* move model helper function in pipeline to EfficiencyMixin
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-02-28 09:26:39 -10:00
c0f5346a20
Fix procecss process ( #6591 )
...
* Fix words
* Fix
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-02-15 19:06:33 -10:00
159885adc6
correct hub_token exposition behaviour (thanks to @bghira). ( #6918 )
2024-02-08 18:38:27 -10:00
30e5e81d58
change to 2024 in the license ( #6902 )
...
change to 2024
2024-02-08 08:19:31 -10:00
97ee616971
add ipo, hinge and cpo loss to dpo trainer ( #6788 )
...
add ipo and hinge loss to dpo trainer
2024-01-31 16:41:31 +05:30
1cab64b3be
Update train_diffusion_dpo.py ( #6754 )
...
* Update train_diffusion_dpo.py
Address #6702
* Update train_diffusion_dpo_sdxl.py
* Empty-Commit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-01-31 12:46:23 +05:30
acd1962769
correct hflip arg ( #6743 )
2024-01-28 17:42:34 +05:30
c101066227
Correct SNR weighted loss in v-prediction case by only adding 1 to SNR on the denominator ( #6307 )
...
* fix minsnr implementation for v-prediction case
* format code
* always compute snr when snr_gamma is specified
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-01-27 09:18:09 +05:30
b4220e97b1
Add Community Example Consistency Training Script ( #6717 )
...
* initial commit for unconditional/class-conditional consistency training script
* make style
* Add entry for consistency training script in community README.
* Move consistency training script from community to research_projects/consistency_training
* Add requirements.txt and README to research_projects/consistency_training directory.
* Manually revert community README changes for consistency training.
* Fix path to script after moving script to research projects.
* Add option to load U-Net weights from pretrained model.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2024-01-26 15:10:57 +02:00
c9081a8abd
[Fix bugs] pipeline_controlnet_sd_xl.py ( #6653 )
...
* Update pipeline_controlnet_sd_xl.py
* Update pipeline_controlnet_xs_sd_xl.py
2024-01-24 09:18:12 +05:30
1f0705adcf
[Big refactor] move unets to unets module 🦋 ( #6630 )
...
* move unets to module 🦋
* parameterize unet-level import.
* fix flax unet2dcondition model import
* models __init__
* mildly depcrecating models.unet_2d_blocks in favor of models.unets.unet_2d_blocks.
* noqa
* correct depcrecation behaviour
* inherit from the actual classes.
* Empty-Commit
* backwards compatibility for unet_2d.py
* backward compatibility for unet_2d_condition
* bc for unet_1d
* bc for unet_1d_blocks
2024-01-23 08:57:58 +05:30
da95a28ff6
[Diffusion DPO] apply fixes from #6547 ( #6668 )
...
apply fixes from #6547
2024-01-22 20:14:54 +05:30
d6a70d8ba8
Bump jinja2 from 3.1.2 to 3.1.3 in /examples/research_projects/realfill ( #6539 )
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/3.1.2...3.1.3 )
---
updated-dependencies:
- dependency-name: jinja2
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-15 16:10:10 +02:00
1dd0ac9401
[DPO Training] pass tracker name as argument ( #6542 )
...
pass tracker name as argumentw
2024-01-12 09:15:39 +05:30
2d1f2182cc
example: Train Instruct pix2 pix with lora implementation ( #6469 )
...
* base template file - train_instruct_pix2pix.py
* additional import and parser argument requried for lora
* finetune only instructpix2pix model -- no need to include these layers
* inject lora layers
* freeze unet model -- only lora layers are trained
* training modifications to train only lora parameters
* store only lora parameters
* move train script to research project
* run quality and style code checks
* move train script to a new folder
* add README
* update README
* update references in README
---------
Co-authored-by: Rahul Raman <rahulraman@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-01-10 06:38:19 +05:30
a483a8eddf
Rename REAMDE.md to README.md
2024-01-05 20:49:44 +05:30
2a97067b84
[Experimental] Diffusion LoRA DPO training ( #6422 )
...
* add: experimental script for diffusion dpo training.
* random_crop cli.
* fix: caption tokenization.
* fix: pixel_values index.
* fix: grad?
* debug
* fix: reduction.
* fixes in the loss calculation.
* style
* fix: unwrap call.
* fix: validation inference.
* add: initial sdxl script
* debug
* make sure images in the tuple are of same res
* fix model_max_length
* report print
* boom
* fix: numerical issues.
* fix: resolution
* comment about resize.
* change the order of the training transformation.
* save call.
* debug
* remove print
* manually detaching necessary?
* use the same vae for validation.
* add: readme.
2024-01-05 16:40:06 +05:30
d700140076
[LoRA deprecation] handle rest of the stuff related to deprecated lora stuff. ( #6426 )
...
* handle rest of the stuff related to deprecated lora stuff.
* fix: copies
* don't modify the uNet in-place.
* fix: temporal autoencoder.
* manually remove lora layers.
* don't copy unet.
* alright
* remove lora attn processors from unet3d
* fix: unet3d.
* styl
* Empty-Commit
2024-01-03 20:54:09 +05:30
9f283b01d2
changed w&b report link ( #6387 )
2023-12-29 19:49:11 +05:30
e7044a4221
multi-subject-dreambooth-inpainting with 🤗 datasets ( #6378 )
...
* files added
* fixing code quality
* fixing code quality
* fixing code quality
* fixing code quality
* sorted import block
* seperated import wandb
* ruff on script
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-29 09:33:49 +05:30
c1e8bdf1d4
Move ControlNetXS into Community Folder ( #6316 )
...
* update
* update
* update
* update
* update
* make style
* remove docs
* update
* move to research folder.
* fix-copies
* remove _toctree entry.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-27 08:15:23 +05:30
0532cece97
Bump transformers from 4.34.0 to 4.36.0 in /examples/research_projects/realfill ( #6255 )
...
Bump transformers in /examples/research_projects/realfill
Bumps [transformers](https://github.com/huggingface/transformers ) from 4.34.0 to 4.36.0.
- [Release notes](https://github.com/huggingface/transformers/releases )
- [Commits](https://github.com/huggingface/transformers/compare/v4.34.0...v4.36.0 )
---
updated-dependencies:
- dependency-name: transformers
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-21 09:03:17 +05:30
0a401b95b7
[Docs] Fix typos ( #6122 )
...
Fix typos and trim trailing whitespaces
2023-12-11 10:55:28 -08:00
75ada25048
Harmonize HF environment variables + deprecate use_auth_token ( #6066 )
...
* Harmonize HF environment variables + deprecate use_auth_token
* fix import
* fix
2023-12-06 22:22:31 +01:00
6b04d61cf6
[Styling] stylify using ruff ( #5841 )
...
* ruff format
* not need to use doc-builder's black styling as the doc is styled in ruff
* make fix-copies
* comment
* use run_ruff
2023-11-20 11:48:34 +01:00
c896b841e4
Set usedforsecurity=False in hashlib methods (FIPS compliance) ( #5790 )
...
* Set usedforsecurity=False in hashlib methods (FIPS compliance)
* update version dependency
* bump hfh version
* bump hfh version
2023-11-17 14:56:58 +01:00
8fcd52febb
Correct code for distributed training of RealFill ( #5740 )
...
Correct code for distributed training
2023-11-13 19:01:15 +01:00
0488810f61
Fix realfill example compatibility with latest torchvision version ( #5736 )
2023-11-13 18:55:17 +01:00
5b087e82d1
Add realfill ( #5456 )
...
* Add realfill
* Move realfill folder
* Fix some format issues
2023-10-30 15:21:40 +01:00
cc12f3ec92
[Examples] Update with HFApi ( #5393 )
...
* update training examples to use HFAPI.
* update training example.
* reflect the changes in the korean version too.
* Empty-Commit
2023-10-16 19:34:46 +05:30
0ea78f9707
chore: fix typos ( #5386 )
...
* chore: fix typos
* Update src/diffusers/pipelines/shap_e/renderer.py
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com >
---------
Co-authored-by: psychedelicious <4822129+psychedelicious@users.noreply.github.com >
2023-10-16 15:23:37 +02:00
02a8d664a2
Min-SNR Gamma: correct the fix for SNR weighted loss in v-prediction … ( #5238 )
...
Min-SNR Gamma: correct the fix for SNR weighted loss in v-prediction by adding 1 to SNR rather than the resulting loss weights
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-10-05 20:52:27 +02:00
dfcce3ca6e
[Research folder] Add SDXL example ( #5275 )
...
* [SDXL Flax] Add research folder
* Add co-author
Co-authored-by: Juan Acevedo <jfacevedo@google.com >
---------
Co-authored-by: Juan Acevedo <jfacevedo@google.com >
2023-10-03 07:51:46 +02:00
bdd16116f3
[Schedulers] Fix callback steps ( #5261 )
...
* fix all
* make fix copies
* make fix copies
2023-10-02 19:52:53 +02:00
cdcc01be0e
[Examples] add compute_snr() to training utils. ( #5188 )
...
add compute_snr() to training utils.
2023-09-27 21:42:20 +05:30
4a06c74547
Min-SNR Gamma: follow-up fix for zero-terminal SNR models on v-prediction or epsilon ( #5177 )
...
* merge with main
* fix flax example
* fix onnx example
---------
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-26 18:14:52 +05:30
24563ca654
SNR gamma fixes for v_prediction training ( #5106 )
...
Co-authored-by: bghira <bghira@users.github.com >
2023-09-20 21:18:56 +01:00
d73e6ad050
guard save model hooks to only execute on main process ( #4929 )
2023-09-08 10:30:06 -07:00
3201903d94
Retrieval Augmented Diffusion Models ( #3297 )
...
* Resetting rdm pr
* Fixed styles
* Fixed style
* Moved to rdm folder+fixed slight errors
* Removed config diff
* Started adding tests
* Adding retrieved images
* Fixed faiss import
* Fixed import errors
* Fixing tests
* Added require_faiss
* Updated dependency table
* Attempt solving consistency test
* Fixed truncation and vocab size issue
* Passed common tests
* Finished up cpu testing on pipeline
* Passed all tests locally
* Removed some slow tests
* Removed diffs from test_pipeline_common
* Remove logs
* Removed diffs from test_pipelines_common
* Fixed style
* Fully fixed styles on diffs
* Fixed name
* Proper rename
* Fixed dummies
* Fixed issue with dummyonnx
* Fixed black style
* Fixed dummies
* Changed ordering
* Fixed logging
* Fixing
* Fixing
* quality
* Debugging regex
* Fix dummies with guess
* Fixed typo
* Attempt fix dummies
* black
* ruff
* fixed ordering
* Logging
* Attempt fix
* Attempt fix dummy
* Attempt fixing styles
* Fixed faiss dependency
* Removed unnecessary deprecations
* Finished up main changes
* Added doc
* Passed tests
* Fixed tests
* Remove invisible watermark
* Fixed ruff errors
* Added prompt embed to tests
* Added tests and made retriever an optional component
* Fixed styles
* Made faiss a dependency of pipeline
* Logging
* Fixed dummies
* Make pipeline test work
* Fixed style
* Moved to research projects
* Remove diff
* Fixed style error
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-04 09:42:04 +02:00
UmerHA