obscura/diffusers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,076 Commits 711 Branches 93 Tags
be4afa0bb4384f201c8fe68af536faffefbae661
Commit Graph

901 Commits

Author SHA1 Message Date
Mark Van Aken be4afa0bb4 #7535 Update FloatTensor type hints to Tensor (#7883) 
* find & replace all FloatTensors to Tensor

* apply formatting

* Update torch.FloatTensor to torch.Tensor in the remaining files

* formatting

* Fix the rest of the places where FloatTensor is used as well as in documentation

* formatting

* Update new file from FloatTensor to Tensor
 2024-05-10 09:53:31 -10:00
Tolga Cangöz c1c42698c9 Remove dead code and fix f-string issue (#7720) 
* Remove dead code

* PylancereportGeneralTypeIssues: Strings nested within an f-string cannot use the same quote character as the f-string prior to Python 3.12.

* Remove dead code
 2024-05-08 13:15:28 -10:00
Hyoungwon Cho c2217142bd Modification on the PAG community pipeline (re) (#7876) 
* edited_pag_implementation

* update

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-05-07 16:35:15 -10:00
Bagheera 8edaf3b79c 7879 - adjust documentation to use naruto dataset, since pokemon is now gated (#7880) 
* 7879 - adjust documentation to use naruto dataset, since pokemon is now gated

* replace references to pokemon in docs

* more references to pokemon replaced

* Japanese translation update

---------

Co-authored-by: bghira <bghira@users.github.com>
 2024-05-07 09:36:39 -07:00
HelloWorldBeginner 58237364b1 Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed. (#7816) 
* Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed.

* fix check code quality

* Decouple the NPU flash attention and make it an independent module.

* add doc and unit tests for npu flash attention.

---------

Co-authored-by: mhh001 <mahonghao1@huawei.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-05-03 08:14:34 -10:00
YiYi Xu 21a7ff12a7 update the logic of is_sequential_cpu_offload (#7788) 
* up

* add comment to the tests + fix dit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-05-01 06:25:57 -10:00
Linoy Tsaban 26a7851e1e Add B-Lora training option to the advanced dreambooth lora script (#7741) 
* add blora

* add blora

* add blora

* add blora

* little changes

* little changes

* remove redundancies

* fixes

* add B LoRA to readme

* style

* inference

* defaults + path to loras+ generation

* minor changes

* style

* minor changes

* minor changes

* blora arg

* added --lora_unet_blocks

* style

* Update examples/advanced_diffusion_training/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* add commit hash to B-LoRA repo cloneing

* change inference, remove cloning

* change inference, remove cloning
add section about configureable unet blocks

* change inference, remove cloning
add section about configureable unet blocks

* Apply suggestions from code review

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-30 09:46:30 +05:30
Nilesh 235d34cf56 Check for latents, before calling prepare_latents - sdxlImg2Img (#7582) 
* Check for latents, before calling prepare_latents - sdxlImg2Img

* Added latents check for all the img2img pipeline

* Fixed silly mistake while checking latents as None
 2024-04-28 14:53:29 -10:00
39th president of the United States, probably 9d16daaf64 Add DREAM training (#6381) 
A new function compute_dream_and_update_latents has been added to the
training utilities that allows you to do DREAM rectified training in line
with the paper https://arxiv.org/abs/2312.00210. The method can be used
with an extra argument in the train_text_to_image.py script.

Co-authored-by: Jimmy <39@🇺🇸.com>
 2024-04-27 07:19:15 +05:30
Steven Liu 7404f1e9dc [docs] Clean up toctree (#7715) 
* toctree

* optim

* feedback

* improve overview
 2024-04-23 09:30:33 -07:00
Abhinav Gopal d1e3f489e9 Animatediff Controlnet Community Pipeline IP Adapter Fix (#7413) 
* fixed encode_image function signature in controlnet animatediff

* copied encode_image from stable diffusion pipeline

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-04-19 15:35:07 -10:00
Sai-Suraj-27 db969cc16d fix: Fixed type annotations for compatability with python 3.8 (#7648) 
* Fixed type annotations for compatability with python 3.8

* Add required imports.
 2024-04-18 19:34:09 -10:00
Dhruv Nair 90250d9e48 Cast height, width to int inside prepare latents (#7691) 
update
 2024-04-18 19:30:39 -10:00
Fabio Rigano b5c8b555d7 Move IP Adapter Face ID to core (#7186) 
* Switch to peft and multi proj layers

* Move Face ID loading and inference to core

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-18 14:13:27 -10:00
UmerHA fda1531d8a Fixing implementation of ControlNet-XS (#6772) 
* CheckIn - created DownSubBlocks

* Added extra channels, implemented subblock fwd

* Fixed connection sizes

* checkin

* Removed iter, next in forward

* Models for SD21 & SDXL run through

* Added back pipelines, cleared up connections

* Cleaned up connection creation

* added debug logs

* updated logs

* logs: added input loading

* Update umer_debug_logger.py

* log: Loading hint

* Update umer_debug_logger.py

* added logs

* Changed debug logging

* debug: added more logs

* Fixed num_norm_groups

* Debug: Logging all of SDXL input

* Update umer_debug_logger.py

* debug: updated logs

* checkim

* Readded tests

* Removed debug logs

* Fixed Slow Tests

* Added value ckecks | Updated model_cpu_offload_seq

* accelerate-offloading works ; fast tests work

* Made unet & addon explicit in controlnet

* Updated slow tests

* Added dtype/device to ControlNetXS

* Filled in test model paths

* Added image_encoder/feature_extractor to XL pipe

* Fixed fast tests

* Added comments and docstrings

* Fixed copies

* Added docs ; Updates slow tests

* Moved changes to UNetMidBlock2DCrossAttn

* tiny cleanups

* Removed stray prints

* Removed ip adapters + freeU

- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components

* Fixed test_save_load_float16

* Make style, quality, fix-copies

* Changed loading/saving API for ControlNetXS

- Changed loading/saving API for ControlNetXS
- other small fixes

* Removed ControlNet-XS from research examples

* Make style, quality, fix-copies

* Small fixes

- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints

* checkin May 11 '23

* CheckIn Mar 12 '24

* Fixed tests for SD

* Added tests for UNetControlNetXSModel

* Fixed SDXL tests

* cleanup

* Delete Pipfile

* CheckIn Mar 20

Started replacing sub blocks  by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`

* check-in Mar 23

* checkin 24 Mar

* Created init for UNetCnxs and CnxsAddon

* CheckIn

* Made from_modules, from_unet and no_control work

* make style,quality,fix-copies & small changes

* Fixed freezing

* Added gradient ckpt'ing; fixed tests

* Fix slow tests(+compile) ; clear naming confusion

* Don't create UNet in init ; removed class_emb

* Incorporated review feedback

- Deleted get_base_pipeline /  get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU

* Make style, quality, fix-copies

* Implemented review feedback

* Removed compatibility check for vae/ctrl embedding

* make style, quality, fix-copies

* Delete Pipfile

* Integrated review feedback

- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`

* Reduced size of test tensors

For this, added `norm_num_groups` as parameter everywhere

* Renamed cnxs-`Addon` to cnxs-`Adapter`

- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up

* Fixed save_pretrained/from_pretrained bug

* Removed redundant code

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-04-16 21:56:20 +05:30
kabachuha 2f23437618 Add (Scheduled) Pseudo-Huber Loss training scripts to research projects (#7527) 
* add scheduled pseudo-huber loss training scripts

See #7488

* add reduction modes to huber loss

* [DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.

pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/c6495def1fbbaf2a0233110d50f976ed61620e83

* [DB Lora] add option for smooth l1 (huber / delta)

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/dd22958caa56e4db885324f76188c13bdf504569

* [DB Lora] unify huber scheduling

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/19a834c3ab448614e8887b07f2bb4e0aaabf0805

* [DB Lora] add snr huber scheduler

Pairing of https://github.com/kohya-ss/sd-scripts/pull/1228/commits/47fb1a68547e76f33cd54a3da8d2c35b9489c56e

* fixup examples link

* use snr schedule by default in DB

* update all huber scripts with snr

* code quality

* huber: make style && make quality

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-13 07:26:08 +05:30
dg845 0bee4d336b LCM Distill Scripts Fix Bug when Initializing Target U-Net (#6848) 
* Initialize target_unet from unet rather than teacher_unet so that we correctly add time_embedding.cond_proj if necessary.

* Use UNet2DConditionModel.from_config to initialize target_unet from unet's config.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-11 07:52:12 -10:00
satani99 37e9d695af Modularize instruct_pix2pix SD inferencing during and after training in examples (#7603) 
* Modularize instruct_pix2pix code

* quality check

* quality check

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-10 11:19:16 +05:30
Christopher Beckham 8e46d97cd8 Add missing restore() EMA call in train SDXL script (#7599) 
* Restore unet params back to normal from EMA when validation call is finished

* empty commit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-09 18:07:55 +05:30
Bagheera 8e963d1c2a 7529 do not disable autocast for cuda devices (#7530) 
* 7529 do not disable autocast for cuda devices

* Remove typecasting error check for non-mps platforms, as a correct autocast implementation makes it a non-issue

* add autocast fix to other training examples

* disable native_amp for dreambooth (sdxl)

* disable native_amp for pix2pix (sdxl)

* remove tests from remaining files

* disable native_amp on huggingface accelerator for every training example that uses it

* convert more usages of autocast to nullcontext, make style fixes

* make style fixes

* style.

* Empty-Commit

---------

Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-02 20:15:06 +05:30
Álvaro Somoza 73ba81090e [Community pipeline] SDXL Differential Diffusion Img2Img Pipeline (#7550) 
* initial-commit pipeline created

* updated README.md
 2024-04-01 18:15:30 -10:00
YiYi Xu 7956c36aaa add a from_pipe method to DiffusionPipeline (#7241) 
* add from_pipe



---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-04-01 13:02:00 -10:00
haikmanukyan 5266ab7935 add HD-Painter pipeline (#7520) 
* add HD-Painter pipeline

* style fixing

* refactor, change doc, fix ruff

* fix docs

* used correct ruff version

---------

Co-authored-by: Hayk Manukyan <youremail@yourdomain.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-04-01 15:10:44 +05:30
Bingxin Ke c2e87869be [Community pipeline] Marigold depth estimation update -- align with marigold v0.1.5 (#7524) 
* add resample option; check denoise_step; update ckpt path

* Add seeding in pipeline to increase reproducibility

* fix typo

* fix typo
 2024-03-30 07:09:02 -10:00
Hyoungwon Cho 9d20ed37a2 Perturbed-Attention Guidance (#7512) 
* pag_initial

* pag_docs

* edit_docs

* custom

* typo

* delete_docs

* whitespace

* make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-30 10:52:51 +05:30
Linoy Tsaban bda1d4faf8 add Instant id sdxl image2image pipeline (#7507) 
* initial commit - instantid img2img

* adapting to img2img

* change add_time_ids

* change add_time_ids

* WIP changes

* add strength to timesteps

* check insightface import

* style

* check insightface import changed to warning

* check insightface import changed to warning

* style

---------

Co-authored-by: apolinário <joaopaulo.passos@gmail.com>
 2024-03-30 10:25:21 +05:30
Bagheera d78acdedc1 apple mps: training support for SDXL (ControlNet, LoRA, Dreambooth, T2I) (#7447) 
* apple mps: training support for SDXL LoRA

* sdxl: support training lora, dreambooth, t2i, pix2pix, and controlnet on apple mps

---------

Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-28 14:26:18 +05:30
Thomas Liang 0cc5630945 [Chore] Fix Colab notebook links in README.md (#7495) 2024-03-27 12:36:36 -10:00
Long(Tony) Lian 5199ee4f7b Fix missing raise statements in check_inputs (#7473) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-26 13:34:28 -10:00
Sayak Paul 288632adf6 [Training utils] add kohya conversion dict. (#7435) 
* add kohya conversion dict.

* update readme

* typo

* add filename
 2024-03-26 17:31:22 +05:30
Ernie Chu 5ce79cbded Update train_dreambooth_lora_sd15_advanced.py (#7433) 
you cannot specify `type="bool"` and `action="store_true"` at the same time.
remove excessive and buggy `type=bool`.

Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
 2024-03-26 12:53:02 +02:00
Marçal Comajoan Cara d52f3e30f8 Fix broken link (#7472) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-26 10:29:08 +05:30
estelleafl 0dd0528851 Small ldm3d fix (#7464) 
* fixed typo

* updated doc to be consistent in naming

* make style/quality

* preprocessing for 4 channels and not 6

* make style

* test for 4c

* make style/quality

* fixed test on cpu

* fixed doc typo

* changed default ckpt to 4c

* Update pipeline_stable_diffusion_ldm3d.py

* fix bug

---------

Co-authored-by: Aflalo <estellea@isl-iam1.rr.intel.com>
Co-authored-by: Aflalo <estellea@isl-gpu33.rr.intel.com>
Co-authored-by: Aflalo <estellea@isl-gpu38.rr.intel.com>
 2024-03-25 15:33:43 -10:00
M. Tolga Cangöz a51b6cc86a [Docs] Fix typos (#7451) 
* Fix typos

* Fix typos

* Fix typos

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-25 11:48:02 -07:00
Sayak Paul e29f16cfaa [Research Projects] ORPO diffusion for alignment (#7423) 
* barebones orpo

* remove reference model.

* full implementation

* change default of beta_orpo

* add a training command.

* fix: dataloading issues.

* interpreting the formulation.

* revert styling

* add: wds full blown version

* fix: per_gpu_batch_siz

* start debuggin

* debugging

* remove print

* fix

* remove filter keys.

* turn on non-blocking calls.

* device_placement

* let's see.

* add bigger training run command

* reinitialize generator for fair repro

* add: detailed readme and requirements

---------

Co-authored-by: Sayak Paul <sayakpaul@Sayaks-MacBook-Pro-2.local>
 2024-03-25 08:37:41 +05:30
YiYi Xu e4356d6488 add a "Community Scripts" section (#7358) 
* add

* add tiling

* fix

* fix

* fix

* give community script its own readme

* Update examples/community/README_community_scripts.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/community/README_community_scripts.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/community/README_community_scripts.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/community/README_community_scripts.md

---------

Co-authored-by: Alexis Rolland <alexis.rolland@ubisoft.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-03-21 10:05:07 -10:00
M. Tolga Cangöz 3028089e5e Fix typos (#7411) 
* Fix typos

* Fix typo in SVD.md
 2024-03-20 18:46:47 -07:00
Sayak Paul e25e525fde [LoRA test suite] refactor the test suite and cleanse it (#7316) 
* cleanse and refactor lora testing suite.

* more cleanup.

* make check_if_lora_correctly_set a utility function

* fix: typo

* retrigger ci

* style
 2024-03-20 17:13:52 +05:30
M. Tolga Cangöz e97a633b63 Update access of configuration attributes (#7343) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-18 08:53:29 -10:00
M. Tolga Cangöz 6a05b274cc Fix Typos (#7325) 
* Fix PyTorch's convention for inplace functions

* Fix import structure in __init__.py and update config loading logic in test_config.py

* Update configuration access

* Fix typos

* Trim trailing white spaces

* Fix typo in logger name

* Revert "Fix PyTorch's convention for inplace functions"

This reverts commit f65dc4afcb.

* Fix typo in step_index property description

* Revert "Update configuration access"

This reverts commit 8d44e870b8.

* Revert "Fix import structure in __init__.py and update config loading logic in test_config.py"

This reverts commit 2ad5e8bca2.

* Fix typos

* Fix typos

* Fix typos

* Fix a typo: tranform -> transform
 2024-03-18 09:48:40 -07:00
Anatoly Belikov 98d46a3f08 delete vae and text encoders after use in SDXL training script (#6693) 
delete vae and text encoders after use

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-18 20:03:53 +05:30
Sayak Paul 76de6a09fb post-release v0.27.0 (#7329) 
* post-release

* quality
 2024-03-18 10:52:20 +05:30
M. Tolga Cangöz 5d848ec07c [Tests] Update a deprecated parameter in test files and fix several typos (#7277) 
* Add properties and `IPAdapterTesterMixin` tests for `StableDiffusionPanoramaPipeline`

* Fix variable name typo and update comments

* Update deprecated `output_type="numpy"` to "np" in test files

* Discard changes to src/diffusers/pipelines/stable_diffusion_panorama/pipeline_stable_diffusion_panorama.py

* Update test_stable_diffusion_panorama.py

* Update numbers in README.md

* Update get_guidance_scale_embedding method to use timesteps instead of w

* Update number of checkpoints in README.md

* Add type hints and fix var name

* Fix PyTorch's convention for inplace functions

* Fix a typo

* Revert "Fix PyTorch's convention for inplace functions"

This reverts commit 74350cf65b.

* Fix typos

* Indent

* Refactor get_guidance_scale_embedding method in LEditsPPPipelineStableDiffusionXL class
 2024-03-14 12:17:35 -07:00
Linoy Tsaban 83062fb872 [Advanced DreamBooth LoRA SDXL] Support EDM-style training (follow up of #7126) (#7182) 
* add edm style training

* style

* finish adding edm training feature

* import fix

* fix latents mean

* minor adjustments

* add edm to readme

* style

* fix autocast and scheduler config issues when using edm

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-14 18:40:14 +05:30
Anatoly Belikov 53e9aacc10 log loss per image (#7278) 
* log loss per image

* add commandline param for per image loss logging

* style

* debug-loss -> debug_loss

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-14 11:41:43 +05:30
Beinsezii d3986f18be Change step_offset scheduler docstrings (#7128) 
* Change step_offset scheduler docstrings

* Mention it may be needed by some models

* More docstrings

These ones failed literal S&R because I performed it case-sensitive
which is fun.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-13 15:12:00 -10:00
jnhuang ac07b6dc6a Fix Wrong Text-encoder Grad Setting in Custom_Diffusion Training (#7302) 
fix index in set textencoder grad

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-03-13 20:22:44 +05:30
Sayak Paul 4fbd310fd2 [Chore] switch to logger.warning (#7289) 
switch to logger.warning
 2024-03-13 06:56:43 +05:30
pravdomil 165af7edd3 Inline InputPadder (#6582) 
inline InputPadder
 2024-03-09 11:24:07 -10:00
pravdomil e64fdcf2ce Fix gmflow_dir (#6583) 
* remove sys.path

* update readme
 2024-03-09 08:53:17 -10:00