* refactor to support patching LoRA into T5
instantiate the lora linear layer on the same device as the regular linear layer
get lora rank from state dict
tests
fmt
can create lora layer in float32 even when rest of model is float16
fix loading model hook
remove load_lora_weights_ and T5 dispatching
remove Unet#attn_processors_state_dict
docstrings
* text encoder monkeypatch class method
* fix test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* dreambooth if docs - stage II, more info
* Update docs/source/en/training/dreambooth.mdx
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update docs/source/en/training/dreambooth.mdx
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update docs/source/en/training/dreambooth.mdx
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* download instructions for downsized images
* update source README to match docs
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* add _convert_kohya_lora_to_diffusers
* make style
* add scaffold
* match result: unet attention only
* fix monkey-patch for text_encoder
* with CLIPAttention
While the terrible images are no longer produced,
the results do not match those from the hook ver.
This may be due to not setting the network_alpha value.
* add to support network_alpha
* generate diff image
* fix monkey-patch for text_encoder
* add test_text_encoder_lora_monkey_patch()
* verify that it's okay to release the attn_procs
* fix closure version
* add comment
* Revert "fix monkey-patch for text_encoder"
This reverts commit bb9c61e6fa.
* Fix to reuse utility functions
* make LoRAAttnProcessor targets to self_attn
* fix LoRAAttnProcessor target
* make style
* fix split key
* Update src/diffusers/loaders.py
* remove TEXT_ENCODER_TARGET_MODULES loop
* add print memory usage
* remove test_kohya_loras_scaffold.py
* add: doc on LoRA civitai
* remove print statement and refactor in the doc.
* fix state_dict test for kohya-ss style lora
* Apply suggestions from code review
Co-authored-by: Takuma Mori <takuma104@gmail.com>
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* DataLoader will now bake in any transforms or image manipulations contained in the EXIF
Images may have rotations stored in EXIF. Training using such images will cause those transforms to be ignored while training and thus produce unexpected results
* Fixed the Dataloading EXIF issue in main DreamBooth training as well
* Run make style (black & isort)
* update IF stage I pipelines
add fixed variance schedulers and lora loading
* added kv lora attn processor
* allow loading into alternative lora attn processor
* make vae optional
* throw away predicted variance
* allow loading into added kv lora layer
* allow load T5
* allow pre compute text embeddings
* set new variance type in schedulers
* fix copies
* refactor all prompt embedding code
class prompts are now included in pre-encoding code
max tokenizer length is now configurable
embedding attention mask is now configurable
* fix for when variance type is not defined on scheduler
* do not pre compute validation prompt if not present
* add example test for if lora dreambooth
* add check for train text encoder and pre compute text embeddings
* 👽 qol improvements for LoRA.
* better function name?
* fix: LoRA weight loading with the new format.
* address Patrick's comments.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* change wording around encouraging the use of load_lora_weights().
* fix: function name.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add: LoRA text encoder support for DreamBooth example.
* fix initialization.
* fix: modification call.
* add: entry in the readme.
* use dog dataset from hub.
* fix: params to clip.
* add entry to the LoRA doc.
* add: tests for lora.
* remove unnecessary list comprehension./
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
Sayak Paul