update

2024-02-19 16:59:11 +00:00 · 2024-02-19 16:13:45 +00:00 · 2024-02-19 15:13:54 +00:00 · 2024-02-19 08:40:48 +00:00 · 2024-02-19 11:11:42 +05:30 · 2024-02-19 10:09:19 +05:30
786 changed files with 5200 additions and 5602 deletions
@@ -32,7 +32,7 @@ jobs:
        run: |
          apt-get update && apt-get install libsndfile1-dev libgl1 -y
          python -m pip install -e .[quality,test]
-          python -m pip install pandas
+          python -m pip install pandas peft
      - name: Environment
        run: |
          python utils/print_env.py
@@ -34,11 +34,6 @@ jobs:
            runner: docker-cpu
            image: diffusers/diffusers-pytorch-cpu
            report: torch_cpu_models_schedulers
          - name: LoRA
            framework: lora
            runner: docker-cpu
            image: diffusers/diffusers-pytorch-cpu
            report: torch_cpu_lora
          - name: Fast Flax CPU tests
            framework: flax
            runner: docker-cpu
@@ -94,14 +89,6 @@ jobs:
          --make-reports=tests_${{ matrix.config.report }} \
          tests/models tests/schedulers tests/others
    - name: Run fast PyTorch LoRA CPU tests
      if: ${{ matrix.config.framework == 'lora' }}
      run: |
        python -m pytest -n 2 --max-worker-restart=0 --dist=loadfile \
          -s -v -k "not Flax and not Onnx and not Dependency" \
          --make-reports=tests_${{ matrix.config.report }} \
          tests/lora
    - name: Run fast Flax TPU tests
      if: ${{ matrix.config.framework == 'flax' }}
      run: |
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -26,9 +26,9 @@ ENV PATH="/opt/venv/bin:$PATH"
 # pre-install the heavy dependencies (these can later be overridden by the deps from setup.py)
 RUN python3.9 -m pip install --no-cache-dir --upgrade pip && \
    python3.9 -m pip install --no-cache-dir \
-    torch==2.1.2 \
+    torch \
-    torchvision==0.16.2 \
+    torchvision \
-    torchaudio==2.1.2 \
+    torchaudio \
    invisible_watermark && \
    python3.9 -m pip install --no-cache-dir \
    accelerate \
@@ -25,9 +25,9 @@ ENV PATH="/opt/venv/bin:$PATH"
 # pre-install the heavy dependencies (these can later be overridden by the deps from setup.py)
 RUN python3 -m pip install --no-cache-dir --upgrade pip && \
    python3 -m pip install --no-cache-dir \
-        torch==2.1.2 \
+        torch \
-        torchvision==0.16.2 \
+        torchvision \
-        torchaudio==2.1.2 \
+        torchaudio \
        invisible_watermark \
        --extra-index-url https://download.pytorch.org/whl/cpu && \
    python3 -m pip install --no-cache-dir \
@@ -25,9 +25,9 @@ ENV PATH="/opt/venv/bin:$PATH"
 # pre-install the heavy dependencies (these can later be overridden by the deps from setup.py)
 RUN python3 -m pip install --no-cache-dir --upgrade pip && \
    python3 -m pip install --no-cache-dir \
-    torch==2.1.2 \
+    torch \
-    torchvision==0.16.2 \
+    torchvision \
-    torchaudio==2.1.2 \
+    torchaudio \
    invisible_watermark && \
    python3 -m pip install --no-cache-dir \
    accelerate \
@@ -25,9 +25,9 @@ ENV PATH="/opt/venv/bin:$PATH"
 # pre-install the heavy dependencies (these can later be overridden by the deps from setup.py)
 RUN python3 -m pip install --no-cache-dir --upgrade pip && \
    python3 -m pip install --no-cache-dir \
-        torch==2.1.2 \
+        torch \
-        torchvision==0.16.2 \
+        torchvision \
-        torchaudio==2.1.2 \
+        torchaudio \
        invisible_watermark && \
    python3 -m pip install --no-cache-dir \
        accelerate \
@@ -1,5 +1,5 @@
 <!---
-Copyright 2023- The HuggingFace Team. All rights reserved.
+Copyright 2024- The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -52,12 +52,16 @@
      title: Image-to-image
    - local: using-diffusers/inpaint
      title: Inpainting
    - local: using-diffusers/text-img2vid
      title: Text or image-to-video
    - local: using-diffusers/depth2img
      title: Depth-to-image
    title: Tasks
  - sections:
    - local: using-diffusers/textual_inversion_inference
      title: Textual inversion
    - local: using-diffusers/ip_adapter
      title: IP-Adapter
    - local: training/distributed_inference
      title: Distributed inference with multiple GPUs
    - local: using-diffusers/reusing_seeds
@@ -321,6 +325,8 @@
        title: Text-to-image
      - local: api/pipelines/stable_diffusion/img2img
        title: Image-to-image
      - local: api/pipelines/stable_diffusion/svd
        title: Image-to-video
      - local: api/pipelines/stable_diffusion/inpaint
        title: Inpainting
      - local: api/pipelines/stable_diffusion/depth2img
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -20,6 +20,24 @@ An attention processor is a class for applying different types of attention mech
 ## AttnProcessor2_0
 [[autodoc]] models.attention_processor.AttnProcessor2_0
 ## AttnAddedKVProcessor
 [[autodoc]] models.attention_processor.AttnAddedKVProcessor
 ## AttnAddedKVProcessor2_0
 [[autodoc]] models.attention_processor.AttnAddedKVProcessor2_0
 ## CrossFrameAttnProcessor
 [[autodoc]] pipelines.text_to_video_synthesis.pipeline_text_to_video_zero.CrossFrameAttnProcessor
 ## CustomDiffusionAttnProcessor
 [[autodoc]] models.attention_processor.CustomDiffusionAttnProcessor
 ## CustomDiffusionAttnProcessor2_0
 [[autodoc]] models.attention_processor.CustomDiffusionAttnProcessor2_0
 ## CustomDiffusionXFormersAttnProcessor
 [[autodoc]] models.attention_processor.CustomDiffusionXFormersAttnProcessor
 ## FusedAttnProcessor2_0
 [[autodoc]] models.attention_processor.FusedAttnProcessor2_0
@@ -29,32 +47,17 @@ An attention processor is a class for applying different types of attention mech
 ## LoRAAttnProcessor2_0
 [[autodoc]] models.attention_processor.LoRAAttnProcessor2_0
 ## CustomDiffusionAttnProcessor
 [[autodoc]] models.attention_processor.CustomDiffusionAttnProcessor
 ## CustomDiffusionAttnProcessor2_0
 [[autodoc]] models.attention_processor.CustomDiffusionAttnProcessor2_0
 ## AttnAddedKVProcessor
 [[autodoc]] models.attention_processor.AttnAddedKVProcessor
 ## AttnAddedKVProcessor2_0
 [[autodoc]] models.attention_processor.AttnAddedKVProcessor2_0
 ## LoRAAttnAddedKVProcessor
 [[autodoc]] models.attention_processor.LoRAAttnAddedKVProcessor
 ## XFormersAttnProcessor
 [[autodoc]] models.attention_processor.XFormersAttnProcessor
 ## LoRAXFormersAttnProcessor
 [[autodoc]] models.attention_processor.LoRAXFormersAttnProcessor
 ## CustomDiffusionXFormersAttnProcessor
 [[autodoc]] models.attention_processor.CustomDiffusionXFormersAttnProcessor
 ## SlicedAttnProcessor
 [[autodoc]] models.attention_processor.SlicedAttnProcessor
 ## SlicedAttnAddedKVProcessor
 [[autodoc]] models.attention_processor.SlicedAttnAddedKVProcessor
 ## XFormersAttnProcessor
 [[autodoc]] models.attention_processor.XFormersAttnProcessor
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -12,11 +12,11 @@ specific language governing permissions and limitations under the License.
 # IP-Adapter
-[IP-Adapter](https://hf.co/papers/2308.06721) is a lightweight adapter that enables prompting a diffusion model with an image. This method decouples the cross-attention layers of the image and text features. The image features are generated from an image encoder. Files generated from IP-Adapter are only ~100MBs.
+[IP-Adapter](https://hf.co/papers/2308.06721) is a lightweight adapter that enables prompting a diffusion model with an image. This method decouples the cross-attention layers of the image and text features. The image features are generated from an image encoder.
 <Tip>
-Learn how to load an IP-Adapter checkpoint and image in the [IP-Adapter](../../using-diffusers/loading_adapters#ip-adapter) loading guide.
+Learn how to load an IP-Adapter checkpoint and image in the IP-Adapter [loading](../../using-diffusers/loading_adapters#ip-adapter) guide, and you can see how to use it in the [usage](../../using-diffusers/ip_adapter) guide.
 </Tip>
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -408,6 +408,91 @@ Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers)
 </Tip>
 ## Using AnimateLCM
 [AnimateLCM](https://animatelcm.github.io/) is a motion module checkpoint and an [LCM LoRA](https://huggingface.co/docs/diffusers/using-diffusers/inference_with_lcm_lora) that have been created using a consistency learning strategy that decouples the distillation of the image generation priors and the motion generation priors.
 ```python
 import torch
 from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter
 from diffusers.utils import export_to_gif
 adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM")
 pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter)
 pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
 pipe.load_lora_weights("wangfuyun/AnimateLCM", weight_name="sd15_lora_beta.safetensors", adapter_name="lcm-lora")
 pipe.enable_vae_slicing()
 pipe.enable_model_cpu_offload()
 output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=1.5,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
 )
 frames = output.frames[0]
 export_to_gif(frames, "animatelcm.gif")
 ```
 <table>
    <tr>
        <td><center>
        A space rocket, 4K.
        <br>
        <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/animatelcm-output.gif"
            alt="masterpiece, bestquality, sunset"
            style="width: 300px;" />
        </center></td>
    </tr>
 </table>
 AnimateLCM is also compatible with existing [Motion LoRAs](https://huggingface.co/collections/dn6/animatediff-motion-loras-654cb8ad732b9e3cf4d3c17e).
 ```python
 import torch
 from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter
 from diffusers.utils import export_to_gif
 adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM")
 pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter)
 pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
 pipe.load_lora_weights("wangfuyun/AnimateLCM", weight_name="sd15_lora_beta.safetensors", adapter_name="lcm-lora")
 pipe.load_lora_weights("guoyww/animatediff-motion-lora-tilt-up", adapter_name="tilt-up")
 pipe.set_adapters(["lcm-lora", "tilt-up"], [1.0, 0.8])
 pipe.enable_vae_slicing()
 pipe.enable_model_cpu_offload()
 output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=1.5,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
 )
 frames = output.frames[0]
 export_to_gif(frames, "animatelcm-motion-lora.gif")
 ```
 <table>
    <tr>
        <td><center>
        A space rocket, 4K.
        <br>
        <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/animatelcm-motion-lora.gif"
            alt="masterpiece, bestquality, sunset"
            style="width: 300px;" />
        </center></td>
    </tr>
 </table>
 ## AnimateDiffPipeline
 [[autodoc]] AnimateDiffPipeline
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The GLIGEN Authors and The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The GLIGEN Authors and The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The Intel Labs Team Authors and HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The Intel Labs Team Authors and HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -0,0 +1,43 @@
 <!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
 an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 -->
 # Stable Video Diffusion
 Stable Video Diffusion was proposed in [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://hf.co/papers/2311.15127) by Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, Robin Rombach.
 The abstract from the paper is:
 *We present Stable Video Diffusion - a latent video diffusion model for high-resolution, state-of-the-art text-to-video and image-to-video generation. Recently, latent diffusion models trained for 2D image synthesis have been turned into generative video models by inserting temporal layers and finetuning them on small, high-quality video datasets. However, training methods in the literature vary widely, and the field has yet to agree on a unified strategy for curating video data. In this paper, we identify and evaluate three different stages for successful training of video LDMs: text-to-image pretraining, video pretraining, and high-quality video finetuning. Furthermore, we demonstrate the necessity of a well-curated pretraining dataset for generating high-quality videos and present a systematic curation process to train a strong base model, including captioning and filtering strategies. We then explore the impact of finetuning our base model on high-quality data and train a text-to-video model that is competitive with closed-source video generation. We also show that our base model provides a powerful motion representation for downstream tasks such as image-to-video generation and adaptability to camera motion-specific LoRA modules. Finally, we demonstrate that our model provides a strong multi-view 3D-prior and can serve as a base to finetune a multi-view diffusion model that jointly generates multiple views of objects in a feedforward fashion, outperforming image-based methods at a fraction of their compute budget. We release code and model weights at this https URL.*
 <Tip>
 To learn how to use Stable Video Diffusion, take a look at the [Stable Video Diffusion](../../../using-diffusers/svd) guide.
 <br>
 Check out the [Stability AI](https://huggingface.co/stabilityai) Hub organization for the [base](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid) and [extended frame](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt) checkpoints!
 </Tip>
 ## Tips
 Video generation is memory-intensive and one way to reduce your memory usage is to set `enable_forward_chunking` on the pipeline's UNet so you don't run the entire feedforward layer at once. Breaking it up into chunks in a loop is more efficient.
 Check out the [Text or image-to-video](text-img2vid) guide for more details about how certain parameters can affect video generation and how to optimize inference by reducing memory usage.
 ## StableVideoDiffusionPipeline
 [[autodoc]] StableVideoDiffusionPipeline
 ## StableVideoDiffusionPipelineOutput
 [[autodoc]] pipelines.stable_video_diffusion.StableVideoDiffusionPipelineOutput
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -167,6 +167,12 @@ Here are some sample outputs:
    </tr>
 </table>
 ## Tips
 Video generation is memory-intensive and one way to reduce your memory usage is to set `enable_forward_chunking` on the pipeline's UNet so you don't run the entire feedforward layer at once. Breaking it up into chunks in a loop is more efficient.
 Check out the [Text or image-to-video](text-img2vid) guide for more details about how certain parameters can affect video generation and how to optimize inference by reducing memory usage.
 <Tip>
 Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers) to learn how to explore the tradeoff between scheduler speed and quality, and see the [reuse components across pipelines](../../using-diffusers/loading#reuse-components-across-pipelines) section to learn how to efficiently load the same components into multiple pipelines.
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -1,4 +1,4 @@
-<!--Copyright 2023 The HuggingFace Team. All rights reserved.
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dhruv Nair	c32abb213f	update	2024-02-19 16:59:11 +00:00
Dhruv Nair	a17d8757ca	update	2024-02-19 16:13:45 +00:00
Dhruv Nair	b544b408a6	update	2024-02-19 15:13:54 +00:00
Dhruv Nair	41d8e074ee	update	2024-02-19 08:40:48 +00:00
Dhruv Nair	d2fc5ebb95	[Refactor] FreeInit for AnimateDiff based pipelines (#6874 ) * update * update * update * update * update * update * update * update * update * update	2024-02-19 11:11:42 +05:30
YiYi Xu	779eef95b4	[from_single_file] pass `torch_dtype` to `set_module_tensor_to_device` (#6994 ) fix Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-19 10:09:19 +05:30
Dhruv Nair	d5b8d1ca04	Fix Pixart Slow Tests (#6962 ) * update * update	2024-02-19 09:50:18 +05:30
Fabio Rigano	eba7e7a6d7	IP-Adapter attention masking (#6847 ) * Add attention masking to attn processors * Update tensor conversion --------- Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-18 18:06:14 -10:00
Sayak Paul	31de879fb4	[IP2P] Make text encoder truly optional in InstructPi2Pix (#6995 ) * make text encoder component truly optional. * more fixes * Apply suggestions from code review Co-authored-by: YiYi Xu <yixu310@gmail.com> --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-18 20:21:22 +05:30
nbpppp	07349c25fe	Fix deprecation warning for torch.utils._pytree._register_pytree_node in PyTorch 2.2 (#7008 ) Fixed deprecation warning for torch.utils._pytree._register_pytree_node in PyTorch 2.2 Co-authored-by: Yinghua <yzho0423@uni.sydney.edu.au>	2024-02-18 20:11:33 +05:30
YiYi Xu	8974c50bff	[SVD] fix a bug when passing image as tensor (#6999 ) * fix * update docstring --------- Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-17 23:10:31 -10:00
Mikhail Koltakov	c18058b405	Fixed typos in dosctrings of __init__() and in forward() of Unet3DConditionModel (#6663 ) * Fixed typos in __init__ and in forward of Unet3DConditionModel * Resolving conflicts --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-17 21:56:16 -10:00
Thomas Lips	2938d5a672	Add documentation for `strength` parameter in `Controlnet_img2img` pipelines (#6951 ) copy docstring for `strength` from stablediffusion img2img pipeline to controlnet img2img pipelines	2024-02-17 21:47:02 -10:00
Sayak Paul	d4ade821cd	start depcrecation cycle for lora_attention_proc 👋 (#7007 )	2024-02-18 13:11:30 +05:30
Steven Liu	3a7e481611	[docs] Video generation (#6701 ) * first draft * fix path * fix path * i2vgen-xl * review * modelscopet2v * feedback	2024-02-16 16:35:37 -08:00
Steven Liu	d649d6c6f3	[docs] Fix callout (#6998 ) Update ip_adapter.md	2024-02-16 10:37:12 -10:00
Bhavay Malhotra	777063e1bf	Update textual_inversion.py (#6952 ) * Update textual_inversion.py * Apply suggestions from code review * Update textual_inversion.py * Update textual_inversion.py * Update textual_inversion.py * Update textual_inversion.py * Update examples/textual_inversion/textual_inversion.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * Update textual_inversion.py * styling --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-16 15:39:51 +05:30
Stephen	104afbce84	Standardize model card for textual inversion sdxl (#6963 ) * standardize model card * fix tags * correct import styling and update tags * run make style and make quality --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-16 14:27:11 +05:30
co63oc	c0f5346a20	Fix procecss process (#6591 ) * Fix words * Fix --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-15 19:06:33 -10:00
Sayak Paul	087daee2f0	add: peft to the benchmark workflow (#6989 )	2024-02-16 09:29:10 +05:30
Paakhhi	7e164d98a8	Fix diffusers import prompt2prompt (#6927 ) * Bugfix: correct import for diffusers * Fix: Prompt2Prompt example * Format style --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-15 15:30:16 -10:00
Sayak Paul	e6d1728e0a	[IP Adapters] feat: allow low_cpu_mem_usage in ip adapter loading (#6946 ) * feat: allow low_cpu_mem_usage in ip adapter loading * reduce the number of device placements. * documentation. * throw low_cpu_mem_usage warning only once from the main entry point.	2024-02-15 15:37:17 +05:30
Linoy Tsaban	8f2c7b4df0	[advanced sdxl lora script] - fix #6967 bug when using prior preservation loss (#6968 ) * fix bug in micro-conditioning of class images * fix bug in micro-conditioning of class images * style	2024-02-15 12:20:05 +05:30
YiYi Xu	2e387dad5f	fix IPAdapter unload_ip_adapter test (#6972 ) add Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-14 20:42:40 -10:00
Steven Liu	9efe1e52c3	[docs] IP-Adapter (#6897 ) * use cases * first draft * fix image links * lcm-lora * feedback * review * feedback * feedback	2024-02-14 13:23:37 -08:00
Sayak Paul	37b09517b9	fix: controlnet inpaint single file. (#6975 )	2024-02-14 19:04:57 +05:30
Sayak Paul	4343ce2c8e	[Core] Harmonize single file ckpt model loading (#6971 ) * use load_model_into_meta in single file utils * propagate to autoencoder and controlnet. * correct class name access behaviour. * remove torch_dtype from load_model_into_meta; seems unncessary * remove incorrect kwarg * style to avoid extra unnecessary line breaks	2024-02-14 10:49:06 +05:30
Younes Belkada	0ca7b68198	[`PEFT` / `docs`] Add a note about torch.compile (#6864 ) * Update using_peft_for_inference.md * add more explanation	2024-02-14 02:29:29 +01:00
Dhruv Nair	3cf4f9c735	Allow passing `config_file` argument to ControlNetModel when using `from_single_file` (#6959 ) * update * update * update	2024-02-13 18:54:53 +05:30
Dhruv Nair	40dd9cb2bd	Move SDXL T2I Adapter lora test into PEFT workflow (#6965 ) update	2024-02-13 17:08:53 +05:30
Dhruv Nair	30bcda7de6	Fix flaky IP Adapter test (#6960 ) update	2024-02-13 17:07:39 +05:30
YiYi Xu	9ea62d119a	[DPMSolverSinglestepScheduler] correct `get_order_list` for `solver_order=2`and `lower_order_final=True` (#6953 ) * add * change default --------- Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-12 22:10:33 -10:00
Dhruv Nair	a326d61118	Fix configuring VAE from single file mixin (#6950 ) * update	2024-02-12 22:10:05 -10:00
Alex Umnov	e7696e20f9	Updated lora inference instructions (#6913 ) * Updated lora inference instructions * Update examples/dreambooth/README.md Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * Update README.md * Update README.md --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-13 09:35:20 +05:30
Piyush Thakur	4b89aeffe1	[Type annotations] fixed in save_model_card (#6948 ) fixed type annotations Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-13 08:56:45 +05:30
Steven Liu	0a1daadef8	[docs] Community pipelines (#6929 ) fix	2024-02-12 10:38:13 -08:00
Sayak Paul	371f765908	[Diffusers -> Original SD conversion] fix things (#6933 ) * fix: bias loading bug * fixes for SDXL * apply changes to the conversion script to match single_file_utils.py * do transpose to match the single file loading logic.	2024-02-12 17:30:22 +05:30
Piyush Thakur	75aee39eac	[Model Card] standardize T2I Adapter Sdxl model card (#6947 ) standardize model card template t21-adapter-sdxl	2024-02-12 16:43:20 +05:30
Dhruv Nair	215e6804d3	Unpin torch versions in CI (#6945 ) * update * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-12 16:01:05 +05:30
Disty0	9254d1f39a	Pass device to enable_model_cpu_offload in maybe_free_model_hooks (#6937 )	2024-02-12 13:42:32 +05:30
Piyush Thakur	e1bdcc7af3	[Model Card] standardize T2I Sdxl Lora model card (#6944 ) * standardize model card template t2i-lora-sdxl * type annotations	2024-02-12 11:45:40 +05:30
Dhruv Nair	84905ca728	Update PixArt Alpha test module to match src module (#6943 ) update	2024-02-12 11:01:33 +05:30
Piyush Thakur	6f336650c3	[Model Card] standardize T2I Sdxl model card (#6942 ) standardize model card template t2i-sdxl	2024-02-12 10:01:20 +05:30
Piyush Thakur	06a042cd0e	[Model Card] standardize T2I Lora model card (#6940 ) standardize model card t2i-lora	2024-02-12 10:01:13 +05:30
Piyush Thakur	8772496586	[Model Card] standardize T2I model card (#6939 ) * standardize model card * fix base_model	2024-02-12 10:00:41 +05:30
dg845	35fd84be27	Replace hardcoded values in SchedulerCommonTest with properties (#5479 ) --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>	2024-02-10 21:34:20 -10:00
YiYi Xu	f2756253e6	Fix a bug in `AutoPipeline.from_pipe` when switching pipeline with optional components (#6820 ) * fix * add tests --------- Co-authored-by: yiyixuxu <yixu310@gmail,com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-09 21:56:52 -10:00
YiYi Xu	0071478d9e	allow attention processors to have different signatures (#6915 ) add Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-09 21:56:19 -10:00
Sayak Paul	7c8cab313e	post release 0.26.2 (#6885 ) * post release * style * Empty-Commit --------- Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>	2024-02-09 07:36:38 -10:00
Sayak Paul	ca9ed5e8d1	[LoRA] deprecate certain lora methods from the old backend. (#6889 ) * deprecate certain lora methods from the old backend. * uncomment necessary things. * safe remove old lora backend 👋	2024-02-09 17:14:32 +01:00
Bingxin Ke	98b6bee1a1	[Community Pipeline][Bug Fix] marigold_depth_estimation: input image value range (#6787 ) [FIX] IMPORTANT: rgb normalization	2024-02-09 16:55:37 +01:00
Dhruv Nair	ab7113487c	More IPAdapter test fixes (#6888 ) update	2024-02-09 13:54:58 +05:30
camaro	59c307f1d5	Standardize model card for Controlnet (#6910 ) * controlnet * style --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-09 12:29:08 +05:30
Sayak Paul	159885adc6	correct hub_token exposition behaviour (thanks to @bghira). (#6918 )	2024-02-08 18:38:27 -10:00
Aryan	7337eea59b	[refactor] unnecessary lines in decode_latents in video pipelines (#6682 ) * refactor decode latents in video pipelines * make fix-copies	2024-02-08 17:10:52 -10:00
camaro	f07899a57c	Standardize model card for Controlnet SDXL (#6908 ) controlnet-sdxl	2024-02-09 07:53:39 +05:30
camaro	a83cc0c0bc	Standardize model card for Controlnet flax (#6909 ) * controlnet-flax * style --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-09 07:52:56 +05:30
C Q	db5194a45d	Fix Compatibility Issues in stable_diffusion_xl_reference.py (#6251 ) * Fix Compatibility Issues in stable_diffusion_xl_reference.py --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-08 10:52:59 -10:00
shaoxiaowang	e6c9c2513f	fix examples/community/pipeline_stable_diffusion_xl_instantid.py (#6759 ) Co-authored-by: wangshaoxiao <wangshaoxiao@xiaomi.com>	2024-02-08 10:09:40 -10:00
Kyunghwan Kim	d643b6691f	Add vae tiling and slicing in img2img and inpaint (#6871 ) * Add vae tiling in img2img and inpaint * Add vae tiling not slicing	2024-02-08 09:50:00 -10:00
Michael	f5c9be3a0a	Remove <cat-toy> validation prompt from examples/textual_inversion/textual_inversion_sdxl.py (#6877 ) Remove <cat-toy> validation prompt from textual_inversion_sdxl.py The `<cat-toy>` validation prompt is a default choice for the example task in the README. But no other part of `textual_inversion_sdxl.py` references the cat toy and `textual_inversion.py` has a default validation prompt of `None` as well. So bring `textual_inversion_sdxl.py` in line with `textual_inversion.py` and change default validation prompt to `None`	2024-02-08 09:46:06 -10:00
Laisky.Cai	1824d0050e	fix: load_image should support PIL Image (#6904 ) Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-08 08:25:37 -10:00
Sayak Paul	30e5e81d58	change to 2024 in the license (#6902 ) change to 2024	2024-02-08 08:19:31 -10:00
Masamune Ishihara	8de78001df	Add fps argument to export_to_gif function. (#6786 )	2024-02-08 21:59:51 +05:30
Patryk Bartkowiak	3ac2357794	changed positional parameters to named parameters like in docs (#6905 ) Co-authored-by: Patryk Bartkowiak <patryk.bartkowiak@tcl.com> Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>	2024-02-08 21:39:03 +05:30
Ehsan Akhgari	17808a091e	Fix bug when converting checkpoint to diffusers format (#6900 ) This fixes #6899.	2024-02-08 18:52:11 +05:30
Sayak Paul	491a933a1b	[I2VGenXL] `attention_head_dim` in the UNet (#6872 ) * attention_head_dim * debug * print more info * correct num_attention_heads behaviour * down_block_num_attention_heads -> num_attention_heads. * correct the image link in doc. * add: deprecation for num_attention_head * fix: test argument to use attention_head_dim * more fixes. * quality * address comments. * remove depcrecation.	2024-02-08 12:30:14 +05:30
Sayak Paul	aa82df52e7	[IP Adapters] introduce `ip_adapter_image_embeds` in the SD pipeline call (#6868 ) * add: support for passing ip adapter image embeddings * debugging * make feature_extractor unloading conditioned on safety_checker * better condition * type annotation * index to look into value slices * more debugging * debugging * serialize embeddings dict * better conditioning * remove unnecessary prints. * Update src/diffusers/loaders/ip_adapter.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * make fix-copies and styling. * styling and further copy fixing. * fix: check_inputs call in controlnet sdxl img2img pipeline --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>	2024-02-08 11:10:10 +05:30
Srimanth Agastyaraju	a11b0f83b7	Fix: training resume from fp16 for SDXL Consistency Distillation (#6840 ) * Fix: training resume from fp16 for lcm distill lora sdxl * Fix coding quality - run linter * Fix 1 - shift mixed precision cast before optimizer * Fix 2 - State dict errors by removing load_lora_into_unet * Update train_lcm_distill_lora_sdxl.py - Revert default cache dir to None --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>	2024-02-08 11:09:29 +05:30
Sayak Paul	1835510524	Remove `torch_dtype` in to() to end deprecation (#6886 ) * remove torch_dtype from to() * remove torch_dtype from usage scripts. * remove old lora backend * Revert "remove old lora backend" This reverts commit `adcddf6ba4`.	2024-02-08 09:38:57 +05:30
camaro	4a3d52850b	fix: keyword argument mismatch (#6895 )	2024-02-08 09:37:56 +05:30
YiYi Xu	97d004b9b4	[ip-adapter] make sure length of `scale` is same as number of ip-adapters when using `set_ip_adapter_scale` (#6884 ) add Co-authored-by: yiyixuxu <yixu310@gmail,com>	2024-02-07 10:13:12 -10:00
Sayak Paul	76696dca55	[Model Card] standardize dreambooth model card (#6729 ) * feat: standarize model card creation for dreambooth training. * correct 'inference * remove comments. * take component out of kwargs * style * add: card template to have a leaner description. * widget support. * propagate changes to train_dreambooth_lora * propagate changes to custom diffusion * make widget properly type-annotated	2024-02-07 15:07:11 +05:30
Félix Sanz	17612de451	fix: typo in callback function name and property (#6834 ) * fix: callback function name is incorrect On this tutorial there is a function defined and then used inside `callback_on_step_end` argument, but the name was not correct (mismatch) * fix: typo in num_timestep (correct is num_timesteps) fixed property name	2024-02-06 12:05:40 -08:00
Dhruv Nair	994360f7a5	Fix last IP Adapter test (#6875 ) update	2024-02-06 08:53:40 -10:00