Waifu-Inpaint-XL

Waifu-Inpaint-XL is an SDXL-based model specifically designed for inpainting anime-themed images.

Overview

Waifu-Inpaint-XL is initialized from WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction, with its input channels expanded from 4 to 9. Among these, 1 additional channel is dedicated to the mask, and 4 extra channels are used for masked latents.

Model Details

• Developed by: ShinoharaHare
• Model type: Diffusion based text-to-image/image-to-image generative model for inpainting
• Language(s) (NLP): English
• License: CreativeML Open RAIL++-M
• Finetuned from: WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction

🧨 Diffusers

Image Mask Inpainted

import torch
from diffusers import StableDiffusionXLInpaintPipeline
from diffusers.utils import load_image

pipeline = StableDiffusionXLInpaintPipeline.from_pretrained(
  'ShinoharaHare/Waifu-Inpaint-XL',
  torch_dtype=torch.half
)
pipeline.to('cuda')

image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/tPo5oPJQpxWamM-tGIYqj.png')
mask_image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/QpmzmgROUM0eP53Cxx2Ih.png', lambda x: x.convert('L'))

inpainted_image = pipeline(
  prompt='blue eyes, holding red spider lily in hand',
  image=image,
  mask_image=mask_image,
  num_inference_steps=28,
  guidance_scale=5.0,
  height=image.height,
  width=image.width,
  generator=torch.Generator(pipeline.device).manual_seed(5)
).images[0]

inpainted_image.show()