Does this work with SDXL?

[tiimgreen]

I tried a quick example by changing the the dimensions to 1024 and the models to be the SDXL base models instead of SDv1.5

pipe = Pipeline.from_pretrained(
    "diffusers/stable-diffusion-xl-1.0-inpainting-0.1",
    torch_dtype=weight_dtype
)
pipe.tokenizer = TokenizerWrapper(
    from_pretrained="stabilityai/stable-diffusion-xl-base-1.0",
    subfolder="tokenizer",
    revision=None
)
add_tokens(
    tokenizer=pipe.tokenizer,
    text_encoder=pipe.text_encoder,
    placeholder_tokens=['P_ctxt', 'P_shape', 'P_obj'],
    initialize_tokens=['a', 'a', 'a'],
    num_vectors_per_token=10
)

But I get a bunch of errors like this:

"\u0002\u0000\u0000\u0000\u0000\u0000\u0000�\tsize mismatch for up_blocks.2.resnets.2.time_emb_proj.bias: copying a param with shape torch.Size([640]) from checkpoint, the shape in current model is torch.Size([320])."