A custom pixel art model created with Alpaca.

With Alpaca, you can create and train new models on your images. This is super useful if you are consistently generating in a specific style, want Alpaca to learn about your own art style, or simply want to explore a style based off of a handful of sample references.

For example, we can imagine that we are creating a pixel art game for which we have already created a few concept images. Wouldn’t it be nice if we could use Alpaca in a way that matched the style of those concept images? This is exactly what custom models enable! Once you have created a custom model, you’ll be able to use it in the plugin in place of the default Alpaca v3 model.

​

Quickstart

We’ll walk you through the steps to create your first model with Alpaca and provide you with an example dataset to get you started.

You can either follow the video below, or keep on reading!

In this example we are going to train a custom model to generate in the style of our custom pixel art dataset. You can view that dataset, and download it to try for yourself here: Pixel Art Dataset.

​

Creating a new model

First, head to the Models section of your dashboard, and click on New Custom Model

Pick a name for your model (e.g. pixel-art) so you can easily identify it, and a Keyword to use when writing your prompt. You can use the default keyword of stylesks.

​

Uploading Images

Next, it’s time to select the images your custom model is going to be trained on. You can use the dataset provided above, or use your own data. You can use anywhere between 15 and 100 images for your model, the more images you add the better your model will be (in general).

Here are a few important recommendations to get the best result:

• All images must have a coherent style
• If the images are too similar (for example you have 6 pictures of the same cat in your data), the model might learn “too much” and only be able to generate images very close to the duplicated data
• The model is trained on square images, if the images you upload are not square, they will automatically be cropped, but you will get better results if you crop them yourself to make sure the subject is in frame. An easy and fast way to correctly crop all your images is to use the free online tool birme.net

Once you have selected your images, click the Next button.

​

Choosing Evaluation Prompts

At the end of the training you will be offered several trained models to choose from. In order to compare their respective quality and identify the best one, you will need to specify a handful of prompts that will be used to generate preview images. For example, with our pixel art project we might write an evaluation prompt like the following:

enigmatic, arcane spellcaster in flowing robes, magical accessories. Wise, intense demeanor, magical runes or sigils, staff, wand, or orb, pointed hat or hood.

You can specify up to 6 different prompts, and a minimum of 1.

If you don’t feel like writing prompts, you can also choose a category from Prompt Examples that matches your data and 6 prompts will be entered for you.

​

Advanced Settings and Launching the Training

Once you click Next, you will be offered the option to choose the recommended training settings, or to tweak them as you want. This is an advanced feature to give you more control over the result, but for now, let’s just keep the recommended settings and click on Launch Training

​

Choosing the Best Model

Once the training is complete, you will be able to choose wich model you prefer from a number of options, by clicking on Model Selection.

Model Selection Screen

The model selection, will present you with multiple options, where for each option you will see an example of what a generation with option would look like using the Evaluation Prompt we entered at the previous step.

It’s important to consider two main aspects when comparing options:

• Which one is following the description of the prompt the best?
• Which one respects the most the style of the images you uploaded the best?

You can click on a specific image, and you will be able to see it in a larger view, making comparing options easier.

Once you select an option, if there are more options to compare you will be asked to choose again, comparing the model you just choosed and a new one. After all the models have been compared you will be asked to confirm your choice, click “Confirm Selection”.

Once confirmed, it will take an additional ~2 minutes to get your model ready, your model will be ready when you see the green indicator on the model card.

Model is Ready to Use

You can now go back to Alpaca’s plugin and start using your model! In the Model dropdown, you should now see your custom model, select it.

Using a Custom Model

Now we can write our prompt to create our first image with our custom models. It differs from writing a prompt for a general model (e.g. Alpaca v3) in two ways:

• You need to include your Keyword somewhere in the prompt to trigger it.
• You shouldn’t need to specify the style in your prompt anymore, since your custom model should have learned it. (for example things like 16 bit pixel art, best quality, fantasy)

Finally click on Generate and enjoy the results!