Skip to main content

How to Run FLUX AI Image Generator with ComfyUI: Complete Setup Guide

· 4 min read
Thamim
R&D @JarvisLabs.ai
Step-by-step guide to set up and run FLUX.1 Schnell for AI image generation using ComfyUI on cloud GPUs. Includes workflows, LoRA integration, and practical examples.

comfyui_flux

Introduction to FLUX​

The FLUX is a new AI model for text-to-image generation. It gained sudden popularity because of its compatibility in generating high-quality images, precise text descriptions, perfect hand poses, and finger alignments. The FLUX model uses an advanced technique called "Flow Matching" with parallel diffusion transformer blocks, which is why it achieved high-quality image generation in a few steps. The Black Forest team released three variants of FLUX.1

  1. FLUX.1 [pro]: This is the most advanced model among the FLUX family, which ranks at the top in the ELO scoring, around 1050 on the chart. We can only access this model via API call.

  2. FLUX.1 [dev]: This model is available for non-commercial purposes. It ranks second in the ELO scoring, around 1040 on the chart. If you are wondering what ELO scoring is, it's a one model vs one model rating system where random users give ratings to both models.

  3. FLUX.1 [schnell]: Free for all under apache-2.0 license. This is the most powerful open-source model and crosses the ELO scoring of around 1030. Perfect for AI content creators who are looking for a powerful model to let their creative juices flow.

All the weights are available on Huggingface.

If you do want to use comfyUI. Use Python code instead, check this blog post for setting up and running flux on Jarvislabs.

If you are new to Jarvislabs​

Follow the below steps to set up your Jarvislabs account.

  • Create a new account here.
  • Setup billing here and add funds to your Jarvislabs.ai wallet.

Launch comfyUI​

Launching ComfyUI on JarvisLabs is pretty simple. Click here and then click "Run on Cloud". Choose the GPU type and storage, then launch the instance. Simple, right?

At JarvisLabs, we provide 50+ custom nodes by default. After launching the instance, it will take one or two minutes to load all the custom nodes. Now click the comfyUI endpoint from the instance page and you will see the comfyUI is running

The FLUX Schnell model will load by default with JarvisLabs, along with its CLIP vision model. All you need to do is download the workflow JSON and run the FLUX model.

If you're facing any error after launching the instance. Check out the FAQs

Download the workflow and trial run​

We can download the JSON from this link. Load this JSON into the comfyUI and you don't need to change anything in the workflow. Just click the queue prompt and yay! you generate your first image using the Flux.

download json

Now type your prompt in the clip encoder. You can add weights and a series of prompts using commas.

type prompt

If you are already familiar with image-generating tools then you will notice that the flux model doesn't require negative prompting because its clip vision model is so powerful.

We can also create memes with Flux because it handles text descriptions in images well. I tried to recreate this meme template, and after several attempts, I got a good result

base meme

Here is what I got

type prompt

prompt: A close-up of a tiny, round hamster with big, shiny black eyes and soft light gray fur. The hamster is the focus of the image, with its adorable, innocent expression highlighted against a simple, plain background. Text saying at the top of the image "me when I" and text at the bottom of the image saying "I lost my fav toy :("

I would recommend you use 24 GB vRAM or more because we provided the Schnell fp16 version which is very big and complex.

Extra resources​

Controlnet and LoRA for Flux dev​

Download the controlnet weights for flux dev. Click this link.

Download the ControlNet flux workflows

Download the flux dev loRA

Conclusion​

The Flux model is powerful for every AI artist and content creator. At Jarvislabs, we made everything super simple for you so you can focus on the creative part, while we take care of the tech you need. I hope this blog helps you understand the potential use cases of the Flux model and how you can leverage cloud GPUs without investing in buying expensive GPUs.

If you're facing any issues while following any of the above steps, please feel free to ping me or my team in the chat. We will be happy to assist you.

See you on the next blog :)