ArtifyAI: Text-to-Image Generation

ArtifyAI is an innovative project that combines the power of Natural Language Processing (NLP) with image generation. This repository implements a pipeline using the T5 Transformer model for text summarization or generation and the Stable Diffusion model for creating images based on the generated text.

Overview

ArtifyAI takes a text input, processes it through a T5 model, and then uses the processed output to generate an image using Stable Diffusion. This allows for seamless conversion of text descriptions into AI-generated images.

Features

Text Processing: Uses T5 (Text-to-Text Transfer Transformer) for summarizing or generating text from user inputs.
Image Generation: Uses Stable Diffusion to create high-quality images from the text processed by the T5 model.
Combined Pipeline: A simple Python function combines these models to produce stunning images from text.

Installation

Prerequisites

To run this project locally, ensure you have the following:

Python 3.7+
CUDA-compatible GPU (for faster performance with Stable Diffusion)
Hugging Face Transformers library
Diffusers for Stable Diffusion
PyTorch with CUDA support (optional for faster image generation)

Step-by-Step Setup

Clone the Repository:

git clone https://github.com/your-username/ArtifyAI.git
cd ArtifyAI

Install Dependencies: It's best to use a virtual environment to manage dependencies.
```
pip install torch transformers diffusers
```

Download the Pretrained Models: You'll need to load the models locally from Hugging Face. You can either download them using the code inside the notebook or by modifying it as follows:

from transformers import T5Tokenizer, T5ForConditionalGeneration
from diffusers import StableDiffusionPipeline
import torch

# Load models
t5_tokenizer = T5Tokenizer.from_pretrained("t5-small")
t5_model = T5ForConditionalGeneration.from_pretrained("t5-small")
ArtifyAI_model = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", torch_dtype=torch.float16)

# Set model to GPU (if available)
ArtifyAI_model = ArtifyAI_model.to("cuda" if torch.cuda.is_available() else "cpu")

Run the Pipeline: A sample pipeline is included in pipeline.py. You can run it using:
```
python pipeline.py
```

Text to Image Generation: You can generate images from text input using the following function:

def t5_to_image_pipeline(input_text):
    # T5 model processing
    t5_inputs = t5_tokenizer.encode(input_text, return_tensors='pt', truncation=True)
    summary_ids = t5_model.generate(t5_inputs, max_length=50, num_beams=5, early_stopping=True)
    generated_text = t5_tokenizer.decode(summary_ids[0], skip_special_tokens=True)

    # Generate image from text using Stable Diffusion
    image = ArtifyAI_model(generated_text).images[0]
    return image

Usage

Run the Jupyter Notebook: You can open ArtifyAI_v1_1.ipynb in Jupyter to run the code interactively.
Save and Load Models: You can modify the notebook to save your models to Google Drive or a local directory.
Custom Inputs: Modify the text input in the pipeline to generate customized images based on different descriptions.

Example

Here's an example of generating an image from text:

image = t5_to_image_pipeline("A futuristic city skyline at sunset")
image.show()

## For Non-Technical Users

Even if you are new to AI, you can use ArtifyAI by following these simple steps:

1. **Install Python**: Download and install Python 3.7+ from the [official Python website](https://www.python.org/downloads/).
   
2. **Install Dependencies**: Follow the steps in the Installation section to install necessary packages using the `pip` command.
   
3. **Run the Code**: You can run the project directly by using the provided code snippets. If you face any issues, you can refer to [Hugging Face](https://huggingface.co/) or [PyTorch](https://pytorch.org/) for troubleshooting.