
Released on October 29th, the 3.5 update included multiple models: Stable Diffusion 3.5 Large, Stable Diffusion 3.5 Large Turbo, and Stable Diffusion 3.5 Medium. Each of these variants can be customized to meet individual hardware needs.

# A Look at the Stable Diffusion 3.5 Variants

**[Stable Diffusion 3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large)**

This base model, features an impressive 8.1 billion parameters and stands out as the most powerful option within the Stable Diffusion family. It prioritizes superior quality and prompt adherence, making it particularly suitable for various professional applications. 3.5 Large boasts a resolution capability of 1 megapixel, it delivers high-quality outputs that meet the demands of various industries, from graphic design to digital content creation. Whether you're working on detailed imagery or need reliable performance for professional projects, this model is engineered to meet and then exceed user expectations.

**Stability AI Highlight**

Analysis shows that Stable Diffusion 3.5 Large excels in prompt adherence and matches larger models in image quality.

**[Stable Diffusion 3.5 Large Turbo](https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo)**

Stable Diffusion 3.5 Large Turbo is designed to create high-quality images with remarkable adherence to prompts, achieving this in just four streamlined steps. This efficiency not only enhances the user experience but also allows for a considerably faster image generation process compared to the original Stable Diffusion 3.5 Large model. Users can expect quicker results without compromising the quality or precision of the images produced, making this version an excellent choice for those seeking both speed and visual fidelity.

**Stability AI Highlight**

Stable Diffusion 3.5 Large Turbo provides some of the fastest inference times for its size, maintaining strong image quality and prompt adherence, even when compared to non-distilled models of similar size.

**[Stable Diffusion 3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium)**

This advanced model boasts an impressive 2.5 billion parameters and incorporates an enhanced MMDiT-X architecture along with cutting-edge training methods. 3.5 Medium is specifically designed to function effortlessly on standard hardware, making it accessible to a wide range of users. Its design philosophy emphasizes a blend of high-quality output and straightforward customization, allowing users to tailor the model's performance to their specific needs.

The model is also capable of generating high-resolution images, with the ability to produce outputs ranging from 0.25 megapixels to 2 megapixels. This versatility in resolution enables users to create visuals suitable for various applications, whether for casual use or more demanding projects. Overall, the model provides an efficient and user-friendly solution for anyone seeking to generate quality images without specialized equipment.

**Stability AI Highlight**

Stable Diffusion 3.5 Medium surpasses other medium-sized models, providing a harmonious blend of prompt adherence and image quality, establishing it as a preferred option for efficient, high-quality performance.

## A Note on Community License

Like earlier versions, Stable Diffusion 3.5 operates under the permissive Stability AI Community License. According to Stability's website, this license permits the following uses:

- **Free for non-commercial use:** Individuals and organizations can use the model free of charge for non-commercial use, including scientific research.
- **Free for commercial use (up to $1M in annual revenue):** Startups, small to medium-sized businesses, and creators can use the model for commercial purposes at no cost as long as their total annual revenue is less than $1M.
- **Ownership of outputs:** Retain ownership of the media generated without restrictive licensing implications.

## A Final Note

Stable Diffusion 3.5 represents a significant advancement, offering a unique combination of prompt adherence and image quality that sets it apart from its competitors. Its ability to deliver high-quality results efficiently makes it an invaluable tool for creators and professionals alike. As the landscape of digital generation continues to evolve, this model positions itself as a reliable and effective choice for achieving outstanding visual outputs.

Excited about new developments in AI and where it is headed next? Stay updated with the latest news and developments in AI and technology through our [blog](https://vast.ai/blog) at [Vast.AI.](http://vast.ai) Interested in running Stable Diffusion on Vast.ai? Check out our [documentation](https://docs.vast.ai/stable-diffusion) for a full walkthrough.


Unleash the power of Stable Diffusion 3.5 on Vast.ai. Explore three models - Large, Large Turbo, and Medium - crafted for exceptional image generation, tailored to your hardware capacity. 

Stable Diffusion 3.5: Stability's Most Powerful AI Model Yet


# Turn Any PDF into an AI-Generated Podcast with Notebook Llama

AI has now gone multimodal: instead of just sending chat messages to a chatbot, there are awesome use cases involving audio, videos and images. One amazing product like this is [NotebookLM](https://notebooklm.google/) from Google which has taken the AI community by storm. It takes in YouTube videos, PDF’s or other documents, and derives insights and even creates a podcast from the notes and original material.

But Meta’s Llama team came out with Notebook Llama, an open source version of this flow, which lets users run their own models to create podcasts on their own material. We will show you how to run it with Vast. In this tutorial, we will show you how to put the world of AI researchers in the palm of your hands for the materials that you care about. Let's dive into how you can set this up using Vast.ai's GPU marketplace.

## What is Notebook Llama?

Notebook Llama is an innovative pipeline that transforms PDF documents into podcast-ready audio content through a series of four notebooks, each handling a specific part of the conversion process:

1. **PDF Pre-processing**: Uses Llama 3.2-3B-Instruct to convert PDFs into clean, structured text
2. **Initial Transcript Generation**: Employs Llama-3.2-8B-Instruct to create a conversational dialogue
3. **Transcript Refinement**: Leverages Llama 3.2 3B Instruct to enhance natural flow and engagement
4. **Text-to-Speech Generation**: Combines Suno/Bark and Parler-TTS for dynamic, two-voice audio

The notebooks that we'll use in this tutorial are slightly modified from their [GitHub counterparts](https://github.com/meta-llama/llama-recipes/tree/main/recipes/quickstart/NotebookLlama). You can find them at the following links:

- [Step 1: PDF Pre-Processing Logic](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/LzgpBE/b0a2e94bf5867bd7be218000a3770576eabbc7c9/files/Step-1%20PDF-Pre-Processing-Logic.json)
- [Step 2: Transcript Writer](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/q7BXoq/9e98ddd2583957116aef559765a55997df4348b5/files/Step-2-Transcript-Writer.json)
- [Step 3: Re-Writer](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/97xXo4/03cb3f5a7730cb6bc770b69c708079af05d50079/files/Step-3-Re-Writer.json)
- [Step 4: TTS Workflow](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/pqBXoe/2d8203c8e76267f14847865386d134619f9aec2a/files/Step-4-TTS-Workflow.ipynb)

## Setting Up Your Environment on Vast.ai
## Setting Up Your Environment on Vast.ai

### Step 1: Select the Right Instance

1. Visit Vast.ai and select a GPU instance with at least 40GB of RAM. We use 40GB as a rough estimate because the notebooks are running multiple small to mid-sized models.
2. Use this base image: https://cloud.vast.ai/?ref_id=62897&template_id=c50a4b1cc2fc37a62e2f3bea7cbd892a
3. Important: Choose "Jupyter Notebook" as the Launch Mode along with direct HTTPS access

### Step 2: Getting into your instance:
Once the instance is running, you can connect directly to it via the console in Vast.ai. More information about this an Jupyter Notebooks on vast can be found [here](https://vast.ai/docs/instance-setup/jupyter)

### Step 3: File Organization

1. Upload the notebook files
2. Create a `resources` folder
3. Upload your target PDF
4. Ensure `requirements.txt` is present

### Step 4: Environment Setup

Once your instance is running:

1. Connect via the console in Vast.ai
2. Navigate to the `app` directory:
```bash
cd app
```

3. Install dependencies:
```bash
pip install -r requirements.txt
pip install git+https://github.com/huggingface/parler-tts.git
sudo apt-get install ffmpeg
```

### Step 5: Hugging Face Authentication

1. Make sure that you have a Hugging Face account. If you don't have one, you can create one [here](https://huggingface.co/).
2. Make sure that you accept the terms for the [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) model.
3. Get your access token from the Hugging Face Settings page
4. Run:
```bash
huggingface-cli login
```
5. Enter your token when prompted

## How It Works: The Pipeline Explained

### Notebook 1: PDF Processing
The first notebook handles the crucial task of converting your PDF into clean, structured text. It uses PyPDF2 for initial extraction and Llama 3.2-3B-Instruct for intelligent text cleaning. What makes this approach unique is its use of a lightweight language model instead of traditional regex-based cleaning.

### Notebook 2: Creating the Conversation
This stage transforms the cleaned text into a natural dialogue using the Llama model. The model creates a two-speaker conversation where:
- Speaker 1 leads and teaches the content
- Speaker 2 asks questions and maintains flow
The output includes natural elements like "umm" and "hmm" to maintain authenticity.

### Notebook 3: Enhanced Rewriting
The third notebook uses Llama 3.2 3B Instruct to refine the conversation further. It's prompted to act as an "Oscar-winning screenwriter," adding:
- Natural dialogue patterns
- Relevant anecdotes and analogies
- TTS-compatible formatting

### Notebook 4: Audio Generation
The final stage brings your podcast to life using two different TTS approaches:
- Speaker 1: Parler-TTS for an expressive, dramatic voice
- Speaker 2: Suno/Bark for a more methodical style

This combination creates a dynamic conversation rather than monotonous single-voice narration.

For this specific notebook, we created a separate speaker description for speaker 2. Each Speaker is also using a different model.

# Output:

Each of these notebooks outputs a file to the resources folder to be used in the next step/notebook. At the end, there will be an MP# file that can serve as a podcast for you to listen to! We've included it in our folder for you to take a listen to.

## Why Use Vast.ai for This Project?

These notebooks leverage multiple large language models and TTS systems, requiring significant GPU resources for running multiple models at the same time. Vast.ai provides:
- Access to high-memory GPUs (40GB+) necessary for running larger models
- Cost-effective GPU rental compared to dedicated hardware or other cloud providers
- Easy-to-use Jupyter notebook support
- Great docker templates for running notebooks and being able to get running quickly in your environment
- Flexible scaling based on your needs

## Trying it out Yourself

Ready to try it yourself? Clone the repository, follow the setup instructions above, and start with the PDF provided to see the results

The beauty of this system is its modularity and its open source nature - you can modify each stage to suit your needs, whether that's using different models, adjusting the conversation style, or tweaking the voice characteristics.

Through the power of Vast.ai's GPU marketplace and the latest AI models, Notebook Llama offers an innovative way to make technical content more accessible and engaging. Give it a try and transform your PDFs into engaging podcast conversations!


As the world's first marketplace for cloud GPU rental, Vast.ai is excited to announce another first: we're expanding our offerings to include AMD support!

Ai

Stable Diffusion 3.5: Stability's Most Powerful AI Model Yet

Turn Any PDF into an AI-Generated Podcast with Notebook Llama