
Vast.ai provides a platform where developers can access a wide selection of GPUs from various hosts and data centers, offering competitive pricing to fit any budget. The cost and availability of GPUs are critical to the success and speed of generative AI projects. We’re excited to announce that [dstack](https://dstack.ai/) now includes integration with our [GPU marketplace](https://vast.ai/). dstack is an open-source cloud platform for training and deploying gen AI models. This integration enables our users to access an extensive selection of affordable GPUs, breaking down the barriers to entry for developers and ensuring that high costs do not stifle innovation.

# Diverse GPU Options at Your Fingertips

Vast.ai’s marketplace stands out by offering a wide selection of GPUs, ensuring that developers can select the best fit for their projects without cost constraints. The platform recommends GPU instances based on a comprehensive quality score but also gives you the flexibility to set a maximum price, ensuring that you stay within your budget.

# Empowering Developers with dstack and Vast.ai

With dstack 0.12.3, accessing Vast.ai's marketplace becomes a seamless part of the development workflow. Now, dstack’s intuitive CLI and API are equipped to tap into Vast.ai's resources, allowing you to train and deploy generative AI models with greater ease and less financial strain. This integration streamlines your development process, enabling you to utilize the cost benefits of Vast.ai without the hassle of complex cloud infrastructure management.

![](/uploads/dstack-top.png)

# Effortless Setup and Deployment

Using Vast.ai with dstack is easy and straightforward:

1. Log into your Vast.ai account and navigate to the 'Account' section to find your API Key.
2. With your API Key, configure the dstack backend according to the provided steps.

![](/uploads/dstack-bottom.png)

3. Once configured, begin using the robust features of dstack’s CLI and API.

This integration not only facilitates [fine-tuning](https://dstack.ai/docs/guides/fine-tuning/) and [deployment](https://dstack.ai/docs/guides/text-generation/) of models but also the creation of [development environments](https://dstack.ai/docs/guides/dev-environments/) that support interactive coding, making it a perfect fit for developers who prefer to work within their favorite IDEs.

# Seamless Switching Between Cloud Providers

The flexibility that dstack offers is unparalleled. You can now effortlessly switch to Vast.ai when it suits your project needs while retaining the option to use other cloud providers as required. This ensures that your workflow is as efficient and cost-effective as possible.

# Streamline Your AI Projects with dstack and Vast.ai

If you're looking for a simple and cost-effective way to train and deploy generative AI models, the combination of dstack and Vast.ai is ready for you. It's designed to save you time and money. With dstack’s new preview versions of the [fine-tuning](https://dstack.ai/docs/guides/fine-tuning/) and [text generation](https://dstack.ai/docs/guides/text-generation/) APIs, your development process just got even better.

Need a hand? Our [community](https://discord.gg/hSuEbSQ4X8) as well as dstack’s [community on Discord](https://discord.gg/u8SmfwPpMd) is available to offer support and answer your questions. We're committed to helping you make the most of what Vast.ai and dstack have to offer.


We’re excited to announce that dstack now includes integration with our GPU marketplace.

Accessing the GPU marketplace with Vast.ai and dstack


# Announcing Virtual Machine Rental on Vast.ai

Have you ever thought about what you could accomplish with more flexibility in your GPU rentals? At Vast.ai, we're always looking for ways to innovate and improve our platform, and today we're excited to announce our latest feature: [virtual machine (VM) rental](https://vast.ai/docs/instance-setup/virtual-machines).

We now offer full VMs alongside Docker-based instance rentals. This new offering is designed to support larger customers and more diverse workflows via GPU remote desktop.

Here's everything you need to know to get started!

## Virtual Machine Rental: Expanding the Possibilities

The new VM rental option will help meet the demand for use cases that Docker can't easily support. **VMs also offer features not available in Docker, such as support for init managers and process tracing.**

### Key Benefits of Virtual Machine Support Include:

- **Enhanced Flexibility:** Users can run low level debugging and performance profiling utilities and are not constrained by container environments.
- **Kernel-space Customizations:** Developers gain granular control over their systems, enabling custom driver installations, kernel tweaks, and integration of specialized frameworks.
- **Faster Iteration:** Rather than pre-building and pushing custom Docker images, users can run Docker inside the instance.
- **Improved Security:** Virtual Machines offers an improved  security posture for sensitive workloads.

Host machines can now tap into these advantages, allowing them to attract more renters and capitalize on the increased need for GPU-powered tasks beyond Docker’s capabilities.

### Important VM Differences to Know:

VMs do come with some trade-offs compared to Docker-based instances. For example, they have slower creation and boot times, higher disk overhead, and users will have to deal with a smaller selection of machines and lesser availability of preconfigured templates. In addition, only SSH launch mode is currently supported.

Notably, the on-start script field allows users to specify a script to run on instance start. Unlike in Docker-based instances, the interpreter must be specified by a shebang.

Another difference is that **the cloud copy utility for VMs only allows migrating the entire VM instead of allowing individual folders to be copied.**

Despite these drawbacks, VMs open up new opportunities for both renters and hosts, offering greater flexibility, broader use cases, and the potential for higher earnings.

### Empowering Renters, Rewarding Hosts

One thing that sets us apart from other providers is our focus on cost-efficiency. We strive to keep rental costs low for users across the board – and in fact, Vast.ai GPU rentals are ~3-5X cheaper than current alternatives.

At the same time, we want to provide hosts with the opportunity to earn more.

As our COO, Travis Cannell, explains, "We've always believed that democratizing GPU access means offering maximum flexibility. Virtual Machine support was the natural next step, providing more control, ease of use, and enabling researchers to build and iterate."

With the opportunity for both renters and hosts to benefit, it's equally important to ensure that setup is seamless and operations remain secure. Here's how we make that happen.

## VMs on Vast: Simple Setup and Strong Security

Getting started with VM rental on Vast.ai is simple. **Our hosting software automatically tests host machines for VM compatibility and enables the feature on any machine where it's supported.** (There is no requirement to be VM compatible, however.)

For new host machines, these tests will be run upon installation. For machines configured before the VM feature release, testing for VM compatibility will happen at a time when the machine is unoccupied.

And that's it! Host machines that don't have VM support enabled will simply be hidden from search results when VM-based templates are selected.

Privacy and security are built into the system, as well. Hosts and users cannot view someone else's VM, and no one has access to see what any other VM is doing. Users can rent VMs with peace of mind knowing that their virtual environment is private and secure.

## Getting Started: Creating a VM

Ready to spin up your first VM? It's as easy as selecting the right template!

1.  Make sure you have an SSH key in your account (guide [here](https://vast.ai/docs/instance-setup/ssh)).
2.  Select the **Ubuntu 22.04 VM template** in the interface or [click here](https://cloud.vast.ai/?ref_id=108031&template_id=e2f271ce754aeef772181fbf9c82a354) to load it.
3.  Adjust the disk storage needed and then rent a GPU!

Note: SSH keys cannot be edited on running VMs, and SSH access is currently the only method preconfigured on VM instances. Also, when you select the correct template, it will automatically filter for machines that support VMs and will launch a VM when you rent with that template.

For more detailed information about VM template customization, see our Docs page [here](https://vast.ai/docs/instance-setup/virtual-machines).

## Power Up with VM Rental on Vast.ai

With our new virtual machine rental feature, we're opening up a world of possibilities for both hosts and renters. Whether you need the flexibility of GPU remote desktops for complex workflows or you're looking to maximize your earning potential as a host, Vast.ai has you covered. We're committed to giving you the tools to succeed – easily, affordably, and securely.

Ready to see what you can accomplish? Experience the [power of VM rental](https://vast.ai/docs/instance-setup/virtual-machines) with Vast.ai today!


At Vast.ai, we're always looking for ways to innovate and improve our platform, and today we're excited to announce our latest feature: virtual machine (VM) rental. 

Announcing Virtual Machine Rental on Vast.ai


Released on October 29th, the 3.5 update included multiple models: Stable Diffusion 3.5 Large, Stable Diffusion 3.5 Large Turbo, and Stable Diffusion 3.5 Medium. Each of these variants can be customized to meet individual hardware needs.

# A Look at the Stable Diffusion 3.5 Variants

**[Stable Diffusion 3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large)**

This base model, features an impressive 8.1 billion parameters and stands out as the most powerful option within the Stable Diffusion family. It prioritizes superior quality and prompt adherence, making it particularly suitable for various professional applications. 3.5 Large boasts a resolution capability of 1 megapixel, it delivers high-quality outputs that meet the demands of various industries, from graphic design to digital content creation. Whether you're working on detailed imagery or need reliable performance for professional projects, this model is engineered to meet and then exceed user expectations.

**Stability AI Highlight**

Analysis shows that Stable Diffusion 3.5 Large excels in prompt adherence and matches larger models in image quality.

**[Stable Diffusion 3.5 Large Turbo](https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo)**

Stable Diffusion 3.5 Large Turbo is designed to create high-quality images with remarkable adherence to prompts, achieving this in just four streamlined steps. This efficiency not only enhances the user experience but also allows for a considerably faster image generation process compared to the original Stable Diffusion 3.5 Large model. Users can expect quicker results without compromising the quality or precision of the images produced, making this version an excellent choice for those seeking both speed and visual fidelity.

**Stability AI Highlight**

Stable Diffusion 3.5 Large Turbo provides some of the fastest inference times for its size, maintaining strong image quality and prompt adherence, even when compared to non-distilled models of similar size.

**[Stable Diffusion 3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium)**

This advanced model boasts an impressive 2.5 billion parameters and incorporates an enhanced MMDiT-X architecture along with cutting-edge training methods. 3.5 Medium is specifically designed to function effortlessly on standard hardware, making it accessible to a wide range of users. Its design philosophy emphasizes a blend of high-quality output and straightforward customization, allowing users to tailor the model's performance to their specific needs.

The model is also capable of generating high-resolution images, with the ability to produce outputs ranging from 0.25 megapixels to 2 megapixels. This versatility in resolution enables users to create visuals suitable for various applications, whether for casual use or more demanding projects. Overall, the model provides an efficient and user-friendly solution for anyone seeking to generate quality images without specialized equipment.

**Stability AI Highlight**

Stable Diffusion 3.5 Medium surpasses other medium-sized models, providing a harmonious blend of prompt adherence and image quality, establishing it as a preferred option for efficient, high-quality performance.

## A Note on Community License

Like earlier versions, Stable Diffusion 3.5 operates under the permissive Stability AI Community License. According to Stability's website, this license permits the following uses:

- **Free for non-commercial use:** Individuals and organizations can use the model free of charge for non-commercial use, including scientific research.
- **Free for commercial use (up to $1M in annual revenue):** Startups, small to medium-sized businesses, and creators can use the model for commercial purposes at no cost as long as their total annual revenue is less than $1M.
- **Ownership of outputs:** Retain ownership of the media generated without restrictive licensing implications.

## A Final Note

Stable Diffusion 3.5 represents a significant advancement, offering a unique combination of prompt adherence and image quality that sets it apart from its competitors. Its ability to deliver high-quality results efficiently makes it an invaluable tool for creators and professionals alike. As the landscape of digital generation continues to evolve, this model positions itself as a reliable and effective choice for achieving outstanding visual outputs.

Excited about new developments in AI and where it is headed next? Stay updated with the latest news and developments in AI and technology through our [blog](https://vast.ai/blog) at [Vast.AI.](http://vast.ai) Interested in running Stable Diffusion on Vast.ai? Check out our [documentation](https://docs.vast.ai/stable-diffusion) for a full walkthrough.


Unleash the power of Stable Diffusion 3.5 on Vast.ai. Explore three models - Large, Large Turbo, and Medium - crafted for exceptional image generation, tailored to your hardware capacity. 

Stable Diffusion 3.5: Stability's Most Powerful AI Model Yet


# Turn Any PDF into an AI-Generated Podcast with Notebook Llama

AI has now gone multimodal: instead of just sending chat messages to a chatbot, there are awesome use cases involving audio, videos and images. One amazing product like this is [NotebookLM](https://notebooklm.google/) from Google which has taken the AI community by storm. It takes in YouTube videos, PDF’s or other documents, and derives insights and even creates a podcast from the notes and original material.

But Meta’s Llama team came out with Notebook Llama, an open source version of this flow, which lets users run their own models to create podcasts on their own material. We will show you how to run it with Vast. In this tutorial, we will show you how to put the world of AI researchers in the palm of your hands for the materials that you care about. Let's dive into how you can set this up using Vast.ai's GPU marketplace.

## What is Notebook Llama?

Notebook Llama is an innovative pipeline that transforms PDF documents into podcast-ready audio content through a series of four notebooks, each handling a specific part of the conversion process:

1. **PDF Pre-processing**: Uses Llama 3.2-3B-Instruct to convert PDFs into clean, structured text
2. **Initial Transcript Generation**: Employs Llama-3.2-8B-Instruct to create a conversational dialogue
3. **Transcript Refinement**: Leverages Llama 3.2 3B Instruct to enhance natural flow and engagement
4. **Text-to-Speech Generation**: Combines Suno/Bark and Parler-TTS for dynamic, two-voice audio

The notebooks that we'll use in this tutorial are slightly modified from their [GitHub counterparts](https://github.com/meta-llama/llama-recipes/tree/main/recipes/quickstart/NotebookLlama). You can find them at the following links:

- [Step 1: PDF Pre-Processing Logic](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/LzgpBE/b0a2e94bf5867bd7be218000a3770576eabbc7c9/files/Step-1%20PDF-Pre-Processing-Logic.json)
- [Step 2: Transcript Writer](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/q7BXoq/9e98ddd2583957116aef559765a55997df4348b5/files/Step-2-Transcript-Writer.json)
- [Step 3: Re-Writer](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/97xXo4/03cb3f5a7730cb6bc770b69c708079af05d50079/files/Step-3-Re-Writer.json)
- [Step 4: TTS Workflow](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/pqBXoe/2d8203c8e76267f14847865386d134619f9aec2a/files/Step-4-TTS-Workflow.ipynb)

## Setting Up Your Environment on Vast.ai
## Setting Up Your Environment on Vast.ai

### Step 1: Select the Right Instance

1. Visit Vast.ai and select a GPU instance with at least 40GB of RAM. We use 40GB as a rough estimate because the notebooks are running multiple small to mid-sized models.
2. Use this base image: https://cloud.vast.ai/?ref_id=62897&template_id=c50a4b1cc2fc37a62e2f3bea7cbd892a
3. Important: Choose "Jupyter Notebook" as the Launch Mode along with direct HTTPS access

### Step 2: Getting into your instance:
Once the instance is running, you can connect directly to it via the console in Vast.ai. More information about this an Jupyter Notebooks on vast can be found [here](https://vast.ai/docs/instance-setup/jupyter)

### Step 3: File Organization

1. Upload the notebook files
2. Create a `resources` folder
3. Upload your target PDF
4. Ensure `requirements.txt` is present

### Step 4: Environment Setup

Once your instance is running:

1. Connect via the console in Vast.ai
2. Navigate to the `app` directory:
```bash
cd app
```

3. Install dependencies:
```bash
pip install -r requirements.txt
pip install git+https://github.com/huggingface/parler-tts.git
sudo apt-get install ffmpeg
```

### Step 5: Hugging Face Authentication

1. Make sure that you have a Hugging Face account. If you don't have one, you can create one [here](https://huggingface.co/).
2. Make sure that you accept the terms for the [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) model.
3. Get your access token from the Hugging Face Settings page
4. Run:
```bash
huggingface-cli login
```
5. Enter your token when prompted

## How It Works: The Pipeline Explained

### Notebook 1: PDF Processing
The first notebook handles the crucial task of converting your PDF into clean, structured text. It uses PyPDF2 for initial extraction and Llama 3.2-3B-Instruct for intelligent text cleaning. What makes this approach unique is its use of a lightweight language model instead of traditional regex-based cleaning.

### Notebook 2: Creating the Conversation
This stage transforms the cleaned text into a natural dialogue using the Llama model. The model creates a two-speaker conversation where:
- Speaker 1 leads and teaches the content
- Speaker 2 asks questions and maintains flow
The output includes natural elements like "umm" and "hmm" to maintain authenticity.

### Notebook 3: Enhanced Rewriting
The third notebook uses Llama 3.2 3B Instruct to refine the conversation further. It's prompted to act as an "Oscar-winning screenwriter," adding:
- Natural dialogue patterns
- Relevant anecdotes and analogies
- TTS-compatible formatting

### Notebook 4: Audio Generation
The final stage brings your podcast to life using two different TTS approaches:
- Speaker 1: Parler-TTS for an expressive, dramatic voice
- Speaker 2: Suno/Bark for a more methodical style

This combination creates a dynamic conversation rather than monotonous single-voice narration.

For this specific notebook, we created a separate speaker description for speaker 2. Each Speaker is also using a different model.

# Output:

Each of these notebooks outputs a file to the resources folder to be used in the next step/notebook. At the end, there will be an MP# file that can serve as a podcast for you to listen to! We've included it in our folder for you to take a listen to.

## Why Use Vast.ai for This Project?

These notebooks leverage multiple large language models and TTS systems, requiring significant GPU resources for running multiple models at the same time. Vast.ai provides:
- Access to high-memory GPUs (40GB+) necessary for running larger models
- Cost-effective GPU rental compared to dedicated hardware or other cloud providers
- Easy-to-use Jupyter notebook support
- Great docker templates for running notebooks and being able to get running quickly in your environment
- Flexible scaling based on your needs

## Trying it out Yourself

Ready to try it yourself? Clone the repository, follow the setup instructions above, and start with the PDF provided to see the results

The beauty of this system is its modularity and its open source nature - you can modify each stage to suit your needs, whether that's using different models, adjusting the conversation style, or tweaking the voice characteristics.

Through the power of Vast.ai's GPU marketplace and the latest AI models, Notebook Llama offers an innovative way to make technical content more accessible and engaging. Give it a try and transform your PDFs into engaging podcast conversations!


As the world's first marketplace for cloud GPU rental, Vast.ai is excited to announce another first: we're expanding our offerings to include AMD support!

Blog

Accessing the GPU marketplace with Vast.ai and dstack

Diverse GPU Options at Your Fingertips

Empowering Developers with dstack and Vast.ai

Effortless Setup and Deployment

Seamless Switching Between Cloud Providers

Streamline Your AI Projects with dstack and Vast.ai

Continue Reading: