
February may be a short month, but we've made the most of it with plenty of updates!

Vast.ai is now offering NVIDIA RTX 5000-series GPUs for rent, including the powerful **RTX 5090**, bringing even more high-performance options to our platform. Plus, we've partnered with **SkyPilot** in an integration that allows you to rent Vast machines through SkyPilot's infrastructure and price optimization framework for even greater flexibility.

Of course, keeping everything running smoothly means staying on top of any technical hiccups. As always, we've been hard at work refining our platform and fixing bugs. Here are some of the latest bug fixes we've rolled out since our last update:

-   Fixed instance-to-instance copy issue.
-   Made various virtual machine (VM) enhancements.
-   Corrected an issue where machine is erroneously marked as "unavailable."
-   Fixed earnings display issue and template category display issue.

### NVIDIA Cloud GPU Updates

At Vast.ai, every update is aimed at making your experience smoother, faster, and more efficient. From expanding our GPU offerings to adding templates and updating guides, our latest improvements are designed to give you even more control and streamline your workflow.

#### Highlights of the latest enhancements:

-   NVIDIA RTX 5000-series GPUs are now available, including the [RTX 5090](https://cloud.vast.ai/?gpu_option=RTX+5090).
-   SkyPilot Integration: Rent Vast machines through SkyPilot’s infrastructure and price optimization framework.
-   Added [vLLM Template](https://cloud.vast.ai/?ref_id=62897&template_id=f4c3ccaf87393c5266748b3b42ba8567) and [Guide](https://docs.vast.ai/vllm-llm-inference-and-serving): Fast and easy-to-use library for LLM inference and serving.
-   Added [Unreal Pixel Streaming Template](https://cloud.vast.ai/?ref_id=62897&template_id=c1c84fd5a4d945fc16012e7ca02d1423): Stream high-quality, interactive 3D content directly to web browsers without requiring end users to install anything.
-   Added [Ollama + WebUI Guide](https://docs.vast.ai/ollama-webui) for running DeepSeek R1.
-   Added [Guide for Creating a Custom Template](https://docs.vast.ai/creating-a-custom-template) from Vast Base Image.
-   Updated Hosting Guide.

Please check out the full change log for our February 2025 updates at the bottom of this page.

### Our Commitment

Whether you're looking to try out the RTX 5090, take advantage of SkyPilot integration, explore our new and updated templates and guides, or just enjoy the many features we continue to refine and expand, we're here to help you get the most out of Vast.ai! We're always happy to support both new and longtime users with affordable, on-demand GPU and virtual machine rental.

Our easy-to-use platform is backed by a responsive customer support team and an active Discord community. Need assistance? Email us at [support@vast.ai](mailto:support@vast.ai) or join our Discord server for helpful resources today!

### Change Log

#### Features:

-   NVIDIA RTX 5000 Series GPUs: Rent Blackwell chips on Vast, including the [RTX 5090](https://cloud.vast.ai/?gpu_option=RTX+5090).
-   SkyPilot Integration: Rent Vast machines through SkyPilot’s infrastructure and price optimization framework.

#### Templates & Guides:

-   Added [vLLM Template](https://cloud.vast.ai/?ref_id=62897&template_id=f4c3ccaf87393c5266748b3b42ba8567) and [Guide](https://docs.vast.ai/vllm-llm-inference-and-serving): Fast and easy-to-use library for LLM inference and serving.
-   Added [Unreal Pixel Streaming Template](https://cloud.vast.ai/?ref_id=62897&template_id=c1c84fd5a4d945fc16012e7ca02d1423): Stream high-quality, interactive 3D content directly to web browsers without requiring end users to install anything.
-   Added [Ollama + WebUI Guide](https://docs.vast.ai/ollama-webui) for running DeepSeek R1.
-   Added [Guide for Creating a Custom Template](https://docs.vast.ai/creating-a-custom-template) from Vast Base Image.
-   Updated VM Desktop Template.
-   Updated Open WebUI Template: Now includes pre-built llama.cpp.
-   Updated ComfyUI & ComfyUI + FLUX.1 Templates.
-   Updated SD Swarm & SD Forge Templates.
-   Updated Fooocus Template.
-   Updated Ubuntu 22.04 VM Template: Updated drivers to CUDA 12.8, preinstalled Docker container toolkit, added Tailscale support.
-   Updated Kohya’s GUI.
-   Updated Oobabooga Template.
-   Updated Hosting Guide.

#### Bug Fixes:

-   Various virtual machine (VM) enhancements.
-   Instance-to-instance copy issue.
-   Machine marked as "unavailable" issue.
-   Earnings display issue.
-   Template category display issue.
-   Session key issue.

#### Miscellaneous Tasks:

-   Added API keys section into CLI page for CLI login command.
-   Added length validation to API keys.  


Rent NVIDIA RTX 5090 GPUs on Vast.ai! Now integrated with SkyPilot for better pricing. New templates, guides & fixes to improve your experience.

February 2025 Product Update


## Introduction

In the world of audio processing and speech recognition, identifying when someone is speaking versus when there's silence or background noise is a critical first step. This process, known as Voice Activity Detection (VAD), serves as the foundation for many speech-related applications, from transcription services to voice assistants. While conceptually simple, implementing an efficient and accurate VAD system can significantly improve downstream tasks and reduce computational costs.

[PyAnnote Audio](https://github.com/pyannote/pyannote-audio), an open-source toolkit built on PyTorch, offers state-of-the-art models for VAD that are both accurate and accessible. Running these models on Vast.ai provides a cost-effective solution for processing large audio datasets without investing in expensive hardware. This combination gives developers and researchers the tools they need to build sophisticated audio processing pipelines at a fraction of the cost of traditional cloud providers.

### Why Voice Activity Detection Matters

VAD provides several key benefits for speech processing pipelines:

1. **Reduced Computation Load**: By filtering out non-speech segments before running speech-to-text (STT) models, we significantly reduce the computational resources needed for transcription.

2. **Improved Accuracy**: Many STT models perform better when processing only speech segments rather than trying to interpret silence or background noise.

3. **Efficient Storage**: Extracting only the speech segments can reduce storage requirements for large audio datasets.

4. **Better User Experience**: For applications like voice assistants or transcription services, VAD helps eliminate unnecessary processing of silence.

### What This Guide Covers

In this guide, we will:

-   Set up the Pyannote Audio VAD pipeline
-   Process audio files to detect speech segments
-   Extract and save only the speech portions of the input audio
-   Visualize the results

The output will be a collection of audio files containing only the detected speech segments from the original recording, making them ready for further processing in speech-to-text pipelines.

### Why VAST.ai

VAST.ai offers a marketplace approach to GPU rentals that provides significant advantages for audio processing tasks. Unlike traditional cloud providers, VAST.ai allows you to:

-   Rent precisely the GPU capacity needed for your workload
-   Access GPU's at more affordable rates
-   Access more types of SKU's of GPU's, particularly lower RAM/more affordable GPUs
-   Avoid long-term commitments for experimental projects

For VAD specifically, VAST.ai offers an ideal balance of performance and cost-effectiveness, as these models benefit from GPU acceleration without requiring the most expensive hardware tiers.

## Download This Notebook

To follow along with this tutorial, you can download the complete Jupyter notebook:

[Notebook](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/6q8akx/6b8dc846a00d27decad5a60a6159466813f6d445/files/pyannote_vad_vast.ipynb)

Having the notebook will allow you to execute the code blocks as you read through this guide, making it easier to understand and experiment with the VAD implementation.

## Choosing an Instance

For running the Pyannote Voice Activity Detection model on VAST.ai, you'll need a relatively modest GPU setup since VAD models are computationally efficient compared to larger AI tasks. Here are the recommended specifications:

-   GPU: A mid-range GPU like an RTX 3060 or 4060 would be sufficient.
-   VRAM: 6-8GB of VRAM should be adequate as the Pyannote VAD model is relatively small.
-   RAM: 8-16GB system RAM is recommended for processing audio files.
-   Storage: At least 10GB for the model, dependencies, and your audio files.
-   CUDA: Make sure the instance has CUDA installed (version 11.0+ recommended).
-   Python: Python 3.8+ with PyTorch installed.

## Selecting an Instance

Follow these steps to set up your environment:

1. Ensure that you have a Vast.ai account
2. Go to the Vast Templates in the Console https://cloud.vast.ai/templates/
3. Select the `PyTorch (CuDNN Runtime)` Template
4. Filter for an instance with:
    - 1 GPU
    - 6-8GB of VRAM
    - 8-16GB system RAM
    - 10GB of storage
5. Select an instance and click rent
6. Install the Vast TLS certificate in your browser to access the notebook server https://docs.vast.ai/instances/jupyter#1SmCz
7. Go to your Instances https://cloud.vast.ai/instances/ and click "Open" to access the jupyter server on your instance.
8. Upload the notebook to the server or create a new notebook

## Installing Dependencies

Let's start by installing the necessary Python packages:

```python
%%bash
pip install pyannote.audio
pip install pydub
pip install librosa
pip install yt-dlp
```

We also need to install FFmpeg for audio processing:

```python
%%bash
apt-get update && apt-get install -y ffmpeg
```

## Setting Up Your Hugging Face Token

Pyannote models are hosted on Hugging Face, so you'll need to set up authentication:

```python
# Make sure you've accepted the user conditions at:
# https://hf.co/pyannote/voice-activity-detection
# https://hf.co/pyannote/segmentation

HF_TOKEN = ""  # Add your token here
```

Ensure that you have accepted the terms for the models at the URLs above. The models are free to use, but you must agree to their terms of service.

## Generating Test Data

For this tutorial, we'll download a sample audio file from Vast.ai's YouTube channel. You can also use your own audio file if you prefer:

```bash
yt-dlp -f "bestaudio" --extract-audio --audio-format wav -o "test.wav" https://www.youtube.com/watch?v=542xENIxKFU
```

## Voice Activity Detection

First, let's set up the VAD pipeline:

```python
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

from pyannote.audio import Pipeline

pipeline = Pipeline.from_pretrained(
        "pyannote/voice-activity-detection",
        use_auth_token=HF_TOKEN
    )

# Move pipeline to appropriate device
pipeline = pipeline.to(device)
```

Next, we'll process our audio file to identify speech segments:

```python
# Process the audio file
audio_file = "test.wav"
output = pipeline(audio_file)

print(f"Processing {audio_file} on {device}")
print("Voice activity segments:")

# Get all speech segments
speech_segments = list(output.get_timeline().support())

for i, speech in enumerate(speech_segments):
    # active speech between speech.start and speech.end
    print(f"Segment {i+1}: Speech from {speech.start:.2f}s to {speech.end:.2f}s (duration: {speech.duration:.2f}s)")
```

The output will look like this when using the default audio file:

```
Processing test.wav on cuda
Voice activity segments:
Segment 1: Speech from 6.78s to 51.62s (duration: 44.84s)
Segment 2: Speech from 53.56s to 54.27s (duration: 0.71s)
Segment 3: Speech from 55.55s to 84.76s (duration: 29.21s)
Segment 4: Speech from 86.53s to 89.03s (duration: 2.50s)
...
```

## Saving Speech Segments

Now we'll create a function to extract the identified speech segments from our audio file:

```python
import os
import shutil
from pydub import AudioSegment


def split_audio_by_segments(audio_path, segments, output_dir="output_segments"):
    """
    Split an audio file into multiple files based on speech segments

    Parameters:
    -----------
    audio_path: str
        Path to the input audio file
    segments: list
        List of speech segments (with start and end attributes)
    output_dir: str
        Directory to save the output segments
    """
    # Clear the output directory if it exists
    if os.path.exists(output_dir):
        shutil.rmtree(output_dir)

    # Create output directory
    os.makedirs(output_dir, exist_ok=True)

    # Load the audio file
    audio = AudioSegment.from_file(audio_path)

    # Extract each segment
    for i, segment in enumerate(segments):
        # Convert seconds to milliseconds
        start_ms = int(segment.start * 1000)
        end_ms = int(segment.end * 1000)

        # Extract segment
        segment_audio = audio[start_ms:end_ms]

        # Generate output filename
        filename = os.path.basename(audio_path)
        name, ext = os.path.splitext(filename)
        output_path = os.path.join(output_dir, f"{name}_segment_{i+1:04d}_{start_ms:08d}ms-{end_ms:08d}ms{ext}")

        # Export segment
        segment_audio.export(output_path, format=ext.replace('.', ''))
        print(f"Saved segment {i+1} to {output_path}")
```

Let's apply this function to extract our speech segments:

```python
split_audio_by_segments(audio_file, speech_segments)
```

This will be the output when using the default audio file:

```
Saved segment 1 to output_segments/test_segment_0001_00006780ms-00051617ms.wav
Saved segment 2 to output_segments/test_segment_0002_00053558ms-00054267ms.wav
Saved segment 3 to output_segments/test_segment_0003_00055549ms-00084760ms.wav
Saved segment 4 to output_segments/test_segment_0004_00086532ms-00089029ms.wav
...
```

## Inspecting Results

To verify our results, we'll create a function to play audio files in our Jupyter environment:

```python
import librosa
from IPython.display import Audio, display

def play_audio(file_path, sr=None):
    """
    Play an audio file in a Jupyter notebook.

    Parameters:
    -----------
    file_path : str
        Path to the audio file to play
    sr : int, optional
        Sample rate to load the audio with. If None, uses the file's native sample rate.

    Returns:
    --------
    Audio widget that can be played in the notebook

    Example:
    --------
    >>> play_audio('path/to/audio.wav')
    """
    # Load the audio file
    y, sr = librosa.load(file_path, sr=sr)


    # Return an audio widget to play the sound
    audio_widget = Audio(data=y, rate=sr)
    display(audio_widget)
```

First, we'll play the original audio file. Listen to the first minute or so to get an idea of what it sounds like before VAD.

```python
play_audio(audio_file)
```

Next, we'll listen to the first three clips to verify that we have isolated the speech in our test file:

```python
import os
audio_dir = "./output_segments/"

audio_files = os.listdir(audio_dir)
audio_files.sort()

n_clips = 3

for fname in audio_files[0:n_clips]:
    play_audio(audio_dir + fname)
```

You'll see that the audio lengths match up with the `speech_segments` output:

```
Segment 1: Speech from 6.78s to 51.62s (duration: 44.84s)
Segment 2: Speech from 53.56s to 54.27s (duration: 0.71s)
Segment 3: Speech from 55.55s to 84.76s (duration: 29.21s)
...
```

Listening to the output files, we can see that we have effectively isolated the speech.

## Conclusion

With this implementation, you now have a working Voice Activity Detection system that can identify and extract speech segments from audio files. This forms an excellent foundation for more advanced audio processing tasks like speech recognition, speaker diarization, or audio content analysis.

Look out for more content from Vast about all these other types of tasks!


Run state-of-the-art Voice Activity Detection (VAD) with PyAnnote Audio on Vast.ai for cost-effective speech processing. Extract speech segments easily!

Voice Activity Detection (VAD) with Pyannote on VAST


## Introduction

With the release of o1 family of models, reasoning has taken the AI world by storm. Instead of immediately answering a prompt, reasoning is where a model is trained to first output it's "thinking" for its answer. This increases performance on the most difficult tasks today, and since it is trained into the model, it is better than asking it to record its "chain of thought" via a prompt.

R1 takes this even further. We now have Open Source models for this type of task, which means:

1. We can retain the "thinking" token outputs which we couldn't do beforehand. This lets us create datasets for finetuning.
2. We can easily fine-tune these models on our own tasks
3. We can run them on more affordable compute to drastically reduce the cost!

This implementation combines three key components: A distilled DeepSeek model for reasoning transparency, a Vast Template for vLLM's optimized inference server for efficient deployment, and Langchain for parsing reasoning tokens versus normal output tokens. Together, they create a production-ready system that can handle both the technical demands of serving a large language model and the practical needs of processing its unique output format.

Vast.ai provides an ideal platform for this setup, offering the necessary GPU resources at a fraction of traditional cloud costs. Its marketplace model and simple Docker integration make it particularly well-suited for deploying and scaling language models, allowing developers to focus on building applications rather than managing infrastructure. The Templates feature from Vast allows for a repeatable deployment with little additional configuration.

This guide demonstrates how to deploy [deepseek/DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) with Vast's templates and integrate them with Langchain for advanced processing capabilities. We'll show you how to leverage vLLM's optimized inference server and create custom parsers to handle DeepSeek's distinctive output format. Feel free to follow along in this [companion notebook](https://nbviewer.org/urls/bitbucket.org/!api/2.0/snippets/jsbcannell/xq8zG5/2cf7d8758235fa5176d9aee8b8d3885128c0109c/files/serving_deepseek_on_vast.ipynb)

## Setting Up the Environment

Before we can deploy our model, we need to set up our Vast.ai environment. First, install the Vast SDK:

```python
%%bash
pip install --upgrade vastai
```

Set up your Vast API key (available from your [Account Page](https://cloud.vast.ai/account/)):

```python
%%bash
# Here we will set our api key
export VAST_API_KEY="<your-key-here>"
vastai set api-key $VAST_API_KEY
```

### Choosing the Right Hardware

Now we are going to search for GPUs on Vast.ai to run the DeepSeek-R1-Distill-Qwen-32B model. This model requires specific hardware capabilities to run efficiently with vLLM's optimizations. Here are our requirements:

1. A minimum of 80GB GPU RAM to accommodate:

    - DeepSeek model weights (32B Parameters)
    - KV Cache for handling of extra long output token lengths

2. A single GPU configuration, as DeepSeek-R1-Distill-Qwen-32B can be efficiently served on one GPU:
   Note: Multi-GPU configurations are supported if higher throughput is needed.

3. A static IP address for:

    - Stable API endpoint hosting
    - Consistent client connections
    - Reliable Langchain integration

4. At least one direct port that we can forward for:

    - vLLM's OpenAI-compatible API server
    - External access to the model endpoint
    - Secure request routing

5. At least 120GB of disk space to hold the model and other things we might like to download

Here's how to search for suitable instances on Vast.ai:

```python
%%bash
vastai search offers "compute_cap >= 750 \
gpu_ram >= 80 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 120 \
rentable = true"
```

### Deploying the Server via Vast Template

We'll use vLLM's OpenAI-compatible server to deploy the DeepSeek model. This setup provides an OpenAI-compatible API endpoint that works seamlessly with existing tools and libraries.

We will do this with a template that:

1. Uses `vllm/vllm-openai:latest` docker image. This gives us an OpenAI-compatible server.
2. Forwards port `8000` to the outside of the container, which is the default OpenAI server port
3. Forwards `--model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --max-model-len 8192  --enforce-eager` on to the default entrypoint (the server itself)
4. Uses `--tensor-parallel-size 1` by default.
5. Uses `--gpu-memory-utilization 0.90` by default
6. Ensures that we have 120 GB of Disk space

```python
%%bash
export INSTANCE_ID= #insert instance ID
vastai create instance $INSTANCE_ID --disk 120 --template_hash eda062b3e0c9c36f09d9d9a294405ded
```

### Verify Setup

After deployment, verify that your server is running correctly with a simple curl test:

```python
%%bash
export VAST_IP_ADDRESS="<your-ip-here>"
export VAST_PORT="<your-port-here>"
curl -X POST http://$VAST_IP_ADDRESS:$VAST_PORT/v1/completions \
     -H "Content-Type: application/json" \
     -d '{
           "model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
           "prompt": "Hello, how are you?",
           "max_tokens": 50
         }'
```

Note: You can find your instance's IP address and port in the [Instances tab](https://cloud.vast.ai/instances/) of the Vast AI Console.

## Implementing Custom Output Parsing

### Install Required Dependencies

```python
%%bash
pip install --upgrade langchain langchain-openai openai
```

### Creating a DeepSeek Output Parser

DeepSeek models output responses in a unique format with separate thinking and response sections. The thinking section, wrapped in `<think>` tags, contains the model's reasoning process, while everything after the closing tag represents the final response. This separation is valuable for understanding how the model reaches its conclusions.

Let's create a custom parser that can handle this format:

```python
from typing import Optional, Tuple
from langchain.schema import BaseOutputParser

class R1OutputParser(BaseOutputParser[Tuple[Optional[str], str]]):
    """Parser for DeepSeek R1 model output that includes thinking and response sections."""

    def parse(self, text: str) -> Tuple[Optional[str], str]:
        """Parse the model output into thinking and response sections.

        Args:
            text: Raw text output from the model

        Returns:
            Tuple containing (thinking_text, response_text)
            - thinking_text will be None if no thinking section is found
        """
        if "</think>" in text:
            # Split on </think> tag
            parts = text.split("</think>")
            # Extract thinking text (remove <think> tag)
            thinking_text = parts[0].replace("<think>", "").strip()
            # Get response text
            response_text = parts[1].strip()
            return thinking_text, response_text

        # If no thinking tags found, return None for thinking and full text as response
        return None, text.strip()

    @property
    def _type(self) -> str:
        """Return type key for serialization."""
        return "r1_output_parser"
```

Our parser does several important things:

1. It looks for the `</think>` tag to identify the boundary between thinking and response
2. It extracts and cleans up the thinking section by removing the `<think>` tags
3. It handles cases where no thinking section is present by returning None
4. It ensures both sections are properly stripped of extra whitespace

### Setting Up the Model Chain

Now that we have our parser, we need to set up a processing chain that connects our model to the parser. This chain will:

-   Send requests to our deployed model
-   Process the responses through our custom parser
-   Return the separated thinking and response sections

Here's how we set it up:

```python
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.runnable import RunnablePassthrough

# Initialize the model with your Vast instance details
VAST_IP_ADDRESS="<your-ip-here>"
VAST_PORT="<your-port-here>"

openai_api_key = "EMPTY"
openai_api_base = f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"

model = ChatOpenAI(
    base_url=openai_api_base,
    api_key=openai_api_key,
    model_name="deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
    max_tokens=8000,
    temperature=0.7
)

# Create prompt template
prompt = ChatPromptTemplate.from_messages([
    ("user", "{input}")
])

# Create parser
parser = R1OutputParser()

# Create chain
chain = (
    {"input": RunnablePassthrough()}
    | prompt
    | model
    | parser
)
```

Let's break down what each component does:

-   The `ChatOpenAI` instance connects to our deployed model using the OpenAI-compatible API
-   The `ChatPromptTemplate` formats our input messages
-   The `RunnablePassthrough` ensures our inputs flow through the chain correctly
-   Finally, our custom parser processes the model's output

### Example Usage

Let's test our setup with a prompt that can demonstrate the model's reasoning capabilities:

```python
prompt_text = "Explain quantum computing to a curious 10-year-old who loves video games."

thinking, response = chain.invoke(prompt_text)
print("\nTHINKING:\n")
print(thinking)
print("\nRESPONSE:\n")
print(response)
```

#### Output:

```
THINKING:

Okay, so I need to explain quantum computing to a 10-year-old who loves video games. Hmm, let's break this down. First, I should think about what a 10-year-old knows. They understand basic concepts like computers, games, maybe some video game terms like pixels or bits. They also enjoy stories or analogies that relate to their interests, so using video games as a context makes sense.

Quantum computing is a complex topic, so I need to simplify it. I remember that classical computers use bits, which are 0s and 1s. Quantum computers use qubits, which can be both 0 and 1 at the same time. That's a key difference. Maybe I can compare it to something in a video game. Maybe like a character that can be in two places at once, or a power-up that gives multiple abilities simultaneously.

I should also mention superposition and entanglement. Superposition is when a qubit can be in multiple states at once, and entanglement is when qubits are connected, so the state of one affects the others. Maybe relate that to something like a power-up that affects multiple things at once or a team of characters working together.

Another important point is that quantum computers can solve certain problems much faster than classical computers. Maybe compare it to a game where you can unlock a level or defeat a boss much quicker with a special power-up. But I should also note that not all problems are better solved by quantum computers, just specific ones.

Putting it all together, the explanation should start with what a classical computer does, then introduce qubits as special bits, explain their unique properties using video game analogies, and then talk about the advantages and limitations. I should keep the language simple, use terms they're familiar with, and make it engaging.

Let me think of an example. Maybe a game where you have to find a key in a maze. A classical computer would check each path one by one, while a quantum computer could check all paths at the same time, making it faster. Or maybe a power-up that lets you see all possible paths at once.

I also want to make it clear that it's not just faster, but it's a different way of computing. Maybe use the idea of magic or special abilities in a game that allow for unique capabilities.

I should avoid technical jargon and keep sentences short. Maybe use a narrative style, like telling a story about a game where quantum computing helps the player win. That could make it more relatable and fun.

So, to sum up, I'll start by comparing classical computers to video game rules, then introduce qubits as special bits that can do multiple things at once, explain superposition and entanglement with game-related examples, and discuss the speed advantages and specific applications. I'll wrap it up by connecting it back to video games, showing how quantum computing could make games more complex or fun in the future.

I think that's a solid approach. Now, I'll structure this into a clear, concise explanation using these points and analogies.

RESPONSE:

Quantum computing is like having a supercharged video game character! Imagine you're playing a game where you need to find a hidden treasure in a maze. A regular computer (like a normal game character) would check each path one by one, which could take a long time. But a quantum computer is like a special character that can check all paths at the same time, making it super fast to find the treasure!

In video games, you sometimes get power-ups that give you special abilities. Quantum computers use something called qubits, which are like those power-ups. Unlike regular bits in a computer that can only be 0 or 1, qubits can be both at the same time. This is called superposition. It's like having a character that can be in two places at once!

Also, qubits can work together in a magical way called entanglement. If one qubit changes, the others change too, no matter how far apart they are. It's like having a team of characters where each member's actions affect the others instantly.

Quantum computers are especially good at solving tricky puzzles or problems that would take a regular computer forever. But they're not better at everything—just certain tasks. So, in the game, your special character would shine in specific levels, making them super fun and challenging!

One day, quantum computing might help create even more exciting games, with puzzles that seem impossible to solve without those special powers. So, think of quantum computing as the ultimate power-up for solving complex problems in the blink of an eye!
```

This example shows how the model:

1. First works through its reasoning in the thinking section
2. Then provides a clear, final response
3. Keeps these two aspects separate for better analysis and control

## Key Features

-   **Separated Thinking and Response**: DeepSeek's unique output format provides insight into the model's reasoning process
-   **Custom Output Parsing**: Langchain integration enables structured parsing of the model's unique output format

## Conclusion

This guide has demonstrated how to deploy and serve the DeepSeek-R1-Distill-Qwen-32B model on Vast.ai. We've shown how to:

-   Set up a cost-effective GPU instance with the right specifications
-   Easilly deploy the model using Vast's Templates with vLLM
-   Create a custom Langchain parser to handle DeepSeek's unique thinking/response format
-   Integrate everything through an OpenAI-compatible API

With this setup on Vast.ai's GPU marketplace, you can now build powerful AI applications with DeepSeek while minimizing both infrastructure complexity and compute costs. The platform's combination of flexible GPU options and simple Docker-based deployment makes it an ideal foundation for serving and scaling language models without the expensive cost model of traditional cloud providers.


Deploy DeepSeek-R1-Distill-Qwen-32B on Vast.ai for cost-effective AI reasoning. Use vLLM, Langchain, and custom parsing to extract structured model insights efficiently.

Serving DeepSeek Models on Vast.ai with vLLM and Langchain!


Vast.ai is excited to announce that our GPUs can now be seamlessly rented through SkyPilot, an open-source framework that automates AI, LLM, and batch job execution across multiple clouds. This integration makes it **easy to access Vast.ai's low-cost, high-performance GPUs while SkyPilot handles provisioning, scaling, and cost optimization.** 

Whether you're training deep learning models, fine-tuning LLMs, or running large-scale batch jobs, this partnership provides an efficient and cost-effective solution for developers everywhere.

### Why This Integration Matters

Running large-scale AI workloads can be expensive and resource-intensive, especially as GPU availability fluctuates across providers. With Vast.ai's extensive GPU supply and SkyPilot's intelligent cloud orchestration, you can enjoy streamlined access to affordable, on-demand compute resources.

SkyPilot automatically selects the most cost-effective and available infrastructure, provisioning GPUs across multiple providers – including Vast.ai – to ensure seamless execution of AI and batch jobs at scale. **As the market leader in low-cost cloud GPU rental, Vast.ai will often be the best option here\!**

We're thrilled to be partnering with SkyPilot and becoming part of its "sky" network. To understand why this integration is so powerful, let's take a closer look at what SkyPilot is all about.

### What Is SkyPilot?

Put simply, SkyPilot is a powerful framework for managing cloud workloads in a way that abstracts away infrastructure complexity. It was developed in the UC Berkeley lab that proposed the concept of "sky computing," which is aimed at making multiple cloud providers function as a **single compute pool**.

With SkyPilot, you can run AI models and batch jobs across numerous cloud and Kubernetes environments. This enables you to focus on running workloads efficiently and not worry about infrastructure management.

Along with the new Vast.ai integration, SkyPilot has introduced some major updates with its [release of v0.8](https://github.com/skypilot-org/skypilot/releases/tag/v0.8.0) – including improvements to managed jobs and Kubernetes support, faster GPU provisioning, and adding ready-to-use LLM recipes. 

### Vast.ai's Role

As we've covered, you can now easily access and manage GPU resources on Vast.ai through SkyPilot. **Vast.ai provides a vast supply of cost-effective GPU compute, while SkyPilot automates the selection and provisioning process to match your workload needs** – allowing you to run 2000+ concurrent jobs and scale efficiently.

With Vast.ai's extensive GPU marketplace, you get reliable, affordable compute on demand. SkyPilot enhances this by simplifying deployment, ensuring a seamless experience while maximizing access and failover protection.

### Getting Started with Vast.ai on SkyPilot

Follow [these steps](https://www.google.com/url?q=https://docs.skypilot.co/en/latest/getting-started/installation.html%23vast&sa=D&source=docs&ust=1740510235958694&usg=AOvVaw1n-2VznvD4On0auFV7ffAU) to start using Vast.ai with SkyPilot:

1. **To configure Vast access, go to the Account page on your Vast console to get your API key. Then, run:**

**pip install "vastai-sdk\>=0.1.12"**  
**echo "\<your\_api\_key\_here\>" \> \~/.vast\_api\_key**

### Vast.ai and SkyPilot: An Ongoing Partnership

Vast.ai is committed to ongoing collaboration with SkyPilot. Our shared mission is to help developers navigate cloud resource challenges and access compute power more affordably and efficiently. We're excited about what this integration means for developers and look forward to helping make it easier and more cost-effective to run LLMs, AI, and batch jobs anywhere, anytime.

Special thanks to the SkyPilot team for making this integration possible\!

Easily rent Vast.ai’s low-cost GPUs via SkyPilot for seamless AI, LLM, and batch job execution with automated provisioning, scaling, and cost optimization.

Vast.ai GPUs Can Now Be Rented Through SkyPilot


The growth of AI is outpacing compute. Today, there's an unprecedented demand for computing power as interest in AI surges worldwide – and servers just can't keep up.

More advanced AI models require increasingly powerful hardware. **Without enough GPUs, these large-scale AI systems struggle to run, and also cannot improve.**

Even OpenAI's CEO, Sam Altman, has [admitted](https://techcrunch.com/2024/10/31/openai-ceo-sam-altman-says-lack-of-compute-is-delaying-the-companys-products/) that computing limitations are slowing down the company's ability to release new products. Microsoft's CFO, Amy Woods, [noted](https://www.microsoft.com/en-us/Investor/events/FY-2025/earnings-fy-2025-q1) last year that demand for AI "continues to be higher" than their "available capacity."

If industry giants like OpenAI and Microsoft are feeling the strain, smaller organizations face even steeper challenges. Without deep pockets or dedicated data centers, they often have no choice but to explore other options.

With GPUs in short supply and demand skyrocketing, many companies are turning to a more flexible, scalable solution: **GPU-as-a-service, or GPUaaS**.

## What Is GPU-as-a-Service?

Simply put, **GPUaaS is a cloud-based solution that provides on-demand access to high-performance GPUs.** It fills a gap in the AI industry by enabling customers to rent the compute power they need, scaling up or down as their workloads demand.

Big cloud providers like AWS, Google Cloud, and Azure offer GPU instances, but GPUaaS goes further. Platforms like our own here at Vast.ai tap into a decentralized network of idle GPUs from data centers, enterprises, and even individuals, making high-performance compute more affordable and widely available.

As a result, GPUaaS addresses a critical need: namely, that **AI infrastructure must be scalable, cost-efficient, and accessible to everyone.**

## Why GPUaaS Is the Key to Scalable AI Infrastructure

AI adoption shows no sign of slowing down – and with it comes fluctuating and unpredictable demand for computing power at any given time. Traditional centralized infrastructure isn't well suited to this type of volatility.

Relying on centralized GPU resources often leads to wasted potential and frustrating roadblocks, such as:

-   **High upfront costs** for purchasing and maintaining GPUs.
-   **Underutilized hardware** during slow periods, resulting in inefficiency and wasted costs.
-   **Bottlenecks** when demand spikes, slowing down innovation and delaying results.
-   **Limited access** to GPUs due to supply chain slowdowns and shortages, impacting AI scalability.

This is where GPU-as-a-Service offers a smarter solution. With GPUaaS, you can:

-   **Quickly scale AI workloads** to match demand without over-provisioning.
-   **Reduce costs** with a flexible pay-as-you-go model while eliminating the risks of heavy upfront investment.
-   **Access worldwide GPU resources**, minimizing latency and enhancing performance.
-   **Decrease energy consumption** by leveraging existing, idle GPUs instead of powering additional servers.

## GPUaaS: The Future of AI Compute is Flexible

As AI continues to advance, organizations that can efficiently scale their compute power will have a clear advantage. GPU-as-a-Service reduces the barriers of cost, accessibility, and scalability, offering an alternative to traditional infrastructure. With GPUaaS, you can tap into a global network of GPUs and ensure you always have the resources you need, when you need them.

So why wait? In a world where AI innovation moves fast, compute should never be the bottleneck. **Vast.ai makes enterprise-grade GPUs accessible and affordable through our decentralized marketplace, offering savings of 5-6X on GPU compute.** Whether you're training cutting-edge models or running large-scale inference, we provide the flexibility and efficiency to keep your AI projects moving forward.

Get started and harness the power of GPUaaS with Vast.ai today!

GPU shortages are slowing AI growth. GPU-as-a-Service (GPUaaS) offers a scalable, cost-effective solution for on-demand AI compute power.

Vast.ai

February 2025 Product Update

Voice Activity Detection (VAD) with Pyannote on VAST

Serving DeepSeek Models on Vast.ai with vLLM and Langchain!

Vast.ai GPUs Can Now Be Rented Through SkyPilot

GPU-as-a-Service: The Scalable Solution to AI's Compute Crisis