
Vast.ai GPUs can be seamlessly rented through SkyPilot, an open-source framework that automates AI, LLM, and batch job execution. This integration makes it easy to access Vast.ai's low-cost, high-performance GPUs while SkyPilot handles provisioning, scaling, and cost optimization.

Whether you're training deep learning models, fine-tuning LLMs, or running large-scale batch jobs, this partnership provides an efficient and cost-effective solution for developers everywhere.

## Why This Integration Matters

Running large-scale AI workloads can be expensive and resource-intensive, especially as GPU availability fluctuates across providers. With Vast.ai's extensive GPU supply and SkyPilot's intelligent cloud orchestration, you can enjoy streamlined access to affordable, on-demand compute resources.

SkyPilot automatically selects the most cost-effective and available infrastructure, provisioning GPUs to ensure seamless execution of AI and batch jobs at scale. As the market leader in low-cost cloud GPU rental, Vast.ai provides an ideal foundation for this approach.

## What Is SkyPilot?

Put simply, SkyPilot is a powerful framework for managing cloud workloads in a way that abstracts away infrastructure complexity. It was developed in the UC Berkeley lab that proposed the concept of "sky computing," which aims to make cloud providers function as a single compute pool.

With SkyPilot, you can run AI models and batch jobs efficiently without worrying about infrastructure management. SkyPilot includes improvements to managed jobs and Kubernetes support, faster GPU provisioning, and ready-to-use LLM recipes.

## Vast.ai's Role

You can easily access and manage GPU resources on Vast.ai through SkyPilot. Vast.ai provides a vast supply of cost-effective GPU compute, while SkyPilot automates the selection and provisioning process to match your workload needs – allowing you to run thousands of concurrent jobs and scale efficiently.

With Vast.ai's extensive GPU marketplace, you get reliable, affordable compute on demand. SkyPilot enhances this by simplifying deployment, ensuring a seamless experience while maximizing access and failover protection.

## Getting Started: Practical Tutorial

Now let's walk through a complete example of deploying a language model on Vast.ai using SkyPilot.

### Setting Up the Environment

First, install SkyPilot with Vast.ai support:

```bash
pip install -U "skypilot[vast]"
pip install "vastai-sdk>=0.1.12"
```

Configure your Vast.ai API key:

```bash
# Get your API key from https://vast.ai → Account → API Keys
echo "YOUR_VAST_API_KEY_HERE" > ~/.vast_api_key
chmod 600 ~/.vast_api_key

# Verify setup
sky check
```

Expected output:

```
Vast: enabled ✓
```

### Creating the YAML Configuration

Create a file named `deepseek-r1-inference.yaml`:

```yaml
resources:
  accelerators: H100:1
  cloud: vast
  image_id: docker:vllm/vllm-openai:latest
  disk_size: 100

run: |
  vllm serve deepseek-ai/DeepSeek-R1-0528-Qwen3-8B \
    --host 0.0.0.0 \
    --port 8000 \
    --max-model-len 4096 \
    --gpu-memory-utilization 0.9 \
    --reasoning-parser deepseek_r1
```

This configuration:

- Requests 1x H100 GPU on Vast.ai
- Uses the pre-built vLLM Docker image
- Allocates 100GB disk space for the model
- Serves the DeepSeek R1 model on port 8000

### Deploying the Service

Launch your service with SkyPilot:

```bash
sky launch deepseek-r1-inference.yaml --cluster deepseek-r1
```

This command will:

1. Find an available H100 GPU on Vast.ai
2. Provision the instance
3. Download the DeepSeek R1 model (~15GB)
4. Start the vLLM API server

The deployment takes 5-10 minutes. Monitor progress with:

```bash
sky logs deepseek-r1 --follow
```

Verify deployment:

```bash
sky status deepseek-r1
```

You should see `Status: UP` when ready.

### SSH Port Forwarding

Since direct port mapping isn't available, use SSH tunneling to access your service:

```bash
ssh -L 8000:localhost:8000 deepseek-r1
```

**Keep this terminal open!** The tunnel needs to stay active for API access.

### Testing the API

In a new terminal, test your deployed service:

```bash
# Test health endpoint
curl http://localhost:8000/health

# List available models
curl http://localhost:8000/v1/models
```

#### Python Client Example

```python
import openai

# Configure client for your local tunnel
client = openai.OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="sk-fake-key"  # vLLM doesn't require real API key
)

# Make a simple request
response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{
        "role": "user",
        "content": "Write a brief history of Los Angeles in one paragraph."
    }],
    max_tokens=300,
    temperature=0.7
)

print(response.choices[0].message.content)
```

### Managing Your Deployment

```bash
# Check status
sky status deepseek-r1

# Stop cluster (saves money)
sky stop deepseek-r1

# Start stopped cluster
sky start deepseek-r1

# Terminate completely
sky down deepseek-r1
```

## Conclusion

You now have a production-ready AI API running on Vast.ai's cost-effective H100 GPUs, managed through SkyPilot's streamlined interface. This combination provides enterprise-grade AI capabilities at a fraction of traditional cloud costs.

The integration between Vast.ai and SkyPilot represents a powerful solution for developers who need reliable, affordable GPU access without the complexity of manual infrastructure management. Start experimenting with your own AI workloads using this cost-effective stack!


Easily rent Vast.ai’s low-cost GPUs via SkyPilot for seamless AI, LLM, and batch job execution with automated provisioning, scaling, and cost optimization.

Vast.ai GPUs Can Now Be Rented Through SkyPilot


## Introduction

OpenAI has just released the GPT-OSS family of models, marking their return to open-weight model releases. The family includes GPT-OSS-120B (120 billion parameters) and GPT-OSS-20B (20 billion parameters), offering developers access to state-of-the-art language capabilities previously locked behind proprietary APIs.

GPT-OSS offers configurable reasoning capabilities with reasoning levels - from "low" for quick responses to "high" for complex problem-solving. The model uses OpenAI's harmony encoding system, which structures conversations and reasoning in a sophisticated format that enables fine-grained control over the model's thinking process.

This breakthrough means you can now:

-   Deploy OpenAI-quality models on your own infrastructure
-   Scale inference according to your exact needs

Vast.ai provides the perfect platform for running GPT-OSS models. With its marketplace of GPUs, you can choose the right hardware for your needs - whether running the efficient 20B model or the powerful 120B variant.

In this guide, we'll show you how to deploy both GPT-OSS models on Vast.ai using vLLM for optimized inference, with a focus on the 120B model. You'll learn how to interact with these models using the harmony encoding system for different reasoning levels.

## Setting Up the Environment

Before we can deploy our model, we need to set up our Vast.ai environment. First, install the Vast SDK:

```bash
# Install required packages
pip install --upgrade vastai
pip install --upgrade openai
pip install --upgrade openai-harmony
```

Set up your Vast API key (available from your [Account Page](https://cloud.vast.ai/account/)):

```bash
# Set your Vast.ai API key
export VAST_API_KEY="" # Your key here
vastai set api-key $VAST_API_KEY
```

## Choosing the Right Hardware

The GPT-OSS models have different hardware requirements:

-   **GPT-OSS-120B**: Requires an H100 GPU with 80GB VRAM
-   **GPT-OSS-20B**: Requires 16GB+ VRAM

For this guide, we'll demonstrate with the 120B model on H100. Let's search for suitable instances:

```bash
# Search for suitable GPU instances
vastai search offers " \
gpu_name = H100_SXM \
geolocation=US \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 120 \
rentable = true"
```

## Deploying GPT-OSS with vLLM

Now let's deploy our GPT-OSS instance using vLLM's optimized inference server. We'll use the latest vLLM image with special GPT-OSS support. The pip install commands are from the model's Hugging Face pages ([120B](https://huggingface.co/openai/gpt-oss-120b), [20B](https://huggingface.co/openai/gpt-oss-20b)).

### Deploying GPT-OSS-120B (H100 recommended)

```bash
# Deploy vLLM instance for 120B model
export INSTANCE_ID= # Insert instance ID from search results

vastai create instance $INSTANCE_ID --image vllm/vllm-openai:latest --env '-p 8000:8000' --disk 160  --onstart-cmd 'uv pip install --system --upgrade transformers kernels torch openai; uv pip install --system --pre vllm==0.10.1+gptoss --extra-index-url https://wheels.vllm.ai/gpt-oss/ --extra-index-url https://download.pytorch.org/whl/nightly/cu128; vllm serve openai/gpt-oss-120b --max-model-len 80000'
```

### Alternative: Deploying GPT-OSS-20B on A100

Here's how to deploy the 20B model on an A100 GPU:

```bash
# Deploy vLLM instance for 20B model
export INSTANCE_ID= # Insert instance ID from search results

vastai create instance $INSTANCE_ID --image vllm/vllm-openai:latest --env '-p 8000:8000' --disk 80  --onstart-cmd 'uv pip install --system --upgrade transformers kernels torch openai; uv pip install --system --pre vllm==0.10.1+gptoss --extra-index-url https://wheels.vllm.ai/gpt-oss/ --extra-index-url https://download.pytorch.org/whl/nightly/cu128; vllm serve openai/gpt-oss-20b'
```

After deployment, wait for the model to download and start serving. You can monitor the logs to see when it's ready. Once running, find your instance's IP address and port from the [Instances tab](https://cloud.vast.ai/instances/) in the Vast AI Console.

## Interacting with GPT-OSS Using Harmony Encoding

GPT-OSS uses OpenAI's harmony encoding system, which provides structured conversation formatting and enables different reasoning levels. Let's set up our client:

```python
from openai import OpenAI
from openai_harmony import (
    load_harmony_encoding,
    HarmonyEncodingName,
    Role,
    Message,
    Conversation,
    SystemContent,
    DeveloperContent,
)

# Your server details
VAST_IP_ADDRESS = ""
VAST_PORT = ""

# Initialize client
client = OpenAI(
    api_key="EMPTY",
    base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
)

# Load harmony encoding
enc = load_harmony_encoding(HarmonyEncodingName.HARMONY_GPT_OSS)

## Simple Chat Function
def chat_gpt_oss(prompt, reasoning="medium"):
    """
    Simple chat with GPT-OSS
    reasoning: "low", "medium", or "high"
    """
    # Create conversation with harmony format
    conversation = Conversation.from_messages([
        Message.from_role_and_content(Role.SYSTEM, SystemContent.new()),
        Message.from_role_and_content(
            Role.DEVELOPER,
            DeveloperContent.new().with_instructions(f"Reasoning: {reasoning}")
        ),
        Message.from_role_and_content(Role.USER, prompt),
    ])

    # Get formatted prompt
    formatted_prompt = enc.render_conversation_for_completion(conversation, Role.ASSISTANT)

    # Make API call
    response = client.completions.create(
        model="openai/gpt-oss-120b",
        prompt=formatted_prompt,
        max_tokens=2000,
        temperature=0.7,
    )

    return response.choices[0].text


prompt = "Explain quantum computing to a 4th grader. Keep in concise."
```

## Comparing Reasoning Levels

One of GPT-OSS's most interesting features is its configurable reasoning levels. Let's see how the model responds differently to the same prompt with low and high reasoning:

### Low Reasoning Response

```python
result = chat_gpt_oss(prompt, "low")
print("----LOW REASONING----")
print(result)
```

Output:

```
----LOW REASONING----
analysisWe need to explain quantum computing to a 4th grader, concise. Use simple analogies, like bits as light switches, qubits as coins that can be heads, tails, or both. Keep short.assistantfinal**Quantum Computing in a Nutshell (for a 4th‑grader)**

1. **Regular computers = light switches**
   - A normal computer works with bits that are either **0** (off) or **1** (on), just like a light switch that's either down or up.

2. **Quantum computers = magic coins**
   - A quantum computer uses **qubits**.
   - Think of a qubit like a coin that can be **heads (0)**, **tails (1)**, **or both at the same time** while it's spinning in the air. That "both at once" is called **superposition**.

3. **Working together**
   - When you have many regular bits, you have to try every possible combination one after another.
   - With many qubits, they can be in many combinations **all at once** because they can be linked together in a special way called **entanglement**. It's like a group of magic coins that always know what the others are doing, even if they're far apart.

4. **Why it matters**
   - Because of superposition and entanglement, a quantum computer can solve some puzzles (like cracking codes or simulating molecules) **much faster** than a regular computer.

**Bottom line:** A regular computer is like a row of on/off switches; a quantum computer is like a bunch of spinning magic coins that can be many things at once, letting it try many solutions at the same time.
```

### High Reasoning Response

```python
result = chat_gpt_oss(prompt, "high")
print("----HIGH REASONING----")
print(result)
```

Output:

```
----HIGH REASONING----
analysisWe need to explain quantum computing to a 4th grader, concise. Use simple analogies, avoid jargon, but keep accurate. Should be short. Provide analogies like coins, dice, superposition, entanglement. Keep concise.assistantfinal**What is a quantum computer?**

Imagine you have a super‑smart robot that can solve puzzles much faster than a regular computer. A quantum computer is that robot, but instead of using normal "on/off" switches (bits) like a regular computer, it uses tiny, magical switches called **qubits**.

### Bits vs. Qubits
- **Bit (regular computer):** Like a tiny light that can be **off (0)** or **on (1)**.
- **Qubit (quantum computer):** Like a tiny spinning top that can be **0**, **1**, **or both at the same time** (this is called *superposition*).

### Why is that cool?
If you have 10 bits, they can show only one of 2¹⁰ = 1,024 possible patterns at a time.
If you have 10 qubits, they can be in all 2¹⁰ = 1,024 patterns **at once**! That lets a quantum computer try many solutions together.

### A Simple Analogy
Think of a maze puzzle:

- **Regular computer:** Sends a single mouse down the maze, step by step, until it finds the exit.
- **Quantum computer:** Sends a whole swarm of mice that can explore every possible path at the same time and tell you the exit almost instantly.

### Entanglement (Teamwork)
Sometimes two qubits become "best friends" and link together, no matter how far apart they are. Changing one instantly changes the other. This is called **entanglement**, and it lets quantum computers coordinate their work in a super‑efficient way.

### Bottom Line
A quantum computer uses qubits that can be 0, 1, or both at once, and they can be entangled with each other. This lets the machine look at many possibilities at the same time, solving certain problems way faster than ordinary computers.

So, it's like a super‑fast, super‑clever puzzle‑solver that can try lots of answers all at once!
```

## Comparing Low vs High Reasoning Outputs

The quantum computing explanations demonstrate the distinct capabilities of GPT-OSS-120B's reasoning levels. With low reasoning, the model produced a concise, structured explanation using simple analogies like "magic coins" and "light switches" - delivering clear concepts in a compact format suitable for quick understanding.

The high reasoning response revealed significantly more sophisticated thinking. The model provided detailed analogies like the maze puzzle with mice, explained mathematical concepts (2¹⁰ patterns), and structured the explanation with clear sections and formatting. Most notably, it offered multiple perspectives on the same concepts - from the spinning top analogy to the teamwork explanation of entanglement.

This difference in reasoning depth represents a significant advancement in model control. Rather than simply adjusting temperature or other parameters, GPT-OSS allows direct specification of how thoroughly the model should think through problems. The low reasoning approach prioritized efficiency while maintaining accuracy, while high reasoning prioritized comprehensive understanding with rich examples and detailed explanations.

## Conclusion

OpenAI's first open-source model thus introduces not just accessibility to advanced language capabilities, but a new paradigm for controlling model reasoning depth - enabling applications to dynamically adjust between quick responses and thorough analysis based on context and requirements.


Learn how to deploy and run OpenAI's GPT-OSS models on Vast.ai using vLLM, with configurable reasoning levels and harmony encoding system.

Running OpenAI's GPT-OSS on Vast.ai


The DeepSeek R1-0528 release introduces a significant improvement to the reasoning capabilities of language models by making it easier to access "thinking" mode without requiring complex prompt engineering or pre-pending thinking tokens. This enhanced model provides transparent, step-by-step reasoning that's particularly valuable for educational applications, complex problem solving, and scenarios where transparency in AI decision-making is crucial.

In this post, we'll explore how to deploy the **DeepSeek-R1-0528-Qwen3-8B** model using **vLLM** on Vast.ai's cloud GPU platform, leveraging the new **qwen3 reasoning parser** that simplifies access to the model's internal thinking process.

---

## The Open Source AI Revolution

DeepSeek R1-0528 represents more than just a technical advancement—it's a significant step toward democratizing advanced AI capabilities. As an **open source model**, it makes these reasoning and advanced AI capabilities more accessible.

Unlike closed-source models that require vendor dependence, usage costs, and data privacy concerns, DeepSeek R1-0528 enables **complete control** over your AI infrastructure. Organizations can now deploy advanced reasoning capabilities locally, maintaining data sovereignty while avoiding variable per-token pricing.

This democratization means organizations can access cutting-edge AI without massive infrastructure investments. Advanced reasoning capabilities are no longer exclusive to large tech companies—anyone with GPU access can deploy their own reasoning-capable AI system.

---

## What Makes DeepSeek R1-0528 Special?

The DeepSeek R1-0528 release addresses a key limitation in previous DeepSeek models: the complexity of accessing the model's internal reasoning process. Previously, users needed to manually prepend "thinking" tokens to model outputs or use complex prompt engineering to see step-by-step reasoning.

---

## Deploying DeepSeek R1-0528 on Vast.ai

### Step 1: Set Up Vast.ai Environment

First, install the Vast CLI and configure your API key:

```bash
pip install --upgrade vastai

export VAST_API_KEY="your_vast_api_key"
vastai set api-key $VAST_API_KEY
```

### Step 2: Find Suitable Hardware

The DeepSeek-R1-0528-Qwen3-8B model requires specific hardware capabilities:

-   **Minimum 24GB GPU RAM** for model weights and KV cache
-   **Single GPU configuration** (sufficient for the 8B parameter model)
-   **Static IP address** for stable API endpoint hosting
-   **Direct port access** for the vLLM OpenAI-compatible server
-   **60GB+ disk space** for model storage and dependencies

Search for suitable instances:

```bash
vastai search offers "compute_cap >= 750 \
gpu_ram >= 24 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 60 \
rentable = true"
```

### Step 3: Deploy the Model

Deploy using the vLLM OpenAI-compatible server with the qwen3 reasoning parser:

```bash
export INSTANCE_ID="your_instance_id"

vastai create instance $INSTANCE_ID \
--image vllm/vllm-openai:latest \
--env '-p 8000:8000' \
--disk 60 \
--args --model deepseek-ai/DeepSeek-R1-0528-Qwen3-8B \
--max-model-len 4096 \
--reasoning-parser qwen3
```

**Key deployment parameters:**

-   `--reasoning-parser qwen3`: Enables the simplified reasoning parser
-   `--max-model-len 4096`: Accommodates longer reasoning sequences
-   `--tensor-parallel-size 1`: Single GPU configuration
-   `--gpu-memory-utilization 0.90`: Optimized memory usage

### Step 4: Get Instance Details and Test Connection

Navigate to the [Instances tab](https://cloud.vast.ai/instances/) in the Vast AI Console and locate your instance. Click the IP address button to view the forwarded ports.

Test the connection with a simple curl command:

```bash
export VAST_IP_ADDRESS="your_ip_address"
export VAST_PORT="your_port"

curl -X POST http://$VAST_IP_ADDRESS:$VAST_PORT/v1/completions \
-H "Content-Type: application/json" \
-d '{"model": "deepseek-ai/DeepSeek-R1-0528-Qwen3-8B", "prompt": "Hello, how are you?", "max_tokens": 50}'
```

---

## Testing the Model's Reasoning Capabilities

### Setup Python Client

```python
import time
from openai import OpenAI

# Replace with your actual Vast.ai instance details
VAST_IP_ADDRESS = "your_ip_address"
VAST_PORT = "your_port"

client = OpenAI(
    api_key="DUMMY",
    base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
)
```

### Basic Functionality Test

```python
start = time.time()
test_resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": "Hello"}]
)
elapsed = time.time() - start

print("Response:", test_resp.choices[0].message.content)
print(f"Elapsed time: {elapsed:.4f} s")
```

### Reasoning Capability Test

This is where DeepSeek R1-0528 truly shines. The model naturally provides step-by-step reasoning without complex prompt engineering:

```python
reasoning_prompt = """
Write the History of Paris in a paragraph, show your thinking
step by step first before paragraph and then write the history
in a paragraph
"""

start = time.time()
resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": reasoning_prompt}]
)
elapsed = time.time() - start

print("Model output:\n", resp.choices[0].message.content)
print(f"Elapsed time: {elapsed:.3f} s")
```

**Expected output structure:**

1. **Thinking process**: The model shows its step-by-step reasoning
2. **Final answer**: A well-structured paragraph about Paris's history

### Mathematical Reasoning Test

```python
math_prompt = "If 8 × 7 = ?, show your work step by step."

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": math_prompt}]
)

print("Mathematical reasoning:\n", resp.choices[0].message.content)
```

The model will naturally break down the problem, show intermediate steps, and arrive at the solution with clear reasoning traces.

---

## Key Benefits of DeepSeek R1-0528

### 1. **Simplified Reasoning Access**

No need to prepend thinking tokens or use complex prompt engineering. The `qwen3` reasoning parser automatically handles the extraction and presentation of reasoning steps.

### 2. **Educational Value**

Perfect for educational applications where students need to understand the problem-solving process, not just the final answer.

### 3. **Transparency and Trust**

Users can see exactly how the model arrives at its conclusions, building trust in AI-powered decision-making systems.

### 4. **Cost-Effective Deployment**

Running on Vast.ai provides significant cost savings compared to major cloud providers while maintaining high performance.

### 5. **Easy Integration**

OpenAI-compatible API ensures seamless integration with existing applications and workflows.

---

## Use Cases

### Educational Applications

-   Step-by-step problem solving for mathematics, science, and logic
-   Teaching critical thinking and reasoning skills
-   Creating interactive learning experiences

### Business Intelligence

-   Transparent decision-making in automated systems
-   Audit trails for AI-powered recommendations
-   Complex analysis with clear reasoning paths

### Research and Development

-   Understanding model behavior and capabilities
-   Testing reasoning patterns across different domains
-   Developing more interpretable AI systems

---

## Conclusion

The DeepSeek R1-0528 release represents a significant step forward in making AI reasoning more accessible and transparent. By combining this powerful model with Vast.ai's cost-effective GPU infrastructure and the simplified qwen3 reasoning parser, developers can now easily deploy transparent, reasoning-capable AI systems.

Whether you're building educational tools, business intelligence systems, or research applications, DeepSeek R1-0528 on Vast.ai provides an excellent foundation for applications that require clear, step-by-step reasoning capabilities.

The elimination of complex prompt engineering for accessing reasoning modes makes this model particularly valuable for production applications where transparency and interpretability are key requirements.

---

## Useful Resources

-   [DeepSeek R1-0528 Model on Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B)
-   [vLLM Documentation](https://docs.vllm.ai/)
-   [Vast.ai GPU Marketplace](https://vast.ai)
-   [OpenAI Python SDK](https://github.com/openai/openai-python)

---

_Ready to deploy transparent, reasoning-capable AI? Get started with DeepSeek R1-0528 on Vast.ai today! 🚀_


Deploy DeepSeek R1-0528 on Vast.ai to access transparent, step-by-step AI reasoning without prompt tricks—ideal for education, research, and business use.

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai


Deep Cogito models are hybrid reasoning Large Language Models (LLMs) uniquely designed to operate in two distinct modes within a single deployment: standard direct answering and detailed step-by-step reasoning. Unlike traditional models that require separate deployments for different reasoning capabilities, Deep Cogito can seamlessly toggle between these modes using simple prompt engineering.

This guide demonstrates how to deploy `deepcogito/cogito-v1-preview-llama-8B` on Vast.ai and leverage its dual reasoning capabilities through vLLM's OpenAI-compatible API.

## Deploying on Vast.ai

First, install the Vast.ai SDK and configure your API key:

```bash
pip install --upgrade vastai
export VAST_API_KEY="YOUR_KEY_HERE"
vastai set api-key $VAST_API_KEY
```

Search for a suitable instance that meets the model's requirements:

```bash
vastai search offers "compute_cap >= 750 \
geolocation=US \
gpu_ram >= 60 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 80 \
rentable = true"
```

Deploy the model using vLLM's OpenAI-compatible API:

```bash
vastai create instance YOUR_INSTANCE_ID --image vllm/vllm-openai:latest --env '-p 8000:8000' --disk 80 --args --model deepcogito/cogito-v1-preview-llama-8B
```

## Using Deep Cogito with OpenAI SDK

Install the OpenAI SDK:

```bash
pip install --upgrade openai
```

### Enabling Thinking Mode

Deep Cogito provides two methods to toggle thinking mode:

1. **Hugging Face Transformers Library Method**:
   When using the Hugging Face Transformers library directly, you can enable thinking mode using the `enable_thinking=True` flag with the tokenizer's `apply_chat_template` method. This activates the model's reasoning process, displaying its thought process within `<thinking>` tags before providing the final answer.

2. **vLLM OpenAI-compatible API Method**:
   When using vLLM's OpenAI-compatible API (as in our Vast.ai deployment), use a system prompt instead. The official prompt is:
   ```
   "Enable deep thinking subroutine."
   ```
   
   For the 8B model specifically, we recommend using this enhanced version for more consistent results:
   ```
   "Enable deep thinking subroutine. \n\n YOU MUST USE THE THINKING SUBROUTINE"
   ```

Here's an implementation example using the OpenAI SDK:

```python
from openai import OpenAI

# Vast.ai instance configuration
VAST_IP_ADDRESS = "YOUR_VAST_IP"
VAST_PORT = "YOUR_VAST_PORT"

# Initialize the OpenAI client
client = OpenAI(
    api_key="EMPTY",  # vLLM doesn't require an API key
    base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
)

# Function for reasoning mode
def chat_with_model_reasoning(prompt):
    response = client.chat.completions.create(
        model="deepcogito/cogito-v1-preview-llama-8B",
        messages=[
            {"role": "system", "content": "Enable deep thinking subroutine. \n\n YOU MUST USE THE THINKING SUBROUTINE"},
            {"role": "user", "content": prompt}
        ],
    )
    return response.choices[0].message.content

# Function for standard mode
def chat_with_model(prompt):
    response = client.chat.completions.create(
        model="deepcogito/cogito-v1-preview-llama-8B",
        messages=[
            {"role": "user", "content": prompt}
        ],
    )
    return response.choices[0].message.content
```

## Response Comparison

The following examples demonstrate how the model's responses differ between thinking mode and standard mode when asked to explain Large Language Models:

### With Thinking Mode:

```
<think>
Okay, I need to give a short introduction to Large Language Models (LLMs). Let me start by recalling what I know about them. LLMs are a type of artificial intelligence that uses neural networks to process and generate human-like text. They're trained on massive amounts of text data from the internet, books, and other sources. This training allows them to understand and generate contextually relevant responses.

Wait, I should probably mention that the training process involves machine learning, specifically deep learning. They use transformer architectures, which were introduced by Google in 2017. Transformers revolutionized NLP by allowing the model to process input in parallel, which makes them more efficient than previous sequential models.

But maybe I'm getting a bit too technical. The user might just want a basic overview. Let me focus on the key aspects. LLMs are known for their ability to understand natural language and generate coherent text. They can engage in conversations, answer questions, write stories, translate languages, and even help with coding. But they also have limitations, like sometimes providing incorrect or biased information.

I should also mention some popular examples, like GPT-3, BERT, and the more recent ones like GPT-4. These models vary in size and capability. For instance, GPT-3 has 175 billion parameters, making it one of the largest models. GPT-4 is even bigger and claimed to have more capabilities, though it's not publicly available yet.

Also, LLMs are used in various applications, such as chatbots, virtual assistants, and content generation tools. They can help with tasks like summarization, question answering, and sentiment analysis. But it's important to note that they're not perfect and can have issues like hallucinations (making up information) or biases inherited from the training data.

Hmm, I might need to simplify this a bit. Maybe start with a basic definition and then highlight their key features and applications. Avoid too much technical jargon unless necessary.

Wait, I should also mention the training data aspect. LLMs learn from enormous datasets, which means they can pick up on language patterns, idioms, and even cultural references. But this also means they can reflect biases present in the data, which is a concern.

In summary, LLMs are advanced AI models that can understand and generate human-like text using deep learning techniques. They excel at language tasks but have limitations. They're widely used in various applications but also require careful consideration of their ethical implications.
</think>

Large Language Models (LLMs) are advanced AI systems that can process and understand language in ways similar to human intelligence. These models use neural networks and are trained on vast amounts of text data from the internet, books, and other sources.

Here are the key things you should know about LLMs:

1. **How they work**: LLMs use a technology called transformer architecture, which allows them to process language in parallel and maintain context in conversations. This means they can understand what you're saying and respond appropriately.

2. **Capabilities**: LLMs are incredibly versatile, able to:
   - Engage in conversations
   - Answer questions
   - Write stories and articles
   - Help with coding
   - Translate languages
   - Summarize text
   - Create content

3. **Examples**: Popular LLMs include GPT-3, BERT, and ChatGPT. These models vary in size and capability, but they all share the ability to understand and generate human-like text.

4. **Important limitations**: While powerful, LLMs aren't perfect. They can:
   - Make up information (hallucinations)
   - Reflect biases from their training data
   - Sometimes provide incorrect answers

5. **Applications**: LLMs are used in:
   - Chatbots
   - Virtual assistants
   - Content generation tools
   - Language translation services
   - Text analysis and processing

As these models continue to evolve, they're becoming increasingly important in our digital world, though we need to remain mindful of their limitations and potential biases.
```

### Without Thinking Mode:

```
Large Language Models (LLMs) are AI systems that use neural networks to process and generate human-like text. They're pre-trained on massive datasets of text and can understand context, generate coherent responses, and perform various language tasks like translation, summarization, and question-answering.

The most well-known LLMs include GPT (Generative Pre-trained Transformer) models created by OpenAI, BERT (Bidirectional Encoder Representations from Transformers) created by Google, and others. They've become increasingly useful for applications like chatbots, content generation, and language understanding tasks.

Key characteristics of LLMs include their ability to understand context, generate human-like text, and adapt to different tasks, though they can also sometimes produce incorrect or biased information.
```

## Conclusion

Deep Cogito on Vast.ai offers a cost-effective solution for accessing a powerful reasoning LLM. The system prompt technique enables visibility into the model's reasoning process, making it particularly valuable for complex tasks such as coding, mathematical problems, and detailed analysis.

While the `enable_thinking` flag isn't available through vLLM's API, the system prompt approach can be highly effective with proper configuration. Users should experiment with prompt variations to achieve optimal results, as factors such as emphasis and repetition can significantly impact the consistency of the thinking mode, especially with the 8B model.

For production environments where understanding the model's reasoning process is crucial, this approach provides a robust and economical solution. We recommend testing different prompt variations to find the optimal configuration for your specific use case, as the effectiveness of thinking mode can vary depending on the complexity of the task and the chosen model size.


Learn how to deploy and interact with Deep Cogito’s hybrid reasoning LLM (cogito-v1-preview-llama-8B) on Vast.ai. This guide covers both standard and deep thinking modes via vLLM’s OpenAI-compatible API.

Running Deep Cogito on Vast.ai


## Introduction

In multi-speaker audio recordings like meetings, podcasts, or interviews, knowing who spoke when is crucial for many applications. This process, known as Speaker Diarization, partitions an audio stream into segments according to speaker identity.

[PyAnnote Audio](https://github.com/pyannote/pyannote-audio), an open-source toolkit built on PyTorch, offers state-of-the-art models for speaker diarization that are both accurate and accessible. Running these models on Vast.ai provides a cost-effective solution for processing large audio datasets without investing in expensive hardware. This combination gives developers and researchers the tools they need to build sophisticated audio processing pipelines at a fraction of the cost of traditional cloud providers.

### Why Speaker Diarization Matters

Speaker Diarization provides several key benefits for audio processing pipelines:

1. **Speaker Identification**: It identifies different speakers in a conversation, meeting, or any multi-speaker audio recording.

2. **Improved Transcription**: When combined with speech-to-text systems, diarization allows for speaker-attributed transcripts, making it clear who said what.

3. **Processing Efficiency**: By segmenting audio by speaker and removing non-speech portions, diarization can significantly reduce the computational load for downstream tasks like speech recognition, allowing these systems to process only relevant speech segments rather than the entire audio file.

4. **Audio Indexing**: Makes audio content searchable by speaker, allowing users to find all segments where a specific person speaks.

### What This Guide Covers

In this guide, we will:

-   Set up the Pyannote Audio Speaker Diarization pipeline
-   Process audio files to detect different speakers and their speaking turns
-   Calculate speaking time for each identified speaker
-   Identify regions with overlapping speech
-   Extract and save speaker-specific segments from the input audio
-   Play and verify the diarization results

The output will be a collection of audio files separated by speaker, making them ready for further processing in speech-to-text pipelines or speaker-specific analysis.

### Why VAST.ai

VAST.ai offers a marketplace approach to GPU rentals that provides significant advantages for audio processing tasks. Unlike traditional cloud providers, VAST.ai allows you to:

-   Rent precisely the GPU capacity needed for your workload
-   Access GPUs at more affordable rates
-   Access more types of GPU SKUs, particularly lower RAM/more affordable GPUs
-   Avoid long-term commitments for experimental projects

For speaker diarization specifically, VAST.ai is particularly well-suited as these models typically require GPU acceleration but don't demand extensive resources. Users can rent just the right amount of computing power for this specific task without overpaying for unused capacity, making it an economical choice for audio processing workflows that might otherwise be cost-prohibitive on traditional cloud platforms.

## Download This Notebook

To follow along with this tutorial, you can download the complete Jupyter notebook:

[Notebook](https://nbviewer.org/urls/bitbucket.org/%21api/2.0/snippets/jsbcannell/q7MzLr/ee586f7e9bfbcdb2fb48f059b7bf64caf92fa3bd/files/pyannote_diarization_vast.ipynb)

Having the notebook will allow you to execute the code blocks as you read through this guide, making it easier to understand and experiment with the diarization implementation.

## Choosing an Instance

For running the Pyannote Speaker Diarization model on VAST.ai, you'll need a relatively modest GPU setup. The pyannote/speaker-diarization-3.1 model runs in pure PyTorch and is designed to be efficient. Here are the recommended specifications:

-   GPU: A low-end GPU like an RTX 3060 or 4060 would be sufficient.
-   VRAM: 6-8GB of VRAM should be adequate as the Pyannote diarization pipeline is relatively efficient.
-   RAM: 8-16GB system RAM is recommended for processing audio files.
-   Storage: At least 10GB for the model, dependencies, and your audio files.
-   CUDA: Make sure the instance has CUDA installed (version 11.0+ recommended).
-   Python: Python 3.8+ with PyTorch installed.

## Selecting an Instance

Follow these steps to set up your environment:

1. Ensure that you have a Vast.ai account
2. Go to the Vast Templates in the Console https://cloud.vast.ai/templates/
3. Select the `PyTorch (CuDNN Runtime)` Template
4. Filter for an instance with:
    - 1 GPU
    - 6-8GB of VRAM
    - 8-16GB system RAM
    - 10GB of storage
5. Select an instance and click rent
6. Install the Vast TLS certificate in your browser to access the notebook server https://docs.vast.ai/instances/jupyter#1SmCz
7. Go to your Instances https://cloud.vast.ai/instances/ and click "Open" to access the jupyter server on your instance.
8. Upload the notebook to the server or create a new notebook

## Installing Dependencies

Let's start by installing the necessary Python packages:

```python
%%bash
pip install pyannote.audio
pip install pydub
pip install librosa
pip install datasets
```

We also need to install FFmpeg for audio processing:

```python
%%bash
apt-get update && apt-get install -y ffmpeg
```

## Setting Up Your Hugging Face Token

Pyannote models are hosted on Hugging Face, so you'll need to set up authentication:

```python
# Make sure you've accepted the user conditions at:
# https://huggingface.co/pyannote/speaker-diarization-3.1
# https://huggingface.co/pyannote/segmentation-3.0

HF_TOKEN = ""  # Add your token here
```

Ensure that you have accepted the terms for the models at the URLs above. The models are free to use, but you must agree to their terms of service.

## Downloading Test Data

For this tutorial, we'll use a sample file from the AMI Meeting Corpus dataset, which is a collection of 100 hours of meeting recordings. This dataset is perfect for testing speaker diarization as it contains natural multi-speaker conversations.

```python
from datasets import load_dataset
import os
import soundfile as sf

# Create a directory to save the files
os.makedirs("ami_samples", exist_ok=True)

# Load the dataset with the correct split
dataset = load_dataset("diarizers-community/ami", "ihm", split="train", streaming=True)

# Load just one sample
n_samples = 1
samples = list(dataset.take(n_samples))

for i, sample in enumerate(samples):
    audio = sample["audio"]
    audio_array = audio["array"]
    sampling_rate = audio["sampling_rate"]

    # Calculate duration in seconds
    duration = len(audio_array) / sampling_rate

    # Use soundfile to save the audio
    output_path = f"ami_samples/sample_{i}.wav"
    sf.write(output_path, audio_array, sampling_rate)

    print(f"Saved {output_path} - Speaker: {sample['speakers']} - Duration: {duration:.2f} seconds")
```

## Speaker Diarization

First, let's set up the Speaker Diarization pipeline:

```python
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

from pyannote.audio import Pipeline

pipeline = Pipeline.from_pretrained(
        "pyannote/speaker-diarization-3.1",
        use_auth_token=HF_TOKEN
)

# Move pipeline to appropriate device
pipeline = pipeline.to(device)
```

Next, we'll process our audio file to identify different speakers and their speaking turns:

```python
# Process the audio file
audio_file = "./ami_samples/sample_0.wav"
print(f"Processing {audio_file} on {device}")
output = pipeline(audio_file)

print("Voice activity segments:")

for segment, _, speaker in output.itertracks(yield_label=True):
    result = f"{segment.start:.2f} --> {segment.end:.2f} (duration: {segment.duration:.2f}s) Speaker: {speaker}"
    print(result)
```

The output will look something like this:

```
Processing ./ami_samples/sample_0.wav on cuda
Voice activity segments:
18.36 --> 18.42 (duration: 0.07s) Speaker: SPEAKER_03
23.01 --> 25.63 (duration: 2.62s) Speaker: SPEAKER_03
27.08 --> 27.64 (duration: 0.56s) Speaker: SPEAKER_05
...
```

## Additional Analytics

Now that we have processed our file, let's explore some useful features of the Pyannote SDK:

### Speaker Time

Here we calculate the total speaking time for each speaker:

```python
for speaker in output.labels():
    speaking_time = output.label_duration(speaker)
    print(f"Speaker {speaker} total speaking time: {speaking_time:.2f}s")
```

This will output something like:

```
Speaker SPEAKER_00 total speaking time: 558.98s
Speaker SPEAKER_01 total speaking time: 18.98s
Speaker SPEAKER_02 total speaking time: 22.88s
Speaker SPEAKER_03 total speaking time: 469.68s
Speaker SPEAKER_04 total speaking time: 698.02s
Speaker SPEAKER_05 total speaking time: 190.70s
Speaker SPEAKER_06 total speaking time: 5.74s
...
```

### Speaker Overlap

Pyannote can identify regions where multiple speakers are talking simultaneously:

```python
overlap = output.get_overlap()
print(f"Overlapping speech regions: {overlap}")
```

This will output the segments where more than one speaker is speaking.

```
Overlapping speech regions:
[[ 00:00:27.672 -->  00:00:27.689]
 [ 00:00:38.337 -->  00:00:38.860]
 [ 00:00:40.395 -->  00:00:40.463]
```

### Filter by Speaker

We can also filter the diarization output to focus on a specific speaker:

```python
speaker = "SPEAKER_06"
speaker_turns = output.label_timeline(speaker)
print(f"Speaker {speaker} speaks at:")
for speaker_turn in speaker_turns:
    print(speaker_turn)
```

This will show the segments where this speaker is speaking.

```
Speaker SPEAKER_06 speaks at:
[ 00:03:45.767 -->  00:03:45.852]
[ 00:03:55.386 -->  00:03:55.521]
[ 00:05:07.257 -->  00:05:07.274]
...
```

## Extracting Speaker Segments

To verify our results and prepare the audio for further processing, let's split the original audio into segments by speaker:

```python
import shutil
from pydub import AudioSegment

def split_audio_by_segments(audio_path, diarization_output, output_dir="output_segments"):
    """
    Split an audio file into multiple files based on diarization output

    Parameters:
    -----------
    audio_path: str
        Path to the input audio file
    diarization_output: Annotation
        Pyannote diarization output
    output_dir: str
        Directory to save the output segments
    """
    # Clear the output directory if it exists
    if os.path.exists(output_dir):
        shutil.rmtree(output_dir)

    # Create output directory
    os.makedirs(output_dir, exist_ok=True)

    # Load the audio file
    audio = AudioSegment.from_file(audio_path)

    # Extract each segment with speaker information
    for i, (segment, _, speaker) in enumerate(diarization_output.itertracks(yield_label=True)):
        # Convert seconds to milliseconds
        start_ms = int(segment.start * 1000)
        end_ms = int(segment.end * 1000)

        # Extract segment
        segment_audio = audio[start_ms:end_ms]

        # Generate output filename with speaker information
        filename = os.path.basename(audio_path)
        name, ext = os.path.splitext(filename)
        output_path = os.path.join(output_dir, f"{name}_segment_{i+1:04d}_{start_ms:08d}ms-{end_ms:08d}ms_speaker_{speaker}{ext}")

        # Export segment
        segment_audio.export(output_path, format=ext.replace('.', ''))
        print(f"Saved segment {i+1} to {output_path} (Speaker: {speaker})")

# Apply the function to our audio file
split_audio_by_segments(audio_file, output)
```

## Inspecting Results

To verify our results, we'll create a function to play audio files in our Jupyter environment:

```python
import librosa
from IPython.display import Audio, display

def play_audio(file_path, sr=None):
    """
    Play an audio file in a Jupyter notebook.
    """
    # Load the audio file
    y, sr = librosa.load(file_path, sr=sr)

    # Display an audio widget to play the sound
    audio_widget = Audio(data=y, rate=sr)
    display(audio_widget)
```

Now, let's listen to a few clips to verify that the speakers were correctly identified and isolated.

```python
import os
audio_dir = "./output_segments/"

audio_files = os.listdir(audio_dir)
audio_files.sort()

n_offset = 21
n_clips = 5

for fname in audio_files[n_offset:n_clips + n_offset]:
    print(f"File: {fname}")

    # Extract speaker info if present in filename
    if "_speaker_" in fname:
        speaker_part = fname.split("_speaker_")[1].split(".")[0]
        print(f"Speaker: {speaker_part}")

    play_audio(audio_dir + fname)
```

Around file 21 or so, we see a 14-second clip of `SPEAKER_00` speaking:

`File: sample_0_segment_0025_00055364ms-00070045ms_speaker_SPEAKER_00.wav Speaker: SPEAKER_00`

A few files later we see a 10-second clip of `SPEAKER_00` speaking again:

`File: sample_0_segment_0029_00071530ms-00081840ms_speaker_SPEAKER_00.wav Speaker: SPEAKER_00`

Note: The filenames may change. Each time we run the model there may be a different number of clips. The diarization model sometimes captures `<1` second of audio and labels it as a speaker.

The 10-second clip is audio of `SPEAKER_00` about the meeting agenda. It is a file of just `SPEAKER_00` talking.

The 14-second clip has two other speakers speaking. This seems like an error at first, but we will notice two other files that capture the additional speakers:

`sample_0_segment_0026_00059228ms-00060325ms_speaker_SPEAKER_02.wav`
`sample_0_segment_0027_00061793ms-00062924ms_speaker_SPEAKER_02.wav`

With further processing, we could remove these from the `SPEAKER_00` file if necessary for our application.

We can also use the Pyannote overlap function to verify that we did have overlapping speech at this time in the audio file.

## Verifying Speaker Overlap

Finally, let's check for regions where multiple speakers are talking simultaneously:

```python
overlap = output.get_overlap()
for overlap_ts in overlap:
    print(f"Overlapping speech regions: {overlap_ts}")
```

Here we see two overlapping segments that match up with the overlapping files we found above.

`sample_0_segment_0026_00059228ms-00060325ms_speaker_SPEAKER_02.wav`
`sample_0_segment_0027_00061793ms-00062924ms_speaker_SPEAKER_02.wav`

```
Overlapping speech regions: [ 00:00:59.228 -->  00:01:00.325]
Overlapping speech regions: [ 00:01:01.793 -->  00:01:02.924]
```

## Conclusion

This tutorial demonstrates how Pyannote's speaker diarization on VAST.ai provides a powerful solution for identifying "who spoke when" in multi-speaker recordings. The implementation offers several advantages:

-   **Accuracy and Efficiency**: Pyannote accurately identifies different speakers and their speaking times, even detecting overlapping speech
-   **Practical Applications**: Enables speaker-attributed transcription, conversation analytics, and content indexing for meetings, podcasts, and interviews
-   **Cost-Effective Processing**: VAST.ai's affordable GPU rentals make processing large audio datasets accessible without expensive hardware investments

The Pyannote diarization model provides impressive results out of the box, and running it on VAST.ai makes it accessible and affordable for a wide range of applications. Whether you're building a meeting transcription service, analyzing call center interactions, or researching conversation dynamics, this approach gives you a solid starting point.

Look out for more content from Vast about other audio processing tasks!


Run speaker diarization with Pyannote on VAST.ai to detect who spoke when in audio files—cost-effective, accurate, and easy to integrate with your pipeline.

Speaker Diarization with Pyannote on VAST


We're pleased to announce that Vast.ai has successfully completed the SOC 2 Type I audit, a significant milestone as we continue to deliver secure and scalable GPU compute to a global user base.

Security remains central to everything we do here at Vast.ai. We've spent the past 6+ years delivering reliable service worldwide while meeting the [highest standards](https://vast.ai/article/security-and-compliance-at-vast-ai) of regulatory compliance. 

### Vast.ai’s Secure Cloud Offering

Our [Trust Center](https://trust.vast.ai/) includes additional information about our controls, compliance stance, FAQs and Subprocessors.

For customers with heightened security and compliance requirements, our [**Secure Cloud**](https://cloud.vast.ai/?secure_cloud=true) offering provides access to GPUs hosted exclusively by our certified datacenter partners. This environment already delivers an added layer of assurance – now further backed by an independent SOC 2 audit on our platform infrastructure.

Achieving **SOC 2 Type I certification** is a clear validation of the controls we've built to protect customer data and ensure the integrity of our infrastructure.

Here's a brief overview of SOC 2 Type I and what it entails.

### What Is SOC 2 Type I?

Put simply, SOC 2 (System and Organization Controls 2\) compliance assesses an organization's security measures. The audit determines whether internal controls are appropriately designed to meet [trust services criteria](https://www.aicpa-cima.com/topic/audit-assurance/audit-and-assurance-greater-than-soc-2) – covering areas like security, confidentiality, privacy, and processing integrity – in order to safeguard customer data.

Developed by the American Institute of Certified Public Accountants (AICPA), the SOC 2 Type I report is essentially a detailed snapshot evaluating the design and implementation of these controls at a specific point in time. Its goal is to ensure that security measures meet rigorous industry standards.

### Why This Matters

Our SOC 2 Type I certification affirms to our customers and partners that we've put strong, well-designed controls in place to protect their data and maintain operational reliability. We take data security seriously and follow industry-recognized best practices.

The Type I audit is only the first step in a broader compliance roadmap. We are currently preparing for the SOC 2 Type II audit, which evaluates how effectively internal controls perform over an extended period. That assessment will validate that we maintain our security practices consistently, both in design and in day-to-day execution.

We're also advancing our compliance efforts across other standards, to help us better serve organizations with strict regulatory requirements.

### Our Ongoing Commitment

Earning this SOC 2 Type I certification reflects our commitment to protecting the privacy and integrity of the data entrusted to us by developers, researchers, and organizations worldwide. As a distributed peer-to-peer platform powering global compute, we take our responsibility seriously. This certification is one step in our ongoing effort to provide users with infrastructure they can trust. 

For more about security and compliance at Vast.ai as well as our datacenter partners, please see our previous post [here](https://vast.ai/article/security-and-compliance-at-vast-ai). To request a copy of our SOC 2 report, feel free to contact us anytime at [compliance@vast.ai](mailto:compliance@vast.ai).

A huge thank-you to our team at Vast.ai and the audit partners who helped us reach this milestone\! 

We'll keep raising the bar – and keep you updated as we do.



Vast.ai has achieved SOC 2 Type I certification, reinforcing its commitment to data security and trusted infrastructure. Learn what this means for users and what’s next on their compliance roadmap.

Posts about: Vast.ai

Vast.ai GPUs Can Now Be Rented Through SkyPilot

Running OpenAI's GPT-OSS on Vast.ai

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai

Running Deep Cogito on Vast.ai

Speaker Diarization with Pyannote on VAST

Vast.ai Achieves SOC 2 Type I Certification

Posts about: Vast.ai

Vast.ai GPUs Can Now Be Rented Through SkyPilot

Running OpenAI's GPT-OSS on Vast.ai

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai

Running Deep Cogito on Vast.ai

Speaker Diarization with Pyannote on VAST

Vast.ai Achieves SOC 2 Type I Certification

Subscribe for our product updates.