
The DeepSeek R1-0528 release introduces a significant improvement to the reasoning capabilities of language models by making it easier to access "thinking" mode without requiring complex prompt engineering or pre-pending thinking tokens. This enhanced model provides transparent, step-by-step reasoning that's particularly valuable for educational applications, complex problem solving, and scenarios where transparency in AI decision-making is crucial.

In this post, we'll explore how to deploy the **DeepSeek-R1-0528-Qwen3-8B** model using **vLLM** on Vast.ai's cloud GPU platform, leveraging the new **qwen3 reasoning parser** that simplifies access to the model's internal thinking process.

---

## The Open Source AI Revolution

DeepSeek R1-0528 represents more than just a technical advancement—it's a significant step toward democratizing advanced AI capabilities. As an **open source model**, it makes these reasoning and advanced AI capabilities more accessible.

Unlike closed-source models that require vendor dependence, usage costs, and data privacy concerns, DeepSeek R1-0528 enables **complete control** over your AI infrastructure. Organizations can now deploy advanced reasoning capabilities locally, maintaining data sovereignty while avoiding variable per-token pricing.

This democratization means organizations can access cutting-edge AI without massive infrastructure investments. Advanced reasoning capabilities are no longer exclusive to large tech companies—anyone with GPU access can deploy their own reasoning-capable AI system.

---

## What Makes DeepSeek R1-0528 Special?

The DeepSeek R1-0528 release addresses a key limitation in previous DeepSeek models: the complexity of accessing the model's internal reasoning process. Previously, users needed to manually prepend "thinking" tokens to model outputs or use complex prompt engineering to see step-by-step reasoning.

---

## Deploying DeepSeek R1-0528 on Vast.ai

### Step 1: Set Up Vast.ai Environment

First, install the Vast CLI and configure your API key:

```bash
pip install --upgrade vastai

export VAST_API_KEY="your_vast_api_key"
vastai set api-key $VAST_API_KEY
```

### Step 2: Find Suitable Hardware

The DeepSeek-R1-0528-Qwen3-8B model requires specific hardware capabilities:

- **Minimum 24GB GPU RAM** for model weights and KV cache
- **Single GPU configuration** (sufficient for the 8B parameter model)
- **Static IP address** for stable API endpoint hosting
- **Direct port access** for the vLLM OpenAI-compatible server
- **60GB+ disk space** for model storage and dependencies

Search for suitable instances:

```bash
vastai search offers "compute_cap >= 750 \
gpu_ram >= 24 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 60 \
rentable = true"
```

### Step 3: Deploy the Model

Deploy using the vLLM OpenAI-compatible server with the qwen3 reasoning parser:

```bash
export INSTANCE_ID="your_instance_id"

vastai create instance $INSTANCE_ID \
--image vllm/vllm-openai:latest \
--env '-p 8000:8000' \
--disk 60 \
--args --model deepseek-ai/DeepSeek-R1-0528-Qwen3-8B \
--max-model-len 4096 \
--reasoning-parser qwen3
```

**Key deployment parameters:**
- `--reasoning-parser qwen3`: Enables the simplified reasoning parser
- `--max-model-len 4096`: Accommodates longer reasoning sequences
- `--tensor-parallel-size 1`: Single GPU configuration
- `--gpu-memory-utilization 0.90`: Optimized memory usage

### Step 4: Get Instance Details and Test Connection

Navigate to the [Instances tab](https://cloud.vast.ai/instances/) in the Vast AI Console and locate your instance. Click the IP address button to view the forwarded ports.

Test the connection with a simple curl command:

```bash
export VAST_IP_ADDRESS="your_ip_address"
export VAST_PORT="your_port"

curl -X POST http://$VAST_IP_ADDRESS:$VAST_PORT/v1/completions \
-H "Content-Type: application/json" \
-d '{"model": "deepseek-ai/DeepSeek-R1-0528-Qwen3-8B", "prompt": "Hello, how are you?", "max_tokens": 50}'
```

---

## Testing the Model's Reasoning Capabilities

### Setup Python Client

```python
import time
from openai import OpenAI

# Replace with your actual Vast.ai instance details
VAST_IP_ADDRESS = "your_ip_address"
VAST_PORT = "your_port"

client = OpenAI(
    api_key="DUMMY",
    base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
)
```

### Basic Functionality Test

```python
start = time.time()
test_resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": "Hello"}]
)
elapsed = time.time() - start

print("Response:", test_resp.choices[0].message.content)
print(f"Elapsed time: {elapsed:.4f} s")
```

### Reasoning Capability Test

This is where DeepSeek R1-0528 truly shines. The model naturally provides step-by-step reasoning without complex prompt engineering:

```python
reasoning_prompt = """
Write the History of Paris in a paragraph, show your thinking 
step by step first before paragraph and then write the history 
in a paragraph
"""

start = time.time()
resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": reasoning_prompt}]
)
elapsed = time.time() - start

print("Model output:\n", resp.choices[0].message.content)
print(f"Elapsed time: {elapsed:.3f} s")
```

**Expected output structure:**
1. **Thinking process**: The model shows its step-by-step reasoning
2. **Final answer**: A well-structured paragraph about Paris's history

### Mathematical Reasoning Test

```python
math_prompt = "If 8 × 7 = ?, show your work step by step."

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
    messages=[{"role": "user", "content": math_prompt}]
)

print("Mathematical reasoning:\n", resp.choices[0].message.content)
```

The model will naturally break down the problem, show intermediate steps, and arrive at the solution with clear reasoning traces.

---

## Key Benefits of DeepSeek R1-0528

### 1. **Simplified Reasoning Access**
No need to prepend thinking tokens or use complex prompt engineering. The `qwen3` reasoning parser automatically handles the extraction and presentation of reasoning steps.

### 2. **Educational Value**
Perfect for educational applications where students need to understand the problem-solving process, not just the final answer.

### 3. **Transparency and Trust**
Users can see exactly how the model arrives at its conclusions, building trust in AI-powered decision-making systems.

### 4. **Cost-Effective Deployment**
Running on Vast.ai provides significant cost savings compared to major cloud providers while maintaining high performance.

### 5. **Easy Integration**
OpenAI-compatible API ensures seamless integration with existing applications and workflows.

---

## Use Cases

### Educational Applications
- Step-by-step problem solving for mathematics, science, and logic
- Teaching critical thinking and reasoning skills
- Creating interactive learning experiences

### Business Intelligence
- Transparent decision-making in automated systems
- Audit trails for AI-powered recommendations
- Complex analysis with clear reasoning paths

### Research and Development
- Understanding model behavior and capabilities
- Testing reasoning patterns across different domains
- Developing more interpretable AI systems

---

## Conclusion

The DeepSeek R1-0528 release represents a significant step forward in making AI reasoning more accessible and transparent. By combining this powerful model with Vast.ai's cost-effective GPU infrastructure and the simplified qwen3 reasoning parser, developers can now easily deploy transparent, reasoning-capable AI systems.

Whether you're building educational tools, business intelligence systems, or research applications, DeepSeek R1-0528 on Vast.ai provides an excellent foundation for applications that require clear, step-by-step reasoning capabilities.

The elimination of complex prompt engineering for accessing reasoning modes makes this model particularly valuable for production applications where transparency and interpretability are key requirements.

---

## Useful Resources

- [DeepSeek R1-0528 Model on Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B)
- [vLLM Documentation](https://docs.vllm.ai/)
- [Vast.ai GPU Marketplace](https://vast.ai)
- [OpenAI Python SDK](https://github.com/openai/openai-python)

---

*Ready to deploy transparent, reasoning-capable AI? Get started with DeepSeek R1-0528 on Vast.ai today! 🚀*

Deploy DeepSeek R1-0528 on Vast.ai to access transparent, step-by-step AI reasoning without prompt tricks—ideal for education, research, and business use.

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai


We're back with a new update from the team at **Vast.ai**\! This month, we've rolled out fresh hardware options, new video-generation tools, and continued improvements behind the scenes.

The big news is that our GPU lineup now includes the **NVIDIA DGX B200** as well as the **RTX PRO 6000 WS**\! These powerful machines open the door to even more demanding workloads, bringing you the compute power required for everything from multi-modal inference to precision-driven simulations.

We also recently reached an exciting milestone: we now have **over one thousand RTX 5090 GPUs available on Vast** – 1,180 and counting.

Here are the details about what's new this month\!

### NVIDIA Cloud GPU Updates

Whether you're training foundation models, rendering high-resolution video, or running inference, our latest hardware additions give you extra flexibility:

-   **NVIDIA DGX B200** – A universal AI supercomputer built on the Blackwell architecture, equipped with **eight GPUs** and **1.4 TB** of GPU memory. With massive bandwidth and compute capacity, it's well suited to handle virtually any enterprise AI workload.

-   **RTX PRO 6000 WS** – A high-end Blackwell-based workstation GPU featuring **96 GB** of GDDR7 memory and up to **4000 TOPS** of AI compute. It can easily handle massive datasets and multi-billion-parameter models while supporting seamless multi-app workflows.

### New Guide and Templates

We've focused on video generation this month, adding a couple of new templates and a guide to help you go from idea to clip as seamlessly as possible:

-   [**Guide to Video Generation**](https://docs.vast.ai/video-generation) – A walkthrough for setting up and using ComfyUI on Vast.ai, so you can create advanced stable diffusion pipelines (ideal for video generation workflows) with this powerful node-based interface.

-   **LTX Video (ComfyUI)** – A ready-to-use template that pairs the LTX Video model with ComfyUI, optimized for consumer-grade GPUs.

-   **Open-Sora** – A template for this open-source video generation model with a streamlined, user-friendly platform that lowers the barrier to efficient, high-quality video content creation.

And that's a wrap for June 2025\!

### Our Commitment

Whether you're looking to launch a B200 instance, experiment with video generation using our latest templates, or scale up with our expanding lineup of RTX 5090 GPUs, we're here to support your work at every stage. Our goal at **Vast.ai** is to make powerful compute accessible, flexible, and easy to use – so you can focus on building, training, and creating.

Need help? Reach out to our customer support team anytime at [support@vast.ai](mailto:support@vast.ai), or connect with the community on our Discord server for tips, troubleshooting, and more.

### Change Log

#### New Templates & Guides:

-   [Guide to Video Generation](https://docs.vast.ai/video-generation)
-   LTX Video (Comfy UI) – Video-Gen model optimized for consumer-grade GPUs.
-   Open-Sora – Democratizing efficient video production for everyone.

#### New GPUs:

-   DGX B200
-   RTX PRO 6000 WS

Plus we now have over one thousand 5090 GPUs available.


New DGX B200 and RTX PRO 6000 WS GPUs, over 1000 RTX 5090s available, and new video generation templates.

June 2025 Product Update


Ten years ago today, our CEO and founder, Jacob Cannell, published ["The Brain as a Universal Learning Machine"](https://www.lesswrong.com/posts/9Yc7Pp7szcjPgPsjf/the-brain-as-a-universal-learning-machine) on LessWrong.com.

Drawing from neuroscience, machine learning, and systems theory, he proposed that **the human brain is not a patchwork of specialized, genetically programmed modules, but a general-purpose learning system** – or, as he put it, "a biological implementation of a _Universal Learning Machine_."

In hindsight, the timing was notable. Cannell published the piece just as deep learning was moving from niche research to broader relevance in artificial intelligence (AI). In the decade since then, rapid progress in scalable, trainable, data-driven models has validated the idea that learning, not pre-engineered structure, is the key to building intelligent systems.

So on the tenth anniversary of its publication, we'd like to revisit the key ideas of the article, reflect on what's aged well, and look ahead at what's still unfolding.

We’d encourage you to read the full article on [Less Wrong](https://www.lesswrong.com/posts/9Yc7Pp7szcjPgPsjf/the-brain-as-a-universal-learning-machine).


A look at the brain as a universal learning machine, and how it can be used to learn about the world.

A Decade Later: Revisiting the Brain as a Universal Learning Machine


Serving multiple machine learning models simultaneously is a key challenge in production AI systems. The increasing size of foundational models and constraints on GPU memory often mean that deploying separate models for each task results in high RAM usage, underutilized hardware, and latency problems caused by repeatedly swapping entire models in and out of GPU memory. This not only increases cost but also complicates scaling.

However, by leveraging Lorax, a framework for dynamically loading LoRA adapters—and Vast.ai's flexible GPU marketplace, developers can efficiently serve several specialized models on a single base model deployment. This results in significantly better hardware utilization, reduced infrastructure costs, and low-latency inference for diverse AI workloads, making this a game-changer for enterprises deploying their own AI infra.

In this blog post, we'll explore how to deploy and run multiple LoRA adapters on a shared base model using Lorax on Vast.ai's cloud platform. This setup enables hosting thousands of fine-tuned models simultaneously, loading task-specific adapters on demand, and seamlessly switching between tasks such as math problem solving and customer support classification with minimal overhead.

---

## What Are LoRA Adapters and Lorax?

- **LoRAs (Low-Rank Adaptation):** Lightweight, task-specific parameter adjustments that adapt a large base model to new tasks with far fewer parameters than full fine-tuning. LoRAs can be swapped in and out without needing to reload the entire large model, saving significant RAM and time.

- **Lorax (LoRA eXchange):** An efficient serving framework that enables running thousands of LoRA adapters on a single GPU by dynamically loading these adapters into a base model at inference time. Lorax maintains high throughput and low latency while drastically reducing the deployment cost of hosting multiple fine-tuned variants.

---

## Deploying Multiple Models Using Lorax and Vast.ai

The following example uses the `mistralai/Mistral-7B-v0.1` base model hosted on Vast.ai, along with two LoRA adapters:

- `predibase/gsm8k`: Designed for solving math problems like the GSM8K benchmark.
- `predibase/customer_support`: Specializes in customer service query classification.

We deploy the base model once and dynamically switch between these specialized LoRA adapters, enabling efficient multi-model serving on the same underlying GPU instance.

---

### Step 1: Set Up Vast.ai Environment

1. **Install Vast CLI and configure API key:**

```bash
pip install vastai==0.2.6

export VAST_API_KEY="your_vast_api_key"
vastai set api-key $VAST_API_KEY
```

2. **Search for a suitable GPU instance:**

The `mistralai/Mistral-7B-v0.1` model requires at least 16GB VRAM, but it's safer to select a larger instance (e.g., 32GB VRAM) for LoRA adapter loading.

```bash
vastai search offers "compute_cap >= 750 \
geolocation=US \
gpu_ram >= 32 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 80 \
rentable = true"
```

3. **Deploy the instance:**

Make sure you have accepted the usage terms for `Mistral-7B-v0.1` on Huggingface before proceeding.

```bash
export INSTANCE_ID=your_instance_id

vastai create instance $INSTANCE_ID --image ghcr.io/predibase/lorax:main \
--env '-p 8080:80 --shm-size 1g -e HUGGING_FACE_HUB_TOKEN="your_hf_token"' \
--disk 80 --args --model-id mistralai/Mistral-7B-v0.1
```

This command runs the Lorax server that hosts the base model and dynamically loads adapters on demand.

4. **Get Instance IP Address and Port**

Now, we need to get our IP address and port to call our model. We first need to wait for our machine to download the image and the model and start serving. This will take a few minutes. The logs will show you when it's done.

Next, go to the [Instances tab](https://cloud.vast.ai/instances/) in the Vast AI Console and find the instance you just created.

At the top of the instance, there is a button with an IP address in it. Click this and a panel will show up of the ip address and the forwarded ports.
You should see something like:

```
Open Ports
XX.XX.XXX.XX:YYYY -> 8000/tcp
```

You will need the IP address (XX.XX.XXX.XX) and the port (YYYY) for the next step.

---

### Step 2: Query LoRA Adapters via HTTP Requests

Lorax supports on-demand loading of LoRAs from the Huggingface Hub, minimizing memory overhead versus deploying multiple large models simultaneously.

First, configure environment variables to connect to your Vast instance and set your Huggingface token:

```python
HF_TOKEN = "your_hf_token"
VAST_IP_ADDRESS = "your_instance_ip"
VAST_PORT = "your_vast_port"
```

#### Example: Math Problem Solving with `predibase/gsm8k`

Prepare a prompt:

```python
question = "James decides to run 3 sprints 3 times a week. He runs 60 meters each sprint. How many total meters does he run a week?"
math_prompt = f"Please answer the following question: {question}\nAnswer"
```

Make a POST request to the `/generate` endpoint specifying the LoRA adapter ID:

```python
import requests

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {HF_TOKEN}"
}
data = {
    "inputs": math_prompt,
    "parameters": {
        "max_new_tokens": 100,
        "adapter_id": "predibase/gsm8k",
        "adapter_source": "hub"
    }
}

url = f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/generate"
response = requests.post(url, headers=headers, json=data)
print("Response:", response.json()["generated_text"])
```

**Response:**

```
He runs 3*3=<<3*3=9>>9 sprints a week
So he runs 9*60=<<9*60=540>>540 meters a week
#### 540
```

#### Example: Customer Support Classification with `predibase/customer_support`

Prepare a customer support transcript prompt:

```python
transcript = "Hi I am having trouble with my account. It says I need to reset my password to log in but I already reset my password."

customer_support_prompt = f"""Consider the case of a customer contacting the support center.
The term task type refers to the reason for why the customer contacted support.

### The possible task types are: account issue, billing issue, product issue, none of the above

Summarize the issue/question/reason that drove the customer to contact support:

Transcript: {transcript}

Task Type:
"""
```

Make the API call:

```python
data = {
    "inputs": customer_support_prompt,
    "parameters": {
        "max_new_tokens": 100,
        "adapter_id": "predibase/customer_support",
        "adapter_source": "hub"
    }
}

response = requests.post(url, headers=headers, json=data)
print("Response:", response.json()["generated_text"])
```

**Response:**

```
account issue
```

---

### Step 3: Access Models Using the OpenAI SDK

Lorax provides compatibility with the OpenAI API, allowing easy integration into existing apps with minimal code changes.

1. **Install OpenAI SDK**

```bash
pip install --upgrade openai
```

2. **Call the math adapter:**

```python
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1",
    default_headers={"Authorization": f"Bearer {HF_TOKEN}"}
)

resp = client.completions.create(
    model="predibase/gsm8k",
    prompt=math_prompt,
    max_tokens=100,
    temperature=0.7
)

print("Response:", resp.choices[0].text)
```

Output:

```
James runs 3 x 3 = <<3*3=9>>9 sprints in a week.
He runs 9 x 60 = <<9*60=540>>540 meters in a week.
#### 540
```

3. **Call the customer support adapter:**

```python
resp = client.completions.create(
    model="predibase/customer_support",
    prompt=customer_support_prompt,
    max_tokens=100,
    temperature=0.7
)
print("Response:", resp.choices[0].text)
```

Output:

```
account issue
```

---

## Why This Approach Matters

Serving multiple specialized AI models traditionally involves deploying each fine-tuned model independently, consuming lots of GPU memory and resulting in higher cost and slower inference when switching tasks.

By contrast, Lorax’s approach uses a single shared base model and dynamically loads lightweight LoRA adapters tailored for different tasks. This yields several benefits:

- **Reduced RAM and GPU usage:** Only the small adapter layers are loaded on demand, not the entire large model.
- **Lower infrastructure cost:** A single GPU instance can serve thousands of specialized models.
- **Faster context switching:** Switching between tasks is near-instant as you don’t reload the full model.
- **Flexibility:** New adapters can be added or updated independently.
- **Ease of integration:** OpenAI-compatible APIs simplify adoption.

Combined with Vast.ai’s cost-effective, flexible GPU rental options, this method provides a scalable, production-ready solution for deploying multi-model AI services with a minimal footprint.

---

## Conclusion

Efficiently serving multiple machine learning models is vital for scaling AI-powered products. The combination of Lorax’s dynamic LoRA adapter serving and Vast.ai’s flexible GPU infrastructure offers a powerful, cost-effective, and scalable way to support many specialized models on a single deployment.

Whether solving math problems, classifying customer inquiries, or other tasks, this approach reduces overhead and unlocks practical multi-model serving for modern AI applications.

Try setting up your own multi-LoRA serving environment today to experience the benefits firsthand!

---

## Useful Resources

- [Lorax GitHub Repository](https://github.com/predibase/lorax)
- [Vast.ai GPU Marketplace](https://vast.ai)
- [Mistral-7B-v0.1 on Huggingface](https://huggingface.co/mistralai/Mistral-7B-v0.1)
- [OpenAI Python SDK](https://github.com/openai/openai-python)

---

_Happy modeling! 🚀_


Serve multiple ML models efficiently using Lorax and LoRA adapters on Vast.ai—cut costs, reduce latency, and maximize GPU use with dynamic adapter loading.

Efficiently Serving Multiple Machine Learning Models with Lorax and vLLM on Vast.ai


Nearly every startup faces the same two pressures: to move fast and make every dollar count.

Time and budget may be limited while demands remain high – especially for teams working with AI, machine learning, or other compute-heavy workloads. For startups, the pressure to deliver quickly and prove technical viability makes access to reliable infrastructure even more critical.

Traditional cloud providers often aren't built with startups in mind. Pricing may be steep or overly complex, resources can be constrained, and it's easy to end up locked into systems that don't scale with your needs. On the other hand, buying your own hardware adds upfront costs and delays, which can be difficult to justify for early-stage teams.

Vast.ai offers another path. **Our market-based cloud GPU rental platform gives startups the flexibility to run powerful workloads without the usual friction.**

Here's a closer look at why startups choose to build with Vast.ai.

### Why Vast.ai Makes Sense for Startups

Startups need infrastructure that keeps up with their pace, budget, and technical demands. The following are some of the reasons Vast.ai hits the mark.

#### 1\. Instant Access to GPUs

With Vast.ai, there are no waitlists or delays. Startups can instantly access powerful GPUs like **H100s**, **A100s**, and **RTX 5090s**, backed by a global network of providers that ensures better availability and built-in redundancy.

Plus, with machines available around the world, startups can deploy GPUs in regions closer to their major user concentrations, reducing network latency for performance-sensitive applications.

#### 2\. Cost-Effective Compute Power

Vast.ai offers significantly cheaper GPU rental compared to traditional cloud providers – **on average 5-6X lower in cost** – making high-performance compute far more accessible.

For bootstrapped startups and solo founders, Vast.ai levels the playing field, enabling smaller teams on a budget to compete in compute-intensive spaces, as well as handle temporary spikes in workload without having to invest in expensive infrastructure.

#### 3\. Flexible Startup-Friendly Billing

Vast.ai's **pay-as-you-go pricing** gives startups the freedom to scale compute resources up or down as needed, with no lock-in contracts. This way, **you only pay for what you use**, and you can quickly adapt to new demands as they arise.

On-demand and interruptible (bid auction) [rental options](https://docs.vast.ai/instances/rental-types) offer even more control – ideal for variable usage patterns and budget-conscious teams. On top of that, real-time cost tracking and detailed resource utilization dashboards help you monitor and optimize performance.

#### 4\. Enterprise-Grade Security

With Vast.ai, **startups can choose the level of trust and security that best fits their needs.** Providers [range](https://vast.ai/compliance) from verified hosts, suitable for general computing needs, to our vetted datacenter partners who maintain third-party compliance certifications (at minimum ISO 27001 certified) and sign comprehensive Data Processing Agreements (DPAs) with Vast.ai.

For sensitive or production workloads, our Secure Cloud tier ensures you’re connecting with partners in certified locations that are current with ISO 27001 and/or Tier 3/4 standards. (You can even filter for it. Look for the blue datacenter label\!)

We've also [completed](https://vast.ai/article/vast-ai-achieves-soc-2-type-1-certication) our SOC 2 Type I certification – part of our ongoing commitment to the highest standards of regulatory compliance.

#### 5\. Developer Control

Vast.ai provides **transparent instance specs** (GPU type, RAM, storage, and more), so startups can choose exactly what fits their workload and budget. Developers can bring their own Docker images, run their own stack, and configure environments to match their specific requirements.

With a low barrier to entry, even early-stage teams without dedicated DevOps support can get up and running quickly. Our selection of pre-configured [templates](https://docs.vast.ai/templates) are like recipes that help users easily set up instances; templates can define the Docker image, environment variables, and launch mode for each new instance.

#### 6\. Ideal for AI Workloads

Vast.ai is well suited for AI startups working on large-scale training, fine-tuning, and inference. Our customizable, on-demand [CPU/GPU clusters](https://vast.ai/clusters) ensure reliable throughput and flexible resource allocations for continuous or large-scale workloads.

With support for large language models (LLMs) and diffusion models, our platform is compatible with all major machine learning (ML) frameworks and offers the flexibility to run custom containers and tooling.

For teams that want to get started quickly, Vast.ai provides **ready-to-use GPU instances pre-configured for frameworks like TensorFlow and PyTorch**, reducing setup time and simplifying deployment.

#### 7\. Built to Experiment

With Vast.ai, **startups can prototype, test, and explore without any long-term commitments** or expensive upfront investment.

For instance, teams can run quick tests and train models on demand, access and evaluate cutting-edge GPUs like the H200 that may otherwise be out of reach, and even spin up high-performance environments on short notice for hackathons and demos.

### The Bottom Line

**Great ideas can come from anywhere, and they deserve the resources to grow.** By making GPU compute power more open and accessible, Vast.ai gives startups and early-stage teams a real shot at achieving their goals.

Built with transparency, accessibility, and security in mind – and aligned with the open-source ethos that many founders value – our platform supports the kind of flexibility and control that your team needs from day one.

Ready to build, test, and scale on your terms? Discover what's possible with Vast.ai\!


Vast.ai offers startups instant, low-cost access to powerful GPUs—perfect for AI, ML, and compute-heavy work—without lock-ins or upfront infrastructure costs.

7 Reasons Why Startups Use Vast.ai


The GeForce RTX 4090 and the A100 both sit at the high end of NVIDIA's GPU lineup, but they're built for very different worlds. 

The **RTX 4090** is a powerful consumer-grade GPU, designed for ultra high-performance gaming, professional creative work, and even some entry-level AI workloads. It's surprisingly capable across a variety of tasks, especially for its price point.

The **A100**, on the other hand, isn't aimed at consumers at all. It's an enterprise-grade GPU built for data centers, research labs, and AI teams running large-scale model training, inference, and simulation workloads – designed to move serious data at serious speed.

In this post, we'll look at how these two machines compare with each other, and why you might choose one over the other based on your specific needs.

### NVIDIA GeForce RTX 4090: A Workhorse with Range

When it launched in September 2022 as the top-tier consumer GPU of its generation, the RTX 4090 quickly became the go-to choice for 4K gaming, 3D rendering, and AI-enhanced creative workflows. 

Built on NVIDIA's **Ada Lovelace** architecture and powered by the **AD102 chip**, the RTX 4090 introduced significant upgrades in ray tracing and Tensor Core acceleration, and delivered notable performance gains with **DLSS 3** in a variety of games.

While it's no longer the flagship GPU of NVIDIA's GeForce line (that title now goes to the RTX 5090, which you can read about [here](https://vast.ai/article/nvidia-geforce-rtx-5090-specs-everything-you-need-to-know)), the RTX 4090 definitely still holds its own. It remains one of the most capable and well-rounded GPUs available at its price point for consumers and technical teams alike.

#### Some of the RTX 4090's key specs and features include:

* **24 GB of GDDR6X memory** with **1,008 GB/s bandwidth** on a **384-bit** interface.  
    
* **16,384 CUDA cores** for high-parallel throughput.  
    
* **128 third-gen RT cores** and **512 fourth-gen Tensor cores** for advanced ray tracing and AI workloads.  
    
* **DLSS 3** support for real-time upscaling and performance boosts in many games.  
    
* **450 watts** of thermal design power (TDP), requiring an **850-watt power supply** due to its high processing capabilities.

If you're looking for serious performance in a desktop form factor, the RTX 4090 continues to deliver. It may not be the newest card on the market, but it remains a workhorse that gets the job done across a wide range of applications.

As capable as the RTX 4090 is, it's not built for everything. If your workloads go beyond what a high-end consumer GPU can handle, the A100 just might bring the compute power you need.

### NVIDIA A100: A Powerhouse with Purpose

The A100 is built for scale. Based on NVIDIA's Ampere architecture and powered by the **GA100 chip**, it's designed to tackle massive workloads – like foundational AI model training – thanks to its raw throughput, high-bandwidth memory, and ability to scale across multi-GPU environments. 

When the A100 launched in June 2021, it introduced **third-gen Tensor Cores**, support for **TF32** precision, and **Multi-Instance GPU** (MIG) capabilities – enabling flexible resource partitioning and efficient parallel processing across shared environments.

While some of its core specs might look modest compared to the RTX 4090, the A100 delivers superior performance in AI and data-intensive workloads because it's optimized where it counts: memory bandwidth, interconnects, and specialized compute.

For instance, the A100's memory clock is much lower than the RTX 4090's on paper (roughly 3 Gbps vs. 21 Gbps). However, the A100 uses **HBM2e memory** with a much wider **5,120-bit interface**. This design allows it to deliver around **2 TB/s** of bandwidth – double the RTX 4090 – despite the lower frequency. It's an approach that prioritizes efficiency and scale.

#### To recap, here are a few notable features of the A100:

* **80 GB of HBM2e memory** with about **2 TB/s of bandwidth** (twice that of the RTX 4090).  
    
* **5,120-bit memory bus**, an exceptionally wide interface enabling fast access to memory, reducing bottlenecks in AI and high-performance computing (HPC) workloads.  
    
* **432 third-gen Tensor Cores**, with high-throughput **TF32** performance for AI and robust **FP64** support for HPC workloads.  
    
* **Multi-Instance GPU (MIG)** support, so it can be partitioned into up to 7 isolated GPU instances – ideal for multi-tenant workloads and fine-grained resource allocation.  
    
* **NVLink and NVSwitch** allow up to 16 A100 GPUs to be interconnected at up to **600 GB/s** of bi-directional bandwidth per GPU.


  
In short, the A100 delivers serious performance for large-scale AI and HPC workloads. But if you're comparing it head-to-head with the RTX 4090 on specs alone, here's how the two stack up.

### RTX 4090 vs. A100: Key Specs Compared

For reference, this table highlights some of the features and specs of the RTX 4090 and A100:

| Specification | RTX 4090 | A100 |
| :---- | :---: | ----- |
| **Architecture** | Ada Lovelace | Ampere |
| **VRAM** | 24 GB GDDR6X | 80 GB HBM2e |
| **Memory Clock** | 1313 MHz 21 Gbps | 1512 MHz 3 Gbps |
| **Memory Bus Width** | 384 | 5120 |
| **Bandwidth** | 1.01 TB/s | 1.94 TB/s |
| **Streaming Multiprocessors (SMs)** | 128 | 108 |
| **Tensor Cores** | 512 | 432 |
| **CUDA Cores** | 16,384 | 6,912 |
| **Ray Tracing Cores** | 128 | N/A |
| **Base Clock** | 2235 MHz | 1065 MHz |
| **Boost Clock** | 2520 MHz | 1410 MHz |
| **Multi-Instance GPUs** | No MIG Support | Up to 7 MIGs @ 10GB each |
| **Thermal Design Power (TDP)** | 450W | 300W |
| **Recommended Power Supply** | 850W | 700W |
| **Launch Date** | Sept. 20, 2022 | June 28, 2021 |

As mentioned, specs are just one part of the equation. Ultimately, choosing the right GPU depends on your workload, environment, and priorities. 

### Use Cases: Which GPU Should You Choose?

Here’s how the strengths of each GPU align with different types of use cases.

#### You should consider the RTX 4090 if you prefer:

* **A powerful, desktop-friendly GPU** for high-end gaming, creative workflows, and light AI experimentation.  
* **Excellent performance-per-dollar in single-GPU setups**, particularly for developers and small teams.  
* **A widely available, less costly option** for rendering, video editing, or running small to medium AI models locally.

#### Whereas the A100 may be a better fit if you need:

* **Scalable performance** for training large models, running high-throughput inference, or powering complex simulations.  
* **A data center-optimized GPU** with advanced memory architecture and precision modes tailored for AI and HPC workloads.  
* **Flexible deployment options**, including support for multi-instance workloads and high-speed interconnects in clustered environments.

\------------------------------

At **Vast.ai**, we understand there are other considerations, as well. Access, flexibility, and affordability are just as important, especially when you're experimenting, scaling, or moving fast. 

That's why our market-based cloud GPU rental platform gives you affordable, on-demand access to RTX 4090s, A100s, and much more. With Vast.ai, you can instantly match the right hardware to your workload – without the upfront investment and overhead of managing your own infrastructure.

Try Vast today and discover what you can achieve with compute power that fits your needs and your budget\!

Compare NVIDIA’s RTX 4090 vs A100 GPUs to find out which is better for gaming, AI, HPC, or creative work—and how to access both on Vast.ai.

NVIDIA RTX 4090 vs. A100: Two Powerhouses, Two Purposes


Welcome back to another update from the team at **Vast.ai**\! Over the past two months, we've been hard at work refining our platform and rolling out features that make your experience on Vast faster, smoother, and more powerful. These updates are part of our ongoing effort to support your projects with scalable, flexible infrastructure you can rely on.

We're also pleased to have achieved [**SOC 2 Type I certification**](https://vast.ai/article/vast-ai-achieves-soc-2-type-1-certication) recently, a clear validation of the controls we've put in place to protect customer data and ensure the integrity of our infrastructure. The independent SOC 2 audit confirms that our security practices meet the highest standards of regulatory compliance – giving you added confidence as you build on our platform.

Another exciting update: [**H200 GPUs**](https://vast.ai/article/nvidia-h100-vs-h200-two-hopper-based-heavyweights) are now available on Vast\! If you're pushing the limits of high-performance compute, these machines are ready to meet the challenge.

Let's take a look at what else is new\!

### NVIDIA Cloud GPU Updates

Whether it's expanded hardware options, new templates, or updated guides, our latest improvements give you a more powerful, customizable environment for every stage of your workflow.

#### Highlights of the latest enhancements:

* **New Templates Added**

  * [***Dia 1.6b TTS***](https://cloud.vast.ai/?ref_id=62897&template_id=6ba071acb5f590913c5eaf6c2dc3d0db) – Convert text to lifelike speech using the open-source text-to-speech model from Nari Labs (full guide [here](https://docs.vast.ai/tts-with-nari-labs-dia))..  
  * [***Tari Universe***](https://cloud.vast.ai/?ref_id=62897&template_id=338f9628e43e586a87d37d18c719291d) – Run crypto mining workloads on Vast via desktop application (note: cryptocurrency funding is required).  
  * [***Pinokio Desktop***](https://cloud.vast.ai/?ref_id=62897&template_id=06c4818322f002904b324165b9dba52f) – Deploy AI applications with verified scripts from trusted publishers, without manual setup.

* **Introducing Local Volumes (CLI-only)**

  * Local Volumes are **persistent Docker volumes** that keep your data intact across restarts, new instances, and failures – so your storage survives instance deletion.  
  * **Benefits:** Share one volume across multiple containers on the same machine, and spin up fresh instances without data-transfer overhead. 

* **Upgraded Instance Portal**

  * For Vast.ai instances using [recommended templates](https://cloud.vast.ai/templates/), instances now launch via **secure Cloudflare tunnels** by default.  
  * Create a user-friendly **custom tunnel URL** that points to a port on your instance.  
  * **Live logs** and **live stats** (CPU, Memory, and Disk) for your instance are now viewable.

* **Vast Teams Improvements**

  * **Smoother invites** (fast and easy team creation), **standalone teams** (no need to convert your individual account), and **context switching** (seamlessly switch between Individual and Team views).  
  * **Access Keys:** Share API Keys and view Team-wide Session Keys as owner.  
  * Various stability and usability improvements.

For more information and helpful links, check out the full change log for our April–May 2025 updates at the bottom of this page.

### Our Commitment

Whether you're trying out the H200, deploying AI applications with just a few clicks, launching a new template, or exploring our latest platform features, we're here to make your experience on **Vast.ai** as smooth and flexible as possible. However you use Vast, our goal remains the same: to deliver reliable, cost-effective compute whenever you need it.

Got questions? Help is always within reach\! Contact our customer support team at [support@vast.ai](mailto:support@vast.ai) anytime, or join our Discord server for additional resources and community support.

### Change Log

#### New Templates

* **Dia 1.6b TTS** – Transform text into ultra-realistic speech using Dia, the open-source text-to-speech model from Nari Labs.  
  * [Full Guide](https://docs.vast.ai/tts-with-nari-labs-dia)  
  * [Launch an Instance with the Dia Template](https://cloud.vast.ai/?ref_id=62897&template_id=6ba071acb5f590913c5eaf6c2dc3d0db)  
* **Tari Universe** – Desktop Crypto mining application you can run on Vast. IMPORTANT: You must fund your Vast account with cryptocurrency if you want to use the service for mining.  
  * [Launch an Instance with the Tari Universe Template](https://cloud.vast.ai/?ref_id=62897&template_id=338f9628e43e586a87d37d18c719291d)  
* **Pinokio Desktop** – Access an extensive library of verified scripts from trusted publishers and deploy AI applications effortlessly without manual setup.  
  * [Launch an Instance on Vast with the Pinokio Desktop Template](https://cloud.vast.ai/?ref_id=62897&template_id=06c4818322f002904b324165b9dba52f)

####  Introducing Local Volumes (CLI only – UI coming soon\!)

**What are Local Volumes?**  
Persistent Docker volumes to keep your data intact across restarts, new instances, and failures.

**Key client benefits:**  
• Persistent storage survives instance deletion.  
• Share one volume across multiple containers on the same machine.  
• Spin up fresh instances on the same machine without data-transfer overhead.

**Get started:**  
• Search for available Volumes via CLI:  vastai search volumes  
• General docs: [https://docs.vast.ai/volumes](https://docs.vast.ai/volumes) 

*Note: Currently, clients can only rent Local Volumes through CLI. UI support and Network Volumes are coming soon.*

#### Teams Improvements

We’ve improved the Vast Teams experience for user access management and collaboration:

* **Smoother Invites:** Get team members connected on Vast quickly.  
* **Standalone Teams:** Create Teams without converting your individual account.  
* **Context Switching:** Seamlessly switch between individual and Team views.  
* **Access Keys:** Share API Keys and view Team-wide Session Keys as owner.  
* **Team Basics:** Transfer credits to/from your team, transfer ownership, and other basics.  
* **Bug Fixes:** Various stability and usability improvements.

[Create a team](https://cloud.vast.ai/manage-members/) and let us know your feedback\!

#### Instance Portal upgrade

We've also rolled out a major upgrade to the **Instance Portal** that appears on Vast.ai instances using [recommended templates](https://cloud.vast.ai/templates/).

**What’s New:**

* **Secure Tunnel URLs:** Instances now launch via secure Cloudflare tunnels by default, eliminating the “not secure” screen.  
* **Custom Tunnel URLs:** Create a user-friendly URL that points to a port on your Instance. Optionally use a named tunnel to use a custom domain.  
* **Live Logs:** See a live stream of log updates on your Instance.  
* **Live Stats:** See live CPU, Memory, and Disk stats for your Instance.

Deploy a recommended template instance or check out the [updated Instance Portal guide](https://docs.vast.ai/instance-portal) to learn more\!

#### Opt in to the new Instances UI

We've rolled out an (optional) UI update as part of our ongoing UI refresh:

* Cleaner instance card design  
* Instance data fetching improvements  
* Various instance UI bug fixes

Opt-in: Go to [Settings \> Early Access](https://cloud.vast.ai/account/) and toggle it on. You can switch back anytime.

#### SOC2 Type I Certification complete.

#### H200s available now\!

Vast.ai adds H200 GPUs, SOC 2 Type I cert, new templates, persistent storage, and better teams & instance tools for faster, secure cloud compute.

Blog

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai

June 2025 Product Update

A Decade Later: Revisiting the Brain as a Universal Learning Machine

Efficiently Serving Multiple Machine Learning Models with Lorax and vLLM on Vast.ai

7 Reasons Why Startups Use Vast.ai

NVIDIA RTX 4090 vs. A100: Two Powerhouses, Two Purposes

April-May 2025 Product Updates

Blog

DeepSeek R1-0528: Enhanced Reasoning with Simplified Thinking Mode on Vast.ai

June 2025 Product Update

A Decade Later: Revisiting the Brain as a Universal Learning Machine

Efficiently Serving Multiple Machine Learning Models with Lorax and vLLM on Vast.ai

7 Reasons Why Startups Use Vast.ai

NVIDIA RTX 4090 vs. A100: Two Powerhouses, Two Purposes

April-May 2025 Product Updates

Subscribe for our product updates.