Running DeepSeek R1 + Crew AI on Vast.ai
August 20, 2025
13 Min Read
By Team Vast
Introduction
This post demonstrates how to deploy DeepSeek-R1-0528-Qwen3-8B using vLLM on Vast.ai and integrate it with Crew AI for multi-agent workflows. The setup creates an OpenAI-compatible API endpoint that Crew AI can use as a custom LLM provider.
What is Crew AI?
Crew AI is an open-source framework for building multi-agent AI systems. It allows you to create autonomous AI agents that can work together on complex tasks, with each agent having its own role, goals, and tools. The framework handles agent coordination, task delegation, and workflow orchestration.
Key features of Crew AI:
- Multi-Agent Systems: Define multiple agents with specific roles and expertise that collaborate to complete tasks
- Task Orchestration: Agents can delegate tasks to each other and work sequentially or in parallel
- Tool Integration: Agents can use custom tools and APIs to interact with external systems
- LLM Flexibility: Works with any OpenAI-compatible API endpoint, including self-hosted models
What is Vast.ai?
Vast.ai provides a marketplace for renting GPU compute power, offering a cost-effective alternative to major cloud providers. It allows us to find and rent specific GPU configurations that match our model's requirements, with the flexibility to select hardware optimized for specific models.
In This Guide
We will:
- Deploy DeepSeek R1 on Vast.ai using vLLM with OpenAI-compatible API
- Configure Crew AI to connect to the custom model endpoint
- Create AI agents powered by DeepSeek's reasoning capabilities
- Build and execute multi-agent workflows for complex tasks
- Demonstrate cost-effective AI agent deployment at scale
Setting Up the Environment
First, install the Vast CLI and configure your API key. You can get your API key on the Account Page in the Vast Console:
#In an environment of your choice
pip install --upgrade vastai
# Set your api key
export VAST_API_KEY="" # Your key here
vastai set api-key $VAST_API_KEY
Next, install the required packages for our Crew AI integration:
pip install --upgrade litellm langchain-openai
pip install --upgrade openai crewai crewai-tools
Choosing the Right Hardware
DeepSeek R1 (8B parameters) with Crew AI multi-agent workflows requires:
- GPU RAM: 24GB minimum for model weights and extended reasoning sequences
- Recommended GPU: RTX 4090, A6000, or RTX A5000
- Disk Space: 60GB for model files and dependencies
- Static IP: Required for stable API endpoint across agent interactions
- Direct Port: Needed to expose vLLM's API server
We'll search for suitable hardware:
vastai search offers "compute_cap >= 750 \
gpu_ram >= 24 \
num_gpus = 1 \
static_ip = true \
direct_port_count >= 1 \
verified = true \
disk_space >= 60 \
rentable = true"
Pricing varies depending on GPU type and availability.
Deploying DeepSeek R1
Choose an instance ID from your search results and deploy using vLLM's OpenAI-compatible server:
export INSTANCE_ID="" # Insert instance ID
# Deploy with the converted Docker command for DeepSeek R1-0528-Qwen3-8B
vastai create instance $INSTANCE_ID --image vllm/vllm-openai:latest --env '-p 8000:8000' --disk 60 --args --model deepseek-ai/DeepSeek-R1-0528-Qwen3-8B --served-model-name custom/deepseek --max-model-len 4096 --reasoning-parser qwen3
Key parameters explained:
--image vllm/vllm-openai:latest: Provides OpenAI-compatible API endpoints--reasoning-parser qwen3: Enables DeepSeek's step-by-step reasoning capabilities--max-model-len 4096: Sufficient context for most multi-agent workflows--served-model-name custom/deepseek: Clean model name for Crew AI integration
The deployment will take 5-10 minutes to download the model and start serving.
Connecting to Your Instance
To get your instance connection details:
- Navigate to the Instances tab
- Find your instance and click the IP address button
- Copy the external IP address and port
You should see something like:
XX.XX.XXX.XX:YYYY -> 8000/tcp
Test the connection to verify your model is ready:
import requests
VAST_IP_ADDRESS = "" # Your instance IP
VAST_PORT = "" # Your instance port
# Check what model your API serves
response = requests.get(f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1/models")
models = response.json()
model_name = models['data'][0]['id']
print(f"API serves model: {model_name}")
Building Multi-Agent Workflows with Crew AI
Now let's create a sophisticated multi-agent research team powered by DeepSeek's reasoning capabilities:
from crewai import Agent, Task, Crew
from langchain_openai import ChatOpenAI
from openai import OpenAI
# Replace with your actual Vast.ai instance IP and port
VAST_IP_ADDRESS = "" # Your instance IP
VAST_PORT = "" # Your instance port
client = OpenAI(
api_key="DUMMY",
base_url=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
)
import litellm
# Your Vast AI vLLM endpoint
litellm.api_base = f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1"
litellm.api_key = "DUMMY" # Required by litellm even if unused
import requests
# Check what model your API serves
response = requests.get(f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1/models")
models = response.json()
model_name = models['data'][0]['id']
print(f"API serves model: {model_name}")
# Create LLM connection for Crew AI
crew_llm = ChatOpenAI(
model=f"openai/{model_name}",
openai_api_base=f"http://{VAST_IP_ADDRESS}:{VAST_PORT}/v1",
openai_api_key="DUMMY",
temperature=0.7,
max_tokens=1000,
request_timeout=60
)
print(f"✅ Using model: openai/{model_name}")
Creating Specialized AI Agents
Define a collaborative research team with distinct roles and expertise:
# Create specialized agents for our research team
research_analyst = Agent(
role="Senior Research Analyst",
goal="Uncover cutting-edge developments in AI and technology",
backstory="""You are a veteran researcher with 15 years of experience in technology analysis.
You're known for your ability to identify important trends and synthesize complex information.""",
llm=crew_llm,
verbose=True
)
tech_writer = Agent(
role="Tech Writer",
goal="Craft compelling narratives about complex technical topics",
backstory="""You are an experienced technical writer who excels at making complex topics
accessible. You have a talent for explaining technical concepts in clear, engaging ways.""",
llm=crew_llm,
verbose=True
)
quality_reviewer = Agent(
role="Editorial Reviewer",
goal="Ensure accuracy and clarity in technical communications",
backstory="""You are a meticulous editor with a keen eye for detail. You ensure all content
is accurate, well-structured, and easy to understand.""",
llm=crew_llm,
verbose=True
)
Executing Complex Research Tasks
Define collaborative tasks that leverage each agent's expertise:
# Define tasks for each agent
research_task = Task(
description="""Research and analyze the impact of open-source LLMs on enterprise AI adoption.
Focus on:
1. Key open-source models being adopted by enterprises
2. Cost comparisons with proprietary solutions
3. Main use cases and success stories
4. Challenges and limitations
Provide a comprehensive analysis with specific examples.""",
expected_output="A detailed research report with findings and insights",
agent=research_analyst
)
writing_task = Task(
description="""Based on the research findings, create a concise executive summary that:
1. Highlights the 3 most important insights
2. Includes specific examples and data points
3. Provides actionable recommendations for enterprises
Make it engaging and accessible to non-technical executives.""",
expected_output="A polished executive summary of 200-300 words",
agent=tech_writer
)
review_task = Task(
description="""Review the executive summary for:
1. Technical accuracy
2. Clarity and readability
3. Logical flow and structure
Provide specific feedback and a final approval.""",
expected_output="Review feedback and final approved version",
agent=quality_reviewer
)
# Create and execute the crew
print("🚀 Starting multi-agent research team...\n")
crew = Crew(
agents=[research_analyst, tech_writer, quality_reviewer],
tasks=[research_task, writing_task, review_task],
verbose=1
)
result = crew.kickoff()
print(f"\n📊 Final Result:\n{result}")
Multi-Agent Research Team Output
This section shows the complete execution of our DeepSeek-powered Crew AI workflow, demonstrating how three specialized agents collaborate to produce comprehensive research and analysis.
🚀 Crew Initialization
The workflow begins with Crew AI setting up the execution environment:
🚀 Starting multi-agent research team...
╭──────────────────────────────────────────── Crew Execution Started ─────────────────────────────────────────────╮
│ │
│ Crew Execution Started │
│ Name: crew │
│ ID: e303ddc0-b84c-4c4f-ba6f-5934c767eceb │
│ Tool Args: │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
📊 Phase 1: Research Analysis
Our Senior Research Analyst agent conducts comprehensive research on open-source LLMs in enterprise environments:
╭─────────────────────────────────────────────── 🤖 Agent Started ────────────────────────────────────────────────╮
│ │
│ Agent: Senior Research Analyst │
│ │
│ Task: Research and analyze the impact of open-source LLMs on enterprise AI adoption. │
│ Focus on: │
│ 1. Key open-source models being adopted by enterprises │
│ 2. Cost comparisons with proprietary solutions │
│ 3. Main use cases and success stories │
│ 4. Challenges and limitations │
│ │
│ Provide a comprehensive analysis with specific examples. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
The Research Analyst produces a comprehensive analysis including:
╭───────────────────────────────────────────── ✅ Agent Final Answer ─────────────────────────────────────────────╮
│ │
│ Agent: Senior Research Analyst │
│ │
│ Final Answer: │
│ **Research Report: Impact of Open-Source Large Language Models (LLMs) on Enterprise AI Adoption** │
│ │
│ **Executive Summary:** │
│ Open-source Large Language Models (LLMs) are rapidly transitioning from research projects to enterprise-grade │
│ tools, offering a compelling alternative or complement to proprietary AI solutions. Enterprises are │
│ leveraging these models for cost savings, customization, avoiding vendor lock-in, and accelerating │
│ innovation. While challenges related to integration, data security, performance tuning, and talent exist, the │
│ trend indicates a significant shift in how businesses approach and deploy AI, particularly for internal │
│ applications, research, and development. │
│ │
│ **1. Key Open-Source Models Being Adopted by Enterprises:** │
[... detailed model output truncated ...]
│ **Conclusion:** │
│ │
│ The adoption of open-source LLMs by enterprises represents a significant shift toward more democratized, │
│ cost-effective, and customizable AI solutions. While challenges exist, particularly around infrastructure, │
│ talent, and integration complexity, the benefits of cost savings, customization, and avoiding vendor lock-in │
│ are driving accelerated adoption across industries. │
│ │
│ Enterprises most successful with open-source LLM adoption tend to have: │
│ 1. Clear use cases with high-volume, consistent usage patterns │
│ 2. Dedicated AI/ML teams or partnerships with specialized vendors │
│ 3. Robust data security and compliance frameworks │
│ 4. Long-term strategic commitment to AI capabilities development │
│ │
│ As the open-source LLM ecosystem continues to mature, with improved tooling, managed services, and more │
│ permissive licensing, enterprise adoption is expected to accelerate significantly over the next 2-3 years. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
✍️ Phase 2: Content Creation
The Tech Writer agent transforms the research into an executive-ready summary:
╭─────────────────────────────────────────────── 🤖 Agent Started ────────────────────────────────────────────────╮
│ │
│ Agent: Tech Writer │
│ │
│ Task: Based on the research findings, create a concise executive summary that: │
│ 1. Highlights the 3 most important insights │
│ 2. Includes specific examples and data points │
│ 3. Provides actionable recommendations for enterprises │
│ │
│ Make it engaging and accessible to non-technical executives. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Executive Summary Output:
╭───────────────────────────────────────────── ✅ Agent Final Answer ─────────────────────────────────────────────╮
│ │
│ Agent: Tech Writer │
│ │
│ Final Answer: │
│ **Executive Summary: Open-Source AI Models Transform Enterprise Operations** │
│ │
│ Open-source large language models (LLMs) are reshaping how enterprises approach artificial intelligence, │
│ offering compelling alternatives to expensive proprietary solutions. Our analysis reveals three critical │
│ insights that should inform every executive's AI strategy. │
│ │
│ **1. Dramatic Cost Reduction Potential** │
[... detailed model output truncated ...]
│ The open-source AI revolution is not a future trend—it's happening now. Enterprises that act decisively to │
│ evaluate and adopt these technologies will gain significant competitive advantages in cost efficiency, │
│ customization capabilities, and strategic control over their AI future. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
🔍 Phase 3: Quality Review
The Editorial Reviewer agent performs final quality assurance:
╭─────────────────────────────────────────────── 🤖 Agent Started ────────────────────────────────────────────────╮
│ │
│ Agent: Editorial Reviewer │
│ │
│ Task: Review the executive summary for: │
│ 1. Technical accuracy │
│ 2. Clarity and readability │
│ 3. Logical flow and structure │
│ │
│ Provide specific feedback and a final approval. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Editorial Review Output:
╭───────────────────────────────────────────── ✅ Agent Final Answer ─────────────────────────────────────────────╮
│ │
│ Agent: Editorial Reviewer │
│ │
│ Final Answer: │
│ **Editorial Review and Final Approval** │
│ │
│ **Review Assessment:** │
│ │
│ **Technical Accuracy: ✅ APPROVED** │
│ The executive summary accurately reflects the research findings with correct data points and realistic cost │
│ projections. The 60-80% cost savings figure aligns with industry reports, and the specific example of the │
│ retail company's $320,000 investment with 9-month break-even is mathematically sound and represents typical │
│ enterprise deployment scenarios. │
│ │
│ **Clarity and Readability: ✅ APPROVED** │
│ The document successfully translates complex technical concepts into accessible business language. Technical │
│ jargon is minimized, and each insight is supported by concrete examples that non-technical executives can │
│ easily understand and relate to their own organizational contexts. │
│ │
│ **Logical Flow and Structure: ✅ APPROVED** │
│ The summary follows a clear progression from identifying opportunities (cost savings) to competitive │
│ advantages (customization) to strategic considerations (control and future-proofing). The actionable │
│ recommendations are appropriately segmented by timeline and provide clear next steps for decision-makers. │
│ │
│ **Minor Enhancement Suggestion:** │
│ Consider adding a brief risk mitigation note about the talent and infrastructure requirements to provide a │
│ balanced perspective for executive decision-making. │
│ │
│ **Final Approved Version:** │
│ │
│ **Executive Summary: Open-Source AI Models Transform Enterprise Operations** │
│ │
│ Open-source large language models (LLMs) are reshaping how enterprises approach artificial intelligence, │
│ offering compelling alternatives to expensive proprietary solutions. Our analysis reveals three critical │
│ insights that should inform every executive's AI strategy. │
│ │
│ **1. Dramatic Cost Reduction Potential** │
│ Enterprises are achieving 60-80% cost savings by transitioning from proprietary AI services to open-source │
│ alternatives. A retail company reduced monthly AI expenses from $45,000 to $10,000 by self-hosting Llama 2 │
│ models, achieving break-even in just 9 months despite a $320,000 initial investment. For high-volume users │
│ processing over 10 million tokens monthly, the economics strongly favor open-source deployment. │
│ │
│ **2. Customization Drives Competitive Advantage** │
│ Unlike closed systems, open-source models can be fine-tuned on proprietary company data, creating unique │
│ competitive advantages. A financial services firm developed custom models for personalized client reporting, │
│ while an e-commerce platform achieved 92% customer satisfaction with automated support handling 70% of │
│ inquiries. This customization capability allows enterprises to embed their specific knowledge, processes, and │
│ brand voice directly into AI systems. │
│ │
│ **3. Strategic Control and Future-Proofing** │
│ Open-source adoption eliminates vendor lock-in while providing complete control over sensitive data and AI │
│ capabilities. A pharmaceutical company now analyzes research literature internally using fine-tuned models, │
│ reducing analysis time from weeks to days while maintaining strict data privacy. This approach future-proofs │
│ AI investments against pricing changes and service discontinuations. │
│ │
│ **Actionable Recommendations** │
│ │
│ **Immediate Actions (Next 90 Days):** │
│ - Identify high-volume, repetitive AI use cases within your organization │
│ - Pilot open-source models for internal applications where data privacy is critical │
│ - Assess current AI spending and usage patterns to quantify potential savings │
│ │
│ **Strategic Investments (6-12 Months):** │
│ - Build internal AI capabilities or establish partnerships with specialized vendors │
│ - Develop data governance frameworks suitable for AI model training │
│ - Create hybrid deployment strategies combining open-source and proprietary solutions based on use case │
│ requirements │
│ │
│ **Long-term Positioning (12+ Months):** │
│ - Establish centers of excellence for AI model development and deployment │
│ - Invest in secure, compliant infrastructure for self-hosted AI models │
│ - Develop proprietary datasets and fine-tuning capabilities as strategic differentiators │
│ │
│ **Implementation Considerations:** │
│ Success requires significant upfront investment in specialized talent and infrastructure. Enterprises should │
│ evaluate their technical capabilities and consider managed service partnerships to mitigate implementation │
│ risks while capturing the strategic benefits of open-source AI adoption. │
│ │
│ The open-source AI revolution is not a future trend—it's happening now. Enterprises that act decisively to │
│ evaluate and adopt these technologies will gain significant competitive advantages in cost efficiency, │
│ customization capabilities, and strategic control over their AI future. │
│ │
│ **EDITORIAL APPROVAL: ✅ APPROVED FOR PUBLICATION** │
│ *This document has been reviewed for accuracy, clarity, and completeness. Ready for executive distribution.* │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
🎯 Workflow Completion
╭──────────────────────────────────────────── Crew Execution Completed ───────────────────────────────────────────╮
│ │
│ │
│ Crew Execution Completed │
│ Total execution time: 0:02:18 │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Results
The workflow output demonstrates the potential of this multi-agent approach clearly. The research analyst produced a comprehensive enterprise AI adoption analysis with specific examples and quantified benefits, diving deep into cost comparisons, use cases, and implementation challenges. The technical writer then transformed this detailed research into executive-ready insights with actionable recommendations segmented by timeline. Finally, the editorial reviewer ensured accuracy and clarity throughout, providing both critique and a polished final version.
Conclusion
This guide demonstrated how to combine DeepSeek R1's advanced reasoning, Crew AI's multi-agent orchestration, and Vast.ai's flexible GPU marketplace to create sophisticated AI workflows that were previously only accessible to organizations with massive infrastructure budgets. The integration shows that complex AI applications no longer require dedicated data centers or long-term hardware commitments.
Vast.ai makes this advanced AI orchestration accessible to any organization by providing on-demand access to the computational power needed without massive upfront investment. Whether you're experimenting with new AI workflows or scaling existing operations, the combination of DeepSeek's reasoning, Crew AI's orchestration, and Vast.ai's flexible infrastructure opens up entirely new possibilities for automated knowledge work.
