Running Oobabooga in API Mode on

- Team Vast

October 28, 2023-Vast

Introduction to Oobabooga

Oobabooga is a tool designed for interacting with Large Language Models (LLMs), capable of understanding and generating text. With its user-friendly web interface, it's perfect for non-technical users interested in harnessing the power of LLMs. Oobabooga supports a wide range of models, excelling in various use-cases like prompting, loading, and managing models.

Why Run Oobabooga in API Mode vs Standard Mode?

Choosing API Mode for Oobabooga on offers several advantages, including enhanced scalability for multitasking, seamless tool integration via user-friendly endpoints, and streamlined workflow automation. Given these benefits, let's dive into how to actually run Oobabooga on

Why Choose for Oobabooga Deployment?

Running Oobabooga can be GPU-intensive, posing challenges for those without high-end GPUs or the financial resources to acquire them. That's where, a cloud-based GPU rental platform, comes in. By deploying Oobabooga on, you gain access to powerful, on-demand GPU resources and pay only for what you use, enabling complex text-generation tasks without hardware limitations.

Launching Oobabooga on A Step-by-Step Guide

Logging In to

  1. Log into your account.
  2. Ensure you have enough credits for the task.

Selecting Oobabooga Template:

  1. Click into the console.
  2. Click ‘EDIT IMAGE & CONFIG.’

  1. Select the recommended template for Oobabooga.

  1. Click on ‘EDIT’

  1. Scroll down to ‘On-start Script’ and type in ‘--api’ before the final semicolon

  1. Click on ‘Select And Save’

Allocating Disk Storage:

  1. Adjust disk storage to a minimum of 80GB.

Choosing a Suitable GPU:

  1. Review the Large Language Model (LLM) leaderboard on Hugging Face to understand the GPU RAM requirements of your chosen model.
  2. Filter for the total GPU RAM on the machine. Oobabooga can combine the GPU RAM on multi-GPU instances to load the model.

Launching Oobabooga Instance:

  1. Click ‘Rent’.

  1. Go to ‘Instances’

  1. Click ‘Open’ on your newly created instance.

Final Thoughts

Oobabooga's user-friendly interface and versatile support for various LLMs make it an invaluable tool for text generation tasks. When considering deployment options, emerges as a cost-effective solution, providing access to powerful GPU resources on-demand. Following this guide, users can easily set up Oobabooga on, bypassing any hardware hurdles, and making using Oobabooga straightforward and simple.

Share on
  • Contact
  • Get in Touch