
# Vast.ai September Updates

It's that time again! We're back with the latest updates here at **Vast.ai**, aimed at bringing you the best possible GPU rental platform experience. This month, we've rolled out numerous template updates as well as added a [new guide](https://vast.ai/docs/use-cases/10-serving-infinity) to our Docs on serving Infinity Embeddings.

As always, addressing bugs is one of our top priorities to ensure seamless performance. Here are some of the latest bug fixes our team has implemented since our last update:

- **Cloud copy** no longer sends wrong parameters.
- **UI login** when CAPTCHA is incorrect should now delete cache/cookies.
- **Billing page** now loads correctly (previously had issues on some browsers).
- Fixed an issue where the **templates page** sometimes rendered before Recommended templates were loaded.

## NVIDIA Cloud GPU Updates

With each update here at Vast.ai, we're focused on improving both the flexibility and capabilities of our platform. It's our hope that these updates will streamline your experience and help maximize your productivity on Vast. Below are some highlights of the latest changes we've made:

- A new guide for [serving Infinity Embeddings](https://vast.ai/article/serving-vllm-embeddings-on-vast-ai) has been added to the Docs section of our website.
- Added separate **8x** and **8x+ GPU #** filters.
- New **GUI & CLI** to change the template of an existing contract (within constraints), then destroy and recreate.
- New **CLI/API** for audit logs of important user commands (create, destroy, etc.) with `user_id`, `timestamp`, `IP`, etc.
- Added new **ComfyUI + FLUX.1** template, **Stable Diffusion WebUI Forge** template, and **Ollama** template.
- Updated **PyTorch** to the most recent compatible version.

For your convenience, you can check out the full changelog for our **September 2024** product updates at the bottom of this page.

## Our Commitment

Our goal at **Vast.ai** is to provide an exceptional and affordable cloud GPU rental service for both renters and hosts. Whether you're beginning a new project or fine-tuning current ones, we're here to support you throughout the entire process. Feel free to reach out any time through the chatbox at the bottom right, send us an email at [support@vast.ai](mailto:support@vast.ai), or join our **Discord server** for ongoing support and resources!

---

## Change Log

### Features:

- Added separate **8x** and **8x+ GPU #** filters.
- New **GUI & CLI** to change the template of existing contracts (within constraints), then destroy and recreate.
- New **CLI/API** for audit logs of important user commands (create, destroy, etc.) with `user_id`, `timestamp`, `IP`, etc.

### Templates / Miscellaneous:

- Added [guide](https://vast.ai/docs/use-cases/10-serving-infinity) for **serving Infinity Embeddings** to the Docs section on the website.
- Updated **PyTorch** to the most recent compatible version.
- Added new **ComfyUI + FLUX.1** template.
- Updated **A1111** template for TLS compatibility.
- Updated **Fooocus** template for TLS.
- Updated **TGI** template (TLS/Auth).
- Updated **Kohya's GUI** template (TLS/Auth).
- Added **Stable Diffusion WebUI Forge** template.
- Added **Ollama** template.

### Bug Fixes:

- Fixed an issue where the **templates page** sometimes rendered before Recommended templates were loaded.
- **UI login** when CAPTCHA is incorrect should now delete cache/cookies.
- **Cloud copy** no longer sends wrong parameters.
- **Billing page** now loads correctly (previously had issues on some browsers).


Discover the latest features and updates from Vast.ai.

September 2024 Product Update


# The much-anticipated NVIDIA GeForce RTX 5090 GPU, expected to launch later this year, has already generated a lot of buzz in the tech world.

With new leaks and rumors surfacing, join us as we explore the latest speculations – and what they could mean for gaming performance and for tech enthusiasts of all stripes.

## Behind the Buzz: RTX 5090's Rumored Features

The fastest consumer graphics card on the market is the NVIDIA GeForce RTX 4090. As its successor, the new flagship [RTX 5090](https://vast.ai/article/nvidia-rtx-4090-vs-5090) will have some pretty big shoes to fill. But based on what we know about it and what's been leaked, we're confident it'll be up to the task.

For one thing, the upcoming RTX 50-series GPU lineup will utilize NVIDIA's new Blackwell architecture line, leaving the Ada Lovelace generation behind. Along with that will reportedly come the GB202 gaming chip – which is expected to have significant performance gains over the current Ada Lovelace AD102 chips.

A reputable hardware leaker has [stated](https://www.dexerto.com/tech/nvidia-rtx-5090-2301788/) that the GB202 chip is set to share the same process node as the AI-focused GB101, and that it will have 12 Graphics Processing Clusters (GPCs), each containing 8 Texture Processing Clusters (TPCs), for a total of 96 TPCs.

Another reliable source indicated that the RTX 5090 will feature a "dense" memory layout with four modules on top, five on each side, and two at the bottom, necessitating a new PCB design. This aligns with rumors of a 512-bit memory bus – up from 384 in the RTX 4090 – since such a dense memory configuration would require significant bit width. Plus, the RTX 5090 could boast well over 1.5 TB/s in memory bandwidth compared to the RTX 4090's 1,008 GB/s.

Further recent leaks suggest that the RTX 5090 will have a base clock speed of ~2900 MHz. That's an incredible 30% increase over the RTX 4090! Not to mention, with boost clock engaged, speeds exceeding 3 GHz are on the horizon.

## Design and Power Enhancements

So what does all of this mean for design? One source [suggests](https://www.dexerto.com/tech/nvidia-rtx-5090-2301788/) that the Founders Edition of the RTX 5090 will feature a new PCB layout, divided into three sections: the main board, a rigid I/O board, and a separate PCIe slot section. This layout could make the RTX 5090 thinner than its predecessors, using a two-slot cooler with a dual-fan setup.

The RTX 5090 is also expected to include Display Port 2.1 outputs, an improvement on the Ada Lovelace series. This will enable it to support gaming monitors that feature increasingly higher resolutions and refresh rates.

The power connector has gotten some criticism, however. The latest rumors suggest that the RTX 5090 will use the same 12VHPWR connector as the RTX 4090 – which has [reportedly](https://www.tomshardware.com/news/nvidia-16-pin-gpu-power-connector-all-we-know) had some problems – but there is speculation that NVIDIA will switch to a more modern connector design in the RTX 50 series.

Power supply manufacturer Seasonic was responsible for one of the [latest leaks](https://www.pcgamesn.com/nvidia/geforce-rtx-5090-power-leak), in which details about the RTX 5090 were inadvertently revealed through the company's PSU wattage calculator online. Specifically, the RTX 5090 is listed in the drop-down menu as having 500W TDP (thermal design power) – a +50W change from the RTX 4090 – as well as a 1 x 16-pin 12VHPWR connector.

## The Next Big Leap in GPU Technology

There's also talk that the RTX 5090 could come with up to 192 streaming multiprocessors (SMs) and an impressive 24,576 CUDA cores if it continues using the 128-cores-per-SM configuration of the RTX 4090. Plus, rumors suggest an upgrade to 192 third-gen ray-tracing cores and 768 fourth-gen Tensor cores, improving significantly from the RTX 4090's 128 and 512, respectively.

This naturally means a major boost to performance. According to some leaks, the RTX 5090 will deliver up to 70% performance improvements – but it remains to be seen whether this is with ray tracing and/or DLSS enabled.

Still, if the RTX 5090 ends up being about 1.7x faster than the current fastest gaming GPUs on the market, then the Blackwell generation will represent one of the biggest leaps forward we've seen in a while.

## Spec Comparison

| Spec                             | RTX 4090          | RTX 5090         |
| -------------------------------- | ----------------- | ---------------- |
| **GPU Architecture**             | Ada Lovelace      | Blackwell        |
| **Streaming Multiprocessors**    | 128               | 192              |
| **CUDA Cores**                   | 16,384            | 24,576           |
| **Ray-Tracing Cores**            | 128               | 192              |
| **Tensor Cores**                 | 512               | 768              |
| **Base Clock**                   | 2.23 GHz          | 2.9 GHz          |
| **Boost Clock**                  | 2.52 GHz          | 3+ GHz           |
| **L2 Cache**                     | 72MB              | 128MB            |
| **Memory**                       | 12 GDDR6X modules | 14 GDDR7 modules |
| **Memory Bandwidth**             | 1,008 GB/s        | 1,532 GB/s       |
| **VRAM**                         | 24 GB             | 28 GB            |
| **Thermal Design Power (TDP)**   | 450W              | 500W             |
| **Support for Display Port 2.1** | No                | Yes              |

As we approach the likely release window of the RTX 5090 in Q4 this year, the wait is almost over…

## A New Era of Consumer-Grade Excellence

To sum up, the NVIDIA GeForce RTX 5090 is set to usher in a whole new generation of state-of-the-art consumer-grade GPU performance. From outstanding 4K resolution to next-gen DLSS 4 (AI-powered upscaling to further enhance performance and image quality), consumers are expecting big things from this graphics card.

It may be seen as a luxury upgrade from the already-exceptional RTX 4090 – but if you're looking for the best in class, the RTX 5090 will certainly deliver. The price, however, will also undoubtedly reflect its top-tier status. For some, the high upfront investment will be out of reach, or won't make sense for their needs.

Here at Vast.ai, we offer a budget-friendly solution for accessing next-level graphics and compute power. Renting GPUs on our platform lets you leverage the latest technology without the high upfront cost and upkeep of buying your own hardware. Ready to experience next-level GPU capabilities or prepare for the arrival of the RTX 5090? Explore Vast today to get the cutting-edge performance you need at an unbeatable price!


NVIDIA RTX 5090: Leaks, Rumors, and Next-Gen GPU Performance


In the complex landscape of data center operations, understanding and adhering to various compliance standards is crucial. This article will explore the differences and similarities between Tier 2/3 data center classifications and compliance standards like HIPAA and ISO 27001, providing insights for businesses navigating these regulations.

# Understanding Data Center Tiers

Data centers are classified into four tiers by the Uptime Institute, based on their uptime and redundancy levels. Tier 2 and Tier 3 data centers are particularly relevant for growing businesses and large organizations.

Tier 2 Data Centers offer a single path for power and cooling but include some redundant and backup components. They promise an uptime of 99.741% per year, translating to up to 22 hours of downtime annually​​.

Tier 3 Data Centers provide multiple paths for power and cooling, with redundant systems that allow maintenance without downtime. They boast an uptime of 99.982% per year, equating to less than 1.6 hours of downtime annually​​.

These tiers offer a balance between performance, reliability, and cost, making them suitable for businesses that require consistent uptime without the need for the extensive redundancies of Tier 4.

## Compliance Standards: HIPAA and ISO 27001

Compliance standards like HIPAA and ISO 27001 are essential for data centers, especially those handling sensitive information.

HIPAA (Health Insurance Portability and Accountability Act) is a U.S. legislation focused on protecting sensitive patient information managed by healthcare organizations. It requires entities to implement physical, administrative, and technical safeguards for Protected Health Information (PHI), which can be supported by the infrastructure and redundancy offered by Tier 2 and Tier 3 facilities​​.

ISO 27001 is an international standard for information security management. It includes 114 controls in areas like access control, physical and environmental security, and incident management. While ISO 27001 certification does not equate to HIPAA compliance, its controls significantly overlap with HIPAA requirements​. Tier 2 and Tier 3 data centers can support ISO 27001 compliance through their security measures, operational sustainability, and staff expertise​​.

# Vast.ai's Datacenter Compliance 

Vast.ai has a network of dozens of datacenters that have ISO 27001 and TIER 2 or 3 ratings. Vast.ai has done due diligence and confirmed that these provider’s equipment is in a facility with an up to date compliance certificate.

To find Vast.ai providers, use the “Secure Cloud (Only Trusted Datacenters)” filter on the search page at https://cloud.vast.ai/create/

Vast.ai secure cloud has providers with at least one of the following certifications:

- ISO 27001
- TIER 2
- TIER 3

For production services where uptime and data security are vital, use the trusted datacenter filter and look for the blue datacenter label.

# Final Thoughts

Choosing a data center that aligns with compliance needs is crucial for businesses. Tier 2 and Tier 3 data centers offer balanced and cost-effective solutions for organizations requiring consistent uptime and robust security measures. Vast.ai's secure cloud offering empowers businesses to navigate this landscape efficiently, ensuring compliance and security. As the digital landscape evolves, staying compliant and resilient remains a top priority, a challenge that Vast.ai is well-equipped to meet.


Navigating Data Center Compliance: Understanding Tier 2/3 and HIPAA/ISO 27001 Standards


As the year comes to a close, we're back one more time with the latest updates here at Vast.ai! At the moment, we're primarily working on a few major features that will be unveiled in due course – but for now, we're pleased to announce the smaller ongoing product updates that continue to make our GPU rental platform better and better over time.

First up are bug fixes. We're proud of our team's response time in getting every hiccup dealt with as quickly as possible, which ensures we're able to deliver a seamless experience for our valued customers. The following are some of our latest fixes:

- Fixed internet speedtest script for Ubuntu 23+
- All Coinbase payments now display a link to the transaction
- Fixed issues with Google Drive token expiration
- Cuda version detection is now fixed for TensorFlow
- Fixed some de-verification issues
- Template editors now accept > and <
- Fixed issues with Dropbox connections

#### NVIDIA Cloud GPU Updates

As mentioned, more changes are on the way! But in the past few weeks, we've implemented some enhancements to our platform that should make Vast even more user-friendly and intuitive for both current and new customers. Here are a few highlights of the latest updates:

- Added RTX 5000 Ada to search page
- Updated CLI documentation - https://vast.ai/docs/cli/commands
- Updated the displayed text for Docker image in the template editor
- Discount factor now visible for reserved instance discounts
- Added machine OS version filter to CLI/API
- Updated recommended templates to use AI Dock's ComfyUI template

Plus, we've updated our [Terms of Service](https://vast.ai/terms) to reflect that credits purchased via credit card cannot be used for cryptocurrency mining. The refund policy has also been clarified. And as usual, you can find the full change log for our December 2023 product updates below for your convenience.

We're proud to be the market leader in low-cost, on-demand cloud GPU rental. With Vast, you can save 5-6X on GPU compute compared to other solutions – all in one simple interface with a supportive Discord community and a responsive customer service team to answer all your questions! We'd love to chat and help you discover what Vast can do for you today. You're welcome to contact us any time via the chatbox at the bottom right or email us at [support@vast.ai](mailto:support@vast.ai). In the meantime, happy holidays!

#### Change Log

- Added RTX 5000 Ada to search page
- Schedule maintenance doesn't require a machine that is listed
- Added strong warning about copying / or /root or /root/.ssh
- Updated our docs to include using the CLI from inside an instance
- Bugfix: fixed internet speedtest script for Ubuntu 23+
- Bugfix: all Coinbase payments now display a link to the transaction
- Bugfix: host end date fix
- Bugfix: Cuda version detection fixed for TensorFlow
- Updated the displayed text for Docker image in the template editor
- Bugfix: host UI schedule maintenance UI button fixed
- Bugfix: display on billing warning message
- Bugfix: fixed some de-verification issues
- Bugfix: template editors now accept > and <
- Bugfix: in the template editor extra filters can be overwritten even if connected to the Docker repo
- Terms of Service has been updated (no mining with credits purchased via credit card; refund policy clarified)
- Reset API key now asks for confirmation
- CLI has more explicit error messages for syntax errors
- Updated CLI documentation - https://vast.ai/docs/cli/commands
- Discount factor now visible for reserved instance discounts
- Added machine OS version filter to CLI/API
- Updated destroyed instance cleanup
- Updated recommended templates to use AI Dock's ComfyUI template
- Bugfix: fixed issues with Google Drive token expiration
- Bugfix: earnings page should not allow you to see charges and payments for export
- Bugfix: kaalia: auto-detection of NVML and bad GPU errors
- Bugfix: kaalia: fix issue with update_scripts not installing on new machines
- Bugfix: fixed issues with Dropbox connections


December product updates


We're pleased to unveil the latest updates here at Vast.ai, geared toward achieving the best possible product experience for new and existing customers alike! Our aim is to provide you with a GPU rental platform that offers best-in-class service at a fraction of the price.

To that end, we work tirelessly to stay on top of the game when it comes to delivering improvements and enhancements. Bug fixes are always first on the agenda so we can ensure our customers enjoy a seamless experience at all times. Here are some of the latest fixes we've implemented:

- Refreshing now no longer deselects the selected template
- Fixed issue with setting internet pricing on host machine page
- Billing page now correctly displays no charges for offline instances
- GPU filters now working for all GPU types; added L40 and A6000ada
- Fixed incorrect invoice charges summation
- Scroll issues on instance page fixed
- Client earnings page now shows payouts for referrals

#### NVIDIA Cloud GPU Updates

Updating our product isn't just about bug fixes – we also extend our focus to implementing meaningful enhancements in a variety of ways. We want all of our customers to be able to make the most of their time on Vast! Below are some highlights of the latest changes:

- Documents and guides for Autoscaler (in beta) are now available – https://vast.ai/docs/autoscaler/introduction

- Contact us (on website chat or via email at contact@vast.ai) if you want to try out the Autoscaler for automating the provisioning of GPU workers for dynamic workloads.

- Added support for Oobabooga and Stable Diffusion for the Autoscaler beta
- Added warning to user for contract expiration
- Added example CLI autoscale script
- Added in PDF and CSV billing reports for clients with a date range picker
- Added a clean modal for hosts to manage machines in the GUI with a beta flag

We've also recently updated our Terms of Service, which can be read [here](https://vast.ai/terms). The full change log for our November 2023 product updates is included at the bottom of this page for your convenience – there are plenty of additional improvements we've made in recent weeks that you can check out below.

If you're new to Vast and looking for a cost-effective cloud GPU rental solution for any number of potential use cases (a few of which we outlined in a [recent blog post](https://vast.ai/article/GPU-Rental-Use-Cases)) we invite you to discover what our platform can do for you! Please reach out at any time via the chatbox at the bottom right or email us at [support@vast.ai](mailto:support@vast.ai) with any questions – we're happy to help!

#### Change Log

- Templates can now specify recommended disk size
- Added docs for Autoscaler (in beta) - https://vast.ai/docs/autoscaler/introduction

- Contact us (on website chat or email at contact@vast.ai) if you want to try out the Autoscaler for automating the provisioning of GPU workers for dynamic workloads.

- Added links to the txn for Crypto payments in billing
- Added confirmation on reboot
- Copy modal now has a pop-up with indication of success or failure
- For reserved instances, a pop-up now appears if there's not enough credit
- Updated our hosting landing page with info on how to become a datacenter partner ([https://vast.ai/hosting](https://vast.ai/hosting))
- Bugfix: issues with de-verification email to hosts
- Bugfix: issue with setting internet pricing on host machine page
- Bugfix: refreshing now no longer deselects the selected template
- Bugfix: payment information on billing page fixed
- Bugfix: fixes for templates with filters, such as GLX desktop
- Bugfix: SSH proxy port was wrong, now fixed
- Added in equivalent CLI command in each template
- /Host/setup is now the only place an account can be converted to a host account
- Added better code snippet text in docs
- Reworked the email notification system for when balance is zero
- Added warning to user for contract expiration
- Added example CLI autoscale script
- Bugfix: args for templates now filter machines properly
- Bugfix: autoscroll issues fixed
- Added in PDF and CSV billing reports for clients with a date range picker
- Added in multi-region support in the geo filter
- Added more clear arrows for the cloud copy UI to make it clear you can swap directions
- Added a warning pop-up for when you set your autobilling threshold above 0 (which could result in an immediate charge)
- Fixed the client referral earnings page to be more clear
- Instance deposits for reserved discounts now show instance ID in billing page
- Add or change invoices CLI/API endpoint to get charge history for a set of instances with contract-based filters (like instance tag for autoscaling groups)
- Bugfix: billing page now correctly displays no charges for offline instances
- Bugfix: fixed overflow/spacing issues on mobile browsers on the homepage
- Bugfix: GPU filters now working for all GPU types; added L40 and A6000ada
- Bugfix: fixed issue with payouts appearing on client billing pages
- Bugfix: invoices now use start date and end date properly
- Added support for Oobabooga and Stable Diffusion for the Autoscaler beta
- Added additional documentation for the Autoscaler
- Added in an easy switch link to the login & register modal
- Added feedback notifications and spinners on PDF exports for invoices
- Bugfix: fixed incorrect invoice charges summation
- Bugfix: fixed scroll issues on instance page
- Bugfix: date range is now properly set for hosts on the invoice/earnings page
- Bugfix: full invoice address now in the PDF invoices
- We have updated our Terms of Service ([https://vast.ai/terms](https://vast.ai/terms))
- Added a clean modal for hosts to manage machines in the GUI with a beta flag
- Added a South America filter
- Added a verification email re-sent button on account page
- Added additional documentation and guides for the Autoscaler beta
- Template name and image are now displayed when available in the UI
- You can now override the entrypoint for args launch instances using the option --onstart-cmd (or aliased as --entrypoint). For example: vastai create instance 7277200 --image pytorch/pytorch --disk 40 --onstart-cmd 'bash' --args -c 'echo hello; sleep infinity;'
- Bugfix: client earnings page now shows payouts for referrals
- Bugfix: frontend GUI now enforcing 8GB minimum disk size
- Bugfix: added additional detections and deletions for instances that failed to load


Update

September 2024 Product Update

NVIDIA RTX 5090: Leaks, Rumors, and Next-Gen GPU Performance

Navigating Data Center Compliance: Understanding Tier 2/3 and HIPAA/ISO 27001 Standards

December product updates

November product updates