
# Latest AI Model Releases: September and October

The world of Open Source AI has gotten many updates in the last month or so. There are now many new models with great quality:speed ratios and models that challenge the frontier of closed source models. This makes it even easier to build applications and automate workflows with open source models, which you can deploy on [Vast.ai](https://vast.ai). Meta, Mistral, and Nvidia have made the biggest waves with their recent releases.

## Meta and Llama 3.2 Models

On September 25th, Meta introduced several exciting [updates](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/) to their AI model lineup, focusing on lightweight models and vision capabilities.

### Lightweight Models

First, Meta released impressive 1B and 3B parameter models that are best in class for their size. These models allow for near-instantaneous interactions on edge devices and can be used for processing large amounts of tokens with simple processing tasks.

The development process for these models involved pruning from Llama 3.1-8B, followed by knowledge distillation using the 8B and 70B models as teachers. The instruction-tuned versions underwent additional training with Supervised Fine-tuning (SFT), Direct Policy Optimization (DPO), and Rejection Sampling, incorporating synthetic data from Llama 3.1-405B.

These small models are easy to download and get playing with locally, but can be leveraged for large scale data processing tasks as they are much less expensive to run and still have great capabilities.

### Vision Models

Meta's new vision models come in 11B and 90B parameter sizes. Interestingly, these are built as adapters for the 8B and 70B text models, respectively, maintaining the same performance on text tasks while adding visual capabilities. This approach saved compute during training and allows for flexible deployment options.

The vision models were created by pretraining adapters and image encoders with noisy image and text data, followed by post-training similar to the text-only models. This has two advantages: For the same compute budget, Meta could train the models on more data, and the models are deployable on the same GPU's at the same time as their text-only models and develoeprs don't have to worry about regressions in their text-only tasks.

### Llama Guard Models

Meta also released new Llama Guard models for safety and moderation tasks, based on the 11B vision model and the 1B and 3B text models. These models are designed to help with safety and moderation tasks at the edge.

## Mistral's AI Innovations

Mistral has been on a similar research path as Meta, releasing models that are focused on similar efforts and qualities, small efficient models for the edge, and multimodal models as drop-in replacements for existing text-only models.

### Pixtral Model

Released on September 17th, [Pixtral](https://mistral.ai/news/pixtral-12b/) is a 12B parameter multimodal model. Similar to Meta's Llama 3.2 Vision approach, Pixtral was created as a drop-in replacement for Mistral-Nemo-11B, a text-only model. It features a 400M image encoder as an adapter, maintaining deployment flexibility. It performs well on a variety of multi-modal tasks.

Pixtral is available under the Apache 2.0 license and can be downloaded for the base model [here](https://huggingface.co/mistralai/Pixtral-12B-Base-2409) and for the instruct model behind a gated release [here](https://huggingface.co/mistralai/Pixtral-12B-2409).

### Mistral-Small v24.09

Also released on September 17th, 2024, [Mistral-Small v24.09](https://mistral.ai/news/september-24-release/) is a 22B parameter text model that bridges the gap between Mistral Nemo-11B and Mistral Large v2. This model offers substantial improvements in reasoning, code generation, and function calling compared to its predecessor. It's available under the Mistral Research License, and can be downloaded [here](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409) behind the research gate.

### Ministral Models

On September 25th, 2024, Mistral introduced two new models: [Ministral 3B and Ministral 8B](https://mistral.ai/news/ministraux/).

Ministral 3B is a compact powerhouse, offering capabilities roughly equivalent to the original Mistral 7B but in just 3B parameters. Its instruct-tuned version outperforms Mistral-7B instruct models and shows superior performance to Llama 3.2-3B. However, it is not availabel on HuggingFace, and requires an Enterprise License from Mistral to access.

Ministral 8B represents a significant upgrade over the Mistral 7B series, offering a substantial leap in capabilities. According to benchmarks, it outperforms Llama 3.1-8B across of series of tasks. You can download the model [here] (https://huggingface.co/mistralai/Ministral-8B-Instruct-2410) behind the gated access and run it on Vast to evaluate it for your workloads.

## Nvidia's Contribution to AI Advancement

Nvidia has also focused on 70B models to compete with frontier models like those from OpenAI and Anthropic.

### Llama 3.1-Nemotron-70B Models

Nvidia has [introduced](https://developer.nvidia.com/blog/new-reward-model-helps-improve-llm-alignment-with-human-preferences/) two notable models based on the Llama 3.1 architecture:

1. **Llama 3.1-Nemotron-70B-Reward**: This model was trained via Reinforcement Learning from Human Feedback (RLHF) to serve as a reward model. It currently holds the top position on the [RewardBench leaderboard](https://huggingface.co/spaces/allenai/reward-bench), showcasing its effectiveness in evaluating AI responses. It was also used to train the Instruct model, showing how well this model can be used to train other models synthetically.

2. **Llama 3.1-Nemotron-70B-Instruct**: Leveraging the reward model mentioned above, this instruct-tuned version has achieved impressive results. It currently leads the [ArenaHard Leaaderboard](https://huggingface.co/spaces/lmarena-ai/chatbot-arena-leaderboard/commit/2bba63b79d685a2d46696c2da0576a86a3cfc7c6), demonstrating its superior performance in complex instruction-following tasks.

The Llama 3.1-Nemotron-70B-Reward model was trained on the [HelpSteer-v2 dataset](https://arxiv.org/abs/2406.08673), which likely contributed to its strong performance as a reward model.

What's particularly noteworthy is that the Llama 3.1-Nemotron-70B-Instruct model has shown outstanding results in automatic benchmarks, outperforming even some of the most advanced models in the field, including Claude 3.5 Sonnet and GPT-4.

These developments from Nvidia show that methods to move beyond the GPT-4 wall of capabilities are not just being developed, but open sourced and available to developers to use wherever they use compute.

## Conclusion:

It's a great time to be a developer using AI. In the past few months, there have been major capability improvements for LLM's and Multimodal models, enabling tasks that were not possible before with open source models. And the release of many smaller language models that are still as capable as previous generation models means that the cost curve is coming down for deploying to production.

At Vast, we're very excited to see these advancements, and you can look out for more updates and building on top of these types of models for specific workflows in the coming months.


Open Source AI has evolved rapidly, with new models offering top quality and speed, challenging closed-source standards.

Latest AI Model Releases: September and October


# Could the End Be Near for the NVIDIA GeForce RTX 4090?

Rumors are circulating that the high-end consumer GPU is on the verge of being discontinued. With production potentially stopping, the 4090 could soon be disappearing off the shelves to make way for NVIDIA's new flagship GPU: the [RTX 5090](https://vast.ai/article/nvidia-rtx-5090-out-by-christmas-rumors).

As with any unconfirmed reports, it's best to take this news with a grain of salt – but the possibility has already sparked plenty of buzz among GPU enthusiasts.

## What's Driving the Rumors?

Reports about the potential discontinuation of the RTX 4090 appear to have [originated](https://wccftech.com/nvidia-geforce-rtx-4090-4090d-discontinued-next-month-in-preparation-for-next-gen-rtx-5090-5090d-gpus/) on China’s Board Channels, a frequent source of supply chain rumors. According to these sources, NVIDIA may stop producing the RTX 4090, along with its China-specific RTX 4090D variant, as early as October, with inventory expected to dwindle through the rest of the year.

But given that it's already late October without any official announcements from NVIDIA, the RTX 4090's availability seems secure in the near term. Still, we can't rule out any sudden shifts as NVIDIA prepares for the release of the next-gen RTX 5000 series.

According to rumors, the launch date of the new [flagship Blackwell RTX 5090 GPU](https://vast.ai/article/nvidia-rtx-5090-out-by-christmas-rumors) will be either in late 2024 or in early January at CES 2025. That timing does line up with NVIDIA potentially pulling the plug on the RTX 4090 sooner rather than later. Plus, the move would allow add-in board (AIB) partners to clear their stock of the RTX 4090 ahead of the RTX 5000-series release.

## Potential Impact on Pricing and Availability

Ending production of the RTX 4090 now (or very soon) would give NVIDIA's partners the holiday season to move existing inventory. This could result in shortages during the lead-up to Christmas as supplies run low, causing prices to spike as stock dwindles.

That said, with the highly anticipated RTX 5090 on the horizon, many buyers may hold off and wait for the new flagship release before making any purchase decisions. There's a chance this could put downward pressure on RTX 4090 prices, making it a more attractive option in the meantime.

It's hard to predict how things will unfold. Without official confirmation from NVIDIA, everything is merely speculation – and for now, it's a waiting game.

## Exploring Your GPU Options

While the future of the RTX 4090 remains uncertain, and the next-gen RTX 5090 will likely come with a steep price tag, one thing is clear: our cloud GPU rental platform allows you to access powerful GPUs at the most competitive rates available.

You don't need to make any huge purchases or commit to a single machine. With Vast.ai, you have the flexibility to choose from a range of GPUs that best fit your project, budget, and performance requirements at any given moment. Whether you're fine-tuning deep learning models, running complex simulations, or pushing your gaming rig to new heights, we're here to connect you with the GPU compute power that meets your exact needs.

[Discover the possibilities](https://vast.ai/) with Vast.ai today.


Rumors are circulating that the high-end consumer GPU is on the verge of being discontinued.

NVIDIA RTX 4090 to Be Discontinued Sooner Rather Than Later?


When it comes to high-performance computing (HPC), the NVIDIA H100 is among the best of the best GPUs on the market. Its predecessor, the NVIDIA A100, is also a very impressive GPU in its own right. From enterprise to exascale, these GPUs are enabling groundbreaking advancements in everything from AI research to complex scientific simulations.

Today we're taking a look at how the H100 and A100 compare with each other. Depending on your use case, one or the other might be better suited for your needs and preferences. So let's dive in!

## Introduction to the NVIDIA H100 and A100

As mentioned, the H100 GPU is the successor to the A100. It was specifically optimized for machine learning and designed to [deliver](https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/) an "order-of-magnitude performance leap for large-scale AI and HPC" over the A100, with "substantial improvements in architectural efficiency."

Based on the innovative Hopper architecture, the H100 can speed up large language models (LLMs) by 30X over the A100, according to NVIDIA. It features fourth-gen Tensor Cores and – unlike the A100 – a dedicated Transformer Engine to solve trillion-parameter language models.

Whether it's used in smaller enterprise systems or massive, unified GPU clusters, the H100 delivers efficient scalability thanks to its combination of fourth-gen NVLink (with 900 GB/s of GPU-to-GPU interconnect), NDR Quantum-2 InfiniBand networking (speeding up communication by every GPU across nodes), and PCIe Gen5. With superior FP64 and FP8 performance, the H100 is ideal for next-gen AI workloads.

The A100 GPU is known for its versatility, however. Built on the Ampere architecture, it was groundbreaking when released in 2020, providing up to 20X higher performance over the prior NVIDIA Volta generation. It can efficiently scale up or be partitioned using Multi-Instance GPU (MIG) into seven isolated GPU instances, just like the H100, which is ideal for shifting workload demands.

In short, the A100 excels in both training and inference tasks, and while it may not offer the sheer power of the H100, its robust FP64 performance makes it a great fit for scientific simulations and data analytics.

## Key Differences: H100 vs. A100

The H100 has an updated chip design and various other factors that distinguish it from the A100. The following are a few that are important to consider:

### Memory Bandwidth

The A100 has a memory bandwidth of about 2 TB/s, which is plenty for most use cases; however, the H100 offers an impressive 3.35 TB/s, making it ideal for workloads where data transfer bottlenecks may occur.

### CUDA Cores and Tensor Cores

A higher number of CUDA and Tensor Cores means a GPU can perform more parallel computations simultaneously. With 14,592 CUDA Cores and 456 Tensor Cores, the H100 outperforms the A100’s 6,912 CUDA Cores and 432 Tensor Cores, delivering superior performance in applications that can leverage increased parallelism.

### Confidential Computing

The H100's Hopper architecture introduced NVIDIA's new confidential computing environment. It protects data-in-use by performing computation in a trusted execution environment (TEE) that is both hardware-based and attested. This means you essentially have absolute security for use cases where security, privacy, and regulatory compliance are paramount.

### Tensor Memory Accelerator (TMA)

Another feature of the H100's Hopper architecture is the TMA unit, which can transfer large blocks of data efficiently between global memory and shared memory. It supports asynchronous copies between thread blocks in a cluster as well, optimizing data movement and memory management for tensor operations. Ultimately, this enhances the efficiency and speed of large-scale AI and deep learning tasks.

### MIG Capability

The Multi-Instance GPU (MIG) feature is present in both the H100 and A100, providing flexible workload management so you can run multiple LLMs in parallel, for instance. The H100's MIG capability is more robust, however, with better resource allocation and more memory and compute power to split.

## Performance Benchmarks

According to NVIDIA's own benchmarks, the H100 offers up to 30X better inference performance than the A100. In another test where the A100 and H100 were used in clusters to train two large language models (LLMs), the H100 demonstrated remarkable speed improvements. The H100's new NVLink Switch System provided an even bigger boost, enabling the H100 cluster to train the models up to nine times faster than the A100 cluster:

![H100 vs A100 Speed Comparison](/uploads/Speed-up-over-A100.png)

## Choosing the Right GPU

So what does all this mean for you? The best GPU for your needs will depend on numerous factors. Here's a quick outline of the various features and specs of the NVIDIA A100 and H100:

| Feature                         | A100                   | H100                    |
|---------------------------------|------------------------|-------------------------|
| **GPU Architecture**            | Ampere                 | Hopper                  |
| **GPU Memory**                  | 40 or 80 GB HBM2e      | 80 MB HBM3              |
| **GPU Memory Bandwidth**        | 1.6 to 2 TB/s          | 3.35 TB/s               |
| **CUDA Cores**                  | 6912                   | 14,592                  |
| **FP64 TFLOPS**                 | 9.7                    | 33.5                    |
| **FP32 TFLOPS**                 | 19.6                   | 67                      |
| **TF32 Tensor Core Flops\***    | 156 \| 312             | 378 \| 756              |
| **FP16 Tensor Core Flops\***    | 312 \| 624             | 756 \| 1513             |
| **FP8 Tensor Core TFLOPS**      | N/A                    | 3958                    |
| **Peak INT8 TOPS\***            | 624 \| 1248            | 3958                    |
| **Media Engine**                | 0 NVENC<br>5 NVDEC<br>5 NVJPEG | 0 NVENC<br>7 NVDEC<br>7 NVJPEG |
| **L2 Cache**                    | 40 MB                  | 50 MB                   |
| **Power**                       | Up to 400 W            | Up to 700 W             |
| **Form Factor**                 | SXM4 - 8-way HGX       | SXM5 - 8-way HGX        |
| **Interconnect**                | PCIe 4.0 x16           | PCIe 5.0 x16            |

\*Without and with structured sparsity.

---

## Use Cases

When it comes to use cases, you may want to consider some general guidelines.

### The A100 is well suited for:
- Versatile AI workloads including training and inference.
- Applications needing a balance of performance and cost-efficiency.
- Scenarios where ultra-high precision isn't the priority.

### The H100 is ideal for:
- Cutting-edge AI training, large-scale simulations, and other HPC workloads.
- Environments where computational speed and memory bandwidth are critical.
- Future-proofing investments in AI infrastructure.

---

When deciding between the A100 and H100, consider your specific workload requirements. If you need top-tier double-precision performance and superior memory bandwidth, or you're dealing with next-gen HPC at datacenter scale and trillion-parameter AI, the H100 is the clear winner. For a more versatile and cost-effective solution that still delivers powerful AI performance, the A100 is a solid choice.

At Vast.ai, we understand the importance of finding the right GPU for your needs as well as your budget. Our cloud GPU rental marketplace offers a variety of machines at the lowest prices possible, saving you 5-6X on GPU compute. Whether you prefer on-demand pricing or interruptible instances through spot auction-based pricing, we've got you covered.

We invite you to discover what Vast can do for you and accelerate your projects with the right GPU today!


Explore the differences between NVIDIA's H100 and A100 GPUs, from architectural advancements to performance benchmarks.

H100 vs A100: Comparing Two Powerhouse GPUs


With all of the GPU options, finding the one that's right for your needs is challenging. Whether you're into high-end cloud gaming, training AI models, or even outfitting a university lab or data center, understanding the nuances between different models is key.

Today, we’re focusing on two powerful GPUs from NVIDIA: the L40 and [L40S](https://cloud.vast.ai/?gpu_option=L40S). While they may not have received as much attention as some of the company's other offerings, they certainly deserve a closer look! These two GPUs, built on the [Ada Lovelace](https://vast.ai/article/rtx-4090-deep-learning) architecture, pack a punch in performance and capabilities.

But how do they stack up against each other? What about compared to other GPUs? And more importantly, which one should you choose for your specific project needs?

## The Basics: NVIDIA L40 vs. L40S

The L40S is essentially an upgrade to the L40 – as well as a close relative to the [RTX 4090 gaming graphics card](https://vast.ai/article/nvidia-rtx-4090-vs-5090).

Initially designed for data center graphics and simulation tasks, the L40 is the engine of NVIDIA Omniverse. It's a powerhouse for extended (XR) and virtual reality (VR) applications, design collaboration, and digital twins, thanks to its advanced RTX and AI capabilities. It enables accelerated ray-traced and path-traced rendering, delivers physically accurate simulations, and generates highly realistic 3D synthetic data for the most demanding Omniverse workloads.

Compared to the previous generation, the L40 GPU delivers 5X higher inference performance for compute-intensive AI workloads – so it can generate high-quality images and immersive visual content lightning fast. It's perfectly suited for image generative AI applications. If that's all you need, then the L40 might be an ideal option for you.

However, the L40S can do all of this and more. It was developed to meet the surging demand for GPUs that can handle the intense computational requirements of machine learning training and inference.

NVIDIA describes the L40S as "the most powerful universal GPU," offering "breakthrough multi-workload performance." Here's a quick snapshot of its performance in a couple of areas:

!["Graph comparing NVIDIA L40S GPU performance for Stable Diffusion image generation and LLM inference, showing significant differences in images per minute and token latency across various configurations."](/uploads/l4-vs-l40S-image-generative-ai.png)

After its release in late 2023, the L40S achieved popularity due to the lack of availability of the NVIDIA A100 and H100 Tensor Core GPUs. There was a massive chip shortage and backlogged shipments, forcing many buyers to turn to other options – like the L40S. Since the L40S can not only be used for the same workloads as the L40 but can also power AI training and inference at a high level like the A100 and H100, it was a logical alternative.

The L40S, A100, and H100 GPUs are therefore in a unique position to be compared with each other. Let's take a look!

## L40S vs. A100 vs. H100: Specs and Performance

Each of the three GPUs brings its own strengths to the table. The table below compares various features and specs between them.

| Feature                | A100          | L40S           | H100          |
|------------------------|---------------|----------------|---------------|
| **GPU Architecture**   | Ampere        | Ada Lovelace   | Hopper        |
| **GPU Memory**         | 40 or 80 GB HBM2e | 48 GB GDDR6    | 80 MB HBM3    |
| **GPU Memory Bandwidth** | 1.6 to 2 TB/s | 864 GB/s       | 3.35 TB/s     |
| **CUDA Cores**         | 6912          | 18,176         | 14,592        |
| **FP64 TFLOPS**        | 9.7           | N/A            | 33.5          |
| **FP32 TFLOPS**        | 19.6          | 91.6           | 67            |
| **TF32 Tensor Core Flops** | 156 | 312 | 183 | 366 | 378 | 756 |
| **FP16 Tensor Core Flops** | 312 | 624 | 362 | 733 | 756 | 1513 |
| **FP8 Tensor Core TFLOPS** | N/A         | 1466           | 3958          |
| **Peak INT8 TOPS**     | 624 | 1248   | 733 | 1466      | 3958          |
| **Media Engine**       | 0 NVENC 5 NVDEC 5 NVJPEG | 0 NVENC 5 NVDEC 5 NVJPEG | 0 NVENC 7 NVDEC 7 NVJPEG |
| **L2 Cache**           | 40 MB         | 96 MB          | 50 MB         |
| **Power**              | Up to 400 W   | Up to 350 W    | Up to 700 W   |
| **Form Factor**        | SXM4 - 8-way HGX | Dual Slot Width | SXM5 - 8-way HGX |
| **Interconnect**       | PCIe 4.0 x16  | PCIe 4.0 x16   | PCIe 5.0 x16  |

*Without and with structured sparsity.

As shown, there are some clear differences in performance when it comes to FP64 (double-precision), FP32 (single-precision), and FP16 (half-precision) computations.

For instance, the L40S does not natively support FP64. This means that it may not perform as well as the A100 and H100 in applications that require high precision. The H100 in particular, with its notably superior FP64 performance, is generally preferred for the most demanding tasks.

In terms of FP32 and FP16 Tensor Core performance, the L40S outperforms the A100 40GB and can hold its own against the H100. However, in memory-intensive machine learning scenarios, the L40S's lower memory bandwidth may counterbalance its performance compared to both the A100 80GB and H100.

Still, the L40S can really shine next to the A100 in certain situations. According to NVIDIA, when it comes to complex AI workloads involving billions of parameters and multiple data modalities, including text and video, the L40S can [achieve](https://nvidianews.nvidia.com/news/nvidia-global-data-center-system-manufacturers-to-supercharge-generative-ai-and-industrial-digitalization) up to 1.2 times the generative AI inference performance and up to 1.7 times the training performance of the A100.

Another point in the L40S's favor? While it may not be able to beat the H100 on certain metrics, it's definitely more flexible. The higher-performance H100 can't be used as an Omniverse server and doesn't support graphics, whereas the L40S can be and does.

## The Bottom Line


The NVIDIA L40 excels in powering image generative AI applications and Omniverse workloads. You may not need it for much else, and if that's the case, the L40 would probably serve you well.

On the other hand, the NVIDIA L40S nicely balances versatility as well as performance. The following are some reasons you might choose it for your next project:

- **You have multi-modal workloads** – You'll achieve remarkable efficiency with the L40S, enabling you to run HPC simulations, train AI models, and render images using the same computing infrastructure. Plus, video output is built in.
- **You need compute power fast in a familiar form factor** – The L40S is designed for rapid deployment within existing systems, offering powerful performance without requiring extensive reconfiguration. Its Dual Slot PCIe form factor ensures compatibility with a wide range of setups.
- **Your workloads don't require extreme precision or maximum speed** – Without the math performance (FLOPS), high-bandwidth memory, and NVLINK of the H100, the L40S's inference and training performance for smaller models is nonetheless perfectly suitable in most cases.
- **You prioritize cost-effectiveness** – The L40S provides an excellent price-to-performance ratio, particularly for enterprises and research institutions that need robust AI capabilities without breaking the budget.

For many of us, cost is a major factor in [GPU decision-making](https://vast.ai/pricing). At Vast.ai, our mission is to democratize AI and bring massive compute power, at an affordable price, to anyone who needs it. You don't have to make a huge hardware investment upfront.

Our cloud GPU rental marketplace offers the lowest prices possible, enabling you to access top-tier machines at any time – and save 5-6X on GPU compute. We offer low-cost on-demand pricing as well as interruptible instances through spot auction-based pricing to bring you even more savings.

Come explore what Vast can do for you and accelerate your projects today!

Comparing NVIDIA L40 vs. L40s – and More


For both technology enthusiasts and industry professionals, keeping a close eye on the latest GPU advancements isn't just a hobby – it's crucial for staying competitive and maximizing your performance and compute power. Here at Vast.ai, we're just as excited to witness the next big leaps forward in graphics technology!

Today, we're looking at two GPUs in particular: the formidable NVIDIA RTX 4090 versus its highly anticipated successor, the not-yet-released RTX 5090. While the RTX 4090 has solidified its reputation for performance, the RTX 5090 is still a subject of much speculation.

As we explore what we know and what we anticipate, keep in mind our knowledge of the RTX 5090 is still evolving. But nonetheless, let's take a look at the facts, the early insights, and emerging rumors…

### Spotlight on NVIDIA GeForce RTX 4090

Known for being the fastest consumer-grade GPU on the market, the RTX 4090 is truly a beast of a machine. Based on the AD102 chip using the Ada Lovelace architecture, it epitomizes super-high-end gaming and professional performance.

The RTX 4090 has a triple-slot design and boasts an incredible 128 dedicated third-gen RT cores and 512 fourth-gen Tensor cores – more than its closest competitors – elevating ray-tracing performance and speeding up AI-driven tasks like DLSS 3 in games.

Plus, it's packed with 16,384 CUDA cores and has 24GB of GDDR6X graphics memory with a maximum bandwidth of 1,008 GB/s and a 384-bit memory bus interface. Error-correcting code (ECC) can be enabled, but this reduces performance while making no difference in GPU rendering – so keeping it turned off is appropriate for most uses when you're prioritizing speed.

Due to its immense processing capability, the RTX 4090 draws a lot of power to deliver maximum performance: 450 watts. It requires an 850-watt power supply and robust cooling solutions, which does increase usage costs, of course.

Launched in September 2022, the RTX 4090 still currently stands as the pinnacle of NVIDIA’s offerings for enthusiasts seeking unmatched graphics performance and computational power.

But that will change in due time.

### Anticipating NVIDIA GeForce RTX 5090

The RTX 5090 is expected to be released later this year – and it's a game-changer, poised to redefine high-end performance with its groundbreaking features and next-gen architecture. That's right, we're now moving beyond NVIDIA's Ada Lovelace and into the era of the Blackwell architecture.

Joining the new Blackwell GPU lineup, the flagship GeForce RTX 5090 will reportedly utilize the "physically monolithic" GB202 die. Among its highlights are a potential 512-bit memory bus and up to 32GB of next-gen GDDDR7 memory (the successor to the GDDR6/X memory used by the RTX 40-series GPUs). Early reports suggest the RTX 5090 could achieve memory bandwidths of up to 1,532 GB/s, a substantial boost from the RTX 4090's 1,008 GB/s​.

Some are expecting to see up to 192 streaming multiprocessors (SMs) in the RTX 5090, with an unprecedented 24,576 CUDA cores – assuming the GB202 chip keeps the 128-cores-per-SM design as seen in the RTX 4090. Plus, rumors say the RTX 5090 will feature 192 third-gen RT cores and 768 fourth-gen Tensor cores, up from 128 and 512 respectively in the RTX 4090.

The RTX 5090 is anticipated to deliver 50-70% performance improvements across the board compared to the RTX 4090 – particularly excelling in 4K resolution. And with 4K 240FPS gaming on the horizon, the RTX 5090 will be perfectly positioned to tackle the most demanding gaming scenarios.

Some are also looking forward to seeing the RTX 5090 support next-gen DLSS 4, NVIDIA's AI-powered upscaling that will further enhance performance and image quality​.

In terms of power consumption, the RTX 5090 will likely follow the trend of its predecessors, with total graphics power (TGP) ranging from 450W to 600W, about on par with the RTX 4090.

For a quick look at some of the anticipated specs of the RTX 5090 compared to the RTX 4090 – based on the latest speculation and leaks – here's a table highlighting the upgrades:

| Spec                   | RTX 4090   | RTX 5090   |
|------------------------|------------|------------|
| Streaming Multiprocessors | 128        | 192        |
| CUDA Cores             | 16,384     | 24,576     |
| Ray-Tracing Cores      | 128        | 192        |
| Tensor Cores           | 512        | 768        |
| Boost Clock            | 2.52 GHz   | 2.9 GHz    |
| L2 Cache               | 72MB       | 128MB      |
| Memory Bandwidth       | 1,008 GB/s | 1,532 GB/s |

### Concluding Thoughts

As we anticipate the release of the RTX 5090, it's important to recognize that the RTX 4090 already delivers exceptional performance, handling everything available at 4K with ease. The RTX 5090, while sure to be extremely impressive, may be seen more as a luxury upgrade rather than a necessity for gamers and professionals who can (or already do) benefit from the power of the RTX 4090.

Cost is another critical factor. The RTX 5090 will undoubtedly be a top-of-the-line machine with a matching price tag. And the RTX 4090 is no slouch, either, still commanding a steep price to this day, nearly two years after its release.

Here at Vast.ai, we take pride in providing access to powerful machines for rent on the cloud at unbeatable prices. GPU rental offers unmatched flexibility, enabling you to utilize state-of-the-art technology without the expense and upkeep of purchasing your own hardware. Whether you're looking to explore the capabilities of the RTX 4090, prepare for the arrival of the RTX 5090, or check out any other machine on Vast, we'd be happy to help you discover the possibilities!


The RTX 4099 is a very powerful card that is good for both gaming and professional use. The RTX 5090 is expected to be even more powerful and will likely be more expensive.

Nvidia

Latest AI Model Releases: September and October

NVIDIA RTX 4090 to Be Discontinued Sooner Rather Than Later?

H100 vs A100: Comparing Two Powerhouse GPUs

Comparing NVIDIA L40 vs. L40s – and More

NVIDIA RTX 4090 vs. the Upcoming RTX 5090