
A new NVIDIA Blackwell GPU with 96GB of VRAM? If rumors are to be believed, that could be exactly what's coming. 

A recent shipping manifest [revealed](https://wccftech.com/nvidia-rtx-pro-6000-x-blackwell-leak-96-gb-gddr7-600w/) an entry for the "**RTX PRO 6000 X Blackwell**," evidently being sent to India "for testing purposes." At least two different shippers have sent the GPUs for testing.

This isn't your typical consumer-grade gaming card, however. The as-yet-unannounced Blackwell-based GPU appears to be aimed at AI and data-intensive workloads. Here's what we know so far about this new entry in NVIDIA's professional lineup.

### NVIDIA RTX Pro 6000 X: What We Know So Far

With the impressive [RTX 5090](https://vast.ai/article/nvidia-geforce-rtx-5090-specs-everything-you-need-to-know) expected to remain at the top of the GeForce 50-series lineup, the professional segment is where we'll likely see ultra-high-end specs that push the limits on VRAM – and this latest shipping log discovery suggests that will be the case. 

According to reports, as mentioned, the RTX Pro 6000 X features a staggering **96GB of GDDR7 memory** on a 512-bit memory bus. That's significantly higher than even the most powerful gaming cards. For example, the RTX 5090 has 32GB of GDDR7 memory – which is more than enough for most purposes – but higher VRAM is obviously beneficial for workloads that require handling massive datasets, such as foundation AI model training and large-scale simulations.

The RTX Pro 6000 X also features a **GB202 die** and **600 watts** of total board power (TBP). This aligns with previous high-end Blackwell architecture designs, like the RTX 5090, which also has the GB202 die and comes in at 575 watts of TBP.

Interestingly, the discovery of the RTX Pro 6000 X on the shipping manifest follows an [earlier sighting](https://wccftech.com/nvidia-rtx-4090-with-96gb-vram-reportedly-exists/) of a 96GB RTX 4090 GPU – perhaps an engineering sample. While NVIDIA hasn't made any announcements, these repeated sightings nonetheless suggest a broader push for high-VRAM GPUs.

### Comparison to Existing RTX Workstation GPUs

We've seen professional-grade GPUs with massive VRAM configurations before, such as the RTX 6000 Ada with 48GB of memory, as well as the A100/A800 series. However, the RTX Pro 6000 X's 96GB of GDDR7 memory would make it one of the highest-capacity workstation GPUs to date. 

Assuming we see the RTX Pro 6000 X emerge, it would set a new benchmark for professional GPUs, doubling the memory of the RTX 6000 Ada while offering the faster speeds and efficiency of GDDR7 memory.

Plus, unlike the A100 and A800 series, which are designed primarily for data centers and enterprise AI workloads, the RTX Pro 6000 X could serve as a more flexible solution. Its ultra-high VRAM would allow professionals to work with larger datasets without necessarily relying on multi-GPU setups. 

### Looking Ahead

NVIDIA hasn't officially announced the RTX Pro 6000 X, so all information should be taken with a grain of salt, but the repeated mentions of this new GPU in shipping logs suggest that an announcement may be imminent. 

For Vast.ai users, this could mean more powerful GPU rental options in the near future. Stay tuned for more developments\! In the meantime, check out our current selection of high-performance GPUs available on Vast.ai for flexible and cost-effective access to cutting-edge hardware today.  

NVIDIA’s rumored RTX Pro 6000 X packs 96GB of GDDR7 VRAM, a GB202 die, and 600W TBP—ideal for AI, data science, and high-end workstation workloads.

NVIDIA RTX Pro 6000 X Rumors: Blackwell GPU with 96GB VRAM Spotted in Shipping Log

Pyannote Audio and NVIDIA’s Sortformer enhance Whisper’s speech recognition by adding speaker diarization. We’ll run them on VAST.ai to create speaker-attributed transcripts.

Implementing Speech-to-Text with Speaker Diarization: Comparing Pyannote and Sortformer on VAST.ai


The long-awaited NVIDIA GeForce RTX 5090 is here – but good luck getting your hands on one. From shipping delays to hardware defects, this GPU launch has been far from smooth.

Controversy aside, it's exciting to finally have confirmed performance details for this high-end graphics card. You can [try it now](https://cloud.vast.ai/?gpu_option=RTX%205090) for yourself on Vast.ai. Here's everything you need to know about the RTX 5090...

### RTX 5090: The Performance King?

As NVIDIA's new flagship GPU, the [RTX 5090](https://vast.ai/pricing/gpu/RTX-5090) is undoubtedly an impressive machine. Is it worth the $1999 price tag, though? That depends on more than just raw power – let's take a look.

#### Official Specs

For starters, here are some of the official specs for the RTX 5090:

-   **Architecture:** Powered by NVIDIA's cutting-edge Blackwell architecture.
-   **Gaming Chip:** Featuring the next-gen GB202 gaming chip.
-   **Memory:** 32GB of GDDR7 memory with a blazing-fast 1,792GB/sec bandwidth, supported by a super-wide 512-bit memory interface for even faster data transfer.
-   **CUDA Cores:** 21,760 cores, ideal for high-performance graphics and AI workloads.
-   **L2 Cache:** Expanded to 98MB, up from 73MB on the RTX 4090, improving memory efficiency and reducing latency.
-   **Display Support:** PCIe Gen 5 with DisplayPort 2.1b connectors supporting up to 8K at 165Hz.
-   **Power Requirements:** 575W total graphics power (TGP), with a 1000W power supply recommended for optimal performance.

Another bonus? The RTX 5090 has a redesigned **Founders Edition** with a two-slot design and a dual flow-through fan configuration – so it's compact enough for small form factor PCs, a remarkable shift from its predecessors.

For reference, here's a quick overview of the RTX 5090 specs:

| Spec                           | RTX 5090                       |
| :----------------------------- | :----------------------------- |
| **CUDA Cores**                 | 21,760                         |
| **Ray Tracing Cores**          | 170 (4th-gen)                  |
| **Tensor Cores**               | 680 (5th-gen)                  |
| **Base Clock**                 | 2,017 MHz                      |
| **Boost Clock**                | 2,407 MHz                      |
| **L2 Cache**                   | 98,304KB                       |
| **ROPs**                       | 176                            |
| **VRAM**                       | 32GB 28Gbps GDDR7              |
| **VRAM Interface**             | 512-bit                        |
| **Memory Bandwidth**           | 1,792 GB/s                     |
| **Interface**                  | 16x PCIe 5.0                   |
| **Power Connectors**           | 1 x 16-pin / 4 x 8-pin adapter |
| **Total Graphics Power (TGP)** | 575W                           |

(For more specifications – and to see how the RTX 5090 stacks up against its younger siblings in the RTX 50-series lineup as well as the RTX 4090 – check out the chart at the bottom of this page.)

#### AI and Professional Workloads

Beyond gaming, the RTX 5090 is well suited for high-performance AI and professional workloads. Its **680 Tensor cores** and **32GB of GDDR7** memory efficiently accelerate deep learning, 3D rendering, and other compute-intensive tasks. With **98MB of L2 cache** and a **512-bit memory interface**, the RTX 5090 delivers faster data access and reduced latency – crucial for AI training, scientific computing, and content creation.

The **5th-gen Tensor cores** boost AI inference efficiency in frameworks like TensorFlow and PyTorch, while **4th-gen RT cores** enhance real-time ray tracing for animation, visual effects, CAD work, and more.

For demanding AI and video workloads, the RTX 5090 more than meets the challenge. In Procyon's AI XL (FP16) test, the RTX 5090 was **40% faster** than the RTX 4090, and in PugetBench's DaVinci Resolve video processing test, it was **12% faster**.

However, cutting-edge performance doesn't necessarily mean a flawless launch\! Early adopters are already encountering serious issues in some cases.

#### Next-Gen Gaming Features

The biggest leap in technology for this GPU comes from DLSS 4's new **Multi Frame Generation**, an AI-driven frame interpolation technique exclusive to the RTX 50 series. It uses AI to generate up to three additional frames between traditionally rendered ones, significantly boosting frame rates with minimal visual artifacts.

Further, the RTX 5090 introduces a range of AI-driven enhancements in what's called **neural rendering**, designed to push in-game realism to new levels. **RTX Neural Shaders** improve texture compression and deliver film-quality lighting and shading in real time, while **RTX Neural Faces** uses generative AI to create lifelike skin, hair, and facial details.

Building on the power of DLSS 4, these neural rendering techniques integrate directly into the graphics pipeline, leveraging NVIDIA’s Tensor cores to enhance visual fidelity. Though game support is still in development, early demos – such as _Half-Life 2 RTX_ – [showcase](https://www.pcgamesn.com/nvidia/geforce-rtx-5090-review) the potential for deeper shadows, realistic translucency in materials, and an overall richer environment.

Another breakthrough is **Mega Geometry**, a feature that uses RT cores to significantly boost the number of triangles in ray-traced scenes. This allows game engines to maintain full geometric detail without sacrificing performance, resulting in more realistic object depth, shadows, and fine details.

The above features position the RTX 5090 as a powerhouse for next-generation gaming visuals... but there's a catch.

##### Caveat: Performance Claims vs. Real-World Results

The way that NVIDIA presents its performance numbers can be misleading. The company leans heavily on Multi Frame Generation-boosted FPS (frames per second) to compare these new GPUs against older ones.

Frame Generation and Multi Frame Generation (MFG) are powerful tools, but they aren't universal fixes. Their impact varies by game – some titles see massive FPS gains with little visual trade-off, while others may introduce artifacts or latency.

In addition, **not all games will support these features**, as DLSS adoption depends on individual developers. A GPU's raw performance still matters. MFG-inflated benchmarks can make performance gains seem more significant, but Frame Generation still needs a strong base frame rate to be effective. In CPU-limited games, or titles that don't support DLSS 4, the real-world performance gap may be far smaller.

Ultimately, while the RTX 5090 excels at AI-driven rendering, **real-world results will depend on game support and how well these features are implemented.**

### Early Defects and Controversies

#### Melting Power Connectors

Reports of **melted power connectors** on the RTX 5090 Founders Edition have [surfaced](https://www.theverge.com/news/609207/nvidia-rtx-5090-power-connector-melting-burning-issues). A number of users have experienced burnt plastic at both the GPU and PSU ends – with user error and third-party cables evidently not to blame.

NVIDIA introduced the updated 12V-2x6 power connector for the RTX 50 series following melting and overheating issues with the RTX 40-series GPUs. The new connector features shorter sensing pins and longer conductor terminals to improve connection reliability. Users can still use existing 12VHPWR cables with the RTX 50 series; however, the RTX 5090 can draw up to 575 watts of power – which is dangerously close to the cable's 600-watt rating.

PSU manufacturers have taken precautions, such as adding visual indicators to ensure a secure connection, but it remains unclear whether this fully resolves the risk of overheating. So far, NVIDIA has declined to comment on the reports.

#### Chip-Level Fault

One of the more unexpected issues plaguing the RTX 5090 launch is what's now being called the **"missing ROPs" defect.** NVIDIA has officially [confirmed](https://www.techradar.com/computing/gpu/some-lucky-rtx-5080-buyers-may-not-be-so-lucky-after-all-as-nvidia-confirms-rare-chip-fault-also-applies-to-this-gpu-as-well-as-the-rtx-5090-and-5070-ti) that a small number of RTX 5090, RTX 5090D, and RTX 5070 Ti cards were shipped with fewer ROPs (Render Output Units) than they should have. These ROPs are basically rendering pipelines for 3D graphics. Having fewer of them slows down PC gaming performance.

While this defect reportedly only impacts an estimated 0.5% of units (or around 1 in 200), affected cards have an average 4% drop in graphical performance. Since it's a hardware issue, it can't be fixed via BIOS or driver updates. However, it has no impact on AI and compute tasks, and NVIDIA has corrected the production anomaly while offering free replacements for affected cards.

Tools like GPU-Z can help verify whether your card has the correct number of ROPs; if fewer than 176, it should probably be replaced.

### Shipping Delays and Stock Shortages

Due to a combination of these production issues as well as high demand and supply chain challenges, the RTX 5090 is currently experiencing **significant shipping delays.** Availability is low and wait times are high. UK retailer Overclockers has been particularly transparent about lead times, [revealing](https://www.pcgamesn.com/nvidia/overclockers-stock-announcement) that it could take anywhere from 3-16 weeks for its customers to receive an RTX 5090 that they've already pre-ordered.

Most other retailers are facing similar challenges, with limited stock and long waitlists becoming prevalent. Speculation about imminent price hikes is already growing.

### Price Increase Rumors

It's no surprise that high demand and limited supply often lead to rising prices. With the RTX 5090, though, there may be another factor at play. DeepSeek's rapid rise in popularity has sparked a huge [surge](https://www.theinformation.com/articles/deepseek-spurs-black-market-demand-for-nvidia-gaming-chips-in-china) in demand for NVIDIA's gaming GPUs in China, as these cards can be repurposed to run DeepSeek's AI models – and provide a workaround for U.S. export restrictions on advanced GPUs.

(For more info on DeepSeek and how to run their LLM on our GPU cloud, see our previous post [here](https://vast.ai/article/deepseek-r1-open-source-disruptor-or-overhyped-upstart).)

With both gamers and AI developers vying for the same hardware, the strain on supply could push prices even higher in the near future. Some forecasts predict RTX 5090 prices could skyrocket to $5,000 or more.

### Final Thoughts: The Power of the RTX 5090 – Without the Hassle

Despite its hurdles, the RTX 5090 represents a major step forward in AI-driven graphics and cutting-edge performance, cementing its place as the most powerful consumer-grade GPU available today.

At the same time, it may only be worth purchasing if you really want the best of the best and you're willing to shell out for it. With a $1,999 price tag and relatively modest raw performance gains over the RTX 4090, the RTX 5090's true potential hinges on the wider adoption of Multi Frame Generation and neural rendering – features that, if fully embraced, could eventually make this flagship GPU a must-have for those willing to invest.

In the meantime, why wait? Skip the delays and high costs – [rent an RTX 5090](https://cloud.vast.ai/?gpu_option=RTX%205090) on Vast.ai for instant access to top-tier performance\! Given its limited availability on store shelves and its 575-watt power draw, our affordable rental options are the most practical way to access the RTX 5090's full potential without the upfront investment.

[Try it out](https://cloud.vast.ai/?gpu_option=RTX%205090) for yourself today\!

\*\*\*\*\*\*\*\*\*\*

#### Additional Specifications:

| Spec                                |   RTX 5090    |   RTX 5080    |  RTX 5070 Ti  |   RTX 5070   |    RTX 4090    |
| :---------------------------------- | :-----------: | :-----------: | :-----------: | :----------: | :------------: |
| **Architecture**                    |     GB202     |     GB203     |     GB203     |    GB205     |     AD102      |
| **VRAM**                            |  32GB GDDR7   |  16GB GDDR7   |  16GB GDDR7   |  12GB GDDR7  |  24GB GDDR6X   |
| **VRAM Speed**                      |    28Gbps     |    30Gbps     |    28Gbps     |    28Gbps    |     21Gbps     |
| **VRAM Bus Width**                  |      512      |      256      |      256      |     192      |      384       |
| **Bandwidth**                       |   1792GB/s    |    960GB/s    |    896GB/s    |   672GB/s    |    1.01TB/s    |
| **Streaming Multiprocessors (SMs)** |      170      |      84       |      70       |      48      |                |
| **Tensor Cores**                    |      680      |      336      |      280      |     192      |      512       |
| **Ray Tracing Cores**               |      170      |      84       |      70       |      48      |      128       |
| **Boost Clock**                     |   2407 MHz    |   2617 MHz    |   2452 MHz    |   2512 MHz   |    2520 MHz    |
| **Total Graphics Power (TGP)**      |     575W      |     360W      |     300W      |     250W     |      450W      |
| **Recommended Power Supply**        |     1000W     |     850W      |     750W      |     650W     |      850W      |
| **Launch Date**                     | Jan. 30, 2025 | Jan. 30, 2025 | Feb. 20, 2025 | Mar. 5, 2025 | Sept. 20, 2022 |
| **Launch Price**                    |    $1,999     |     $999      |     $749      |     $549     |     $1,599     |


NVIDIA’s RTX 5090 delivers cutting-edge AI and gaming performance, but launch issues, high costs, and stock shortages make it hard to get. Try it now on Vast.ai!

NVIDIA GeForce RTX 5090 Specs: Everything You Need to Know


# NVIDIA GeForce RTX 5090: Release Announced

NVIDIA has officially unveiled the **GeForce RTX 5090**, the latest in its groundbreaking lineup of GPUs. The release date for this highly anticipated graphics card has been set for **January 30, 2025**, with availability expected alongside the RTX 5080, while the RTX 5070 Ti and RTX 5070 will follow in February.

During his CES keynote yesterday, NVIDIA CEO Jensen Huang introduced the RTX 5090 with a bold claim: the GPU is **two times faster than the RTX 4090,** thanks to DLSS 4 and the Blackwell architecture. But that's just the beginning.

## Official RTX 5090 Specs

Here are a few of the official specifications for the long-awaited **RTX 5090** (see more at the bottom of this page!):

-   **Architecture:** Built on the advanced Blackwell architecture.
-   **Memory:** 32GB of GDDR7 memory with an impressive memory bandwidth of 1,792GB/sec.
-   **CUDA Cores:** 21,760 cores, designed for maximum computational power.
-   **Power Requirements:** 575W total graphics power (TGP) with a recommended 1000W PSU.
-   **Display Support:** PCIe Gen 5 with DisplayPort 2.1b connectors supporting up to 8K at 165Hz.
-   **Gaming Chip:** Featuring the cutting-edge GB202 gaming chip.

In addition to a massive performance boost, the RTX 5090 features a redesigned **Founders Edition** with two double flow-through fans and a 3D vapor chamber, making it sleek enough to fit into small form factor PCs – a surprising change from the bulky RTX 4090.

These specs establish the RTX 5090 as the ultimate tool for gamers, creators, and professionals who demand peak performance.

## Beyond the RTX 5090: The RTX 50-Series Lineup

The younger siblings in the 50-series lineup – the RTX 5080, RTX 5070 Ti, and RTX 5070 – will also deliver outstanding capabilities:

-   **RTX 5080:** 16GB of GDDR7 memory, 10,752 CUDA cores, and a memory bandwidth of 960GB/sec. NVIDIA claims this card is twice as fast as the RTX 4080, with a total power consumption of 360 watts and recommended 850W PSU.
-   **RTX 5070 Ti:** 16GB of GDDR7, 8,960 CUDA cores, and a memory bandwidth of 896GB/sec, plus 300W TGP and 750W PSU.
-   **RTX 5070:** 12GB of GDDR7, 6,144 CUDA cores, and a bandwidth of 672 GB/sec, plus 250W TGP with just 650W PSU. It boasts "RTX 4090 performance at $549," according to Huang.

All three serve as excellent choices for those seeking high performance without committing to the premium price of the flagship model. – which brings us to the cost of these GPUs...

## RTX 50-Series Pricing and Availability

The RTX 5090 will debut at **$1,999** and will hit shelves on January 30, 2025, alongside the RTX 5080 priced at $999. The RTX 5070 Ti and RTX 5070, priced at **$749** and **$549** respectively, will launch in February.

With this lineup, NVIDIA is offering a range of options to meet different performance needs and budgets. Let's take a look at what else sets the RTX 50-series apart from its predecessors.

## DLSS 4 and Neural Tech: What's New?

During his keynote speech, Huang showcased the RTX Blackwell GPUs with an impressive real-time rendering demo that highlighted the latest advancements. The demonstration featured RTX Neural Materials, RTX Neural Faces, text-to-animation capabilities, and **DLSS 4** in action.

As Huang put it, "The new generation of DLSS can generate beyond frames – it can predict the future. We used GeForce to enable AI, and now AI is revolutionizing GeForce."

**Multi Frame Generation** is a highlight of DLSS 4, using AI to extrapolate and insert up to three additional frames between traditionally rendered frames, boosting frame rates by up to 8x.

A standout feature, **RTX Neural Shaders**, compresses in-game textures more efficiently and delivers film-quality shading and lighting in real time. Another feature, called **RTX Neural Faces**, uses generative AI to create lifelike skin, hair, and facial details.

Not only that, but NVIDIA is also bringing the RTX 50-series to laptops starting in March 2025. The laptop version of the RTX 5090 will feature 24GB of GDDR7 memory, making it a powerhouse for users on the go.

## Experience Cutting-Edge Performance Now with Vast.ai

While the RTX 5090 is set to make a big splash in the GPU market, **you can already access top-tier computing power right now with Vast.ai.** Our platform offers flexible GPU rentals, enabling you to tackle your most demanding projects without breaking the bank or waiting for new hardware.

Explore our options and elevate your workflow with Vast. Don’t let hardware limitations hold you back – get started in just minutes, today!

### Additional Specifications:

| Spec                                    | RTX 5090            | RTX 5080            | RTX 5070 Ti        | RTX 5070        
| ----------------------------------------| ------------------- | ------------------- | ------------------ | ------------------ |
| **Architecture**                        | GB202               | GB203               | GB203              | GB205              |
| **VRAM**                                | 32GB                | 16GB                | 16GB               | 12GB               |
| **VRAM Speed**                          | 28Gbps              | 30Gbps              | 28Gbps             | 28Gbps             |
| **VRAM Bus Width**                      | 512                 | 256                 | 256                | 192                |
| **Bandwidth**                           | 1792 GB/s           | 960 GB/s            | 896 GB/s           | 672 GB/s           |
| **Streaming Multiprocessors (SMs)**     | 170                 | 84                  | 70                 | 48                 |
| **Tensor Cores**                        | 680                 | 336                 | 280                | 192                |
| **Ray Tracing Cores**                   | 170                 | 84                  | 70                 | 48                 |
| **Boost Clock**                         | 2407 Mhz            | 2617 Mhz            | 2452 Mhz           | 2512 Mhz           |
| **Total Graphics Power (TGP)**          | 575W                | 360W                | 300W               | 250W               |
| **Recommended Power Supply**            | 1000W               | 850W                | 750W               | 650W               |
| **Launch Date**                         | Jan. 30, 2025       | Jan. 30, 2025       | Feb. 2025          | Feb. 2025          |
| **Launch Price**                        | $1,999              | $999                | $749               | $549               |

NVIDIA has officially unveiled the GeForce RTX 5090, the latest in its groundbreaking lineup of GPUs.

NVIDIA GeForce RTX 5090: Release Announced


# Latest AI Model Releases: September and October

The world of Open Source AI has gotten many updates in the last month or so. There are now many new models with great quality:speed ratios and models that challenge the frontier of closed source models. This makes it even easier to build applications and automate workflows with open source models, which you can deploy on [Vast.ai](https://vast.ai). Meta, Mistral, and Nvidia have made the biggest waves with their recent releases.

## Meta and Llama 3.2 Models

On September 25th, Meta introduced several exciting [updates](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/) to their AI model lineup, focusing on lightweight models and vision capabilities.

### Lightweight Models

First, Meta released impressive 1B and 3B parameter models that are best in class for their size. These models allow for near-instantaneous interactions on edge devices and can be used for processing large amounts of tokens with simple processing tasks.

The development process for these models involved pruning from Llama 3.1-8B, followed by knowledge distillation using the 8B and 70B models as teachers. The instruction-tuned versions underwent additional training with Supervised Fine-tuning (SFT), Direct Policy Optimization (DPO), and Rejection Sampling, incorporating synthetic data from Llama 3.1-405B.

These small models are easy to download and get playing with locally, but can be leveraged for large scale data processing tasks as they are much less expensive to run and still have great capabilities.

### Vision Models

Meta's new vision models come in 11B and 90B parameter sizes. Interestingly, these are built as adapters for the 8B and 70B text models, respectively, maintaining the same performance on text tasks while adding visual capabilities. This approach saved compute during training and allows for flexible deployment options.

The vision models were created by pretraining adapters and image encoders with noisy image and text data, followed by post-training similar to the text-only models. This has two advantages: For the same compute budget, Meta could train the models on more data, and the models are deployable on the same GPU's at the same time as their text-only models and develoeprs don't have to worry about regressions in their text-only tasks.

### Llama Guard Models

Meta also released new Llama Guard models for safety and moderation tasks, based on the 11B vision model and the 1B and 3B text models. These models are designed to help with safety and moderation tasks at the edge.

## Mistral's AI Innovations

Mistral has been on a similar research path as Meta, releasing models that are focused on similar efforts and qualities, small efficient models for the edge, and multimodal models as drop-in replacements for existing text-only models.

### Pixtral Model

Released on September 17th, [Pixtral](https://mistral.ai/news/pixtral-12b/) is a 12B parameter multimodal model. Similar to Meta's Llama 3.2 Vision approach, Pixtral was created as a drop-in replacement for Mistral-Nemo-11B, a text-only model. It features a 400M image encoder as an adapter, maintaining deployment flexibility. It performs well on a variety of multi-modal tasks.

Pixtral is available under the Apache 2.0 license and can be downloaded for the base model [here](https://huggingface.co/mistralai/Pixtral-12B-Base-2409) and for the instruct model behind a gated release [here](https://huggingface.co/mistralai/Pixtral-12B-2409).

### Mistral-Small v24.09

Also released on September 17th, 2024, [Mistral-Small v24.09](https://mistral.ai/news/september-24-release/) is a 22B parameter text model that bridges the gap between Mistral Nemo-11B and Mistral Large v2. This model offers substantial improvements in reasoning, code generation, and function calling compared to its predecessor. It's available under the Mistral Research License, and can be downloaded [here](https://huggingface.co/mistralai/Mistral-Small-Instruct-2409) behind the research gate.

### Ministral Models

On September 25th, 2024, Mistral introduced two new models: [Ministral 3B and Ministral 8B](https://mistral.ai/news/ministraux/).

Ministral 3B is a compact powerhouse, offering capabilities roughly equivalent to the original Mistral 7B but in just 3B parameters. Its instruct-tuned version outperforms Mistral-7B instruct models and shows superior performance to Llama 3.2-3B. However, it is not availabel on HuggingFace, and requires an Enterprise License from Mistral to access.

Ministral 8B represents a significant upgrade over the Mistral 7B series, offering a substantial leap in capabilities. According to benchmarks, it outperforms Llama 3.1-8B across of series of tasks. You can download the model [here] (https://huggingface.co/mistralai/Ministral-8B-Instruct-2410) behind the gated access and run it on Vast to evaluate it for your workloads.

## Nvidia's Contribution to AI Advancement

Nvidia has also focused on 70B models to compete with frontier models like those from OpenAI and Anthropic.

### Llama 3.1-Nemotron-70B Models

Nvidia has [introduced](https://developer.nvidia.com/blog/new-reward-model-helps-improve-llm-alignment-with-human-preferences/) two notable models based on the Llama 3.1 architecture:

1. **Llama 3.1-Nemotron-70B-Reward**: This model was trained via Reinforcement Learning from Human Feedback (RLHF) to serve as a reward model. It currently holds the top position on the [RewardBench leaderboard](https://huggingface.co/spaces/allenai/reward-bench), showcasing its effectiveness in evaluating AI responses. It was also used to train the Instruct model, showing how well this model can be used to train other models synthetically.

2. **Llama 3.1-Nemotron-70B-Instruct**: Leveraging the reward model mentioned above, this instruct-tuned version has achieved impressive results. It currently leads the [ArenaHard Leaaderboard](https://huggingface.co/spaces/lmarena-ai/chatbot-arena-leaderboard/commit/2bba63b79d685a2d46696c2da0576a86a3cfc7c6), demonstrating its superior performance in complex instruction-following tasks.

The Llama 3.1-Nemotron-70B-Reward model was trained on the [HelpSteer-v2 dataset](https://arxiv.org/abs/2406.08673), which likely contributed to its strong performance as a reward model.

What's particularly noteworthy is that the Llama 3.1-Nemotron-70B-Instruct model has shown outstanding results in automatic benchmarks, outperforming even some of the most advanced models in the field, including Claude 3.5 Sonnet and GPT-4.

These developments from Nvidia show that methods to move beyond the GPT-4 wall of capabilities are not just being developed, but open sourced and available to developers to use wherever they use compute.

## Conclusion:

It's a great time to be a developer using AI. In the past few months, there have been major capability improvements for LLM's and Multimodal models, enabling tasks that were not possible before with open source models. And the release of many smaller language models that are still as capable as previous generation models means that the cost curve is coming down for deploying to production.

At Vast, we're very excited to see these advancements, and you can look out for more updates and building on top of these types of models for specific workflows in the coming months.


Open Source AI has evolved rapidly, with new models offering top quality and speed, challenging closed-source standards.