Nvidia unveils its new NVLM 1.0 AI model, rivaling the likes of OpenAI’s GPT-4

Nvidia has released its powerful open-source artificial intelligence model that could outpace the likes of OpenAI’s GPT-4.

The company’s new NVLM 1.0 family of open-source multimodal large language models (LLMs), with its flagship model, NVLM-D-72B, has around 72 billion parameters.

According to Nvidia’s research team, the new AI model excels in vision-language tasks while maintaining and even improving text-only performance compared to their LLM backbones. In their paper, the researchers state: “We introduce NVLM 1.0, a family of frontier-class multimodal large language models that achieve state-of-the-art results on vision-language tasks, rivaling the leading proprietary models (e.g., GPT-4o) and open-access models.”

Unlike some of the other proprietary models where there is a significant decline in text performance over time, the NVLM-D-72B reportedly increased its accuracy by an average of 4.3 points across key text benchmarks.

The LLM was also able to interpret charts and tables, analyze images, understand memes, code software, as well as solve mathematical problems. The model weights are publicly available on Hugging Face and Nvidia says it will eventually release the training code.

AI researchers on X have called the release “wild,” and praised its ability to understand visual data. One user wrote: “Wow! Nvidia just published a 72B model with is ~on par with llama 3.1 405B in math and coding evals and also has vision ?”

NVLM by NVIDIA is wild. And Open. Check it out.https://t.co/fYpagW4Kog pic.twitter.com/r9V8uamGVf

— Alex Zhavoronkov, PhD (aka Aleksandrs Zavoronkovs) (@biogerontology) October 2, 2024

Wow. New NVIDIA 72B model rivals Llama's 405B!