The company says the 132 billion parameter DBRX model surpasses popular open-source LLMs like LLaMA 2 70B, Mixtral, and Grok-1 across language understanding, programming, and maths tasks. It even outperforms Anthropic’s closed-source model Claude on certain benchmarks.

DBRX demonstrated state-of-the-art performance among open models on coding tasks, beating out specialised models like CodeLLaMA despite being a general-purpose LLM. It also matched or exceeded GPT-3.5 across nearly all benchmarks evaluated.

The state-of-the-art capabilities come thanks to a more efficient mixture-of-experts architecture that makes DBRX up to 2x faster at inference than LLaMA 2 70B, despite having fewer active parameters. Databricks claims training the model was also around 2x more compute-efficient than dense alternatives.

“DBRX is setting a new standard for open source LLMs—it gives enterprises a platform to build customised reasoning capabilities based on their own data,” said Ali Ghodsi, Databricks co-founder and CEO.

DBRX was pretrained on a massive 12 trillion tokens of “carefully curated” text and code data selected to improve quality. It leverages technologies like rotary position encodings and curriculum learning during pretraining.

Customers can interact with DBRX via APIs or use the company’s tools to finetune the model on their proprietary data. It’s already being integrated into Databricks’ AI products.

“Our research shows enterprises plan to spend half of their AI budgets on generative AI,” said Dave Menninger, Executive Director, Ventana Research, part of ISG. “One of the top three challenges they face is data security and privacy.

“With their end-to-end Data Intelligence Platform and the introduction of DBRX, Databricks is enabling enterprises to build generative AI applications that are governed, secure and tailored to the context of their business, while maintaining control and ownership of their IP along the way.”

Partners including Accenture, Block, Nasdaq, Prosus, Replit, and Zoom praised DBRX’s potential to accelerate enterprise adoption of open, customised large language models. Analysts said it could drive a shift from closed to open source as fine-tuned open models match proprietary performance.

Mike O’Rourke, Head of AI and Data Services at NASDAQ, commented: “Databricks is a key partner to Nasdaq on some of our most important data systems. They continue to be at the forefront of the industry in managing data and leveraging AI, and we are excited about the release of DBRX.

“The combination of strong model performance and favourable serving economics is the kind of innovation we are looking for as we grow our use of generative AI at Nasdaq.”

You can find the DBRX base and fine-tuned models on Hugging Face. The project’s GitHub has further resources and code examples.

(Photo by Ryan Quintal)

See also: Large language models could ‘revolutionise the finance sector within two years’

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Databricks claims DBRX sets ‘a new standard’ for open-source LLMs appeared first on AI News.

The release fulfils Musk’s promise to open source Grok in an effort to accelerate AI development and adoption.

XAI announced the move in a blog post, stating: “We are releasing the base model weights and network architecture of Grok-1, our large language model. Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.”

Grok had previously only been available through Musk’s social network X as part of the paid X Premium+ subscription. By open sourcing it, xAI has empowered developers, companies, and enthusiasts worldwide to leverage the advanced language model’s capabilities.

The model’s release includes its weights, which represent the strength of connections between its artificial neurons, as well as documentation and code. However, it omits the original training data and access to real-time data streams that gave the proprietary version an advantage.

Named after a term meaning “understanding” from Douglas Adams’ Hitchhiker’s Guide series, Grok has been positioned as a more open and humorous alternative to OpenAI’s ChatGPT. The move aligns with Musk’s battle against censorship, “woke” ideology displayed by models like Gemini, and his recent lawsuit claiming OpenAI violated its nonprofit principles.

While xAI’s open source release earned praise from open source advocates, some critics raised concerns about potential misuse facilitated by unrestricted access to powerful AI capabilities.

You can find Grok-1 on GitHub here.

(Image Credit: xAI)

See also: Anthropic says Claude 3 Haiku is the fastest model in its class

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Elon Musk’s xAI open-sources Grok appeared first on AI News.

The ambitious expansion was announced by former Tesla staff scientist Remi Cadene in a post on X:

In keeping with Hugging Face’s ethos of open source, Cadene stated the robot project would be “open-source, not as in Open AI” in reference to OpenAI’s legal battle with Cadene’s former boss, Elon Musk.

Cadene – who will be leading the robotics initiative – revealed that Hugging Face is hiring robotics engineers in Paris, France.

A job listing for an “Embodied Robotics Engineer” sheds light on the project’s goals, which include “designing, building, and maintaining open-source and low cost robotic systems that integrate AI technologies, specifically in deep learning and embodied AI.”

The role involves collaborating with ML engineers, researchers, and product teams to develop innovative robotics solutions that “push the boundaries of what’s possible in robotics and AI.” Key responsibilities range from building low-cost robots using off-the-shelf components and 3D-printed parts to integrating deep learning and embodied AI technologies into robotic systems.

Until now, Hugging Face has primarily focused on software offerings like its machine learning codebase and open-source chatbot. The robotics project marks a significant departure into the hardware realm as the startup aims to bring AI into the physical world through open and affordable robotic platforms.

(Photo by Possessed Photography on Unsplash)

See also: Google engineer stole AI tech for Chinese firms

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Hugging Face is launching an open robotics project appeared first on AI News.

The geospatial model, built from NASA’s satellite data, will be the largest of its kind on Hugging Face and marks the first-ever open-source AI foundation model developed in collaboration with NASA.

Jeff Boudier, head of product and growth at Hugging Face, highlighted the importance of information sharing and collaboration in driving progress in AI. Open-source AI and the release of models and datasets are fundamental in ensuring AI benefits as many people as possible.

Climate science faces constant challenges due to rapidly changing environmental conditions, requiring access to the latest data. Despite the abundance of data, scientists and researchers struggle to analyse the vast datasets effectively. NASA estimates that by 2024, there will be 250,000 terabytes of data from new missions.

To address this issue, IBM embarked on a Space Act Agreement with NASA earlier this year—aiming to build an AI foundation model for geospatial data.

By making this geospatial foundation model openly available on Hugging Face, both companies aim to promote collaboration and accelerate progress in climate and Earth science.

Sriram Raghavan, VP at IBM Research AI, commented:

“The essential role of open-source technologies to accelerate critical areas of discovery such as climate change has never been clearer.

By combining IBM’s foundation model efforts aimed at creating flexible, reusable AI systems with NASA’s repository of Earth-satellite data, and making it available on the leading open-source AI platform, Hugging Face, we can leverage the power of collaboration to implement faster and more impactful solutions that will improve our planet.”

The geospatial model, jointly trained by IBM and NASA on Harmonized Landsat Sentinel-2 satellite data (HLS) over one year across the continental United States, has shown promising results. It demonstrated a 15 percent improvement over state-of-the-art techniques using only half the labelled data.

With further fine-tuning, the model can be adapted for various tasks such as deforestation tracking, crop yield prediction, and greenhouse gas detection.

IBM’s collaboration with NASA in building the AI model aligns with NASA’s decade-long Open-Source Science Initiative, promoting a more accessible and inclusive scientific community. NASA, along with other federal agencies, has designated 2023 as the Year of Open Science, celebrating the benefits of sharing data, information, and knowledge openly.

Kevin Murphy, Chief Science Data Officer at NASA, said:

“We believe that foundation models have the potential to change the way observational data is analysed and help us to better understand our planet.

By open-sourcing such models and making them available to the world, we hope to multiply their impact.”

The geospatial model leverages IBM’s foundation model technology and is part of IBM’s broader initiative to create and train AI models with transferable capabilities across different tasks.

In June, IBM introduced watsonx, an AI and data platform designed to scale and accelerate the impact of advanced AI with trusted data. A commercial version of the geospatial model, integrated into IBM watsonx, will be available through the IBM Environmental Intelligence Suite (EIS) later this year.

By leveraging the power of open-source technologies, this latest collaboration aims to address climate challenges effectively and contribute to a more sustainable future for our planet.

(Photo by Markus Spiske on Unsplash)

See also: Jay Migliaccio, IBM Watson: On leveraging AI to improve productivity

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The event is co-located with Digital Transformation Week.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post IBM and Hugging Face release AI foundation model for climate science appeared first on AI News.

The Llama 2 models range in size from 7-70 billion parameters, making them a formidable force in the AI landscape.

According to Meta’s claims, these models “outperform open source chat models on most benchmarks we tested.”

The release of Llama 2 marks a turning point in the LLM (large language model) market and has already caught the attention of industry experts and enthusiasts alike.

The new language models offered by Llama 2 come in two variants – pretrained and fine-tuned:

• The pretrained models are trained on a whopping two trillion tokens and have a context window of 4,096 tokens, enabling them to process vast amounts of content at once.
• The fine-tuned models, designed for chat applications like ChatGPT, have been trained on “over one million human annotations,” further enhancing their language processing capabilities.

While Llama 2’s performance may not yet rival OpenAI’s GPT-4, it shows remarkable promise for an open-source model.

The Llama 2 journey started with its predecessor, LLaMA, which Meta released as open source with a non-commercial license in February.

However, someone leaked LLaMA’s weights to torrent sites, leading to a surge in its usage within the AI community. This laid the foundation for a fast-growing underground LLM development scene.

Open-source AI models like Llama 2 come with their share of advantages and concerns.

On the positive side, they encourage transparency in terms of training data, foster economic competition, promote free speech, and democratise access to AI. However, critics point out potential risks, such as misuse in synthetic biology, spam generation, or disinformation.

To address such concerns, Meta released a statement in support of its open innovation approach, emphasising that responsible and open innovation encourages transparency and trust in AI technologies.

Despite the benefits of open-source models, some critics remain sceptical, especially regarding the lack of transparency in the training data used for LLMs. While Meta claims to have made efforts to remove data containing personal information, the specific sources of training data remain undisclosed, raising concerns about privacy and ethical considerations.

With the combination of open-source development and commercial licensing, Llama 2 promises to bring exciting advancements and opportunities to the AI community while simultaneously navigating the challenges of data privacy and responsible usage.

(Photo by Joakim Honkasalo on Unsplash)

See also: Anthropic launches ChatGPT rival Claude 2

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The event is co-located with Digital Transformation Week.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Meta launches Llama 2 open-source LLM appeared first on AI News.

“The AI Act will define how the world regulates AI and we need to get it right, for developers and the open-source community,” said Dohmke.

Dohmke was born and grew up in Germany but now lives in the US. As such, he is all too aware of the widespread belief that the EU cannot lead when it comes to tech innovation.

“As a European, I love seeing how open-source AI innovations are beginning to break the narrative that only the US and China can lead on tech innovation.”

“I’ll be honest, as a European living in the United States, this is a pervasive – and often true – narrative. But this can change. And it’s already beginning to, thanks to open-source developers.”

AI will revolutionise just about every aspect of our lives. Regulation is vital to minimise the risks associated with AI while allowing the benefits to flourish.

“Together, OSS (Open Source Software) developers will use AI to help make our lives better. I have no doubt that OSS developers will help build AI innovations that empower those with disabilities, help us solve climate change, and save lives.”

A risk of overregulation is that it drives innovation elsewhere. Startups are more likely to establish themselves in countries like the US and China where they’re likely not subject to as strict regulations. Europe will find itself falling behind and having less influence on the global stage when it comes to AI.

“The AI Act is so crucial. This policy could well set the precedent for how the world regulates AI. It is foundationally important. Important for European technological leadership, and the future of the European economy itself. The AI Act must be fair and balanced for the open-source community.

“Policymakers should help us get there. The AI Act can foster democratised innovation and solidify Europe’s leadership in open, values-based artificial intelligence. That is why I believe that open-source developers should be exempt from the AI Act.”

In expanding on his belief that open-source developers should be exempt, Dohmke explains that the compliance burden should fall on those shipping products.

“OSS developers are often volunteers. Many are working two jobs. They are scientists, doctors, academics, professors, and university students alike. They don’t usually stand to profit from their contributions—and they certainly don’t have big budgets and compliance departments!”

EU lawmakers are hoping to agree on draft AI rules next month with the aim of winning the acceptance of member states by the end of the year.

“Open-source is forming the foundation of AI innovation in Europe. The US and China don’t have to win it all. Let’s break that narrative apart!

“Let’s give the open-source community the daylight and the clarity to grow their ideas and build them for the rest of the world! And by doing so, let’s give Europe the chance to be a leader in this new age of AI.”

GitHub’s policy paper on the AI Act can be found here.

(Image Credit: Collision Conf under CC BY 2.0 license)

Relevant: US and EU agree to collaborate on improving lives with AI

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post GitHub CEO: The EU ‘will define how the world regulates AI’ appeared first on AI News.

The tool helps to protect users from certain unsolicited photos – and we’re not just talking about genitalia, but also shirtless selfies and photos of firearms.

When a suspect image is received by a user, it’s blurred to allow the recipient to view it, block it, or report the sender.

Online harassment is difficult to counter entirely but AI is proving a powerful tool in helping to protect users, especially the vulnerable. Shocking research in 2020 found that 75.8 percent of girls between the ages of 12 and 18 have been sent unsolicited nude images.

By open-sourcing its AI tool, Bumble can help to protect more online users.

“It’s our hope that the feature will be adopted by the wider tech community as we work in tandem to make the internet a safer place,” explained Bumble in a blog post.

Human content moderators see the worst of the web day in and day out. Spending your days reviewing abuse, torture, massacres, beheadings, and more is bound to take a serious mental toll. As a result, content moderators often require therapy and is a role associated with one of the highest suicide rates.

Relying solely on AI moderation is problematic. Human moderators, for example, can understand context and tell the difference between content exposing war crimes and that of terrorist propaganda glorifying hate and violence.

AI tools like the one open-sourced by Bumble can help to protect moderators by blurring the content while still harnessing the unique skills of humans.

In its blog post, Bumble explained how it traversed the trade-offs between performance and the ability to serve its user base at scale:

“We implemented (in its latest iteration) an EfficientNetv2-based binary classifier: a convolutional network that has faster training speed and overall better parameters efficiency. It uses a combination of better designed architecture and scaling, with layers like MBConv (that utilizes 1×1 convolutions to wide up the space and depth-wise convolutions for reducing the number of overall parameters) and FusedMBConv (that merges some steps of the vanilla MBConv above for faster execution), to jointly optimize training speed and parameter efficiency.

The model has been trained leveraging our GPU powered data centers in a continuous exercise of dataset, network and hyperparameters (the settings used to speed up or improve the training performance) optimization.”

Bumble says its tool, both offline and online, achieves “world class performance” of over 98 percent accuracy in both upsample and production-like settings.

You can find Bumble’s tool on GitHub here. The tool has been released under the Apache License so anyone can implement it as is for blurring lewd images or can fine-tune it with additional training samples.

(Image Credit: Bumble)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Bumble open-sources its lewd-spotting AI tool appeared first on AI News.

PyTorch Live was unveiled during PyTorch Developer Day and enables anyone to build mobile ML demo apps using JavaScript, the world’s most popular programming language.

While on-device AI demos cannot currently be shared, Meta says that functionality is on the way. Developers can start building custom machine learning models to later share with the broader PyTorch community.

PyTorch was publicly launched by Meta back in January 2017, when the company was still known as Facebook. The open-source machine learning library quickly became a firm favourite among the developer and data science communities.

As the PyTorch name suggests, the main library’s interface is designed around Python but it also has a C++ interface. 

The once-dominant machine learning library, TensorFlow, had a two-year headstart on PyTorch but has been falling behind in usage in recent years.

In 2018, GitHub’s Octoverse report highlighted the growth of PyTorch as an open-source project outpacing that of TensorFlow. PyTorch grew by 2.8x that year compared to TensorFlow’s still not insubstantial 1.8x.

That edge for PyTorch appears to be eating into TensorFlow’s early mover advantage.

TensorFlow appeared in three times more job listings in Indeed, Monster, SimplyHired, and LinkedIn as PyTorch in April 2019. However, TensorFlow’s edge in job-listing mentions dropped to 2x in 2020.

Over the past year, PyTorch has also overtaken TensorFlow in worldwide Google searches:

PyTorch Live looks set to accelerate the success of the machine learning library. The tools use React Native for building cross-platform visual user interfaces and PyTorch Mobile powers on-device inference.

Anyone wanting to get started with PyTorch Live can do so through its command-line interface setup and/or its data processing API.

(Image Credit: Meta)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo North America on 11-12 May 2022.

The post Meta releases PyTorch Live for creating mobile ML demos ‘in minutes’ appeared first on AI News.