GPT-4o, where the “o” stands for “omni,” is designed to cater to a broader spectrum of input and output modalities. “It accepts as input any combination of text, audio, and image and generates any combination of text, audio, and image outputs,” OpenAI announced.

Users can expect a response time as quick as 232 milliseconds, mirroring human conversational speed, with an impressive average response time of 320 milliseconds.

Pioneering capabilities

The introduction of GPT-4o marks a leap from its predecessors by processing all inputs and outputs through a single neural network. This approach enables the model to retain critical information and context that were previously lost in the separate model pipeline used in earlier versions.

Prior to GPT-4o, ‘Voice Mode’ could handle audio interactions with latencies of 2.8 seconds for GPT-3.5 and 5.4 seconds for GPT-4. The previous setup involved three distinct models: one for transcribing audio to text, another for textual responses, and a third for converting text back to audio. This segmentation led to loss of nuances such as tone, multiple speakers, and background noise.

As an integrated solution, GPT-4o boasts notable improvements in vision and audio understanding. It can perform more complex tasks such as harmonising songs, providing real-time translations, and even generating outputs with expressive elements like laughter and singing. Examples of its broad capabilities include preparing for interviews, translating languages on the fly, and generating customer service responses.

Nathaniel Whittemore, Founder and CEO of Superintelligent, commented: “Product announcements are going to inherently be more divisive than technology announcements because it’s harder to tell if a product is going to be truly different until you actually interact with it. And especially when it comes to a different mode of human-computer interaction, there is even more room for diverse beliefs about how useful it’s going to be.

“That said, the fact that there wasn’t a GPT-4.5 or GPT-5 announced is also distracting people from the technological advancement that this is a natively multimodal model. It’s not a text model with a voice or image addition; it is a multimodal token in, multimodal token out. This opens up a huge array of use cases that are going to take some time to filter into the consciousness.”

Performance and safety

GPT-4o matches GPT-4 Turbo performance levels in English text and coding tasks but outshines significantly in non-English languages, making it a more inclusive and versatile model. It sets a new benchmark in reasoning with a high score of 88.7% on 0-shot COT MMLU (general knowledge questions) and 87.2% on the 5-shot no-CoT MMLU.

The model also excels in audio and translation benchmarks, surpassing previous state-of-the-art models like Whisper-v3. In multilingual and vision evaluations, it demonstrates superior performance, enhancing OpenAI’s multilingual, audio, and vision capabilities.

OpenAI has incorporated robust safety measures into GPT-4o by design, incorporating techniques to filter training data and refining behaviour through post-training safeguards. The model has been assessed through a Preparedness Framework and complies with OpenAI’s voluntary commitments. Evaluations in areas like cybersecurity, persuasion, and model autonomy indicate that GPT-4o does not exceed a ‘Medium’ risk level across any category.

Further safety assessments involved extensive external red teaming with over 70 experts in various domains, including social psychology, bias, fairness, and misinformation. This comprehensive scrutiny aims to mitigate risks introduced by the new modalities of GPT-4o.

Availability and future integration

Starting today, GPT-4o’s text and image capabilities are available in ChatGPT—including a free tier and extended features for Plus users. A new Voice Mode powered by GPT-4o will enter alpha testing within ChatGPT Plus in the coming weeks.

Developers can access GPT-4o through the API for text and vision tasks, benefiting from its doubled speed, halved price, and enhanced rate limits compared to GPT-4 Turbo.

OpenAI plans to expand GPT-4o’s audio and video functionalities to a select group of trusted partners via the API, with broader rollout expected in the near future. This phased release strategy aims to ensure thorough safety and usability testing before making the full range of capabilities publicly available.

“It’s hugely significant that they’ve made this model available for free to everyone, as well as making the API 50% cheaper. That is a massive increase in accessibility,” explained Whittemore.

OpenAI invites community feedback to continuously refine GPT-4o, emphasising the importance of user input in identifying and closing gaps where GPT-4 Turbo might still outperform.

(Image Credit: OpenAI)

See also: OpenAI takes steps to boost AI-generated content transparency

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post GPT-4o delivers human-like AI interaction with text, audio, and vision integration appeared first on AI News.

Algorithmia specialises in artificial intelligence operations and management. The company is backed by Google LLC and focuses on simplifying AI projects for enterprises just getting started.

Diego Oppenheimer, CEO of Algorithmia, says:

“Organisations have specific needs when it comes to ML model monitoring and reporting.

For example, they are concerned with compliance as it pertains to external and internal regulations, model performance for improvement of business outcomes, and reducing the risk of model failure.

Algorithmia Insights helps users overcome these issues while making it easier to monitor model performance in the context of other operational metrics and variables.” 

Insights aims to help enterprises to monitor the performance of their machine learning models. Many organisations currently don’t have that ability, or use a complex variety of tools and/or manual processes.

Operational metrics like execution time and request identification are combined with user-defined metrics such as confidence and accuracy to identify data skews, negative feedback loops, and model drift.

Model drift, in layman’s terms, is the degradation of a model’s prediction power due to changes in the environment—which subsequently impacts the relationship between variables. A far more detailed explanation can be found here for those interested.

Algorithmia teamed up with monitoring service Datadog to allow customers to stream operational – as well as user-defined inference metrics – from Algorithmia, to Kafka, and then into Datadog.

Ilan Rabinovitch, Vice President of Product and Community at Datadog, comments:

“ML models are at the heart of today’s business. Understanding how they perform both statistically and operationally is key to success.

By combining the findings of Algorithmia Insights and Datadog’s deep visibility into code and integration, our mutual customers can drive more accurate and performant outcomes from their ML models.”

Through integration with Datadog and its Metrics API, customers can measure and monitor their ML models to immediately detect data drift, model drift, and model bias.

(Photo by Chris Liverani on Unsplash)

Interested in hearing industry leaders discuss subjects like this? Attend the co-located 5G Expo, IoT Tech Expo, Blockchain Expo, AI & Big Data Expo, and Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London, and Amsterdam.

The post Algorithmia announces Insights for ML model performance monitoring appeared first on AI News.

There are two new chips in the 7-series – the Snapdragon 730 and 730G – along with one in the 6-series, the Snapdragon 665.

Kedar Kondap, VP of Product Management at Qualcomm, says:

“With the introduction of the Snapdragon 730, 730G and 665 Mobile Platforms, we are bringing features such as sophisticated AI, exceptional gaming and advanced camera capabilities to a broad spectrum of devices at exceptional performance.

Each iteration of Snapdragon drives immense innovation that will surpass customer expectations.”

The base Snapdragon 730 boasts double the AI processing power of its predecessor using Qualcomm’s 4th-gen multi-core AI Engine. A Qualcomm Hexagon 688 processor inside the 730 supports improved base scalar and Hexagon Vector eXtensions (HVX) performance, as well as the new Hexagon Tensor Accelerator.

Qualcomm has also packed in the Spectra 350 which contains a dedicated ISP for computer vision. It claims to provide up to four times as much power savings as its predecessor.

As for the Snapdragon 730G, that ‘G’ stands for ‘gamer’ and intends to boost the gaming performance of devices packing it.

While the 730 features the Adreno 618 GPU, the first time a Qualcomm 7-series chip has supported the Vulkan 1.1 graphics API, the 730G claims to offer 15 percent faster rendering. Furthermore, Qualcomm claims it’s “collaborated with consumers’ favorite game-makers to optimize Snapdragon 730G for a selection of the world’s top-rated games.”

A dedicated ‘Jank Reducer’ feature boasts that it can reduce stutter by up to 90 percent in games running at 30FPS.

Perhaps most impressive, however, is the Snapdragon 665 which brings more capable AI processing abilities to lower-end devices.

While the new 7-series chips feature the 4th-gen, the Snapdragon 665 is packing the 3rd-gen of Qualcomm’s AI Engine with the Hexagon 686 DSP. It too claims to offer double the performance of its predecessor.

Packing the Spectra 165 ISP, the Snapdragon 665 provides AI-powered scene recognition and auto adjustments like HDR. To account for the growing number of unique camera setups on today’s smartphones, it supports up to three cameras which can include telephoto, wide-angle, and ultra-wide lenses.

Qualcomm has also partnered with Morpho to enhance the camera functionality of the Snapdragon chips.

Toshi Torihara, VP of Morpho, commented:

“Morpho is proud to collaborate with Qualcomm to integrate our image processing software on mobile. With the Snapdragon 665 mobile platform, we brought MovieSolid, our state-of-the-art image stabilisation implementation, to deliver industry-leading video experiences to the mobile camera industry.

We believe this partnership will further enhance smartphone camera capabilities, bringing mobile imaging & AI technologies to the next level.”

For gamers, the Snapdragon 665 features the Adreno 610 GPU which also supports the Vulkan 1.1 graphics API to help provide an improved gaming experience for those who on lower-end devices.

Commercial devices based on the Snapdragon 730, 730G, and 665 are expected to be available in mid-2019.

Interested in hearing industry leaders discuss subjects like this and their use cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, Blockchain Expo, and Cyber Security & Cloud Expo.

The post Qualcomm boosts Snapdragon AI performance with new chips for high and mid-tier devices appeared first on AI News.