AI Performance Standards on PCs – How a Non-Profit Leads the Way?

AI performance

Most read

Loading Most Ready posts..


  • MLCommons, renowned for its cloud-based AI benchmarks, is creating an MLPerf Client working group to establish benchmarks for measuring AI capabilities on desktops, laptops, and workstations.
  • The move is driven by the rising significance of running AI workloads locally on devices, with the benchmarks aimed at aiding consumers in making informed decisions about their AI-driven applications.
  • The first benchmark focuses on generative AI text-generation, involving collaborations with industry giants like Microsoft, Nvidia, and Qualcomm, while Apple remains noticeably absent from the initiative.

In a significant move to cater to the evolving landscape of artificial intelligence, MLCommons, a nonprofit organization renowned for its cloud computing-based AI benchmarks, has announced the formation of a groundbreaking initiative. The MLPerf Client working group aims to set the gold standard for measuring AI performance on personal computing devices. As AI increasingly shifts from the cloud to local device processing, the benchmarks seek to provide valuable insights into the capabilities of desktops, laptops, and workstations, influencing consumers’ choices in a rapidly advancing technological era.

MLPerf client benchmarks – Unveiling the next frontier for AI performance

With the growing significance of artificial intelligence in various facets of our lives, MLCommons recognizes the need to extend its influence beyond cloud-based standards. The MLPerf Client working group, established with the primary goal of introducing benchmarks for local AI workloads, is set to revolutionize the way consumers evaluate the performance of their personal computing devices. Unlike conventional benchmarks, the new standards are envisioned to be scenario-based, focusing on real-world applications and drawing inspiration from feedback within the community.

The inaugural benchmark introduced by MLCommons concentrates on generative AI text-generation, a pivotal aspect of contemporary AI applications. This benchmark assesses the performance of laptops, desktops, and workstations running Meta Platforms Inc.’s Llama 2. The collaboration with industry leaders like Microsoft and Qualcomm indicates a concerted effort to optimize Llama 2 for Windows and specific mobile devices. The scenario-based approach ensures that the benchmarks are not just theoretical but grounded in the practical needs and experiences of users.

The participation of major companies such as Advanced Micro Devices, Arm, ASUSTek Computing, Dell Technologies, Intel, Lenovo, and Nvidia in the MLPerf Client working group underscores the industry-wide acknowledgment of the importance of local AI processing. With MLCommons steering the direction, these benchmarks are poised to become the go-to guide for businesses and consumers navigating the complex landscape of AI-capable personal computing devices.

Industry Perspectives – Shaping the future of AI performance

According to the executive director of MLCommons, David Kanter, he conveyed optimism about the initiative, emphasizing that large language models serve as a natural and exciting starting point for the MLPerf Client working group. The focus on client systems is timely, reflecting the increasing importance of AI in day-to-day computing experiences. Nvidia’s director of performance benchmarking, Jani Joki, highlights the crucial role MLPerf benchmarks have played in measuring advancements in machine learning within data centers and anticipates a similar impact on client systems.

Interestingly, Apple Inc. is notably absent from the list of collaborators, raising questions about the potential exclusion of MacBook devices from these benchmarks. As generative AI becomes a staple in the tech landscape, Apple might find itself compelled to participate in the future iterations of MLPerf Client benchmarks, especially if the ability to run generative AI locally becomes a decisive factor for consumers.

As MLCommons sets the stage for a new era in AI performance evaluation on personal computing devices, the notable absence of Apple Inc. prompts a critical question: How will the company respond to the evolving landscape of generative AI, and will it eventually join the ranks of those contributing to the establishment of client system benchmarks? In a world where the efficiency of AI on personal devices holds increasing importance, the trajectory of Apple’s involvement could significantly impact the choices available to consumers in the realm of AI-powered computing.

Disclaimer. The information provided is not trading advice. Cryptopolitan.com holds no liability for any investments made based on the information provided on this page. We strongly recommend independent research and/or consultation with a qualified professional before making any investment decisions.

Share link:

Aamir Sheikh

Amir is a media, marketing and content professional working in the digital industry. A veteran in content production Amir is now an enthusiastic cryptocurrency proponent, analyst and writer.

Stay on top of crypto news, get daily updates in your inbox

Related News

Subscribe to CryptoPolitan