File:Performance of AI models on various benchmarks from 1998 to 2024.png

Original file (1,466 × 817 pixels, file size: 137 KB, MIME type: image/png)

Summary

Description
English: Figure 2. Performance of AI models on various benchmarks from 1998 to 2024, including computer vision (MNIST, ImageNet), speech recognition (Switchboard), natural language understanding (SQuAD 1.1, MMLU, GLUE), general language model evaluation (MMLU, Big-Bench, and GPQA), and mathematical reasoning (MATH). Many models surpass human-level performance (black solid line) by 2024, demonstrating significant advancements in AI capabilities across different domains over the past two decades. Data are from (94) for MNSIT, Switchboard, ImageNet, SQuAD 1.1, 2 and GLUE. Data for MMLU, Big Bench, GPQA are from the relevant papers (95, 96, 97).
Date
Source International Scientific Report on the Safety of Advanced AI, interim report, May 2024
Author

CHAIR Prof. Yoshua Bengio, Université de Montréal / Mila - Quebec AI Institute EXPERT ADVISORY PANEL Prof. Bronwyn Fox, The Commonwealth Scientific and Industrial Research Organisation (CSIRO) (Australia) André Carlos Ponce de Leon Ferreira de Carvalho, Institute of Mathematics and Computer Sciences, University of São Paulo (Brazil) Dr. Mona Nemer, Chief Science Advisor of Canada (Canada) Raquel Pezoa Rivera, Federico Santa María Technical University (Chile) Dr. Yi Zeng, Institute of Automation, Chinese Academy of Sciences (China) Juha Heikkilä, DG Connect (European Union) Guillaume Avrin, General Directorate of Enterprises (France) Prof. Antonio Krüger, German Research Center for Artificial Intelligence (Germany) Prof. Balaraman Ravindran, Indian Institute of Technology, Madras (India) Prof. Hammam Riza, KORIKA (Indonesia) Dr. Ciarán Seoighe, Science Foundation Ireland (Ireland) Dr. Ziv Katzir, Israel Innovation Authority (Israel) Dr. Andrea Monti, University of Chieti-Pescara (Italy) Dr. Hiroaki Kitano, Sony Group (Japan) [Interim] Mary Kerema, Ministry of Information Communications Technology and Digital Economy (Kenya) Dr. José Ramón López Portillo, Q Element (Mexico) Prof. Haroon Sheikh, Netherlands’ Scientific Council for Government Policy (Netherlands) Dr. Gill Jolly, Ministry of Business, Innovation and Employment (New Zealand) Dr. Olubunmi Ajala, Innovation and Digital Economy (Nigeria) Dominic Ligot, CirroLytix (Philippines) Prof. Kyoung Mu Lee, Department of Electrical and Computer Engineering, Seoul National University (Republic of Korea) Ahmet Halit Hatip, Turkish Ministry of Industry and Technology (Republic of Turkey) Crystal Rugege, National Center for AI and Innovation Policy (Rwanda) Dr. Fahad Albalawi, Saudi Authority for Data and Artificial Intelligence (Kingdom of Saudi Arabia) Denise Wong, Data Innovation and Protection Group, Infocomm Media Development Authority (IMDA) (Singapore) Dr. Nuria Oliver, ELLIS Alicante (Spain) Dr. Christian Busch, Federal Department of Economic Affairs, Education and Research (Switzerland) Oleksii Molchanovskyi, Expert Committee on the Development of Artificial intelligence in Ukraine (Ukraine) Marwan Alserkal, Ministry of Cabinet Affairs, Prime Minister’s Office (United Arab Emirates) Saif M. Khan, U.S. Department of Commerce (United States) Dame Angela McLean, Government Chief Scientific Adviser (United Kingdom) Amandeep Gill, UN Tech Envoy (United

Nations)

Licensing

This file is licensed under the United Kingdom Open Government Licence v3.0.
You are free to:
  • copy, publish, distribute and transmit the Information;
  • adapt the Information;
  • exploit the Information commercially and non-commercially for example, by combining it with other Information, or by including it in your own product or application.
You must, where you do any of the above:
  • acknowledge the source of the Information in your product or application by including or linking to any attribution statement specified by the Information Provider(s) and, where possible, provide a link to this licence;
  • If the Information Provider does not provide a specific attribution statement, you must use the following:
Contains public sector information licensed under the Open Government Licence v3.0.
This licence does NOT cover:
  • personal data in the Information;
  • Information that has not been accessed by way of publication or disclosure under information access legislation (including the Freedom of Information Acts for the UK and Scotland) by or with the consent of the Information Provider;
  • departmental or public sector organisation logos, crests and the Royal Arms except where they form an integral part of a document or dataset;
  • military insignia;
  • third party rights the Information Provider is not authorised to license;
  • other intellectual property rights, including patents, trade marks, and design rights; and
  • identity documents such as the British Passport.
Consult this guide for full details.
Note: Since 2010, almost all information owned by the UK Crown is offered for use and re-use under the Open Government Licence by authority of The Controller of His Majesty's Stationery Office.info
See also: Meta for information on usage on Wikimedia wikis.

Captions

Figure 2. Performance of AI models on various benchmarks from 1998 to 2024.

Items portrayed in this file

depicts

Neural scaling law

data size

140,006 byte

height

817 pixel

width

1,466 pixel

media type

image/png

checksum

6394a38ccc2050d91c28631d4817420a065051c2

determination method or standard: SHA-1

File history

Click on a date/time to view the file as it appeared at that time.

Date/TimeThumbnailDimensionsUserComment
current22:52, 24 September 2024Thumbnail for version as of 22:52, 24 September 20241,466 × 817 (137 KB)Cosmia NebulaUploaded a work by CHAIR Prof. Yoshua Bengio, Université de Montréal / Mila - Quebec AI Institute EXPERT ADVISORY PANEL Prof. Bronwyn Fox, The Commonwealth Scientific and Industrial Research Organisation (CSIRO) (Australia) André Carlos Ponce de Leon Ferreira de Carvalho, Institute of Mathematics and Computer Sciences, University of São Paulo (Brazil) Dr. Mona Nemer, Chief Science Advisor of Canada (Canada) Raquel Pezoa Rivera, Federico Santa María Technical University (Chile) Dr. Yi Zeng, In...

Global file usage

The following other wikis use this file:

Metadata

This file contains additional information, probably added from the digital camera or scanner used to create or digitize it.

If the file has been modified from its original state, some details may not fully reflect the modified file.

Retrieved from "https://en.wikipedia.org/wiki/File:Performance_of_AI_models_on_various_benchmarks_from_1998_to_2024.png"