"...AI foundation models might be Australia's best chance of reversing the productivity slump - something that's happening in most advanced-economies worldwide. It's a general-purpose technology that's useful in every company and every career."
— CSIRO Director of Digital, National Facilities & Collections, Elanor Huntington
What are AI foundation models?
Artificial intelligence (AI) foundation models are typically trained on vast troves of data using unsupervised learning at scale and support wide ranging downstream tasks, many of which may not have been envisaged by the developers. They're a powerful and novel development in the AI field.
Since the publication of a paper on transformer architecture (Vaswani et al., 2017), there's been an explosion of AI foundation models. Researchers from Germany identified 125 such models with 73 per cent from the US and 15 per cent from China (AKI, 2023). The ChatGPT tool which is an interface for the large language model GPT-4 was released in November 2022. In two months, it had more than 100 million active monthly users.
What do they mean for Australia?
There's a growing body of evidence that foundation and generative AI (GAI) is associated with productivity uplift. However, productivity gains are not assured. When used to augment tasks beyond its known capabilities, GAI was associated with a 19 per cent increase in errors by management consultants (Dell'Acqua et al., 2023).
"But they are rarely made in Australia. The vast majority come from the United States and China. Our industry is becoming increasingly dependent on these models. On one hand this is an opportunity. All these great tools are appearing at our fingertips. But there is also risk. Will these models boost the competitive interest of Australian companies and workers? Will they behave according to our norms, cultural and ethical expectations? Can we be adapting, fine-tuning or building our own AI foundation models to support our own industry and workers? If so, how should we go about doing this? What's Australia's best placement in the global value chain for AI foundation models? There are many more questions. And we are at the beginning of a journey," Elanor Huntington, CSIRO Director of Digital, National Facilities & Collections.
About this report
We prepared this report to inform industry, government and community organisations about the strategic implications of AI foundation models for Australia.
This report helps frame the issues and alert Australian industry, community and government to the opportunities and risks associated with AI foundation models. We want to share what we're seeing and explore the sorts of strategies and policies organisations across the globe are implementing.
Download the report
Download the Artificial Intelligence foundation models report PDF (1 MB)