Generation STEM 
evaluation report 2019–22

Building a bigger, stronger and more diverse 
STEM pipeline in New South Wales

Impact and Evaluation, CSIRO Education and Outreach

March 2023



Generation STEM is managed by CSIRO and made possible 
by the NSW Government’s $25 million endowment to 
the Science and Industry Endowment Fund (SIEF).

Citation

Banks C, O’Brien M, and Miller K (2023) Generation 
STEM evaluation report 2019–22: Generation STEM. 
Canberra, Australia: CSIRO.

Copyright

© Commonwealth Scientific and Industrial Research 
Organisation (CSIRO) 2023. To the extent permitted by 
law, all rights are reserved, and no part of this publication 
covered by copyright may be reproduced or copied in any 
form or by any means except with the written permission 
of CSIRO.

Disclaimer

CSIRO advises that the information contained in this 
publication comprises general statements based on social 
research. Every effort has been made to ensure the accuracy 
of the findings presented in this report. However, given 
its interim nature and other constraints inherent with the 
methodologies, the reader should exercise some caution 
when interpreting the findings in this report. 

CSIRO is committed to providing web-accessible content 
wherever possible. If you are having difficulties with 
accessing this document, please contact csiro.au/contact.

Acknowledgments

The authors would like to thank the large number 
of Generation STEM participants and stakeholders 
who have engaged in this evaluation by completing 
surveys, interviews, and focus groups and/or supplying 
the Commonwealth Scientific and Industrial Research 
Organisation (CSIRO) with education data. Their time and 
effort to has helped inform the insights discussed in this 
report. Thanks to the CSSHREC CSIRO Human Research 
Ethics Committee and the New South Wales (NSW) 
Department of Education research unit for providing 
approvals and useful feedback on the ethical and research 
components of the project.

In particular, the authors would like to acknowledge the 
time and expertise of Myra Singh, a proud Wiradjuri 
woman. Myra is an Academic Coordinator for CSIRO’s Young 
Indigenous Women’s STEM Academy and her assistance 
with the analysis of qualitative data collected from 
participating Aboriginal students and Community members 
for the Deadly in Generation STEM (DiGS) program has 
helped inform the insights discussed. 

The Impact & Evaluation team acknowledges the 
Traditional Owners of the lands with whom this project is 
collaborating, their vibrant living cultures and knowledge 
systems and acknowledge the Countries on which the 
evaluation work took place. We pay our respects to Elders 
past and present and thank all community members who 
provide the leadership to ensure meaningful and effective 
engagement with Aboriginal and Torres Strait Islander 
communities.

CSIRO acknowledges the contributions of former and 
current members of the Impact and Evaluation team 
who supported the program monitoring and evaluation 
methodologies, data collection, and analyses, particularly 
Kate Miller. We also acknowledge the significant 
contributions of former and current members of the 
Generation STEM program team who supported the 
program monitoring and evaluation methodologies, data 
collection, and analyses. We also thank the Consultative 
Council of Generation STEM, who provided oversight and 
strategic advice to the evaluation and data insights projects, 
particularly Ian Oppermann and David Wright.



Contents
Introduction.................................................................................................................................................3

Purpose of this report........................................................................................................................................................4Evaluation approach...........................................................................................................................................................6STEM Community Partnerships Program.....................................................................................9Methodology.......................................................................................................................................................................9Process...............................................................................................................................................................................10Impact and outcomes.......................................................................................................................................................16Discussion..........................................................................................................................................................................25Recommendations............................................................................................................................................................29Generation STEM Links.........................................................................................................................31Methodology.....................................................................................................................................................................31Process...............................................................................................................................................................................31Impact and outcomes.......................................................................................................................................................36Discussion..........................................................................................................................................................................39Recommendations............................................................................................................................................................39Deadly in Generation STEM................................................................................................................41Methodology.....................................................................................................................................................................41Process...............................................................................................................................................................................41Impact and outcomes.......................................................................................................................................................44Discussion..........................................................................................................................................................................45Recommendations............................................................................................................................................................45Conclusion..................................................................................................................................................49References.................................................................................................................................................50Appendices.................................................................................................................................................51Appendix A. Proposed school-level data analysis for STEM CPP..................................................................................51Appendix B. STEM CPP student survey data 2022..........................................................................................................53Appendix C. STEM CPP teacher survey data 2022..........................................................................................................60Appendix D. STEM CPP industry mentor survey data 2022...........................................................................................65Appendix E. Generation STEM Links student survey data 2022....................................................................................70Appendix F. Generation STEM Links supervisor survey data 2022...............................................................................77

Figures

Figure 1. Generation STEM program model...........................................................................................................................3Figure 2. Generation STEM impact pathway..........................................................................................................................4Figure 3. Likelihood that industry contact would recommend STEM CPP to colleagues..................................................11Figure 4. Net percentage change in student interest and knowledge (2021 and 2022)...................................................16Figure 5. Net percentage change in awareness and intentions..........................................................................................18Figure 6. Student responses to ‘STEM subjects are important to my future study and career’ (2022)............................19Figure 7. Student self-reported confidence in skill areas (2022).........................................................................................20Figure 8. STEM CPP program intensity over academic year................................................................................................26Figure 9. STEM subject enrolments in Australia...................................................................................................................27Figure 10. Generation STEM Links student applicant demographics.................................................................................33Figure 11. Student applicant areas of interest versus industry project focus....................................................................34Figure 12. Generation STEM Links priority cohorts.............................................................................................................35Figure 13. Change in student confidence in STEM skills and knowledge..........................................................................37Figure 15. STEM Evidence Tool prototype............................................................................................................................48Figure 16. Breadth versus depth in STEM education programs..........................................................................................49

Tables

Table 1. Summary of evaluation outcomes (does not include implementation outcomes)...............................................2Table 2. Generation STEM interim evaluation overview.......................................................................................................2Table 3. Key evaluation questions...........................................................................................................................................7Table 4. Evaluation rubric for outcome evaluation questions..............................................................................................7Table 5. STEM CPP monitoring and evaluation methods.......................................................................................................9Table 6. STEM CPP implementation.......................................................................................................................................10Table 7. What happened to students at different levels of STEM interest (2022)..............................................................17Table 8. Teacher perceptions of students’ awareness levels...............................................................................................19Table 9. Teachers’ perceptions of students’ skill improvements (2022).............................................................................20Table 10. Net percentage change before and after STEM CPP by gender..........................................................................23Table 11. Cost per participant................................................................................................................................................24Table 12. Effectiveness-cost ratio..........................................................................................................................................24Table 13. Student categories for strategic focus..................................................................................................................26Table 14. Year 12 subject enrolment trends in Australia.....................................................................................................27Table 15. Data collection methods for Generation STEM Links evaluation.......................................................................31Table 16. Generation STEM Links placement data...............................................................................................................33

Generation STEM evaluation report 2019–22


Acronyms

ABN Australian Business Number

CALD Culturally and linguistically diverse

CESE Centre for Education Statistics 
and Evaluation

CEO Chief Executive Officer

CSIRO Commonwealth Scientific and 
Industrial Research Organisation

DiGS Deadly in Generation STEM

HSC High School Certificate

ICSEA Index of Community 
Socio‑educational Advantage

MOU Memorandum of Understanding

NAPLAN National Assessment Program – 
Literacy and Numeracy

NESA New South Wales Education 
Standards Authority

NSW New South Wales

PhD Doctor of Philosophy

RDA Regional Development Association

SES Socio-economic status

SHAE Academy Moree Sports, Health, Arts 
and Education Academy

SIEF Science and Industry Endowment Fund

STEM Science, Technology, Engineering, 
and Mathematics

STEM CPP STEM Community Partnerships Program

TAFE Technical and Further Education

TPL Teacher professional learning

VET Vocational education and training

WIL Work integrated learning



Executive summary

Generation STEM is a New South Wales (NSW) Government 
initiative to attract, support and retain NSW students 
into Science, Technology, Engineering, and Mathematics 
(STEM) education and career paths. Generation STEM is 
made possible through the NSW Government’s $25 million 
endowment to the Science and Industry Endowment Fund 
(SIEF). The initiative launched in 2018 and to date comprises 
four programs: STEM Community Partnerships Program 
(CPP), Deadly in Generation STEM (DiGS), Generation STEM 
Links, and Data Insights. A monitoring and evaluation 
process has been undertaken with the first three programs; 
a description of the progress to date and evaluation 
assessments are presented in this report. 

The key results of the implementation and outcome 
evaluation include:

• strong implementation outcomes were observed, 
including significant scaling up of STEM CPP and 
formation of multiple industry partnerships, and 
successful first year implementation of Generation 
STEM Links and DiGS
• increased teacher capacity, particularly in the area 
of inquiry-based learning for STEM CPP
• increased student interest and knowledge of STEM, 
particularly among those with lower initial interest levels 
and female students
• increased awareness of and intention to pursue STEM 
careers among student participants
• engagement of students in hands-on, real world 
STEM activities
• strengthened knowledge and understanding of culture 
and Indigenous knowledges
• successful targeting of Generation STEM Links to 
students in lower socio-economic status (SES) areas
• increased confidence in skills, including problem solving 
and working in teams.


The cost per participant ratio and effectiveness-cost ratio 
for STEM CPP were reasonable and broadly in line with 
other STEM programs; the cost per participant ratios for 
DiGS was higher in comparison, reflecting the pilot phase 
of the program and a different type of program model 
(i.e., narrower and deeper focus on individual students). 
The cost per placement ratio for Generation STEM Links was 
also relatively higher, reflecting the pilot phase, the subsidy 
component, and that significant work goes into the student 
and industry placement process, placement facilitation, 
monitoring, and interventions as appropriate.

Due to the stage of the initiative, there was insufficient 
evidence at this time to conclusively determine whether 
STEM CPP led to increased numbers of students taking 
STEM or the diversity and number of high potential 
students participating in STEM after Year 10. There was also 
insufficient evidence to determine outcomes for Generation 
STEM Links but initial results indicate the program is set up 
to achieve positive outcomes going forward.

A summary of the assessment of outcomes for STEM CPP 
and Generation STEM Links is presented in Table 1.

The Data Insights project progressed substantially, with 
initial results from the predictive analytics providing useful 
evidence and relatively accurate ability to predict whether 
students in the middle years of high school would pursue 
STEM in Year 12. Evidence X developing a prototype STEM 
Evidence Tool and the establishment of a co-design process.

A number of challenges arose among STEM CPP, 
Generation STEM Links, and DiGS, including successfully 
targeting students disengaged from STEM, program 
fidelity, sustainability of outcomes, and establishing and 
maintaining service delivery partner relationships.

The findings were based on a range of data sources. Table 2 
outlines these sources along with program maturity levels 
and evaluation type for each program. Relevant programs 
were assessed against a set of evaluation questions using 
an evaluation rubric: insufficient evidence, objectives not 
met, met objectives, and exceeded objectives.

Key recommendations for the three programs focused 
on refinement of the program model and targeting 
(STEM CPP), utilising industry mentors more effectively 
(STEM CPP), greater focus on vocational education and 
training (VET) (Generation STEM Links), investigation 
of culturally and linguistically diverse (CALD) students’ 
placement rates (Generation STEM Links), discussion with 
young people on design and delivery of camps/immersion 
days (DiGS), and ensuring emerging outcomes are 
reflected (DiGS).



Table 1. Summary of evaluation outcomes (does not include implementation outcomes)

PROGRAM

QUESTION AREA

ASSESSMENT

STEM CPP

Increased student interest, knowledge and understanding of STEM

Met objectives

Increased student awareness about STEM education and career pathways

Met objectives

Increased student transferable skills

Met objectives

Increased overall number of students participating in STEM education 
after Year 10

Insufficient evidence 
(data available later in 2023)

Increased diversity and number of high potential students participating in 
STEM education after Year 10

Insufficient evidence 
(data available later in 2023)

Cost per participant

$1,665 (2019)

$1,467 (2020)

$805 (2021)

$589 (2022)

Generation 
STEM Links

Increased students’ (a) technical and enterprise skills (b) awareness about 
potential STEM career pathways and (c) commitment to work in STEM

Met objectives

Increased the number and type of tertiary students working in STEM jobs

Insufficient evidence (too early to tell)

Increased the capacity of industry

Insufficient evidence

Cost per placement (July to December 2022)

$8,377





Table 2. Generation STEM interim evaluation overview

GENERATION STEM COMPONENT

DATA SOURCES

PROGRAM MATURITY LEVEL

EVALUATION TYPE

STEM Community Partnerships 
Program

• Participant and stakeholder 
surveys and interviews
• Administrative data
• Case studies
• Program staff feedback
• Cost/outcome analysis


Mature program (from 2019)

Implementation and 
outcome evaluation

Generation STEM Links

• Administrative data
• Participant surveys and interviews
• Program team feedback


New program, 
implemented from 2022

Implementation 
and early outcomes 
evaluation 

Deadly in Generation STEM

• Administrative data
• Student yarning session
• Community stakeholder focus group
• Program team feedback


New program, 
implemented from 2022

Implementation 
evaluation

Data insights

• Administrative data
• Program staff feedback


In development, commenced in 
late 2021

Progress update







Introduction

Generation STEM is a NSW Government initiative to 
attract, support and retain NSW students into STEM and 
school, into further education and into employment. 
The NSW Government has made a ten year, $25 million 
endowment to the Science and Industry Endowment 
Fund (SIEF) to establish Generation STEM. Generation 
STEM takes a location-based approach with the program 
being delivered in regions in NSW where there is a 
current and future need for STEM skills. Generation STEM 
empowers young people with the relevant STEM skills to 
pursue a STEM career. The ambition is to build a strong 
community of STEM‑capable citizens to fuel local industry. 
CSIRO, under the steerage of SIEF and in consultation with 
the NSW Government, is working to design and deliver 
Generation STEM.

Generation STEM comprises several programs and activities 
organised in three focus areas (see Figure 1). This report 
is organised around individual programs rather than the 
broader focus areas, primarily because each program is at a 
different level of maturity and it is difficult to make overall 
assessments of these general focus areas at this stage of the 
initiative. The final evaluation report in 2027 aims to have 
sufficient data in each focus area, due to greater program 
maturity and the progression of data requests.

STEM CPP creates partnerships between local schools 
and industry, with the goal of highlighting local STEM 
careers and opportunities and providing avenues for 
students to develop their STEM skills in an engaging and 
rewarding way. STEM inquiry-based projects are developed 
by students with the guidance of industry mentors and 
teachers, where students also receive opportunities to 
build their STEM skills and awareness through a range 
of activities, including site visits, work experience, 
masterclasses and VET programs.

Deadly in Generation STEM aims to increase engagement 
and retention of Aboriginal and/or Torres Strait Islander 
students in STEM educational pathways, STEM employment, 
and/or future education through culture and on Country. 
Currently run on Dharawal Country (Illawarra-Shoalhaven 
region) and on Kamilaroi Country (Moree), the program 
brings together Aboriginal and/or Torres Strait Islander 
students with CSIRO project officers, STEM mentors, 
cultural mentors, and community stakeholders to deliver 
workshops and run hands-on activities through culture 
and on Country.

Generation STEM Links provides internships to help 
tertiary students gain workplace skills and transition into 
STEM jobs after graduation. It also aims to build a pool of 
STEM-capable professionals for the future of NSW STEM 
industries through its partnerships. Generation STEM Links 
is a hands-on internship program that pairs NSW students 
in their penultimate to final year of study in STEM degrees 
and qualifications with industry to allow both sides to learn, 
connect, and innovate. The program also seeks to provide 
additional recruitment pathways for businesses that may 
not have considered a tertiary student intern.

Data Insights comprises two projects aiming to increase 
the evidence base and its use in improving STEM education 
outcomes. The first is a data analytics project to understand 
the factors (barriers and facilitating) leading to STEM 
education outcomes. The second is called Evidence X, 
which seeks to build an evidence platform/set of tools that 
will assist Generation STEM, and the sector more broadly, 
design and evaluate the success of programs. It is planned 
that the analytics project will eventually form part of 
Evidence X.

Figure 1. Generation STEM program model

Attract, support and retain 
NSW students in STEM

Focus area 1

Building the 
STEM pipeline

Increasing the overall 
quantity and diversity of 
students entering STEM 
educational pathways.

Focus area 2

Transition to 
employment

Supporting STEM 
tertiary students 
(from university and 
VET) in the transition 
to STEM employment.

Increasing diversity

Generation STEM aims to increase participation of 
students from groups underrepresented in the STEM 
pipeline: Aboriginal and/or Torres Strait Islander 
peoples, women, people from low socio-economic 
areas and people from regional and remote areas.

Location priorities

Generation STEM is established in areas where it can 
have the greatest impact – growing communities with 
growing STEM industries. Taking a community-driven 
approach to ensure high impact and relevance.



Purpose of this report

Monitoring and evaluating Generation STEM’s 
implementation and impact is an integral 
aspect of the Generation STEM strategy. 
CSIRO’s approach to planning, monitoring and 
evaluating impact is based on the concept that 
there must be a clear pathway leading from the 
impact back to the activities of the program. 
The Generation STEM Impact Pathway (Figure 
2) is used to articulate these relationships. 
STEM CPP, Generation STEM Links, and DiGS 
have individual impact pathways that elucidate 
program‑specific impact chains. This impact 
model approach enables evidence-based changes 
to be made, if needed, providing the opportunity 
for continuous improvement. Impact pathway 
planning and monitoring provides a means to 
identify and articulate key project activities and 
objectives; as well as the opportunity to monitor 
progress towards desired goals, and implement 
required changes as identified through this 
tracking process to enhance the capacity to 
achieve key project objectives.

Monitoring and evaluation are being undertaken 
over the life of Generation STEM and will include:

• monitoring, evaluating, and reporting on the 
initiative and its activities
• making recommendations for future action.


The overall performance of Generation STEM will 
be assessed against the intended implementation 
goals and program outcomes across its 10 year 
span. Data are being collected throughout the 
10 years in order to measure progress towards 
achieving these outcomes. The monitoring and 
evaluation framework described above will be 
used to assess the degree to which outcomes 
have been achieved. 

A key challenge of impact evaluation is 
clarifying the intended impacts, which includes 
consideration of the attribution and the 
alternative reality without the Generation STEM 
intervention (counterfactual). It is important to 
acknowledge that collecting evidence of the 
state of the counterfactual alongside monitoring 
will be important and is subject to the availability 
of data. It is acknowledged that it will be difficult 
to measure and to determine attribution of each 
outcome to Generation STEM’s activities. 

Figure 2. Generation STEM impact pathway

Impact statement: Generation STEM is a 10-year initiative aimed at 
attracting, supporting, and retaining NSW students in STEM at school, 
and into further education and employment in NSW.

Participation: Who we need to reach across the various parts of the pathway?

SIEF, Generation 
STEM team, CEdO, 
delivery partner(s), 
tertiary institutions 
and employers

Generation STEM 
team, CEdO, 
delivery partner(s), 
tertiary institutions, 
employers

Generation STEM 
team, delivery 
partner(s), tertiary 
institutions, employers 
and students

INPUTS

What we invest

ACTIVITIES

What we do

OUTPUTS

Our deliverables

Funding – CSIRO/
Industry

Staff – Generation 
STEM team, 
CEdO M&E team, 
delivery partner(s) 
and industry

Resources – 
participating 
businesses, tertiary 
institution and CSIRO 
(materials, equipment 
and facilities etc.)

CSIRO corporate 
services (e.g., finance, 
HR, comms etc.)

CSIRO research, 
reputation, 
national footprint

Governance – NSW 
Department of 
Education, SIEF, 
Consultative Council

CSIRO contract 
management

Facilitate relationships 
between CSIRO, 
industry, and 
education institutions

Recruit industry, 
education intuitions 
and students (with 
a specific location 
focus**)

Engage government 
and other community 
stakeholders

Develop resources 
and educational 
material

Develop and collect 
data for monitoring 
and evaluation

Communications 
and marketing

STEM Community 
Partnerships Program 
(STEM CPP)

STEM Links

Deadly in Generation 
STEM (DIGS)

Data Insights

Program monitoring 
and evaluation, 
analysis, interim and 
final reports

Events 
communications and 
marketing collateral, 
(including for targeted 
equity groups)

 Generation STEM programs

 Students/young people

 Industry

 Education sector

 Policy and STEM program providers

 Direct influence on outcomes

 Indirect influence on outcomes

 Minimal influence on outcomes



Generation STEM team, CEdO, delivery partner(s), tertiary institutions, employers and students

OUTCOMES

The uptake, adoption or consumption of our work

BENEFITS

Eco, environ, soc

Shorter term (direct) outcomes

Longer term (indirect) outcomes

Participants* have 
increased awareness 
of STEM, and its 
associated education 
pathways and careers

Participants have 
greater exposure to 
STEM experiences 
relevant to their 
interests and/or 
everyday life

Participants improve 
their enterprise and/or 
technical STEM skills

NSW STEM industry 
develops stronger 
connections with local 
education institutions 
and community

Educators are more 
aware of the different 
approaches/activities 
available to help engage 
students in STEM

Generation STEM 
continues to improve 
based on evidence

Sustainability and 
legacy outcomes TBC

Participants have 
increased confidence 
about their abilities 
in STEM

Participants have an 
increased interest in, and 
attitude towards STEM

Participants have an 
increased aspiration to 
study and work in STEM

Participants have 
an increased work 
readiness skills

NSW STEM industry 
have increased 
opportunities to 
collaborate with local 
education institutions 
and students

Educators have greater 
confidence and skills to 
use different approaches/
activities to help engage 
students in STEM

Educators and 
policy makers better 
understand what factors 
influence young peoples’ 
STEM education and 
career pathways

Sustainability and 
legacy outcomes TBC

More participants 
pursue STEM 
education pathways

More participants have 
the appropriate STEM 
skills for the NSW 
STEM jobs in demand

NSW STEM industry 
better align student 
engagement 
opportunities with 
current and future STEM 
employment demands

NSW STEM industry 
better align student 
engagement 
opportunities to attract 
underrepresented 
groups into STEM

School and tertiary 
systems are better 
able to promote and 
support STEM

Educators and policy 
makers design more 
effective STEM 
education interventions 
to increase STEM uptake 
among young people

Sustainability and 
legacy outcomes TBC

More people are STEM 
literate in their tertiary 
education and/or 
careers

More people are 
employed in NSW-
based STEM jobs**

More NSW STEM 
workers remain 
employed in STEM jobs

NSW STEM industry 
are able to hire 
more people with 
the technical skills 
in demand

Economic – increased 
size of the STEM-skill 
workforce in NSW

Social – increased 
awareness of STEM 
and its place in 
NSW society

*Generation STEM participants 
include underrepresented groups 
in STEM, in particular: females, 
Aboriginal and/or Torres Strait Islander 
people, people in regional and/or 
remote areas, and people from low 
socioeconomic backgrounds.

**A focus is placed on STEM jobs 
associated with Special Activation 
Precincts, other regions with 
upcoming industry development and 
infrastructure investments, or industries 
where there is a shortage of STEM 
professionals in NSW.



Evaluation approach

This evaluation is based on a set of key evaluation 
questions (see Table 3), impact pathways for each program 
and Generation STEM overall, and an evaluation rubric. 
Each program has been assessed against several evaluation 
questions using all available evidence to date, with a focus 
on more recent data for the outcome evaluation questions. 
As each program is at different stages of maturity, different 
types of evaluation questions have been covered in this 
report. For example, for STEM CPP, the most mature of 
the Generation STEM programs, both implementation and 
outcome questions were examined; while for Deadly in 
Generation STEM, which was implemented in 2022, 
only implementation questions were assessed.

Assessments against the outcome evaluation questions 
employed an evaluation rubric (see Table 4) and were 
made using data from a multi-method approach, including 
self‑report surveys, interviews, focus groups, and 
administrative data. Where no pre‑determined metric or 
goals were identified for the program, the Impact and 
Evaluation team based the assessment on a reasonable 
standard based on existing research and evaluation 
literature. For example, increasing student interest in STEM 
was based partially on whether students’ self-reported 
increases in STEM interest were statistically significant 
and with moderate effect sizes1


.

A process of triangulation aimed to position the findings 
within the broader context of the Generation STEM strategy 
to help guide the discussion of the findings and to inform 
key implications and recommendations. Triangulation 
of the results involved:

• scanning key strategic and operational documents 
to identify relevant priorities; and
• positioning the findings of the evaluation against 
the priorities of the program’s funder.


1 Effect sizes are a measure of how meaningful the relationship between two variables or the difference between groups is. Cohen (1988) classified effect sizes 
as small (0.2), medium (0.5), and large (>0.8).



Table 3. Key evaluation questions

PROGRAM(S)

Process – implementation and monitoring

STEM CPP

Generation STEM Links

DiGS

How was the program implemented?

Which aspects are working well? Which aspects can be improved? How?

Outcome – impact and outcomes 

STEM CPP

To what extent has STEM CPP increased students’: (a) interest, knowledge and understanding of STEM; 
(b) awareness about STEM education and career pathways; and (c) transferrable skills?

To what extent has STEM CPP increased the (a) overall number of students participating in STEM education 
after Year 10; and (b) diversity and number of high-potential students participating in STEM education 
after Year 10?

Generation STEM Links

To what extent has Generation STEM Links increased students’: (a) technical and enterprise skills; 
(b) awareness about potential STEM career pathways; and (c) commitment to work in STEM?

To what extent has Generation STEM Links increased the number and type of tertiary students working 
in STEM jobs?

To what extent has Generation STEM Links increased the capacity of industry?

What unexpected impacts has the program had?

Economic – investment and outputs 

STEM CPP

Generation STEM Links

DiGS

To what extent is the relationship between inputs and outputs been cost-effective and to 
expected standards?





Table 4. Evaluation rubric for outcome evaluation questions

OUTCOME EVALUATION RUBRIC 

Insufficient evidence

Did not meet objective

Met objective

Exceeded objective

There was not enough evidence 
to make an assessment for this 
question at the current time.

The program did not meet 
the objectives set out in the 
Generation STEM strategy 
based on all available evidence.

The program met the objectives 
set out in the Generation 
STEM strategy based on all 
available evidence.

The program substantially 
exceeded the objectives set out 
in the Generation STEM strategy 
based on all available evidence.





Note: Objectives may be quantifiable and based on pre-determined targets or goals or be based on overall qualitative assessments and comparisons with 
the existing evaluation and research literature.



STEM Community 
Partnerships Program

Methodology

The evaluation of the STEM CPP program is based on the 
data collection methods outlined in Table 5. The CSIRO 
Human and Interdisciplinary Research Ethics Committee 
provided approval to undertake all monitoring and 
evaluation activities. The NSW Department of Education, 
relevant Catholic Dioceses, and individual independent 
schools also provided permissions to conduct these 
activities in schools.

The suite of self‑report surveys, interviews, case studies, 
and program administrative data provides a rich and 
diverse range of data to assess program implementation 
and outcomes. The process of obtaining data indicating 
whether STEM subject enrolments have increased at STEM 
CPP schools compared to other schools is still in progress.

Table 5. STEM CPP monitoring and evaluation methods

METHODOLOGY

SAMPLE

NOTES

Student self-report online 
general survey

2021 = 123

2022 = 220

Total = 343

Some surveys were distributed prior to 2021 but the number of responses were 
small and unrepresentative of the program model due to COVID-19 disruptions.

Post event surveys

2021 = 165

2022 = 176

Total = 346

Post-event surveys were distributed after key events and activities, such as 
career expos and STEM Taster experiences.

Teacher survey

2021 = 7

2022 = 44

Total = 52

Teachers involved in the program were offered an opportunity to respond to a 
voluntary, online survey seeking their feedback and perceptions of the program. 
In 2022, surveys with less than three question responses were excluded.

Teacher interview

2021 = 3

2022 = 5

Total = 8

A small set of teachers (who indicated willingness to be interviewed in the 
teacher survey).

Industry stakeholder survey

2021 = 4

2022 = 32

Total = 36

In 2022, surveys with less than three question responses were excluded.

Industry stakeholder interview

2021 = 2

2022 = 6

Total = 8

A small set of industry stakeholders were interviewed.

Council (local government) survey

2021 = 70

2022 = 9

Total = 79

In 2022, surveys with less than three question responses were excluded.

Case studies

2021 = 1

2022 = 2

Total = 3

Case studies were undertaken of participating schools.

Program administrative data

2019 to 
2022

Details of locations, numbers of classes, events, etc. have been collected 
and collated by the Generation STEM program team.

School-level data*

Not 
available

CSIRO and the NSW Government (Centre for Educational Statistics and 
Evaluation (CESE)) and NSW Education Standards Authority (NESA)) have been 
in discussions on signing an MOU to obtain school-level data to assess the 
impact of STEM CPP on school-level STEM subject selection. The proposed 
statistical methodology is outlined in Appendix A.





Note: Report appendices include: STEM CPP student (Appendix B), teacher (Appendix C), and industry mentor (Appendix D) survey data, and Generation 
STEM Links student (Appendix E) and teacher (Appendix F) survey data.



Process

How is STEM CPP being implemented? 

Assessments of implementation effectiveness are based 
on program administrative data, feedback from the 
Generation STEM team, analysis of participant interviews 
and surveys, and observation by the Impact and Evaluation 
team. After a relatively slow roll-out and significant 
disruption due to COVID-19 up to and including 2020, the 
implementation and scaling up of the STEM CPP program 
has been comparatively swift in 2021 and 2022. As can 
be seen in Table 6, the pace of the roll out has increased 
substantially from 2021 to 2022 in terms of the numbers of 
students, schools, STEM professionals, industry partners, 
etc. The program was launched in 2018 but the gradual 
implementation and COIVD-19 disruptions mean the 
program model can only be considered to be implemented 
fully since 2021, which is the reasoning for focusing on the 
data from those two years (2021 and 2022).

The program team has also expanded and been 
restructured to better meet the needs of the program; 
for example, by establishing an Industry Engagement 
position to focus on building and maintaining industry 
relationships and collaborations. STEM CPP also saw 
numerous operational activities and improvements that 
demonstrated the maturity of the program; for example, 
revising the initiative website, improved communications 
and promotions, stakeholder engagement and feedback 
opportunities, and refinement of the program model and 
elements (e.g., STEM Taster Program, Careers Expo, Gender 
Inclusive Classroom professional development).

Table 6. STEM CPP implementation

ELEMENT

2021

2022

PER CENT CHANGE

Student participants

1,122

2,231

+98.8

Schools

47

77

+63.8

School retention rate (proportion of the schools 
from previous year participating in current year)

87%

91%

+4.0

STEM professionals

29

60

+106.9

Virtual work experience opportunities

27

44

+63.0

Local Government Areas

7

12

+71.4

Industry partners (active)

14

61

+335.7





Delivery partnership pilot

In 2022 the STEM CPP program focused on expanding 
the program beyond Western Sydney, with a successful 
expansion into the Central Coast region through a 
non‑financial collaborative partnership with a key 
community stakeholder. Exploratory discussions were 
undertaken in parallel in Wagga Wagga and Central West, 
with a formal delivery partnership being established in 
Central West. This partnership was the first subcontractor 
agreement established for delivery of STEM CPP.

The delivery partner in Central West (a Regional 
Development Association or RDA) was identified organically 
through conversations between the CSIRO program 
leadership and other local stakeholders. It was identified 
that the delivery partner had strong relationships and 
expertise both in industry engagement and STEM education 
programs locally. Recognising the time involved in building 
trust and credibility with local stakeholders in an already 
congested space there was a strong rationale for partnering 
with an existing local organisation with established 
relationships rather CSIRO starting fresh with this process.

Further, the STEM CPP model is designed so that Council 
is engaged as a central collaborator. Partnering with an 
RDA meant that they could assume both roles – program 
delivery and central point of collaboration across the 
region. Although this partnership was established 
with a strong rationale through existing connections, 
factors beyond the control of CSIRO meant that the 
partnership was not as fruitful as hoped in expanding the 
program into the Central West region (2 schools were 
actively involved in 2022). The contractual partnership 
arrangement was formally ended by CSIRO in late 2022. 



Despite this, there remains a positive relationship 
between CSIRO and the local delivery partner reflecting 
the strength of the relationship building to support 
effective implementation of the partnership approach. 
Insights amongst the program team have identified a 
number of areas where the approach could be strengthened 
in order to support greater success with future attempts to 
establish successful delivery partnerships. These include:

• recognising the challenges of recruiting appropriately 
skilled Project Officers in regional areas and exploring 
different ways to attract candidates
• test approaches to embedding local Project Officers 
into the CSIRO STEM CPP team to build capability and a 
greater sense of shared understanding and ownership 
of the program i.e., inducting the Project Officer 
through CSIRO
• strengthen contractual arrangements to support 
accountability towards achieving partnership objectives
• recognise the exploratory nature of this activity 
i.e., delivery partner model hasn’t been trialled in CSIRO 
Education and Outreach before, other programs run in 
regional areas successfully; however, these have CSIRO 
project officers on the ground
• consider developing an approach to service delivery 
contracts that more formally and methodically assesses 
the potential fit of the model, provider, context, 
and timing
• consider the visibility of CSIRO within a delivery 
partnership and how this can be maximised to leverage 
off trust and recognition within the region.


How is STEM CPP facilitating NSW STEM 
industry connections and collaboration 
opportunities with local education 
institutions and students?

Over 80 per cent of industry stakeholders reported that 
they were likely or very likely to recommend involvement 
in the STEM CPP program to industry colleagues (Figure 3). 
Amongst the survey sample (n = 32), the majority of 
respondents reported increased engagement with 
schools (73 per cent), with fewer indicating increased 
engagement between community members and industry 
(33 per cent respectively).

Six interviews were undertaken in 2022 with industry 
stakeholders, including some with successful mentoring 
partnerships with schools, some who attended career 
expos, and some who had been matched with schools for a 
mentoring partnership without success. One representative 
lauded the information provided about involvement in 
the program:

So, it gave us all the options on how we could 
participate. It talked about mentoring, it talked 
about the expo, it talked about the showcase, 
it talked about site tours, which we have scheduled 
as well during this year but then we had to 
reschedule due to COVID, so it’s been penciled in for 
next year already. So, it did give us a lot of options 
on how we could contribute to the program.

Industry representative

Figure 3. Likelihood that industry contact would recommend 
STEM CPP to colleagues



The survey and interview results indicate that where 
industry was successfully engaged with students, the 
experience was overwhelmingly positive for the business 
and for the professionals themselves. For example:

I went to the most recent display out at 
Campbelltown when they had their showcase, and 
I was just so inspired by these young kids who – as 
I was walking past their displays, they were almost 
tugging at me, saying, “Come and have a look at 
this. Come and have a look at this. Come and have 
a look at this.” And honestly, you feel so good. 
You love it. You know, I just – and they pull you 
out the front and you take a few photographs with 
the local dignitaries, and that’s fine, but the real 
stars are the kids, right? And you’re in a room full 
of – probably literally over 100 in that room, at 
the Campbelltown Arts Centre, and so, so proud 
of them, right?

Industry mentor

A new element of STEM CPP in 2022 was a partnership 
that was formed with TAFE NSW to deliver the STEM 
Taster Program to increase awareness about VET pathways 
into STEM careers. A total of 40 students took part in 
the program across two TAFE campuses. The experience 
offered hands-on, practical activities and interactions with 
STEM industry professionals, including in IT, manufacturing, 
aviation, and health. Students who responded to a 
voluntary post-activity survey (n = 16) indicated high levels 
of satisfaction, with 75 per cent either ‘very’ or ‘extremely’ 
satisfied with the program. The practical aspects of the 
experience were particularly useful as demonstrated 
by students:

[The best part was] all the practical 
work (e.g., making things, exercises, 
seeing/using equipment, etc.). 

Student

The subjects were interesting and the opportunities 
to look and do things that we normally wouldn’t 
be able to do in school.

Student

Student survey responses indicated increased levels of 
interest and positive intention around STEM as well, 
including 100 per cent of respondents saying they were 
more interested in learning about STEM (n = 12).

Which aspects are working well? 
Which aspects can be improved? How?

This section outlines a number of areas of consideration. 
The structure comprises areas for consideration that 
encompass both aspects that are working well and aspects 
that could be improved within those topics areas. 

Roll-out, reach, and diversity of schools

STEM CPP has been effective in reaching a large number 
of schools and students in 2021 and 2022. The program 
is succeeding in engaging a set of schools (via school 
leadership), many individual teachers, and industry and 
local government partners to deliver a relatively cohesive 
program. The geographic reach of the program is also 
expanding beyond the initial Western Sydney focus area 
into Central Coast (7 schools) and Central West (2 schools). 
The process of reflection and refinement is also working 
well, with numerous program improvements made based 
on double-loop learning processes. For example, based 
on recommendations on a 2021 Insights Report by the 
Impact and Evaluation team, recommended actions are 
well-advanced, including those related to increasing gender 
diversity, mentor matching processes, and the effectiveness 
of certain program elements (e.g., Masterclass). 
The diversity of schools involved is also encouraging, with 
43 per cent of active participating schools in 2022 having an 
Index of Community Socio-Educational Advantage (ICSEA) 
score below 1000 (34 out of 78).

As a voluntary program, the program has been delivered to 
schools, and a small set of teachers2


 within those schools 
who volunteer, or are sometimes asked or told, to take part. 
Many of these teachers are interested in and often already 
engaged in STEM, although a minority are told by their 
Head of Department or Principal to take part. This presents 
an issue that the most interested and/or engaged teachers 
in a school may be volunteering to be a part of the 
program, meaning that it’s possible that some teachers 
and students more in need of support may be missing out.

2 A small proportion of participants were careers advisors.



Building capability amongst teachers

Building the capability of teachers to effectively teach STEM 
and to identify and support students who are interested 
in STEM can have a ‘multiplier’ effect and is the principal 
way the program can achieve sustainable outcomes. 
This is because STEM-capable teachers will continue to 
support multiple cohorts of students for many years after 
involvement in the program. It’s also important because 
of the high proportion of STEM teachers who are teaching 
‘out of field’ in Australia, with one report estimating a 
19 per cent probability that mathematics would be taught 
by an out of field teacher (Shah, Richardson & Watt, 2020). 
Given the inquiry-project is the central feature of STEM CPP, 
it is important to consider how difficult teachers found it to 
embed the inquiry-based learning project into their regular 
teaching practice. In 2022 (n=36), most surveyed teachers 
found it ‘easy’ (47 per cent) or ‘very easy’ (14 per cent), but 
a significant minority found it ‘difficult’ (39 per cent) to 
integrate. This reflects the diversity of teacher participants 
in terms of levels of experience, confidence, and familiarity 
with inquiry and STEM-related pedagogies.

More broadly, teachers were asked the degree to which 
STEM CPP events and activities had on their teaching 
of STEM. The proportion of respondents who felt each 
element had made ‘a significant’3


 difference were:

• 42 per cent: Delivering the inquiry-based learning 
project (n=36)
• 42 per cent: Attending the teacher professional 
learning workshop (n=33)
• 35 per cent: Engaging with industry mentors (n = 26)
• 30 per cent: Attending a teacher networking event (n=20)


It is encouraging that many teachers found the inquiry 
learning project and other elements of the program 
impactful in their teaching. Comparatively, the teacher 
networking event had less of a perceived impact and was 
attended by fewer respondents.

Sustainability and fostering STEM culture 
within schools

The sustainability of STEM CPP has multiple components, 
including schools continuing with the program while 
its running, and schools building a STEM culture that 
can effectively assist students pursue STEM beyond 
participation in the program. STEM school culture is 
defined as the overall STEM beliefs, values, practices, 
and resources in a school (White, Marshall & Alston, 2019).

Future Generation STEM evaluation work will include 
explicit assessment of STEM school culture to understand 
whether programs are shifting culture, which would be a 
significant sustainability goal. Within the program lifespan, 
the majority of teachers (in 2022) felt that their schools 
would likely continue to participate in STEM CPP (n=40): 
53 per cent (very likely), 35 per cent (likely), 10 per cent 
(possible), and 3 per cent (unlikely).

Another feature of the sustainability of programs is the 
degree to which teacher participants share their knowledge 
and skills with their peers, and also the degree to which 
they encourage peers to get involved in the program. 
A ‘willingness to recommend’ question was included in 
the teacher survey to assess this propensity. Most teachers 
would recommend STEM CPP to other teachers (n=40): 
48 per cent (very likely), 43 per cent (likely), and 10 per cent 
(possible). This demonstrates that the program is appealing 
to teachers and is providing perceived value.

The majority of teachers who were interviewed as part of 
the evaluation indicated that their schools did not have 
an active industry mentor relationship; and therefore, 
these relationships were not able to be explored in depth 
at this time. However, the teacher survey data and a few 
of the teacher interviews indicated that when industry 
connections were facilitated, there was a perceived 
significant impact on students’ interest in their projects and 
STEM careers more broadly, indicating that this relationship 
may be valued sufficiently to continue beyond the program.

3 Based on a three point scale: Has made a significant difference, has made some difference, hasn’t really made a difference



Key factors influencing program fidelity

A number of factors affected the program’s fidelity, that is, 
the degree to which the program was delivered as intended. 
Some issues were largely operational; for example, some 
schools being unable to book excursions of most interest 
due to demand (e.g., Western Sydney Airport). Others were 
more central issues. For example, based on teacher 
interviews, it was apparent that early career teachers faced 
more barriers in maintaining the fidelity of the program as 
they were less likely to have the capacity and confidence to:

• liaise with industry
• incorporate inquiry-based learning into the classroom
• support students to think creatively and explore ideas.


Although this is not a surprising finding, it may behove 
the program team to consider differentiation of teacher 
support; for example, offering more or different assistance 
to early career teachers. The new content specialist position 
on the Generation STEM program team is an opportunity to 
address this issue.

Finally, the way the program has been implemented means 
that schools can select from a menu of different activities 
they want to be involved in (e.g., mentor relationship, 
site visit(s), work experience, virtual work experience, 
showcase event, etc.), with only the inquiry project being 
a mandatory aspect of the program. Although this degree 
of program flexibility has been highlighted as a positive 
by some participants, it can also lead to large differences 
in how the program operates and the likelihood of 
achieving student, teacher, school, and industry outcomes. 
For example, an area of consideration is the difference in 
how lower ICSEA schools implement the program compared 
to higher ICSEA schools that have more resources and 
capacity. There is anecdotal evidence that some higher 
ICSEA schools may have the capacity to fully integrate STEM 
CPP into their curriculum and schedules, while some lower 
ICSEA schools may only expose students to specific aspects 
of the program, with much less integration and connection 
to the curriculum and therefore potentially less impact. 
The implication of this is the program team may need to 
differentiate their approaches and support depending on 
the capacity of the school.

Leveraging peer-based learning opportunities

Some teachers reported that opportunities for students to 
learn from other schools was valuable (e.g., to see what 
ideas other students came up with inspired them; meeting 
students from other schools; challenging stereotypes of 
kids who like STEM). Within schools, teachers reported that 
students learnt a lot from working together and focusing on 
what different strengths people could bring to a problem 
to create the solution, often resulting in working with 
students other than their friends in order to be successful 
with their inquiry project.



Many teachers in the interviews reported that networking 
activities (in addition to teacher professional learning 
(TPL)) were valuable, although the survey data indicated 
lower relative valuing of these activities compared to 
other activities (see ). Related to this, for those teachers 
struggling with incorporating inquiry-based learning 
into classroom, TPL could be strengthened to spotlight 
schools that have had success and what that has involved. 
Another option is involving more experienced teachers in 
a ‘lessons learnt / what I’d do differently’ type of setting 
or buddying them up with less experienced teachers. 
Resources could be developed so that teachers have 
something to refer to in order to support translation of 
learnings from the TPL. Another consideration is building 
on existing peer support that is happening, particularly 
for the schools already established in delivering STEM CPP 
within their school (teachers mentoring and engaging new 
teachers into the program).

Maximising industry interest for student benefit

The current approach to mentorship is to focus on 
supporting and building capacity amongst teachers. 
Many teachers report that when they have had a strong 
partnership with an industry professional, this has 
been due to the response of students’ face-to-face 
opportunities to engage with real-life STEM professionals. 
These professionals have done some of the heavy 
lifting with students in terms of supporting the creative 
thinking, ideas, initial research phase, and in some 
schools, have come back to work with student groups 
directly. Teachers reported benefiting from other teachers 
(see above re peer‑based learning re what works) more than 
they reported learning from / being supported by mentors. 
Research has shown that mentors can provide a range of 
emotional and technical support to students, which can 
raise their self-efficacy, skills, confidence, and ability to 
navigate STEM pathways (Aitkens et al., 2020; Jin, 2021; 
Millar, Hobbs, Speldewinde & van Driel, 2022).

Direct relationships with mentors, not mediated by a 
teacher for example, have shown to be effective; in 
addition, having mentors and mentees with shared 
beliefs, values and backgrounds works well.4


 One teacher 
highlighted the benefit of the STEM professional expertise:

What’s really valuable about our partnership with…
[CSIRO] has been the expertise…and the real-world 
settings. The kids were very excited to be shown 
through a laboratory, for example. The students 
loved being taken to a real site and getting that. 
We do inquiry-based learning anyhow but having 
the access to expertise, and localised expertise as 
well, that’s what makes it really full of potential 
for the students.

Teacher

4 Research also shows that other ‘what works’ elements to STEM mentoring: Mentors being involved at particularly important times, for example in years of 
subject selection in high school; Mentors that provide technical/professional (e.g., career resources, research) and socioemotional/psychological support 
(e.g., warm/friendly, good listeners); Fostering student confidence and self-efficacy; Facilitating the valuing of STEM among students; Having a focus on 
the growth and development of students; Mentor-mentee dynamics that are personal and reciprocal in nature (mutual exchange of energy and support); 
and Making STEM seem relatable and possible (especially for girls), for example by providing counter-narratives to stereotypes.



Impact and outcomes

Question 1a: To what extent has STEM CPP increased students’ interest, 
knowledge and understanding of STEM?

Student and teacher participants were asked about any 
changes in relation to interest in learning about STEM 
in 2021 and 2022 (post-retrospective survey). As seen in 
Figure 4, in most areas of student interest and knowledge, 
there was a net percentage increase (the proportion 
of students who indicated a positive change minus the 
proportion who indicated a negative change). Overall, 
the 2021 results were stronger compared to 2022. Given 
the program delivery was largely similar between years, 
this may be due to a post-lockdown ‘bump’, with students 
conflating satisfaction with more normalcy in schooling 
with participation in the program; alternately, it could be 
due to differences in the samples. However, only some 
areas of interest and knowledge demonstrated statistically 
significant changes for 2021 (Expect to do well in STEM 
subjects, STEM important for society, STEM useful in 
everyday life) and for 2022 (STEM useful in everyday life).

Figure 4. Net percentage change in student interest and knowledge (2021 and 2022)

Surveyed teachers also indicated that they felt their 
students were more interested in STEM as a result of 
participating in STEM CPP. Specifically, in 2022 teachers 
identified different aspects of STEM CPP that they felt made 
their students ‘a bit’ or ‘much more’ interested in STEM:

• 91 per cent: Attending the showcase event (n=35)
• 89 per cent: Visiting a local STEM industry or 
worksite (n=19)
• 89 per cent: Completing the inquiry-based learning 
project (n=38)
• 82 per cent: Interacting with industry mentors (n=33)
• 81 per cent: Completing the STEM work experience (n=16)




Overall, according to students and teachers, STEM CPP 
is contributing to many students’ increased interest, 
knowledge, and understanding of STEM in many areas, 
particularly practical and contextual ones, such as 
illuminating to students how STEM is useful in everyday life 
and how STEM is important for society. One explanation 
for why STEM CPP is not resulting in changes across all 
areas is that students already had high levels of interest, 
knowledge, and understanding, and therefore the 
program was unlikely to improve those levels overall. 
This ceiling effect was exemplified in the 2022 student 
survey: 31 per cent of students were already at the 
highest self-reported level of interest in STEM before 
participating in STEM CPP; therefore, there was no potential 
for positive change in the post-retrospective survey. 
Table 6 demonstrates this issue: the biggest increases in 
interest were seen among students with lower (but not the 
lowest) levels of pre‑existing interest in STEM. The students 
with the lowest and highest levels of interest before STEM 
CPP were least likely to shift their perception. This result 
also relates to the issue of program targeting, as there are 
minimal benefits to investing in students who are already 
highly interested in STEM and likely to continue that 
pathway regardless of the program. Therefore, the program 
objectives are most strongly being met by the students with 
lower levels of pre‑existing interest (and other attributes).

Question 1a assessment: To what extent has STEM CPP increased students’ interest, knowledge and 
understanding of STEM?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective





Table 7. What happened to students at different levels of STEM interest (2022)

PERCENTAGE AFTER PARTICIPATING IN STEM CPP

LEVEL OF INTEREST BEFORE PARTICIPATING IN STEM CPP

INCREASED
INTEREST

SAME
INTEREST

DECREASED
INTEREST

0 – lowest interest (n = 9)

0%

100%

n/a

1 (n = 16)

50%

38%

13%

2 (n = 25)

56%

24%

20%

3 (n = 39)

23%

49%

28%

4 (n = 56)

27%

41%

32%

5 – highest interest (n=65)

n/a

89%

11%

All levels (n=210)

22%

58%

20%





Note: changes of 50% or greater highlighted in green.



Question 1b: To what extent has STEM CPP increased students’ awareness about STEM 
education and career pathways?

Participating students were asked several questions 
about their awareness of STEM education and career 
pathways (see Figure 5), and their intentions for following 
those pathways. In both the 2021 and 2022 cohorts, there 
was a substantial net positive change before and after 
participating in the program, particularly for ‘awareness of 
the type of jobs I could work in’ with a 41 per cent positive 
increase in 2021, and 27 per cent in 2022. There were 
smaller, but still positive, net percentage changes in the 
area of ‘would like to have a job in STEM’: 11 per cent in 
2021 and 12 per cent in 2022. There may have been a ceiling 
effect in this area as well, with 24 per cent of surveyed 
students in 2022 already at the highest self‑reported level 
of intention to want a STEM career. There was a small 
to no net change in the question about ‘STEM subjects 
are important to my future study and career’: 0 per cent 
change in 2021 and an 8 per cent net increase in 2022. 
In 2022, 31 per cent of students already thought STEM 
subjects were critically important to their future study 
and career, compared to only 4 per cent who thought it 
wasn’t important.

As can be seen in Figure 6, the majority of students 
already thought STEM subjects were important before 
participating in STEM CPP. The distribution of responses 
is very negatively skewed, reflecting the pre-existing high 
levels of engagement of students in STEM. This distribution 
demonstrates one of the challenges the program has 
in changing perceptions and intentions (compared to a 
more ‘normal’ distribution that might be expected in a 
random sample of students), but also to the issues with the 
program targeting approach and implementation model. 
That is, all the students in a particular class receive a similar 
‘intervention’ regardless of their pre-existing attitudes, 
although it is accepted that students would engage to 
different levels in the common activities.

Teachers were asked about their perceptions of 
improvements, if any, that STEM CPP activities had on 
their students’ awareness of STEM education and career 
pathways. As can be seen in Table 8, a large majority 
of teachers in 2022 felt that students’ awareness had 
increased, including around a third that thought it had 
increased ‘significantly.’

Figure 5. Net percentage change in awareness and intentions



Figure 6. Student responses to ‘STEM subjects are important to my future study and career’ (2022)

Table 8. Teacher perceptions of students’ awareness levels

AWARENESS OF STEM EDUCATION PATHWAYS

AWARENESS OF POTENTIAL STEM CAREERS

No improvement

3%

0%

Some improvement

30%

31%

Moderate improvement

35%

33%

Significant improvement

33%

36%

Total

100%

100%





Question 1b assessment: To what extent has STEM CPP increased students’ awareness about STEM education 
and career pathways?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







Question 1c: To what extent has STEM CPP increased students’ transferrable skills?

The extent to which student participants increased their 
transferrable skills was based on self-report surveys and 
also teacher perceptions. The majority of teachers indicated 
(in 2022) that students had improved skills as a result 
of participating in STEM CPP, with the most ‘significant 
improvements’ in problem solving (54 per cent) and 
communicating ideas (49 per cent) (see Table 9).

Students were asked whether their confidence in different 
skill areas had increased as a result of participating in 
STEM CPP. As can be seen in Figure 7, most students in 
2022 reported increases in ‘working in a team with others’ 
(91 per cent), problem solving abilities (89 per cent), 
and ‘communicating ideas to others’ (87 per cent). 
Male and female students reported similar levels 
of confidence improvements.

Table 9. Teachers’ perceptions of students’ skill improvements (2022)

WORKING IN 
A TEAM WITH 
THEIR PEERS

COMMUNICATING THEIR 
IDEAS EFFECTIVELY 
TO OTHERS

USING CRITICAL THINKING 
TO REASON AND 
DRAW CONCLUSIONS

CREATIVELY THINKING 
ABOUT WAYS TO 
SOLVE PROBLEMS

No improvement

0%

3%

3%

3%

Some improvement

16%

15%

18%

10%

Moderate improvement

41%

33%

42%

33%

Significant improvement

43%

49%

37%

54%

Total

100%

100%

100%

100%





Figure 7. Student self-reported confidence in skill areas (2022)



A STEM teacher involved in STEM CPP exemplified many 
comments by other teachers on the transferable ‘soft’ 
skills that students were able to develop as a result of 
the program:

They really learned; they were able to meet new 
people...they learned how to communicate with 
each other. They had to learn to collaborate with 
each other rather than just sticking to their friends. 
Like each time they did a different group activity, 
they had to mix and collaborate with other students 
even if they didn’t want to. They had to learn to 
collaborate with each other, how to build teamwork 
skills, how to solve problems and challenges and 
be able to work out how to…build different parts 
of models to save time.

STEM teacher/coordinator

Beyond skills, one coordinator in a low SES school also 
identified how the virtual work experience component of 
STEM CPP may have built a sense of STEM identity through 
interactions with the scientists:

But the boys that took part in the earlier – like, 
the stage 1 phase...they’re not knowing yet what 
direction they’re taking and to be able to do that 
virtual work experience and also have relationship – 
obviously virtual relationship but they really liked 
the scientists who were guiding them through the 
experience. They liked conducting that more adult 
conversation. Didn’t have those same assessment 
stressors or anything. It was a really validating 
experience for them to have that.

Coordinator low-SES, high-CALD school

Question 1c assessment: To what extent has STEM CPP increased students’ transferrable skills?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







Question 2a: To what extent has STEM CPP increased the overall number of students 
participating in STEM education after Year 10?

STEM CPP works with students in Years 9 and 10 who 
are already in STEM-related classes (47 per cent in core 
STEM subject classes, and 53 per cent in elective or 
extra‑curricular classes). In addition, the program works 
through teachers at the class level rather than with 
individual students, making the tracking of students post 
involvement in the program challenging. However, a 
memorandum of understanding (MOU) is in development 
that seeks to obtain data from all schools in New South 
Wales that will allow a school-level analysis of STEM 
subject selection (see Appendix A). This analysis will need 
to exclude Year 10 students (because they have already 
made their subject selections before most of STEM CPP 
activities have taken place) and therefore there is a 2 to 
3 year lag between participation (in Year 9) and subject 
selection data (in Years 11 and 12). There is also a lag time 
to collect and provide the data to CSIRO for analysis. 
For example, the Year 11 subject selection data for STEM 
CPP Year 9 participants in 2021 will likely not be available 
until mid‑2024.

What can be said about increases in participation in STEM 
after year 10 currently is based on the theory of change 
of the program, which posits that increased interest, 
knowledge, and skills will lead to increased probability 
of engagement and retention in STEM. Teachers were 
asked in a 2022 online survey ‘Overall, what impact has 
STEM CPP had on your students’ likelihood of studying 
STEM after Year 10?’ Of the 47 responses, 79 per cent 
said ‘moderate impact’, 14 per cent said ‘slight impact’, 
5 per cent said ‘no impact’, and 2 per cent said ‘significant 
impact’. This indicates that teachers felt that STEM CPP was 
likely to be a factor, but possibly not the deciding factor, 
of whether students continued on to STEM. This aligns 
with contemporary research that highlights a myriad of 
contributing influences on decisions to enter and/or stay 
in STEM education pathways, including individual, family, 
school, and cultural/society level factors (many of which 
are outside the control of STEM education providers) 
(Murphy et al., 2019; Edwards et al., 2023). 

The analysis presented in previous questions gives 
a partial picture to address this question, including 
evidence of overall increases in confidence, interest, 
and intention; however, it will not be possible to assess 
this question robustly until the school-level data from 
the NSW Department of Education is made available 
for the evaluation.

Question 2a assessment: To what extent has STEM CPP increased students’ transferrable skills?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







Question 2b: To what extent has STEM CPP increased the diversity and number 
of high‑potential students participating in STEM education after Year 10?

At this point in the life of STEM CPP, there is insufficient 
evidence to assess whether the diversity and number 
of ‘high potential’ students has increased after Year 10. 
There has been some debate within the program about 
what ‘high potential’ means and, given its ambiguous 
meaning and rationale for why it was included as a goal 
for the program, this aspect of the question has been put 
on hold. To clarify, some staff believe that all students 
have potential for STEM (but perhaps not an interest in 
it), and that delineating between ‘high potential’ and 
‘low potential’ is not productive. However, there is a 
rationale for considering under-represented cohorts as 
‘high potential’, such as young women and students from 
regional/remote and/or low socio-economic areas, because 
these cohorts are of course equally capable of succeeding 
in STEM education and career pathways but have not been 
represented at equivalent levels to their counterparts for a 
range of reasons.

Because it is difficult to track students after participation 
in the program and even more difficult to attribute the 
ongoing STEM engagement of former participants to the 
program compared to a range of other influences, the 
best proxy at this point is to assess student self-reported 
changes in attitudes and intentions that could reasonably 
lead to future STEM engagement. Table 10 shows the net 
changes in STEM interest and intention to pursue a STEM 
career among male/female students (for 2022). CALD status 
was not analysed at this time but may be presented in 
future reports. As can been seen, STEM CPP has resulted in 
positive self-reported net changes among young women 
in interest (2021 and 2022) and intention to work in STEM 
(2022). These changes were greater compared to male 
students (except for intention to have a STEM job in 2021).

Table 10. Net percentage change before and after STEM CPP by gender

2021 FEMALES

2021 MALES

2022 FEMALES

2022 MALES

Interest in learning about STEM

+24%

+14%

+13%

-9%

Wanting to have a job working in STEM

-2%

+19%

+19%

+7%





Question 2b assessment: To what extent has STEM CPP increased the diversity and number of high-potential 
students participating in STEM education after Year 10?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







Economic

To what extent has the relationship 
between inputs and outputs been 
cost‑effective and to expected standards?

STEM CPP has taken several years to implement into a 
fully formed program, but it is now useful to consider 
cost to outputs. There are several ways to assess the cost 
to output/outcome ratios. The first is a simple cost per 
participant ratio. As can be seen in Table 11, in 2019 the 
total cost of delivering STEM CPP (labour and operational 
service delivery, not including management and evaluation 
costs) divided by the number of student participants 
was $1,665. This reduced each year to a current ratio of 
$589 in 2022, a reduction of 64 per cent. This is indicative 
of the program gaining economies of scale, that is, 
delivering to more schools with similar investment in staff 
and operational costs. It is likely not useful to compare 
these costs per participant figures with other programs 
because of the many differences in context, organisational 
accounting, and program type; instead, it is more useful to 
monitor them annually as the program grows in order to 
understand trends and potential economies of scale.

It is much more challenging to assess cost-outcome ratios 
for STEM CPP, primarily because there is an absence 
of longer term outcome data available at this time. 
However, to provide some baselines and initial indications 
of cost versus outcome, a calculation has been made 
on the effect size of increases in self-reported student 
intention to have a job in STEM and interest in STEM 
(see Table 12 – for 2021 and 2022 only, when outcome data 
is available). If the central focus of STEM CPP is to increase 
the number of students in STEM education and career 
pathways, then increases in self‑reported interest and 
intention is a reasonable proxy for longer term outcomes 
measured directly. It is assumed that the sample of survey 
respondents is representative of the overall population of 
program participants.

The measure of cost effectiveness is an ‘effectiveness-cost 
ratio’, which is calculated by dividing the effect size for 
different outcomes by the annual cost per participant. 
Some comparative data exists using this measure for raising 
student achievement in mathematics (Yeh, 2010) as a broad 
comparison. Of the 25 initiatives reviewed in this study, 
the median annual cost per participant was $1,006 United 
States dollars, and the median cost-effectiveness ratio was 
0.000083. Although STEM CPP does not have a focus on 
raising achievement, it is clear that the effectiveness-cost 
ratio is relatively strong in the areas of raising interest 
(0.000409 and 0.000044) and intention to have a job in 
STEM (0.000315 and 0.000170). These metrics will continue 
to be monitored as the program continues to be delivered.

Table 11. Cost per participant

CALENDAR YEAR

TOTAL SERVICE DELIVERY 
COST (ACTUAL)

STUDENT 
PARTICIPANTS

ANNUAL COST 
PER PARTICIPANT

2019

$487,953

293

$1,665

2020

$366,788

250

$1,467

2021

$902,903

1,122

$805

2022

$1,315,019

2,231

$589





Note: The service delivery cost is split between Generation STEM funding, and CSIRO contributions in the form of ‘overheads.’ From 2019 to 2022, 
the Generation STEM proportion of the funding was 79 per cent, 70 per cent, 66 per cent, and 66 per cent, respectively.

Table 12. Effectiveness-cost ratio

CALENDAR YEAR

COST PER PARTICIPANT

OUTCOME AREA AND EFFECT SIZE

EFFECTIVENESS-COST RATIO

2021

$805

Interest 0.3296

Intention 0.2539

0.000409

0.000315

2022

$589

Interest 0.0257

Intention 0.1003

0.000044

0.000170





Note: effect sizes calculated using Wilcoxon Signed Rank Test.



Discussion

Industry partnerships and real-world STEM

Research consistently finds that STEM education and 
learning is effectively experienced in a hands-on approach 
that is student-centric, facilitates inquiry-based learning, 
and provides clear links between STEM skills and knowledge 
and everyday environments (Morris et al., 2021). There is 
also emphasis on the enrichment to STEM education for 
students if industry and STEM mentors played a more active 
role in their learning as a way of bridging the gap between 
classroom STEM subjects and awareness of real-world STEM 
application (Morris et al., 2021). However, there is also 
the acknowledgement that bringing STEM into everyday 
life or in partnership with industry is not within reach of 
many teachers or schools, primarily due to lack of time or 
resources, lack of interaction with enrichment programs 
or lack of awareness from non‑specialist or non-science 
teachers teaching STEM subjects, especially in pre-year 10 
classes (Hackling et al., 2014).

Areas for future consideration

STEM CPP has been successfully designed and implemented 
in a range of schools and has been productive in partnering 
and understanding the needs of local councils and 
industries. The program delivery is efficient and there 
is evidence of positive student, teacher, and industry 
outcomes, although the key question of whether the 
program is resulting in increased numbers of students 
in STEM remains largely unanswered at this point. At 
approximately mid-way through Generation STEM funding, 
there are a number of issues the evaluation has highlighted 
that warrant further investigation and/or decisions for the 
remainder of the program funding in order to maximise 
benefits and program effectiveness, including:

• refining the program model to target and focus on 
students who are not interested in STEM but could be
• differentiated teacher training, student 
activities, and industry partnerships to focus on 
under‑represented students
• refine program goals to include retention of students in 
STEM in addition to increasing the number of students 
in STEM
• prepare teachers to place value on the inquiry-based 
skills rather than the particular industry/discipline to 
maximise industry connection opportunities
• provide case studies of successful industry partnerships 
to teachers and potential industry partners and highlight 
key factors contributing to this
• consider the proximity between mentor locations and 
schools they are matched with. Distance is a major 
barrier to face-to-face incursions
• undertake greater assessment of teacher interest 
and desire for industry mentorship to better gauge 
partnership readiness
• focus resources on matching teachers with greatest 
interest/investment in achieving success from 
industry partnerships
• support existing industry relationships to leverage 
peer referral opportunities within their companies/
organisations to engage potential mentors or school 
presenters with different STEM skills and experience
• develop an opt in program offering for schools to elect 
for a STEM industry professional to visit the classroom 
(as opposed to schools visiting sites or an established 
industry mentorship) to enable face-to-face engagement 
between industry and students.


Differentiation and targeting

The current STEM CPP program model results in many 
interested schools and interested/engaged teachers 
signing up. In addition, many of the students that are 
taking part are pre-existing STEM students with high 
levels of interest in STEM already. To increase the number 
of students in STEM, programs must be clear about their 
focus. STEM CPP’s program model is operationally efficient 
but may not be maximising effectiveness in targeting 
resources to those students most in need or most amenable 
to support. Table 13 outlines a conceptual grouping of 
four cohorts of students that are either currently engaged 
(groups 1 and 2) or not engaged in STEM (groups 3 and 4).

To achieve the goal of increasing the number of students 
in STEM, only the targeting of group 3 (not engaged in 
STEM but potential to be engaged) will result in an overall 
increase in STEM students in New South Wales compared 
to the counterfactual of doing nothing. However, STEM 
CPP spreads considerable effort and investment (but not 
exclusively) to groups 1 and 2. Although it is important 
to retain the students in group 2, this will not lead to 
overall increases; it will only prevent decreases. The STEM 
CPP could make adjustments to better target effort and 
resources through differentiation of resources and training 
and to take a more outcome, rather than operational, 
focus to scaling up. That is, focus on reaching and 
supporting group 2 and 3 students, and less on group 1. 
Another consideration may be to consider explicitly adding 
‘retain’ students in STEM along with ‘increase’ students in 
STEM as a program goal in the impact pathway(s), although 
this will be even more challenging to measure accurately.



Table 13. Student categories for strategic focus

GROUP

POTENTIAL STRATEGY

PROGRAM 
INVESTMENT 
LEVEL

WILL PROGRAM INCREASE 
NUMBER OF STUDENTS 
ON STEM PATHWAYS?

1. Students engaged in STEM, 
and likely to remain engaged

Continue to offer general opportunities 
and promote access to existing events, 
activities, resources

Low

No

2. Students engaged in STEM, 
but at (preventable) risk of not 
remaining in STEM5




Provide targeted support and opportunities 
and increase capabilities, motivation, and 
continue to build STEM identity

Medium

No, but important to retain 
in STEM (prevent decrease)

3. Students not engaged in STEM, 
but potential to be engaged

Provide targeted engagement activities 
and support and begin to build and verify 
STEM identity

High

Yes

4. Students not engaged in STEM, with 
little to no interest in pursuing STEM

Focus on raising overall levels of STEM 
awareness and skills that will be useful in any 
career in the future (i.e., 21st Century skills)

Low

No





Year levels targeted

Another program design feature that may be worthy of 
consideration is that Year 10 students’ inclusion in the 
program means that they have already made subject 
selections for Year 11 before the majority of the STEM 
CPP program activities have occurred in that academic 
school year. It should be noted that this is more prevalent 
in schools in their first year of involvement in STEM CPP, 
as existing schools have more activities available in terms 
1 and 2. The general level of program intensity over the 
academic year is shown in Figure 8, noting that some 
major activities such as the Careers Expo do occur in 
term 2. If the goal of STEM CPP is to influence STEM subject 
selection, then Year 10 may be somewhat too late to change 
attitudes and behaviours. In this regard, the program team 
is already considering broadening the program inclusion 
criteria to include Year 8 students.

STEM CPP is also a ‘year-based’ program; that is, it works 
with cohorts of students for one year. It is acknowledged 
that some students may take part in multiple years but 
this only occurs through happenstance rather than an 
intentional multi-year strategy. This year-based focus makes 
it difficult to track students from an evaluation perspective, 
but also difficult to maintain strengths and work with 
individual students over time. Building and maintaining 
a student’s STEM identity takes time and continuous 
effort, and consideration may need to be given on how 
to maintain support for participants, especially students 
from under-represented cohorts, over multiple years.

Figure 8. STEM CPP program intensity over academic year

To reverse the trend in declining STEM subject enrolments 
(see Figure 10) and reach recent high proportions 
(e.g., 2012 in Mathematics), New South Wales would need 
an additional 2,886 students to take Mathematics, 1,499 to 
take Sciences, and 4,329 to take Technologies (based on 
2021 figures) (see Table 14). Given there are around 
500 secondary schools in New South Wales, this equates to 
around 3 (Sciences) to 9 (Technologies) students per school. 
The STEM CPP program may consider an approach that 
targets a smaller number of students, but provides more 
in-depth, longer-lasting support. In addition, the team 
could work with schools to set targets for STEM enrolments. 

5 This excludes individuals that make an informed decision not to pursue STEM based on personal choice, rather than negative experiences, barriers, lack of 
information or opportunity, etc.



Related to this, the program team have observed that 
there may be a shift in focus of students, with Year 9 
students more focused on potential STEM jobs, and Year 10 
students more focused on subject selection. This could 
have implications for tailoring STEM CPP more to different 
ages or year levels to capitalise on these variations in 
interest areas.

Scale of problem and tailoring program

To reverse the trend in declining high school STEM 
subject enrolments (see Figure 9) and reach recent high 
proportions (e.g., 2012 in Mathematics), New South 
Wales would need an additional 2,886 students to 
take Mathematics, 1,499 to take Sciences, and 4,329 to 
take Technologies (based on 2021 figures) (see Table 
14). Given there are around 500 secondary schools in 
New South Wales, this equates to around 3 (Sciences) 
to 9 (Technologies) students per school. Although 
these numbers would not necessarily meet the growing 
demand for STEM, it does demonstrate the realistic scale 
of addressing the problem at the level of individual high 
schools. The STEM CPP program may consider an approach 
that targets a smaller number of students, but provides 
more in-depth, longer-lasting support. In addition, the team 
could work with schools to set targets for STEM enrolments. 
Related to this, the program team have observed that 
there may be a shift in focus of students, with Year 9 
students more focused on potential STEM jobs, and Year 10 
students more focused on subject selection. This could 
have implications for tailoring STEM CPP more to different 
ages or year levels to capitalise on these variations in 
interest areas.

Figure 9. STEM subject enrolments in Australia

Table 14. Year 12 subject enrolment trends in Australia

MATHEMATICS

SCIENCES

TECHNOLOGIES

Current rate (2021 in Australia)

67.7%

50.0%

28.0%

Recent high rate (Australia)

72.9%

(2012)

52.7%

(2017)

35.8%

(2012)

Total Year 12 students (2021) (Australia)

179,023

179,023

179,023

Number of additional Year 12 students needed 
to achieve recent high rate (Australia)

9,309

4,834

13,964

New South Wales share (approximately 31%)

2,886

1,499

4,329

Average number of students per high school (~500 high schools)

6

3

9







Factors that STEM CPP can influence

Regardless of student interest in STEM and intention to 
pursue STEM, there are a number of individual, school level, 
and systemic factors, such as subject weighting for ATAR 
success and parental background, that heavily influence 
student subject selection (Jeffries, Curtis & Conner, 2020; 
Palmer, Burke & Aubusson, 2017). The program’s inclusion of 
industry mentors, inquiry-based learning, showcase events, 
etc. covers a range of these factors. However, there may 
be an opportunity to further focus the program to address 
other factors. For example, there may be an opportunity 
to further inform and influence what students study at the 
tertiary level. In addition, in line with the principle of ‘you 
can’t be what you can’t see’ (Gladstone & Cimpian, 2021) 
it may be useful to examine the gender of industry mentors. 
The program team have reported that STEM CPP is actively 
encouraging representation from women during site visits 
and discussion panels and are actively seeking more female 
STEM mentors.

In both 2021 and 2022, student survey data indicated 
strongly that girls’ confidence was lower than their 
counterparts in almost all STEM subjects, and that the 
program was able to lift these self-reported confidence 
levels; therefore, further refining the program to 
focus on this strength-based area may be useful. 
For example, having more materials and training on 
how to engage young women in STEM. Of course, 
with greater targeting and focus, there may be fewer 
economies of scale and less ability to grow the program 
geographically and to a greater number of schools. 
However, the ability to genuinely and sustainably build 
individual students’ interest, engagement, and identity, 
and bridging the gap between self-perception and actual 
ability, requires substantial focus and targeting to achieve 
longer-term outcomes. One strength of the program that 
has been particularly encouraging has been changing 
students’ perceptions of the relevance of STEM to 
everyday life, potentially as a result of demonstrating the 
practical applications of STEM above and beyond theory, 
and also the focus on the inquiry projects. In relation to 
sustainability, the current model is heavily reliant on the 
interest and engagement levels of individual teachers; it 
may be effective to consider more school-wide activities 
that encourage a change in culture at the school level.

Finally, research shows that interest in subject matters 
begins in early childhood education, where student 
perception of knowledge-building and learning experiences 
begin to form in the classroom (Stephenson et al., 
2021). This is also where the gap in STEM interest and 
confidence begins to develop between girls and boys, 
where a mixture of unconscious gender biases, culture, 
the social environment and inadequate school initiatives 
cause a lag in the participation and engagement of girls 
in STEM-related activities (Speldewinde & Campbell, 2021; 
Stephenson et al., 2021). This gap, when left undisturbed, 
continues to widen through primary school and high 
school, especially with the absence of targeted STEM 
programs designed exclusively for girls (McKinnon, 2020). 
STEM CPP may consider earlier engagement with students 
in lower high school or primary school.



Recommendations

The recommendations for STEM CPP comprise several 
primary ones that focus on the refinement of the program 
model and a number of secondary ones that focus on more 
operational areas.

Primary recommendations

• Continue to build on the commendable implementation 
and delivery of STEM CPP in terms of resource 
development, relationship building, and ability to raise 
interest and intention levels among student participants.
• Consider refining the program model to target and focus 
on students who are not interested in STEM but could 
be. This could involve fewer students being involved, 
potentially offering more individualised rather than 
class-based programming.
• Consider offering more differentiated teacher training, 
student activities, and industry partnerships to focus on 
the ‘group 3’ students.


Secondary recommendations

• Refine the program goals to include ‘retention’ of 
students in STEM in addition to ‘increasing’ the number 
of students in STEM.
• Explore ways to utilise potential industry mentors 
when the first match doesn’t work out with the school. 
These mentors could be redirected into other activities 
to maximise positive engagement.
• Prepare teachers to place value on the inquiry-based 
skills rather than the particular industry/discipline to 
maximise industry connection opportunities.
• Provide case studies of successful industry partnerships 
to teachers and potential industry partners and highlight 
key factors contributing to this.
• Consider the proximity between mentor locations and 
schools they are matched with. Distance is a major 
barrier to face-to-face incursions.
• Undertake greater assessment of teacher interest 
and desire for industry mentorship to better gauge 
partnership readiness.
• Focus resources on matching teachers with greatest 
interest/investment in achieving success from 
industry partnerships.
• Support existing industry relationships to leverage 
peer referral opportunities within their companies/
organisations to engage potential mentors or school 
presenters with different STEM skills and experience.
• Develop an opt in program offering for schools to elect 
for a STEM industry professional to visit the classroom 
(as opposed to schools visiting sites or an established 
industry mentorship) to enable face-to-face engagement 
between industry and students.




Credit: Charles Sturt University



Generation STEM Links

Methodology

The evaluation of the Generation STEM Links program is 
based on a mixed methods approach outlined in Table 15. 
Data were collected on a rolling basis throughout 2022 
from student and industry participants via a brief voluntary 
online survey. This was distributed by email from the 
Impact and Evaluation team once both parties had met the 
exit requirements from the program. If indicated in their 
survey response, a voluntary semi-structured interview 
may be undertaken by the Impact and Evaluation team 
with students and industry supervisors. The CSSHREC 
provided approval to undertake all monitoring and 
evaluation activities.

Process

How was the program implemented? 

The Generation STEM Links program model involves 
attracting industry partners and matching them with a 
pool of students. The students are recruited via tertiary 
education providers in NSW, with an assessment of their 
suitability being undertaken, and offering industry partners 
a short-list of potential interns that have been sourced 
and vetted by the program. Throughout the program, 
industry partners and students are also provided support 
and mentoring, including check-in meetings, resolving any 
issues that arise among students and placement employers, 
and ensuring students are experiencing expected benefits 
from the program.

Requirements of the program include:

• 200 hours of internship must be completed
• students must be formally employed by the business 
and paid at least the minimum wage of $25 per hour 
from the business (Generation STEM Links provides a 
$2,500 placement grant to the business on completion 
of the internship)
• the business must provide proof of payment 
to CSIRO before the grant subsidy is paid.


Table 15. Data collection methods for Generation STEM Links evaluation

METHOD

SAMPLE

NOTES (IF APPLICABLE)

Student self-report online general survey

11

8 students indicated their willingness to be contacted for an interview.

1 student indicated that they did not wish to participate in a destination 
survey in future years.

Only two female students are included.

An additional survey was partially completed 
(i.e., 11 complete surveys, 1 partially completed)

Student semi-structured interview

2

Industry supervisor survey

10

Industry supervisor interview

3

Case studies

3*

*Case studies were developed using the data collected in the respective 
student and industry interviews 

Program administrative data

n/a







Program design

Program staff report that during the design phase of 
Generation STEM Links, it was identified that the main 
bottleneck for the program would be finding and 
onboarding industry partners to host interns. In addition, 
a key learning was that Generation STEM Links would need 
to be differentiated from unpaid ‘Work Integrated Learning 
(WIL)’ programs primarily delivered by universities and 
VET providers such as TAFE, which often provide limited 
matching of students and businesses and often offered 
little to no support throughout the placement. As a result, 
Generation STEM Links went with a fully facilitated model 
in which the program supports the company to refine its 
‘on the job’ project brief, advertise to potential students, 
review applications, conduct screening interviews and 
present a shortlist to the business. In the Generation STEM 
Links program model, the business chooses the student and 
presents the employment contract. Once the student has 
been confirmed, there is a CSIRO-led onboarding meeting, 
a mid-term follow-up, and close-out meetings. There are 
also interventions where necessary when assistance is 
requested by business or student.

The facilitated model is more resource intensive than 
WIL programs but can give businesses assurances that the 
process will minimise risk and that the significant resources 
it puts into the hosting will result in a well-matched, 
engaged student. The program team feel that this has 
become a strong value proposition when attracting new 
businesses and has had the follow-on benefit that many 
businesses have returned for subsequent internships. 
Repeat industry internships are more efficiently delivered 
as they are well prepared for specific projects through to 
onboarding and the internships themselves.

Program guideline changes

At the outset of the program implementation in 2021, 
only STEM businesses in Generation STEM’s location 
priorities were eligible to participate in the program. 
During the year, these guidelines were streamlined 
and simplified, including expanding the geographical 
scope of the program to organisations anywhere in 
NSW. In addition, the eligibility criteria were simplified 
to cover any organisation with an Australian Business 
Number (ABN) and a STEM need within their business. 
To ensure suitability of the placement opportunities for 
students, a further criterion was included that at least one 
employee must have the relevant STEM skills, experience, 
and qualifications to supervise a student placement. 
Program staff report that the geographical expansion of 
the program to all of NSW (i.e., eliminating complicated 
location boundary criteria) has had the greatest impact 
in increasing interest from businesses.



Applications and coverage

Students enter the program through two ways: through 
an application in response to an advertised opportunity 
(n = 117 in 2022) or through an application to be kept on file 
for future opportunities (n = 154 in 2022). As the program 
grew and more industry partners have been onboarded, 
the ‘on file’ students have been sourced for potential 
matches to new opportunities. Figure 10 shows the 
attributes of Generation STEM Links applicants by gender. 
Male applicants outnumbered female applicants by a factor 
of two. Substantial proportions of applicants were from 
lower socio-economic areas and were culturally diverse. 
Only one in ten applicants were provided a placement, 
demonstrating how competitive the program has been 
in its first year. It should be noted that less than half of 
the applicants applied for advertised industry positions, 
with the remainder lodging expressions of interest for 
future opportunities.

Table 16 shows the number of placement locations, 
industry sectors, educational institutions and placement 
organisations involved in Generation STEM Links, 
demonstrating a diversity of geography and sector. 
In addition, most common industry sector for student 
placements was advanced manufacturing (n=12) and 
professional and financial services (n=4).

Figure 10. Generation STEM Links student applicant demographics

Note: Total eligible students applied n = 271. For each demographic category, the bar represents the percentage of all applicants in that category. 
For example, 43 per cent of applicants were culturally diverse, 12 per cent of applicants were female and culturally diverse, and 30 per cent of applicants 
were male and culturally diverse. The ‘given placement’ bars represent the proportion of applicants in that category that were placed: 11 per cent of all 
applicants were placed, 14 per cent of all female applicants were placed, and 10 per cent of all male applicants were placed.

Table 16. Generation STEM Links placement data

STAKEHOLDERS AND LOCATIONS

PLACEMENT LOCATIONS (7)

INDUSTRY SECTORS (7)

EDUCATIONAL INSTITUTIONS (9)

Central Sydney

Western Sydney

Illawarra-Shoalhaven

Riverina-Murray

Hunter

New England

Southern Sydney

Advanced manufacturing (12 placements)

Professional and financial services 
(4 placements)

Agribusiness and food (3 placements)

Mining (2 placements)

Healthcare (1 placement)

Sustainability (1 placement)

Other (7 placements)

Charles Sturt University

Macquarie University

TAFE Ultimo

University of Newcastle

University of New England

University of New South Wales

University of Technology Sydney

University of Wollongong

Western Sydney University







Which aspects are working well? 
Which aspects can be improved? How?

Approximately 54 per cent (n=141) of the student 
applications received for the Generation STEM Links 
program involved students studying engineering 
(see Figure 11). One third of the students involved in the 
placements in 2022 (n=10) are studying engineering. 
Engineering degrees require an internship as compulsory 
learning; therefore; these students are actively seeking 
internship programs to fulfil their academic requirements 
for graduation. Students report that of the internship 
programs on offer, Generation STEM Links has a strong 
value proposition offering the greatest opportunity to gain 
hands-on experience and develop transferable skills. In 
addition, engineering overall has a culture of WIL, which 
makes it easier to find industry partners looking to take on 
student interns.

Figure 11. Student applicant areas of interest versus industry project focus

Note: When applying, students are asked to indicate three preferred interest areas. Data represented is inclusive of all three areas (student identified interest 
areas n = 695; Industry project identified areas n=138).

The other ones I looked at I was more so just 
applying for trying to get the internship. 
Wasn’t really anything that was direct. I was, 
“If I get it, I get it.” Might not enjoy it but it covers 
the hours to meet the requirement. But this is 
definitely [the] ideal case.”

Student

I put mechanical or mechatronics because that’s the 
two majors that I do. But other than that, I wasn’t 
too fussed on exactly what it was. I was just looking 
for an opportunity. I had other internships lined up 
as well, but I just found that with this one, especially 
with the backing of CSIRO and it being in the 
industry that was more interesting to me compared 
to the other one, the work just seems better, and 
I could learn more from the senior engineer here.

Student



Offering a paid internship opportunity is a significant 
incentivising component of the Generation STEM Links 
model, as many internships elsewhere in the sector are 
unpaid. The majority of applicants (79 per cent) lived 
in a lower socio-economic area (according to postcode 
indexation) which suggests that this program is addressing 
a known barrier for participation in voluntary internships. 

Although the program is attracting students from culturally 
and linguistically diverse backgrounds (43 per cent, 
n = 114), this is not being reflected as proportionately 
in the placements that have been undertaken to date 
(see Figure 12 below). Of the 29 placements, only 17 per cent 
(n = 5) of students identified as culturally and linguistically 
diverse. However, due to the two different applicant 
streams (specific applications and general pool), the 
proportions were higher when looking at students who 
specifically applied for the 29 placements: 117 students 
applied, 47 (or 40 per cent) were culturally diverse, and 
those 47 students only applied for 13 of the placements. 
Looking at it in this way, CALD students were self-selecting 
out of many placement opportunities. Nevertheless, this 
general issue suggests an opportunity to identify why 
these differences are reflected and apply continuous 
improvement processes to address this. One potential 
explanation is that many CALD student applicants may 
have been located in Western Sydney where there were 
fewer placement opportunities. In addition, the program 
team reports that there were some challenges identifying 
CALD students and clarifying the number of applicants, 
specifically that some CALD students are not self-identifying 
in their applications.

Figure 12. Generation STEM Links priority cohorts

Note: Total eligible applications n = 271; Total placements n =29. The data in the chart represents the proportion of applicants and placements 
per demographic category. For example, 29 per cent of all applicants were female, and 13 per cent of the students that receive placements 
were female.



Impact and outcomes

To what extent has Generation STEM Links increased students’ (a) technical and enterprise 
skills (b) awareness about potential STEM career pathways and (c) commitment to work 
in STEM?

This program is in its infancy and there are only early 
indications and small amounts of data to assess the 
achievement of outcomes at this point. However, there 
are insights reflected in the data to suggest the effect the 
program may have as it continues to scale as planned. 

Although the sample size is too small to conduct tests 
of statistical significance, students reported a net 
positive percentage change in all domains (see Figure 14) 
measuring the impact the work placement had on 
students’ skills, understanding, and interest in pursuing 
STEM jobs. The greatest change was observed for 
students’ self‑reported core work-readiness skills 
(e.g., communication skills, working in a team, 
planning etc.) 

Self-report qualitative data from students indicated that 
skills, awareness, and commitment to STEM increased 
as a result of participating in the program. One student 
highlighted how their career options had widened due to 
the work placement:

This placement has broadened my perceived career 
options. I am passionate about the environment 
and originally, I wanted to use my degree in 
the renewable energy sector. Having worked at 
[company] for six months, I am now considering 
my career options within the food and beverage 
industry. I would also be very interested in 
transitioning into the design of machines like 
the ones I have been working on in the plant.

Student

Another student reported increased confidence in both 
their skills and career choice:

Now after the placement I felt much more 
confidence in what I do and my choice of choosing 
courses. I was once a Biochemistry major who 
decided to make a shift into the tech industry. So, 
through being exposed to the various technical 
challenges in the business, solving them and 
creating new products, it’s given me much more 
confidence knowing that what I do serve the 
company’s customer pool in a meaningful manner.

Student

One student expressed the benefit of seeing what a STEM 
job is like in real life:

It has shown me the type of work an engineer 
would do. At the same time, it has shown me the 
possibility for career development in the future into 
work other than technical design work.

Student

While another student emphasised the application of skills 
as being particularly beneficial:

It was refreshing to see the application 
of developed skills.

Student



The self-report survey (n=11) provided evidence that 
participating students overall felt a net positive 
change among all areas (see Figure 13), particularly 
confidence in core work-readiness skills (78 per cent 
net positive change in the sample), knowing about 
the reality of working in STEM (67 per cent), and 
professional working relationships (67 per cent). 
These practical skills and networking‑related changes 
emphasise the benefits of the real-world context of 
Generation STEM Links; areas that were more generic 
(e.g., knowing about STEM area of interest, confidence 
in tertiary studies) exhibited lower net changes.

Figure 13. Change in student confidence in STEM skills and knowledge

Assessment: To what extent has Generation STEM Links increased students’ (a) technical and enterprise skills 
(b) awareness about potential STEM career pathways and (c) commitment to work in STEM?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







To what extent has Generation STEM Links increased the number and type of tertiary 
students working in STEM jobs?

The program has not reached a sufficient scale to meaningfully 
assess this outcome. There are emerging indications that 
the program may offer an opportunity to broaden the 
recruitment pipeline of businesses from the traditional 
source of universities to include more VET students to 
meet demand. The Generation STEM Links program team 
is currently exploring ways to engage with the VET sector 
more deeply. Industry partners highlighted this potential:

We are exploring more university and even TAFE 
placements in other areas [outside engineering] 
within our company.

Industry partner

But where we were having issues with getting the 
right people was actually in our manufacturing 
group, where we actually physically make [product] 
here in [location]. And I said to my colleague “A 
TAFE student would be more suitable – for the roles 
we had in mind – than a university degree student”, 
but we hadn’t sort of tried TAFE.

Industry partner

Assessment: To what extent has Generation STEM Links increased the number and type of tertiary students 
working in STEM jobs?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective





To what extent has Generation STEM Links increased the capacity of industry?

The program appears to be meeting the needs of several of 
the host organisations involved, with 86 per cent (n = 12 out 
of 14) of students who have completed their placement by 
December 2022 being offered further employment by the 
industry partner (9 of the students accepted those offers, 
with the other 3 students being unable to accept due to 
location issues and a student deciding to change their focus). 
All of the businesses who have hosted at least one complete 
placement have offered further employment to an intern.

Even more importantly now, it is so difficult 
to attract staff into the manufacturing area. 
Unfortunately, ‘manufacturing’ isn’t one of those 
buzz words that people look for a career in.

Industry partner

Although the program has not reached a sufficient scale 
to meaningfully assess this outcome, qualitative insights 
indicate the program offers the potential to increase the 
capacity of industry. Primarily, this appears to be through 
a highly facilitated recruitment pipeline that leverages the 
trust and recognition of CSIRO.

The candidates we were offered were all good but 
the candidate we have chosen is an exceptional fit 
for our business. I doubt we would have been able 
to achieve this without the aid of the CSIRO.

Industry partner

Assessment: To what extent has Generation STEM Links increased the capacity of industry?

ASSESSMENT

Insufficient evidence

Did not meet objective

Met objectives

Exceeded objective







Economic

To what extent has the relationship 
between inputs and outputs been 
cost‑effective and to expected standards?

An initial (baseline) cost per placement ratio was calculated 
for Generation STEM Links for the last six months of the 
2022 calendar year (July to December), after the bulk of 
establishment work had been done and the majority of 
service delivery effort was directed towards placements. 
The total costs to deliver the placements program, including 
operational, labour, and subsidy costs related to service 
delivery, was $242,9406


. The total number of completed 
placements in the second half of 2022 was 297


, resulting 
in a cost per placement of $8,377. It should be noted that 
placements involve significant effort to promote, select 
students, source industry partners, provide training and 
information, etc. Therefore, the cost per ‘placement’ is an 
outcome unit that encompasses multiple people, processes, 
and activities, and is not directly comparable to different types 
of programs, such as STEM CPP. For context, a recent study 
found that the average cost of hiring a candidate in Australia 
was $23,860 in 2021, with an average time to hire of 40 days 
(Morris-Reade, 2022). The cost per placement is a valuable 
measure because as the program scales up, processes are 
streamlined, and industry partnerships are solidified, it will 
be important to track anticipated ratio decreases. 

Discussion

Initial insights from the program implementation in 2022 
indicate the program application process is accessible and that 
there is demand for the students that the program targets. 
Many of the program applicants and the program placements 
were engineering focused, which suggests that the program is 
effectively reaching engineering students, offering an attractive 
value proposition to both student and industry stakeholders. 
It may be worth considering how other disciplines can be 
more represented. However, as employment outcomes in 
some non‑engineering areas are not overly strong.

The program matching has ensured either relative or, as 
seen with women, a higher proportion reflected in the 
placement cohort (38 per cent) than the application cohort 
(29 per cent).

In contrast, the program matching has resulted in 
less students from CALD backgrounds included in the 
placement cohort (17 per cent) than the application cohort 
(43 per cent). It is important that this difference is further 
explored so that potential barriers (e.g., language) can be 
understood and addressed, where possible.

Self-report data from industry stakeholders indicates that the 
value proposition of the program for employers is access to a 
high quality recruitment pipeline. Despite some organisations 
preferring to assess the applications for their industry project 
themselves, the majority are assessed and interviewed 
initially by the CSIRO program team prior to a matching 
interview with the potential employer, which provides an 
opportunity for students to ask questions and for the industry 
partner to identify their specific needs. Industry reported that 
their participation was influenced by the financial subsidy 
and the reputation of CSIRO. It will be important to monitor 
industry participation data as the program continues to 
assess the sustainability of the program model. 

Recommendations

• Investigate the benefits in increasing program focus on 
attracting and matching VET students with placements to 
meet industry demand in manufacturing.8



• Identify potential barriers or bias in the placement 
assessment process and final industry partner selections 
that may be contributing to the discrepancy between 
CALD student involvement in the placement pool 
compared to representation within the applicant pool.
• Increase workflow automation in the program application 
and matching process to support scaling the program by 
reducing the resource intensiveness of these processes.9



• Continue to undertake monitoring of program 
expenditure to ensure the cost/outcome and cost/benefit 
ratios are within expected ranges, and to quantify 
impacts on industry.
• Leverage the new dedicated Industry Engagement 
Manager to increase participation of different industry 
sectors involved in the program (e.g., biochemistry 
(see Figure 12)) to support expansion of placement 
opportunities. In addition, the program could target 
locations that are identified as important for reaching 
under-represented student cohorts.


6 Around 66.8% was provided by Generation STEM funding and 33.2% was provided by CSIRO in the form of an ‘overhead’ cost.

7 An additional 31 placements were in the pipeline in 2022, and if completed, will be counted in the 2023 calculation.

8 The program team report that working with the VET sector has been challenging due to the highly decentralised nature of TAFE NSW campuses, with no 
straightforward way of advertising or recruiting students across the whole sector or regionally. In addition, the capability of individual campuses and cohorts 
is difficult to ascertain.

9 The program team feel that improving program efficiencies was a significant focus in 2022 with iterative continuous improvement across the program 
based on real-time learning about processes such as how to best advertise jobs, streamlining application processes and questions, and onboarding industry 
partners. The team report that they are reasonably confident that the ‘big wins’ were made in 2022.



Deadly in Generation STEM

Methodology

The Deadly in Generation STEM (DiGS) program was 
implemented for the first time in 2022 in two NSW 
locations – Kamilaroi Country (Moree) and Dharawal 
Country (Illawarra). It was decided to focus data collection 
activities for the purposes of evaluation with the 
participants and community stakeholders in Illawarra as 
the primary case study location for the program. This was 
due primarily to timing and resource challenges. As a 
result, participant insights reflect the local experience in 
Illawarra only, although it is likely that some findings would 
be generalisable.

Due to the impacts of COVID-19 on delivery of DiGS in 
Illawarra in 2022, there were only a small number of 
students involved in the program by the end of the year 
(n = 7). The key stakeholders involved in delivering the 
program, including a CSIRO staff member, a STEM mentor, 
and a cultural mentor, recommended that that they lead the 
yarning with students, and an informal yarning session with 
community stakeholders, given the existing relationships 
and familiarity. It was felt that it would be better for the 
local Indigenous people who run the program to elicit 
feedback on the program with participants, rather than the 
non-Indigenous evaluation team who had no relationships 
with students for 2022.

Data collection activities were undertaken in late 2022 and 
consisted of:

• yarning session with students (where parental/guardian 
consent has been obtained) who had been involved 
in the program. This session was facilitated by a local 
cultural mentor recognised as a community leader, and 
a local Aboriginal STEM professional mentor, who were 
both guided by questions developed by the Impact and 
Evaluation team. Students were split into two groups, 
according to gender, with a facilitator each
• yarning session with community members (e.g., parents, 
teachers, education workers, etc.) facilitated by the 
CSIRO Project Officer based in Illawarra
• very brief online survey for adult community members 
accessible via QR code to Qualtrics survey platform as an 
additional method of sharing their feedback, especially if 
they may not have felt comfortable saying something in 
the group setting
• discussion between Impact and Evaluation team and 
the Generation STEM leadership to explore the delivery 
partnership in Moree.


Program implementation reflections gathered by 
Generation STEM leadership with key stakeholder in Moree.

Process

How was DiGS implemented?

In both Moree and Illawarra, the DiGS program model 
was co-designed in collaboration with community 
stakeholders. The community identified their needs 
and the program elements they felt were important. 
Across the two locations, there were many commonalities 
that emerged from the co-design process although 
there were some differences in the models that were 
implemented as a result. In Moree, the model involved an 
in-school component which was delivered via the Inquiry 
for Indigenous Science Students (I²S²) program, overseen 
by CSIRO. Originally, both locations intended to undertake 
a camp; however, this was not feasible, due to risk of 
disruptions relating to COVID-19. Instead, both locations 
adapted to deliver four themed Immersion Days spread over 
a number of months.



Kamilaroi Country (Moree) 

It is useful to understand and briefly document the process 
in Moree10


. Originally, both locations had a dedicated 
Project Officer working with a key community stakeholder 
acting in a central collaborator capacity (a similar role as 
the Councils in the STEM CPP project). The community 
engagement process had identified the importance of 
involving the Moree Sports, Health, Arts and Education 
(SHAE) Academy due to their connections with the 
community and with young people, including the familiarity 
of the location. CSIRO and the SHAE Academy entered into 
a collaboration agreement, with the CSIRO Project Officer 
having responsibility for delivering the project and working 
closely with the Academy. The first of the four Immersion 
Days with students were successfully delivered in Moree 
in collaboration with the Academy prior to the departure 
of the Project Officer (who moved to CSIRO’s Young 
Indigenous Women’s STEM Academy team). The CSIRO team 
did look to find a suitable replacement candidate in Moree; 
however, no candidates applied for the position. To address 
this delivery capacity gap, CSIRO initiated delivery partner 
conversations with SHAE Academy, noting that CSIRO would 
need to maintain oversight of the STEM expertise for the 
in-school component. Implementation of this delivery 
partnership was somewhat challenging because of a lack of 
capacity in SHAE Academy, as was formalising an agreement 
between partners. The interim solution was to extend 
the collaboration agreement with some funding included 
to deliver the remaining Immersion Days. All remaining 
Immersion Days were held, however both CSIRO and 
SHAE Academy observed that the quality of outputs and 
outcomes were not as a strong as the first day, when the 
CSIRO Project Officer was involved.

Despite some of the barriers with establishing and 
implementation of the formal delivery partnership, a strong 
relationship has been maintained between CSIRO and SHAE 
Academy. Staff from the Academy have emphasised the 
ongoing collaborative involvement of a central community 
stakeholder as critical to the success of a program such as 
DiGS to provide stability and continuity. There is an ongoing 
shared interest in continuing to work together to deliver a 
STEM engagement program for Aboriginal young people 
on Kamilaroi Country. 

Dharawal Country (Illawarra)

Delivery of the program in 2022 involved a close 
collaborative partnership between a locally recognised 
cultural mentor, a CSIRO Project Officer, and a local 
STEM mentor. 

Originally, a partnership with the University of Wollongong 
was intended as an element of the implementation of 
the DiGS program in Illawarra. Due to local stakeholder 
relationships and interdependencies, this partnership was 
not able to be established.

In contrast to Moree, the program implementation in 
Illawarra was supported by a dedicated Project Officer 
throughout 2022. In addition, there was a clearly recognised 
cultural mentor in the community to lead and facilitate 
the Indigenous knowledge framework underpinning 
the program.

What were the particular elements of 
the program and context that made a 
difference?

Based on the yarning session, observational data, 
and program team feedback:

Common themes from both locations

• Hands on activities were perceived to be effective to 
engage students, such as weaving, making medicine tea, 
throwing boomerangs, catching fish.
• Local knowledge holders were important to engage with 
the students.
• Strengthening knowledge and understanding of culture 
and history leads to increased pride and confidence 
in identity.


10 Evaluation reports are often the only public-facing source of information about program processes that can provide valuable learning opportunities 
for future programs.



Kamilaroi Country (Moree) 

• Hosting at SHAE Academy, which is a culturally safe 
and secure organisation and site.
• The program works best when CSIRO can do the 
groundwork to coordinate local knowledge holder’s 
involvement once introductions and connections have 
been made through the Academy.
• The structure of four separate days made it difficult to 
engage with Moree Secondary School – the camp in 
2023 will be a preferable structure because it will allow 
engagement with a wider range of schools.
• Potentially more differentiation to students with 
different levels of interest. For example, the majority of 
students were highly engaged in the hands-on activities, 
but there was a high degree of variability observed in 
engagement levels during other activities.
• Starting and ending days with cultural knowledge and 
Dreaming stories worked well.
• Not using ‘STEM’ acronym and/or language but rather 
encouraging students to walk in the footsteps of their 
ancestors as the First Scientists, Bakers, Engineers and 
Mathematicians – encouraging cultural pride and their 
responsibility to care for Country.


Dharawal Country (Illawarra)

• Meeting students from other schools and forming 
connections with other Aboriginal young people 
they otherwise might not have met. Several students 
mentioned this:


We pretty much don’t – say like us boys, we rarely 
ever get together. Like I’ve never been – besides 
footy, I’ve never met never met them in my life. 
Probably wouldn’t have talked to them either too…
now I know where they’re from, they’re Indigenous. 
I treat them like my cousins and my family.

Student

And if you’re ever stuck on something or you need 
help with like an Indigenous thing, I could call 
[Student] or I can call [Student] and they’ll pretty 
much help me if I need to – because I know they 
probably would know, or give me a sense of idea 
what I should need to know to learn it.

Student

Being outside and on Country and grounding STEM in an 
Indigenous framework made it seem less boring and more 
relevant to students. For example:

Yeah. I thought we were just going to be like sitting 
in a room doing boring stuff.

Student

Understanding how to use technology to care for Country, 
for example, drones to monitor ocean patterns or 
observe koalas.



Impact and outcomes

What outcomes are emerging from the 
initial implementation of DiGS?

Students involved in the yarning session held on Dharawal 
Country (Illawarra) reported an increased understanding 
of Indigenous STEM knowledges as a result of their 
participation. They discussed how this knowledge made 
them feel, reflecting a greater sense of belonging and/or 
wellbeing. This was highlighted by several students:

It shows how advanced we were…It shows how 
intelligent and caring our ancestors were.

Student

Through the program, I’ve definitely got a bigger 
sense of pride in my culture. Especially day one, just 
how [cultural mentor] was telling us how important 
and impactful our people were definitely inspired 
something that makes me think, damn, we are 
important people. It’s really an amazing culture 
to be a part of. So definitely a bigger sense of 
pride in my Aboriginality. And also, a fair bit more 
understanding in it. So, I know more about who I 
am through that now. I know where my people are 
from. That sort of stuff. I know more people like me 
through the program.

Student

Some of the things [cultural mentor] showed 
us, showed that we’re not nothing. Some of the 
techniques they use, like back in…ages away, 
definitely impact like how things are now.

Student

Discussion amongst the community stakeholders 
highlighted that some students have an increased 
confidence and interest in STEM, which is evident at 
home and at school. For example:

Personally, for my child, he’s gone from like a [class 
five] and he’s made – I think this has opened up his 
mind and he’s moved to the top ten…The whole 
school is talking about how – they just don’t know 
where he comes from. And I think this has opened 
up the importance and just made it more relatable.

Parent

Students reported an increased connection to local 
community and Country, including increased cultural 
awareness. A few students emphasised this:

Yeah. The way you connect things from our culture 
and then in modern day time.

Student

The biggest thing we learnt about was definitely 
local culture and what our people have used their 
whole lives. How that impacts modern day society.

Student

Students involved in the yarning discussed how they had 
engaged in knowledge sharing and capacity building across 
the community.

Even that, just do that, refreshes everyone and gets 
them talking and as soon as like all the parents 
came, we went with our parents and we went back, 
and we [told our parents what we heard.] Most of us 
probably misheard or didn’t hear what Uncle said or 
what you said, or the other people said. It’s that – 
like other people are hearing what you maybe didn’t 
hear. Or you heard what they didn’t hear. And you 
just teach each other.

Student

Finally, a cost per participant ratio was calculated to 
establish a baseline for future years. The service delivery 
costs for 2022 were $344,030 and the total student 
participants who completed the program were 27, resulting 
in a cost per participant of $12,742. This ratio would be 
expected to decrease as the first year of program delivery is 
resource intensive, for example, setting up partnerships and 
developing processes and resources. In addition, a number 
of participants withdrew from the program throughout 
the year.



Discussion

The available data suggests that the piloted model for the 
DiGS program has been effective in achieving some of the 
shorter-term outcomes reflected in the Impact Pathway for 
individual student participants. In particular, participating 
in the program appears to have increased understanding 
of Indigenous STEM knowledges amongst students and 
increased connection to local community and Country. 
The students’ reflections suggest that this understanding 
may have extended to others in their families and 
community, through opportunities for shared reflection 
and connection that the Immersion Days incorporated.

There was a rich discussion between the students involved 
in the male student group regarding the benefits of 
the camp model versus the interconnected Immersion 
days. The students demonstrated a lot of thought and 
consideration regarding these factors, suggesting that 
further exploration with the students would be beneficial 
to continue to co-design the DiGS program.

Although both male and female students were involved 
in the qualitative data collection, the depth of discussion 
was noticeably greater amongst the male students. 
The implementation of the program in Illawarra primarily 
involved male knowledge holders and STEM role models. 
Although no attribution can be made with certainty, similar 
gender considerations as have been applied in the STEM 
CPP program may be relevant for future delivery of DiGS.

Insights from community stakeholders was limited; 
however, early indications point to the program model 
being well received, with ongoing interest in its availability 
within both regions. For ongoing implementation of 
the project and subsequent evaluation, consideration 
is needed as to how to respond to feasibility factors to 
support embedding Indigenous experts into the Impact 
and Evaluation team, especially for data collection 
activities. Although the students were split into male and 
female groups for the student focus group discussions, 
there is richer data evident amongst the male participants. 
Both facilitators were local Aboriginal cultural and STEM 
mentors, however both were men. It is worth considering 
whether a female led discussion may have yielded greater 
insights from the female students.

Recommendations

What can be strengthened or adapted to 
reflect initial community feedback?

• Undertake further discussion with young people 
regarding the appropriateness of camps and/or 
Immersion days to explore the considerations identified 
by the participants in Illawarra.
• Involve young people as champions and ambassadors 
of the program to engage potential participants.11



• Consider adapting impact pathway to include 
emerging outcomes of importance or undertaking 
further co-design to reflect on how this is valued, 
such as transferable skills versus knowledge sharing 
and teaching.


Specific to Kamilaroi Country (Moree)

• SHAE Academy to take over organisation of consent 
and information forms so that they can communicate 
with individual families and coordinate this process 
through their relationships and proven place within the 
community.12


 Although this approach would increase the 
cultural safety for families, if this approach was adopted, 
any processes would need to be reviewed by the CSIRO 
Privacy team prior to use.
• CSIRO to work with SHAE Academy Chief Executive 
Officer (CEO) as closely as possible to shape the localised 
approach to recruitment of new Project Officer for 2023 
as the recruitment approach in 2022 is likely to have been 
a barrier for potential candidates.


Learning and reflections gathered among Generation STEM 
team and SHAE Academy implemented into planning for 
2023 and built into sustainability considerations.

• Risk engagement of students within Group 3 
(see Table 13) in Moree if activities aren’t well 
planned and executed, as young people are already 
highly disengaged.


11 The program team reports this is being planned for Illawara in 2023.

12 The program team reports this happened in the later immersion days in 2022.



Specific to Dharawal Country (Illawarra)

• Plan Immersion Days or future days on camp with 
students’ experience of moving through the day(s) in 
mind (including behavioural and cultural considerations).


I felt like day two needed to be split into two. Just 
like another day, because there was just that much 
activity going on, it was kind of hard to navigate. 
And like [another student] said, everyone got a bit 
restless by the end.

Student

Ensure communications about the program highlight the 
student reported benefits (see Impact and Outcomes above) 
and address barriers (perceived and systemic).

…with my daughter, she was worried that it was all 
about maths. And she doesn’t like maths. “I can’t do 
it. I can’t go to that program….”

Parent

Also, maybe like the education level and some of 
the kids feeling like that’s just for the nerdy kids. So 
that stigma. And I guess the other thing, the money 
side of things as well. Like, “Oh, I’d never be able to 
afford to pay for a uni degree” or “I can’t. How am 
I meant to get there? I haven’t got the money.” All 
that sort of stuff.

Community member 

Things we could improve, the shirts. 
They’re really ugly.

Student

Ensure effective dissemination of information with 
stakeholders to contribute to increased participation (e.g., 
let schools already engaged in the program know that they 
can send more students).

Reach students earlier within the educational setting, for 
example in Year 7 before school attendance may start to 
drop in Years 9 and 10.13




But the start of school’s really – that’s really 
important once they finish primary school as well, 
because a lot of the time kids can go off on a path 
that may not be the one that they were in when they 
were in primary school. Because there’s all those 
different relationships and friendship groups. And 
then that has a big impact on where they go from 
there. It’s not about the education, it’s about the 
friendship groups a lot.

Community member

13 The program team reports that a primary school pilot is being trialled in 2023 with the Inquiry for Indigenous Science Students teacher professional learning.



Data insights

The governance body overseeing the Generation STEM 
initiative (Consultative Council), which includes members 
from industry and the NSW Government, in addition to 
the Trustee of SIEF, have expressed an interest in building 
a better evidence base for ‘what works’ in STEM education 
and also a platform to utilise this evidence to understand 
the degree to which Generation STEM programs 
are effective.

To address this need, two data insights projects have been 
established and, although still in their nascent stages, 
produced some initial findings up to early 2023.

Data analytics

A project to apply predictive analytics to an education 
dataset to understand the factors that influence STEM 
education decisions and outcomes in high school was 
established in late 2021. Human research ethics approval 
was obtained from the CSIRO Human Research Ethics 
Committee and a collaboration agreement was signed 
with a Catholic education diocese in New South Wales to 
undertake the analytics using available data from their 
schools. Two Doctor of Philosophy (PhD) students14


 from 
the Australia National University with expertise in artificial 
intelligence and predictive analytics were engaged by 
CSIRO to undertake the analyses. Initial analyses have been 
completed with the final analyses to be concluded by April 
2023 and presented in a separate report or research article.

The dataset comprised 51,851 students from 73 schools, 
although there were significant amounts of missing data 
(primarily from students entering and leaving the Diocese’s 
system). More detailed analyses will be outlined in a 
separate report, but initial exploratory results indicate:

• Higher academic performance in STEM subjects up to 
year 10 is positively correlated with becoming an High 
School Certificate (HSC) STEM student.
• Female students tend to perform better in HSC STEM 
subjects, even though a higher proportion of HSC STEM 
students are male.
• Students with parents with a qualification of bachelor’s 
degree and above are more likely to select STEM for 
HSC and to perform better in those subjects than 
other students.
• Most students do not ‘change course’, that is, students 
that don’t take STEM subjects up to year 10 do not 
usually go on to do HSC STEM subjects.
• HSC STEM subjects are difficult; most students who 
did HSC STEM subjects did academically better in their 
non‑STEM HSC subjects.


As of February 2023, the analytics portion of this project 
had only very initial results using machine learning models. 
The objective was to predict whether a student followed an 
HSC STEM stream (defined as taking either 50 or 80 per cent 
or more STEM subjects in year 12) based on available 
independent variables, such as previous years’ academic 
performance, personal attributes, family attributes, and 
school details. A total of 12 machine learning models were 
tested resulting in accuracy levels between 66 per cent 
and 72 per cent for the 50 per cent definition, and up to 
85 per cent for the 80 per cent definition. That is, based 
on a student’s pre-Year 12 data, the models could predict 
whether the student would be a STEM HSC student 66 to 
85 per cent of the time. The best performing model for the 
50 per cent definition was the ‘Lass-Logistic regression with 
important variables’ (0.7251) followed by ‘Random forest’ 
(0.7191). In general, anything greater than 70 per cent 
can be considered good model performance; however, 
further refinements will be made to attempt greater 
accuracy levels.

The implementation learnings from the analytics project to 
date include:

• Processes to obtain datasets are relatively lengthy and 
multi-faceted, but the close relationship and support 
from staff at the Diocese made it feasible.
• Missing data are problematic but not unsurmountable; 
the data that were ‘missing’ comprised a small portion 
of data that were absent due to non-entry or other 
quality control reasons, and a large portion were ‘gaps’ 
as a result from students moving in and out of the 
Diocese’s education system.
• The top-level results of the analyses confirmed much 
of what is known about STEM education outcomes; 
however, drilling down further into analyses of particular 
groups and locations may provide new insights.


14 Vimu Inguruwattage and Chathura Nagoda Gamage



Evidence X

A second project was established to help build an 
evidence base for STEM education interventions; that is, 
better understand what interventions work and why and 
achieve an increased ability to assess an intervention’s 
effectiveness. The first step in this project was to build 
a prototype of an online, interactive evidence tool that 
would allow users (program designers, program funders, 
and program evaluators) to understand the current state 
of evidence and inform their decision-making (see screen 
shot in Figure 15). The prototype involved classifying a 
set of research articles by a number of different variables 
including the category of outcome (attitude, behaviour, 
skill, knowledge). Each ‘bubble’ summarises a research 
study linking an intervention (e.g., a program) or a factor 
(e.g., a personal attribute) with STEM-education related 
outcomes and provides a link to the original research. 
Users can see the evidence linkage and what the strength 
of the evidence is (weak, promising, strong). This prototype 
was designed using R Shiny and was developed through 
a research collaboration project with CSIRO’s Information 
Management and Technology unit. There are some 
technical limitations with the software that was used, 
and therefore different options will be explored as the 
project progresses.

The next stage in the process involves a co-design phase 
led by an external consultancy to bring together key 
stakeholders in the STEM education sector to design 
a platform of evidence/set of tools. This phase will be 
completed by the end of 2023.

Figure 15. STEM Evidence Tool prototype



Conclusion

The goals of Generation STEM are complex and challenging 
to achieve. As can be seen in Figure 3, the impact pathway 
for Generation STEM identifies increasing the number 
of participants pursuing STEM education pathways and 
ultimately the number of people employed in NSW‑based 
STEM jobs and the STEM literacy of all people. These goals 
aim to address the ‘leaky’ STEM ‘pipeline’, and there 
are no easy fixes because the influences on student 
behaviours and decisions are so diverse and multi-layered. 
Among these myriad of factors, having a STEM identity is 
of central importance to attracting and retaining students 
in STEM; that is, increasing the effectiveness of teaching 
STEM will have little impact if students don’t believe 
STEM is a place for them. Despite a relatively slow roll-
out, Generation STEM is beginning to achieve a number 
of short-term implementation objectives and outcomes, 
such as student’s self-reported interest and intention to 
pursue STEM. The place-based focus of the initiative has 
seen strong relationships developed with communities and 
relevant industries in Western Sydney, Illawarra, Moree, 
and a number of other locales. STEM CPP is reaching 
over 2,000 students per year, and although Generation 
STEM Links and DiGS are much smaller in breadth, they 
perhaps have more potential for depth in terms of potential 
outcomes among individual students and schools. 
Figure 16 outlines a model (adapted from Farazi, 
Gopalakrishnan, & Perez-Luno, 2019 by the first author 
of this report) comprising four program model types of 
different depths (impact) and breadth (size of cohort). 
Generation STEM Links and DiGS aim to be ‘gorge’ 
programs, working with fewer students and investing more 
per student, to achieve substantial individual outcomes 
over time. Currently, it is a question whether STEM CPP 
is a lagoon or gorge program; the solution may be a 
lagoon program that works with individual schools to 
become gorge-like in their outcomes, particularly from 
a sustainability perspective. Growing the breadth to more 
schools, locations, and industry partners is casting a wide 
net, when perhaps a more targeted one may be more 
effective to address the complexities of STEM identity 
formation and verification among students and building 
a STEM school culture that can be sustained beyond the life 
of the program. Regardless of this over-arching question, 
all three programs have been successfully designed 
and implemented and are well-placed to achieve their 
goals by 2027.

Figure 16. Breadth versus depth in STEM education programs

Large cohort (breadth)

Low 
intensity 
(depth)

High 
intensity 
(depth)

Small cohort (breadth)

Lagoon programs

Impact

Budget size: $$

Examples: campaigns, 
events, citizen science, 
marketing, awards

Ocean programs

Impact

Budget size: $$$$$

Examples: 
education system

Gorge programs

Impact

Budget size: $$

Examples: work 
experience, longer-term 
support and mentoring

Pond programs

Impact

Budget size: $

Examples: Targeted online 
resources, one-off teacher 
professional learning



References

Atkins, K., Dougan, B.M., Dromgold-Sermen, M.S. et al. 
(2020). “Looking at Myself in the Future”: How mentoring 
shapes scientific identity for STEM students from 
underrepresented groups. International Journal of STEM 
Education, 7(42). https://doi.org/10.1186/s40594-020-00242-3

Cohen J. (1988). Statistical Power Analysis for the Behavioral 
Sciences. New York, NY: Routledge Academic

Edwards, D., Buckley, S., Chiavaroli, N., Rothman, S. & 
McMillan, J. (2023). The STEM pipeline: pathways and 
influences on participation and achievement of equity 
groups. Journal of Higher Education Policy and Management. 
https://doi.org/10.1080/1360080X.2023.2180169

Farazi, M.S., Gopalakrishnan, S., & Perez-Luno, A. (2019). 
Depth and breadth of knowledge and the governance of 
technology alliances. Journal of Engineering and Technology 
Management, 54, 28–40. https://doi.org/10.1016/j.
jengtecman.2019.08.002

Gladstone, J.R. & Cimpian, A. (2021). Which role models are 
effective for which students? A systematic review and four 
recommendations for maximizing the effectiveness of role 
models in STEM. International Journal of STEM Education, 
8(59). https://doi.org/10.1186/s40594-021-00315-x

Hackling, M., Murcia, K., West, J., & Anderson, K. (2014). 
Optimising STEM education in WA schools. Edith Cowan 
University Research Online. Retrieved from https://ro.ecu.
edu.au/cgi/viewcontent.cgi?article=7940&context=ecuworkspost2013

Han, J. Kelley, T. & Knowles, G. (2021). Factors influencing 
student STEM learning: Self-efficacy and outcome 
expectancy, 21st Century skills, and career awareness. 
Journal for STEM Education Research, 4, 117–137.
https://doi.org/10.1007/s41979-021-00053-3

Jeffries, D., Curtis, D.D. & Conner, L.N. (2020). 
Student factors influencing STEM subject choice in Year 
12: a Structural equation model using PISA/LSAY data. 
International Journal of Science and Mathematics Education, 
18, 441–461. https://doi.org/10.1007/s10763-019-09972-5

Jin, Q. (2021). Supporting Indigenous students in science 
and STEM education: A systematic review. Education 
Sciences, 11(9), 555. https://doi.org/10.3390/educsci11090555

McKinnon. (2022). The absence of evidence of the 
effectiveness of Australian gender equity in STEM 
initiatives. The Australian Journal of Social Issues, 57(1), 
202–214. https://doi.org/10.1002/ajs4.142

Millar, V., Hobbs, L., Speldewinde, C. & van Driel, J. (2022), 
Stakeholder perceptions of mentoring in developing girls’ 
STEM identities: “you do not have to be the textbook 
scientist with a white coat”, International Journal of 
Mentoring and Coaching in Education, 11(4), 398–413. 
https://doi.org/10.1108/IJMCE-11-2021-0100

Morris, J., Slater, E., Boston, J., Fitzgerald, M., & Lummis, 
G. (2021). Teachers in conversation with industry scientists: 
Implications for STEM education. International Journal of 
Innovation in Science and Mathematics Education, 29(1), 
46–57. https://doi.org/10.30722/IJISME.29.01.004

Morris-Reade, R. (2022). The cost of hiring new workers 
doubles to more than $23,000. IT Brief, 17 March 2022. 
Retrieved from https://itbrief.com.au/story/the-cost-of-
hiring-new-workers-doubles-to-more-than-23-000 

Murphy, S., MacDonald, A., Wang, C.A., & Danaia, L. (2019). 
Towards an understanding of STEM engagement: A review of 
the literature on motivation and academic emotions. Canadian 
Journal of Science, Mathematics and Technology Education, 
19, 304–320. https://doi.org/10.1007/s42330-019-00054-w

Palmer, T.A., Burke, P.F. & Aubusson, P. (2017). Why school 
students choose and reject science: A study of the 
factors that students consider when selecting subjects. 
International Journal of Science Education, 39(6), 645–662. 
http://dx.doi.org/10.1080/09500693.2017.1299949

Shah, C., Richardson, P., & Watt, H. (2020): Teaching ‘out of 
field’ in STEM subjects in Australia: Evidence from PISA 2015, 
GLO Discussion Paper, No. 511. Global Labor Organization 
(GLO), Essen. Retrieved from https://www.econstor.eu/
bitstream/10419/215639/1/GLO-DP-0511.pdf 

Speldewinde, C. & Campbell, C. (2021). Bush kinders: 
enabling girls’ STEM identities in early childhood. Journal 
of Adventure Education and Outdoor Learning, 1(1), 1–16.
https://doi.org/10.1080/14729679.2021.2011337

Stephenson, T., Fleer, M., & Fragkiadaki, G. (2021). Increasing 
girls’ STEM engagement in early childhood: Conditions 
created by the Conceptual Playworld model. Research in 
Science Education, 52(4), 1243–1260. https://doi.org/10.1007/
s11165-021-10003-z

White, C., Marshall, J.C., & Alston, D. (2019). Empirically 
supporting school STEM culture—The creation and 
validation of the STEM Culture Assessment Tool (STEM-CAT). 
School Science and Mathematics, 119, 299–311. 

Yeh, S. (2010). The cost effectiveness of 22 approaches for 
raising student achievement. Journal of Education Finance, 
36(1), 38–75. 



Appendices

Appendix A. Proposed school-level data analysis for STEM CPP

Evaluation question: Does STEM CPP contribute to higher uptake of STEM elective 
subjects by students in intervention schools?

Overall analysis model – difference-in-difference regression analysis

Analysis 1

Intervention group: STEM CPP schools in 2019 with Year 9 
students (see list below)

Comparison group: all other schools in NSW with 
Year 9 students

Controls: school ICSEA, school Australian Statistical 
Geography Standard remoteness area, school size 
(enrolments 2021), per cent Aboriginal and/or Torres Strait 
Islander, per cent language background other than English, 
sector (government, Catholic, independent), single or 
mixed gender, National Assessment Program – Literacy and 
Numeracy (NAPLAN) Year 9 numeracy

Outcome measures: (1) percentage of Year 11 students 
completing any STEM subject electives (2) percentage 
of Year 11 students completing 2 or more STEM 
subject electives



Analysis 2

As above, except Intervention group schools are from 2021 (see list below), and the comparison group excludes a number 
of schools involved in the program that have dropped out or joined after 2021.

OUTCOME

GROUPS

Percentage of Year 11 students completing 
any STEM subject elective

Analysis 1: Year 11 STEM subject enrolments in 2019 STEM CPP schools 
and Year 11 STEM subject enrolments in all other schools

Analysis 2: Year 11 STEM subject enrolments in 2021 STEM CPP schools 
and Year 11 STEM subject enrolments in all other schools (with some 
exclusions)

Percentage of Year 11 students completing 2 
or more STEM subject electives

Analysis 1: Year 11 STEM subject enrolments in 2019 STEM CPP schools 
and Year 11 STEM subject enrolments in all other schools

Analysis 2: Year 11 STEM subject enrolments in 2021 STEM CPP 
schools and Year 11 STEM subject enrolments in all other schools 
(with some exclusions)





COEFFICIENT

CALCULATION

INTERPRETATION

β0

B

Baseline average

β1

D-B

Time trend in comparison group

β2

A-B

Difference between two groups pre-STEM CPP

β3

(C-A)-(D-B)

Difference in changes over time





Regression equation: Y= β0 + β1*[Time] + β2*[Intervention] + β3*[Time*Intervention] + β4*[Covariates]+ε



Appendix B. STEM CPP student survey data 2022

WHAT YEAR ARE THEY CURRENTLY IN 

NUMBER 

PROPORTION 

Year 9 

123 

56% 

Year 10 

97 

44% 

(n=220 for all questions including blank responses)





GENDER 

NUMBER 

PROPORTION 

Female 

113 

51% 

Male 

102 

46% 

Other (non-binary, intersex, indeterminate, etc.) 

4 

2% 

Prefer not to say 

1 

0% 





ABORIGINAL AND/OR TORRES STRAIT ISLANDER 

NUMBER 

PROPORTION 

Yes 

7 

3% 

No 

209 

95% 

Prefer not to say 

4 

2% 





CULTURALLY AND LINGUISTICALLY DIVERSE 

NUMBER 

PROPORTION 

Yes 

107 

49% 

No 

107 

49% 

Prefer not to say 

6 

3% 





Student post-school destination 

INTENTION TO STUDY AT TAFE OR UNIVERSITY 

NUMBER 

PROPORTION 

No 

2 

1% 

Unsure / don’t know yet 

33 

15% 

TAFE, but unsure what to study 

6 

3% 

TAFE, and know what to study 

3 

1% 

University, but unsure what to study 

95 

43% 

University, and know what to study 

81 

37% 





Factors influencing student subject selection

WHAT THEIR PARENTS AND FAMILY THINK 

NUMBER 

PROPORTION 

No impact 

55 

25% 

Slight impact 

109 

50% 

Big impact 

48 

22% 

Unsure 

8 

4% 







WHAT THEIR FRIENDS WILL BE STUDYING 

NUMBER 

PROPORTION 

No impact 

126 

57% 

Slight impact 

67 

30% 

Big impact 

13 

6% 

Unsure 

14 

6% 





HOW INTERESTING THEY FIND THE SUBJECT 

NUMBER 

PROPORTION 

No impact 100

6 

3% 

Slight impact 

36 

16% 

Big impact 

173 

79% 

Unsure 

5 

2% 





HOW WELL THEY THINK THEY’LL DO IN THE SUBJECT 

NUMBER 

PROPORTION 

No impact 

11 

5% 

Slight impact 

71 

32% 

Big impact 

125 

57% 

Unsure 

13 

6% 





RELEVANCE OF THE SUBJECT TO THEIR DAILY LIFE 

NUMBER 

PROPORTION 

No impact 

33 

15% 

Slight impact 

90 

41% 

Big impact 

81 

37% 

Unsure 

16 

7% 





RELEVANCE OF THE SUBJECT TO WHAT THEY WANT TO DO FOR A JOB 

NUMBER 

PROPORTION 

No impact 

8 

4% 

Slight impact 

31 

14% 

Big impact 

171 

78% 

Unsure 

10 

5% 





PRE-REQUISITES FOR TAFE OR UNIVERSITY 

NUMBER 

PROPORTION 

No impact 

20 

9% 

Slight impact 

57 

26% 

Big impact 

123 

56% 

Unsure 

20 

9% 







WHICH TEACHER WILL BE TEACHING THE SUBJECT 

NUMBER 

PROPORTION 

No impact 

62 

28% 

Slight impact 

97 

44% 

Big impact 

44 

20% 

Unsure 

17 

8% 





Student confidence studying STEM subjects 

SCIENCE 

NUMBER 

PROPORTION 

Not very confident 

23 

11% 

Somewhat confident 

71 

32% 

Confident 

123 

56% 

N/A

2 

1% 





MATHEMATICS 

NUMBER 

PROPORTION 

Not very confident 

24 

11% 

Somewhat confident 

68 

31% 

Confident 

125 

57% 

N/A

2 

1% 





TECHNOLOGY SUBJECTS 

NUMBER 

PROPORTION 

Not very confident 

21 

10% 

Somewhat confident 

72 

33% 

Confident 

120 

55% 

N/A

6 

3% 





iSTEM

NUMBER 

PROPORTION 

Not very confident 

27 

12% 

Somewhat confident 

80 

37% 

Confident 

81 

37% 

N/A

31 

14% 





STEM-RELATED VET SUBJECTS 

NUMBER 

PROPORTION 

Not very confident 

30 

14% 

Somewhat confident 

86 

39% 

Confident 

51 

23% 

N/A

52 

24% 







Student intention to study STEM subjects in Years 11 and 12 

GENERAL MATHEMATICS 

NUMBER 

PROPORTION 

No 

54 

25% 

Yes, potentially 

41 

19% 

Yes, definitely15


 

111 

51% 

Not applicable/unsure

13 

6% 





ADVANCED OR EXTENSION MATHEMATICS 

NUMBER 

PROPORTION 

No 

48 

22% 

Yes, potentially 

42 

19% 

Yes, definitely

121 

55% 

Not applicable/unsure

8 

4% 





SCIENCE SUBJECTS 

NUMBER 

PROPORTION 

No 

42 

19% 

Yes, potentially 

36 

16% 

Yes, definitely

135 

62% 

Not applicable/unsure

6 

3% 





TECHNOLOGY SUBJECTS 

NUMBER 

PROPORTION 

No 

66 

30% 

Yes, potentially 

50 

23% 

Yes, definitely

86 

39% 

Not applicable/unsure

17 

8% 





STEM-RELATED VET SUBJECTS 

NUMBER 

PROPORTION 

No 

95 

43% 

Yes, potentially 

51 

23% 

Yes, definitely

31 

14% 

Not applicable/unsure

42 

19% 





STEM-RELATED SCHOOL APPRENTICESHIPS OR TRAINEESHIPS 

NUMBER 

PROPORTION 

No 

103 

47% 

Yes, potentially 

54 

25% 

Yes, definitely

28 

13% 

Not applicable/unsure

34 

16% 





15 This question option combines ‘yes’ and ‘yes, definitely’ from the Year 9 and Year 10 surveys



Change in students’ interest and attitudes towards STEM

I AM INTERESTED IN LEARNING ABOUT STEM

NUMBER

PROPORTION

Negative change

43

20%

No change

121

58%

Positive change

46

 22%*

Effect sizes – All: 0.035; Females: 0.106; Males: 0.027





I THINK STEM IS USEFUL IN EVERYDAY LIFE

NUMBER

PROPORTION

Negative change

24

11%

No change

113

54%

Positive change

73

 35%*

Effect sizes – All: 0.195; Females: 0.300; Males: 0.101





I FIND STEM SUBJECTS EXCITING

NUMBER

PROPORTION

Negative change

46

22%

No change

123

59%

Positive change

41

 20%*

Effect sizes – All: 0.022; Females: 0.073; Males: 0.031





I THINK STEM SUBJECTS ARE IMPORTANT 
TO MY FUTURE STUDY AND CAREER

NUMBER

PROPORTION

Negative change

42

20%

No change

110

52%

Positive change

58

28%*

Effect sizes – All: 0.044; Females: 0.062; Males: 0.021





I FIND STEM SUBJECTS DIFFICULT

NUMBER

PROPORTION

Negative change

47

22%

No change

104

50%

Positive change

59

28%*

Effect sizes – All: 0.051; Females: 0.048; Males: 0.046





I THINK STEM IS IMPORTANT FOR SOCIETY

NUMBER

PROPORTION

Negative change

31

15%

No change

117

56%

Positive change

62

 30%*

Effect sizes – All: 0.115; Females: 0.146; Males: 0.093







I AM AWARE OF THE TYPE OF STEM JOBS I CAN WORK IN 

NUMBER

PROPORTION

Negative change

28

13%

No change

97

46%

Positive change

85

 40%*

Effect sizes – All: 0.237; Females: 0.270; Males: 0.211





I EXPECT TO DO WELL IN STEM SUBJECTS

NUMBER

PROPORTION

Negative change

32

15%

No change

123

59%

Positive change

55

 26%*

Effect sizes – All: 0.099; Females: 0.135; Males: 0.073





I WOULD LIKE TO HAVE A JOB WORKING IN STEM

NUMBER

PROPORTION

Negative change

27

13%

No change

130

62%

Positive change

53

 25%*

Effect sizes – All: 0.073; Females: 0.101; Males: 0.051





Impact of STEM CPP activities on students’ interest in STEM

COMPLETING INQUIRY PROJECT

NUMBER

PROPORTION

Less interested

5

2%

No change in interest 

32

16%

A bit more interested 

83

40%

Much more interested 

70

34%

Not applicable/unsure 

15

7%





INTERACTING WITH INDUSTRY MENTOR

NUMBER

PROPORTION

Less interested

7

3%

No change in interest 

36

18%

A bit more interested 

58

28%

Much more interested 

69

34%

Not applicable/unsure 

35

17%







WATCHING MASTERCLASS VIDEO

NUMBER

PROPORTION

Less interested

11

5%

No change in interest 

52

25%

A bit more interested 

38

19%

Much more interested 

39

19%

Not applicable/unsure 

65

32%





ATTENDING BUSINESS SITE/WORKPLACE

NUMBER

PROPORTION

Less interested

5

2%

No change in interest 

31

15%

A bit more interested 

50

24%

Much more interested 

65

32%

Not applicable/unsure 

54

26%





PARTICIPATING IN THE SHOWCASE

NUMBER

PROPORTION

Less interested

8

4%

No change in interest 

30

15%

A bit more interested 

54

26%

Much more interested 

70

34%

Not applicable/unsure 

43

21%





PARTICIPATING IN WORK EXPERIENCE

NUMBER

PROPORTION

Less interested

2

2%

No change in interest 

12

14%

A bit more interested 

24

28%

Much more interested 

21

24%

Not applicable/unsure 

28

32%





ATTENDING A TAFE TESTER EVENT

NUMBER

PROPORTION

Less interested

0

0%

No change in interest 

0

0%

A bit more interested 

6

40%

Much more interested 

9

60%

Not applicable/unsure 

0

0%







Appendix C. STEM CPP teacher survey data 2022

STEM SUBJECTS TAUGHT*

NUMBER

PROPORTION

Sciences

34

72%

Mathematics

5

11%

Technology and applied studies

8

17%

iSTEM

9

19%

Other STEM subjects (including VET in school)

4

9%

STEM-based extra-curricular activities (e.g., school clubs)

15

32%

Other

4

9%

(n=47) *Teachers were able to choose multiple choices for this question





SCHOOL FOCUS ON PROMOTING AND SUPPORTING STUDENT 
ENGAGEMENT IN STEM EDUCATION AND ACTIVITIES

NUMBER

PROPORTION

Moderate

32

70%

Strong focus

10

22%

Very strong focus

4

9%

(n=46)





IMPACT OF STEM CPP ON STUDENTS’ LIKELIHOOD 
OF STUDYING STEM AFTER YEAR 10

NUMBER

PROPORTION

No impact

2

4%

Slight Impact (e.g., a few students)

6

13%

Moderate impact (e.g., some students)

34

74%

Significant impact (e.g., a lot of students)

1

2%

Unsure

3

7%

(n=46)







STEM CPP activities available at the school and their impact on student interest in STEM

COMPLETING THE INQUIRY-BASED LEARNING PROJECT

NUMBER

PROPORTION

Has not changed their interest

2

5%

Has made them less interested in STEM

2

5%

Has made them a bit more interested in STEM

21

55%

Has made them much more interested in STEM

13

34%

(n=38)





INTERACTING WITH INDUSTRY MENTORS

NUMBER

PROPORTION

Has not changed their interest

6

18%

Has made them less interested in STEM

0

0%

Has made them a bit more interested in STEM

18

55%

Has made them much more interested in STEM

9

27%

(n=33)





ATTENDING THE SHOWCASE EVENT

NUMBER

PROPORTION

Has not changed their interest

2

6%

Has made them less interested in STEM

1

3%

Has made them a bit more interested in STEM

13

37%

Has made them much more interested in STEM

19

54%

(n=35)





VISITING A LOCAL STEM INDUSTRY OR WORKSITE

NUMBER

PROPORTION

Has not changed their interest

2

11%

Has made them less interested in STEM

0

0%

Has made them a bit more interested in STEM

8

42%

Has made them much more interested in STEM

9

47%

(n=19)





COMPLETING STEM WORK EXPERIENCE

NUMBER

PROPORTION

Has not changed their interest

3

19%

Has made them less interested in STEM

0

0%

Has made them a bit more interested in STEM

7

44%

Has made them much more interested in STEM

6

38%

(n=16)







Impact of STEM CPP activities on students’ skills and awareness of STEM

WORKING IN A TEAM WITH THEIR PEERS

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

6

16%

Moderate improvement

15

41%

Significant improvement

16

43%

(n=37)





COMMUNICATING THEIR IDEAS EFFECTIVELY TO OTHERS

NUMBER

PROPORTION

No improvement

1

3%

Some improvement

6

15%

Moderate improvement

13

33%

Significant improvement

19

49%

(n=39)





USING CRITICAL THINKING TO REASON AND DRAW CONCLUSIONS

NUMBER

PROPORTION

No improvement

1

3%

Some improvement

7

18%

Moderate improvement

16

42%

Significant improvement

14

37%

(n=38)





CREATIVELY THINKING ABOUT WAYS TO SOLVE PROBLEMS

NUMBER

PROPORTION

No improvement

1

3%

Some improvement

4

10%

Moderate improvement

13

33%

Significant improvement

21

54%

(n=39)





AWARENESS OF STEM EDUCATION PATHWAYS 

NUMBER

PROPORTION

No improvement

1

3%

Some improvement

12

30%

Moderate improvement

14

35%

Significant improvement

13

33%

(n=40)







AWARENESS OF POTENTIAL STEM CAREERS

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

12

31%

Moderate improvement

13

33%

Significant improvement

14

36%

(n=39)





Impact of STEM CPP activities that teachers have participated in

DELIVERING THE INQUIRY-BASED LEARNING PROJECT

NUMBER

PROPORTION

Has not really made a difference

4

11%

Has made some difference

17

47%

Has made a significant difference

15

42%

(n=36)





ATTENDING THE TEACHER PROFESSIONAL LEARNING WORKSHOP

NUMBER

PROPORTION

Has not really made a difference

4

12%

Has made some difference

15

45%

Has made a significant difference

14

42%

(n=33)





ATTENDING A TEACHER NETWORKING EVENT

NUMBER

PROPORTION

Has not really made a difference

3

15%

Has made some difference

11

55%

Has made a significant difference

6

30%

(n=20)





ENGAGING WITH INDUSTRY MENTORS

NUMBER

PROPORTION

Has not really made a difference

3

12%

Has made some difference

14

54%

Has made a significant difference

9

35%

(n=26)







DIFFICULTY OF EMBEDDING THE INQUIRY-BASED LEARNING 
PROJECT INTO NORMAL TEACHING PRACTICE

NUMBER

PROPORTION

Easy

17

43%

Very easy

5

13%

Difficult 

14

35%

Very difficult

0

0%

Unsure

4

10%

(n=40)





Recommending STEM CPP

LIKELIHOOD OF SCHOOL CONTINUING WITH STEM CPP

NUMBER

PROPORTION

Likely

14

35%

Very likely

21

53%

Possible

4

10%

Unlikely

1

3%

(n=40)





LIKELIHOOD OF RECOMMENDING STEM CPP TO OTHER TEACHERS

NUMBER

PROPORTION

Likely

17

43%

Very likely

19

48%

Possible

4

10%

Unlikely

0

0%

(n=40)







Appendix D. STEM CPP industry mentor survey data 2022

INDUSTRY/SECTOR OF ORGANISATION

NUMBER

PROPORTION

Manufacturing and Advanced manufacturing

9

26%

Education

5

14%

Government

5

14%

Water treatment

2

6%

Aviation

2

6%

Construction

2

6%

Engineering

2

6%

Astronomy

1

3%

Telecommunications

1

3%

Automotive

1

3%

Community

1

3%

Waste management

1

3%

Health

1

3%

Information technology

1

3%

Entertainment rides

1

3%

(n=35)





OVERALL EXPERIENCE WITH STEM CPP

NUMBER

PROPORTION

Far short of your expectations

4

13%

Short of your expectations

4

13%

Met your expectations

15

47%

Exceeded your expectations

6

19%

Greatly exceeded your expectations

3

9%

(n=32)





DIFFICULTY OF BECOMING AN INDUSTRY MENTOR WITH STEM CPP

NUMBER

PROPORTION

Very easy

11

34%

Easy

15

47%

Difficult

2

6%

Very difficult

0

0%

Unsure 

4

13%

(n=32)







Activities undertaken as industry mentor with STEM CPP

MENTORING TEACHERS AND/OR STUDENTS

NUMBER

PROPORTION

Yes 

15

56%

No

12

44%

(n=27)





PRESENTING AT A CAREERS DAY

NUMBER

PROPORTION

Yes 

8

31%

No

18

69%

(n=26)





CONDUCTING AN INDUSTRY SITE VISIT

NUMBER

PROPORTION

Yes 

8

31%

No

18

69%

(n=26)





ATTENDING THE VIRTUAL SHOWCASE

NUMBER

PROPORTION

Yes 

11

41%

No

16

59%

(n=27)





OTHER

NUMBER

PROPORTION

Yes 

11

52%

No

10

48%

(n=21)





Confidence in supporting teachers and/or students through activities

MENTORING TEACHERS AND/OR STUDENTS

NUMBER

PROPORTION

Somewhat confident

4

27%

Confident

11

73%

Not applicable/unsure

0

0%

(n=15)





PRESENTING AT A CAREERS DAY

NUMBER

PROPORTION

Somewhat confident

0

0%

Confident

8

100%

Not applicable/unsure

0

0%

(n=8)







CONDUCTING AN INDUSTRY SITE VISIT

NUMBER

PROPORTION

Somewhat confident

0

0%

Confident

7

100%

Not applicable/unsure

0

0%

(n=7)





ATTENDING THE VIRTUAL SHOWCASE

NUMBER

PROPORTION

Somewhat confident

1

9%

Confident

9

82%

Not applicable/unsure

1

9%

(n=11)





OTHER

NUMBER

PROPORTION

Somewhat confident

2

20%

Confident

8

80%

Not applicable/unsure

0

0%

(n=10)





Industry mentor support from STEM CPP 

I RECEIVED ENOUGH SUPPORT TO BE AN EFFECTIVE 
INDUSTRY MENTOR WITH STEM CPP

NUMBER

PROPORTION

Agree

18

58%

Strongly agree

5

16%

Disagree

4

13%

Strongly disagree

2

6%

Unsure

2

6%

(n=31)





Change in engagement of industry mentor’s organisation with other stakeholders

SCHOOLS

NUMBER

PROPORTION

It has remained the same

17

55%

It has increased

11

35%

It has increased significantly

1

3%

Not applicable/unsure

2

6%

(n=31)







INDUSTRY

NUMBER

PROPORTION

It has remained the same

21

68%

It has increased

6

19%

It has increased significantly

0

0%

Not applicable/unsure

4

13%

(n=31)





GOVERNMENT REPRESENTATIVES

NUMBER

PROPORTION

It has remained the same

22

71%

It has increased

4

13%

It has increased significantly

0

0%

Not applicable/unsure

5

16%

(n=31)





LOCAL COUNCIL

NUMBER

PROPORTION

It has remained the same

23

74%

It has increased

4

13%

It has increased significantly

0

0%

Not applicable/unsure

4

13%

(n=31)





COMMUNITY MEMBERS

NUMBER

PROPORTION

It has remained the same

21

70%

It has increased

5

17%

It has increased significantly

1

3%

Not applicable/unsure

3

10%

(n=30)







Recommending STEM CPP

LIKELIHOOD OF CONTINUING AS INDUSTRY MENTOR

NUMBER

PROPORTION

Likely

10

32%

Very likely

16

52%

Possible

3

10%

Unlikely

1

3%

Unsure

1

3%

(n=31)





LIKELIHOOD OF RECOMMENDING STEM CPP TO OTHER COLLEAGUES

NUMBER

PROPORTION

Likely

9

29%

Very likely

13

42%

Possible

6

19%

Unlikely

3

10%

Unsure

0

0%

(n=31)







Appendix E. Generation STEM Links student survey data 2022

GENDER

NUMBER

PROPORTION

Female

3

25%

Male

9

75%

(n=12)





ABORIGINAL AND/OR TORRES STRAIT ISLANDER

NUMBER

PROPORTION

Yes

0

0%

No

12

100%





DISABILITY

NUMBER

PROPORTION

Yes

1

8%

No

11

92%





CULTURALLY AND LINGUISTICALLY DIVERSE

NUMBER

PROPORTION

Yes

3

25%

No

9

75%





Work placement allocation

INDUSTRY

NUMBER

PROPORTION

Professional and financial services

3

25%

Mining

2

17%

Sustainability

1

8%

Advanced manufacturing

1

8%

Agribusiness and food

1

8%

Other

4

33%





HOURS WORKED IN A WEEK

NUMBER

PROPORTION

10 hours or less

1

8%

Between 10 and 20 hours

2

17%

Between 20 and 35 hours

4

33%

More than 35 hours

5

42%





Students’ experience with their work placement

I FOUND THE APPLICATION PROCESS EASY

NUMBER

PROPORTION

Agree

2

15%

Strongly agree

9

69%

Disagree

1

8%

Unsure/hard to say

1

8%







I RECEIVED TIMELY INFORMATION ABOUT THE APPLICATION PROCESS

NUMBER

PROPORTION

Agree

4

31%

Strongly agree

7

54%

Disagree 

1

8%

Unsure/hard to say

1

8%





I WAS SATISFIED WITH THE INTERVIEW AND MATCHING PROCESS

NUMBER

PROPORTION

Agree

3

23%

Strongly agree

7

54%

Disagree 

0

0%

Unsure/hard to say

3

23%





Student experience throughout the work placement

OVERALL EXPERIENCE WITH THE WORK PLACEMENT

NUMBER

PROPORTION

Short of my expectations

1

8%

Met my expectations

4

33%

Exceeded my expectations

7

58%





I HAD ADEQUATE ACCESS AND SUPPORT FROM MY SUPERVISOR

NUMBER

PROPORTION

Agree

3

25%

Strongly agree

9

75%

Disagree

0

0%

Unsure/hard to say

0

0%





I WAS GIVEN MEANINGFUL WORK, TASKS AND ACTIVITIES

NUMBER

PROPORTION

Agree

2

17%

Strongly agree

7

58%

Disagree

2

17%

Unsure/hard to say

1

8%





I WAS ABLE TO APPLY MY THEORETICAL 
KNOWLEDGE IN A WORK‑BASED SETTING

NUMBER

PROPORTION

Agree

7

58%

Strongly agree

5

42%

Disagree 

0

0%

Unsure/hard to say

0

0%







I FELT I WAS GIVEN APPROPRIATE LEVELS OF RESPONSIBILITY

NUMBER

PROPORTION

Agree

5

42%

Strongly agree

6

50%

Disagree 

0

0%

Unsure/hard to say

1

8%





I WAS GIVEN WORK OPPORTUNITIES THAT 
WERE RELEVANT TO MY FIELD OF STUDY

NUMBER

PROPORTION

Agree

7

58%

Strongly agree

5

42%

Disagree 

0

0%

Unsure/hard to say

0

0%





THE WORK I DID CONTRIBUTED TO THE WIDER TEAM

NUMBER

PROPORTION

Agree

3

25%

Strongly agree

7

58%

Disagree 

0

0%

Unsure/hard to say

2

17%





Student confidence in their STEM skills and knowledge AFTER their work placement

I FEEL CONFIDENT ABOUT MY TERTIARY STUDIES

NUMBER

PROPORTION

Strongly agree (5)

3

33%

Agree (4)

2

12%

Neither agree nor disagree (3)

3

33%

Disagree (2)

1

11%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I AM CONFIDENT ABOUT MY CORE WORK-READINESS SKILLS 
(E.G., COMMUNICATION SKILLS, WORKING IN A TEAM, PLANNING ETC.)

NUMBER

PROPORTION

Strongly agree (5)

3

33%

Agree (4)

4

44%

Neither agree nor disagree (3)

2

22%

Disagree (2)

0

0%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%







I AM CONFIDENT ABOUT MY TECHNICAL STEM SKILLS

NUMBER

PROPORTION

Strongly agree (5)

3

33%

Agree (4)

3

33%

Neither agree nor disagree (3)

2

22%

Disagree (2)

0

0%

Strongly disagree (1)

1

11%

Unsure/hard to say (0)

0

0%





I KNOW ABOUT MY STEM AREA OF INTEREST

NUMBER

PROPORTION

Strongly agree (5)

3

33%

Agree (4)

2

22%

Neither agree nor disagree (3)

2

22%

Disagree (2)

2

22%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I KNOW ABOUT THE REALITY OF WORKING IN STEM

NUMBER

PROPORTION

Strongly agree (5)

4

44%

Agree (4)

2

22%

Neither agree nor disagree (3)

3

33%

Disagree (2)

0

0%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I WANT TO PURSUE WORKING IN STEM

NUMBER

PROPORTION

Strongly agree (5)

4

44%

Agree (4)

2

22%

Neither agree nor disagree (3)

3

33%

Disagree (2)

0

0%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I HAVE PROFESSIONAL WORKING RELATIONSHIPS

NUMBER

PROPORTION

Strongly agree (5)

2

22%

Agree (4)

3

33%

Neither agree nor disagree (3)

4

44%

Disagree (2)

0

0%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%







Student confidence in their STEM skills and knowledge BEFORE their work placement

I FEEL CONFIDENT ABOUT MY TERTIARY STUDIES

NUMBER

PROPORTION

Strongly agree (5)

2

22%

Agree (4)

3

33%

Neither agree nor disagree (3)

1

11%

Disagree (2)

2

22%

Strongly disagree (1)

1

11%

Unsure/hard to say (0)

0

0%





I AM CONFIDENT ABOUT MY CORE WORK-READINESS SKILLS 
(E.G., COMMUNICATION SKILLS, WORKING IN A TEAM, PLANNING ETC.)

NUMBER

PROPORTION

Strongly agree (5)

1

11%

Agree (4)

2

22%

Neither agree nor disagree (3)

2

22%

Disagree (2)

3

33%

Strongly disagree (1)

1

11%

Unsure/hard to say (0)

0

0%





I AM CONFIDENT ABOUT MY TECHNICAL STEM SKILLS

NUMBER

PROPORTION

Strongly agree (5)

0

0%

Agree (4)

2

22%

Neither agree nor disagree (3)

4

44%

Disagree (2)

3

33%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I KNOW ABOUT MY STEM AREA OF INTEREST

NUMBER

PROPORTION

Strongly agree (5)

1

11%

Agree (4)

2

22%

Neither agree nor disagree (3)

4

44%

Disagree (2)

2

22%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%







I KNOW ABOUT THE REALITY OF WORKING IN STEM

NUMBER

PROPORTION

Strongly agree (5)

1

13%

Agree (4)

0

0%

Neither agree nor disagree (3)

2

25%

Disagree (2)

4

50%

Strongly disagree (1)

1

13%

Unsure/hard to say (0)

0

0%





I WANT TO PURSUE WORKING IN STEM

NUMBER

PROPORTION

Strongly agree (5)

3

33%

Agree (4)

2

22%

Neither agree nor disagree (3)

3

33%

Disagree (2)

1

11%

Strongly disagree (1)

0

0%

Unsure/hard to say (0)

0

0%





I HAVE PROFESSIONAL WORKING RELATIONSHIPS

NUMBER

PROPORTION

Strongly agree (5)

0

0%

Agree (4)

1

13%

Neither agree nor disagree (3)

2

25%

Disagree (2)

1

13%

Strongly disagree (1)

1

13%

Unsure/hard to say (0)

3

38%





Value of work placement interactions for students

PARTICIPATING IN A WORKPLACE INDUCTION SESSION

NUMBER

PROPORTION

Slightly valuable

3

25%

Moderately valuable

5

42%

Very valuable

2

17%

Not applicable/this did not happen

2

17%





PARTICIPATING IN ONSITE TRAINING

NUMBER

PROPORTION

Slightly valuable

1

8%

Moderately valuable

4

33%

Very valuable

6

50%

Not applicable/this did not happen

1

8%







INFORMAL MENTORING FROM YOUR SUPERVISOR

NUMBER

PROPORTION

Slightly valuable

0

0%

Moderately valuable

1

8%

Very valuable

11

92%

Not applicable/this did not happen

0

0%





RECEIVING CONSTRUCTIVE FEEDBACK ABOUT YOUR PERFORMANCE

NUMBER

PROPORTION

Slightly valuable

0

%

Moderately valuable

1

8%

Very valuable

11

92%

Not applicable/this did not happen

0

%





INTERACTIONS WITH OTHER TEAM MEMBERS

NUMBER

PROPORTION

Slightly valuable

2

17%

Moderately valuable

1

8%

Very valuable

9

75%

Not applicable/this did not happen

0

0%





SUPPORT FROM THE CSIRO PROGRAM TEAM DURING THE PLACEMENT

NUMBER

PROPORTION

Slightly valuable

3

25%

Moderately valuable

5

42%

Very valuable

4

33%

Not applicable/this did not happen

0

0%





Recommending Generation STEM Links

LIKELIHOOD OF RECOMMENDING GENERATION STEM LINKS TO 
OTHER TERTIARY STUDENTS STUDYING A STEM QUALIFICATION

NUMBER

PROPORTION

Likely

1

8%

Extremely likely

11

92%

Unlikely

0

0%







Appendix F. Generation STEM Links supervisor survey data 2022

INDUSTRY OF BUSINESS

NUMBER

PROPORTION

Manufacturing

4

44%

Information technology

3

33%

Mining

1

11%

Software solutions

1

11%





DIFFICULTY OF BECOMING AN INDUSTRY 
SUPERVISOR WITH STEM LINKS

NUMBER

PROPORTION

Easy

5

63%

Very easy

3

38%





SATISFACTION WITH STUDENT PERFORMANCE

NUMBER

PROPORTION ion

Moderately satisfied

4

50%

Very satisfied

1

38%

Extremely satisfied

3

13%





OVERALL EXPERIENCE WITH STEM LINKS STEM Links

NUMBER

PROPORTION

Short of your expectations

2

25%

Met your expectations

3

38%

Exceeded your expectations

1

13%

Greatly exceeded your expectations

2

25%





Change in student’s skills and awareness of STEM

WORKING COLLABORATIVELY IN A TEAM WITH THEIR COLLEAGUES

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

2

25%

Moderate improvement

2

25%

Significant improvement

3

38%

Unsure/hard to say

1

13%





COMMUNICATING THEIR IDEAS EFFECTIVELY TO OTHERS

NUMBER

PROPORTION

No improvement

1

13%

Some improvement

1

13%

Moderate improvement

3

38%

Significant improvement

2

25%

Unsure/hard to say

1

13%







USING THEIR TECHNICAL STEM SKILLS TO COMPLETE TASKS

NUMBER

PROPORTION

No improvement

1

13%

Some improvement

0

0%

Moderate improvement

3

38%

Significant improvement

4

50%

Unsure/hard to say

0

0%





USING CRITICAL THINKING TO REASON AND DRAW CONCLUSIONS

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

2

25%

Moderate improvement

1

13%

Significant improvement

5

63%

Unsure/hard to say

0

0%





CREATIVELY THINKING ABOUT WAYS TO SOLVE PROBLEMS

NUMBER

PROPORTION

No improvement

1

13%

Some improvement

2

25%

Moderate improvement

1

13%

Significant improvement

4

50%

Unsure/hard to say

0

0%





AWARENESS OF STEM EDUCATION PATHWAYS

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

0

0%

Moderate improvement

3

38%

Significant improvement

0

0%

Unsure/hard to say

5

63%





AWARENESS OF POTENTIAL STEM CAREERS

NUMBER

PROPORTION

No improvement

0

0%

Some improvement

0

0%

Moderate improvement

3

38%

Significant improvement

0

0%

Unsure/hard to say

5

63%







Supervising early career students

PREVIOUS EXPERIENCE WITH SUPERVISING EARLY CAREER STUDENTS

NUMBER

PROPORTION

I sometimes supervised early career students

2

25%

I regularly supervised early career students

5

63%

I have never supervised early career students

1

13%





IMPACT OF GENERATION STEM LINKS ON CONFIDENCE 
IN SUPERVISING EARLY CAREER STUDENTS

NUMBER

PROPORTION

No impact

1

13%

Slight impact

2

25%

Moderate impact

2

25%

Significant impact

3

38%





Impact of Generation STEM Links on business engagement

CHANGE IN BUSINESS ENGAGEMENT WITH THE TERTIARY SECTOR

NUMBER

PROPORTION

It has improved somewhat

4

50%

It has remained the same

1

13%

It has improved significantly 

2

25%

Unsure/hard to say

1

13%





Recommending Generation STEM Links

LIKELIHOOD OF CONTINUING AS A BUSINESS 
WITH GENERATION STEM LINKS

NUMBER

PROPORTION

Possible

2

25%

Likely

3

38%

Extremely likely

3

38%

Unlikely 

0

0%





LIKELIHOOD OF RECOMMENDING GENERATION STEM 
LINKS TO OTHER BUSINESSES AND/OR COLLEAGUES

NUMBER

PROPORTION

Possible

2

25%

Likely

2

25%

Extremely likely

4

50%

Unlikely 

0

0%







Contact us

1300 363 400
+61 3 9545 2176
csiro.au/contact
csiro.au


For further information

Education and Outreach

Christopher Banks

+61 7 3833 5999

christopher.banks@csiro.au

csiro.au/education

As Australia’s national science agency, 
CSIRO is solving the greatest 
challenges through innovative 
science and technology.

CSIRO. Creating a better future 
for everyone.