Indigenous STEM Education Project Science Pathways for Indigenous Communities Case study evaluation report August 2020 Citation Cherry, K., Banks, C. Gilbert, & C. Fidler, J. (2020). Science Pathways for Indigenous Communities case study evaluation report. CSIRO. Canberra. Copyright © Commonwealth Scientific and Industrial Research Organisation 2020. Acknowledgements Acknowledgement of Country Aboriginal and Torres Strait Islander peoples have longstanding scientific-knowledge traditions. These traditions have developed knowledge about the world through: • observation, using all the senses • prediction and hypothesis • testing (trial and error) and • making generalisations within specific contexts. These scientific methods have been practised and transmitted from one generation to the next and contribute to ways of knowing the world that are unique as well as complementary to Western scientific knowledge. A deep respect for these Aboriginal and Torres Strait Islander cultural practices and knowledge underpins the philosophy and practice of the Indigenous STEM Education Project. Recognition of traditional contexts for technologies and concepts and their application in the past, present and future—including supporting modern STEM career pathways for Aboriginal and Torres Strait Islander students—reaffirm the ingenuity and creativity of Aboriginal and Torres Strait Islander peoples' knowledge systems. The Indigenous STEM Education Project team acknowledges the Traditional Owners of the lands with whom this Project is collaborating and their vibrant living cultures and knowledge systems. In particular, we acknowledge the Martu, Nyangumarta Luritja knowledge holders from the three schools included in the case study. We pay our respects to Elders past and present, and we thank all community members who are providing the leadership to ensure meaningful and effective engagement with Aboriginal and Torres Strait Islander communities for the six distinct, but complementary, STEM education programs that make up this Project. CSIRO acknowledges that Aboriginal and Torres Strait Islander peoples make extraordinary contributions to Australia in cultural, economic and scientific domains; for example, incorporating Indigenous knowledges of ecological and social systems is vital to the achievement of sustainable development. Other acknowledgements CSIRO wishes to acknowledge the invaluable contribution of Aboriginal and Torres Strait Islander scientists, educators and program leaders—without their knowledge and leadership, the development and implementation of the Indigenous STEM Education Project would not have been possible. Specifically, CSIRO gratefully acknowledges the invaluable contributions of students, teachers, principals, parents, rangers, Elders, Traditional Owners and community members who were involved in the case study visits, whose voices and perspectives made the research possible. In addition, CSIRO acknowledges the advice and guidance of the reference groups for this research. Their wisdom in relation to respectfully engaging Aboriginal and Torres Strait Islander students in the research was enormously helpful and is appreciated. The authors would like to thank the external peer reviewer, Dr Libby Lee-Hammond from Murdoch University, and Renate Hays and Dr Tom Keenan from the CSIRO Planning, Performance and Impact unit for their peer review of this report. Their valuable suggestions made a significant contribution. The Project Steering Committee’s leadership of the Indigenous STEM Education Project is gratefully acknowledged. We also thank the Science Pathways for Indigenous Communities program team, David Broun, Cameron Hugh, Fiona Webb, Meg Mooney and Chris Deslandes, for their constructive feedback and vital assistance in planning and conducting the research activities. We thank the principals of the three schools that were visited for the case study; their invaluable assistance and advice made the research possible. Finally, CSIRO acknowledges the contributions of former members of the Program team and evaluation teams who led or supported the Program monitoring and evaluation methodologies, data collection and analysis, including Marian Heard, Therese Postma, Dr Kirsten Sadler and Dr Michael Tynan. A special acknowledgement goes to Dr Prem Mudhan, who conducted one of the site visits while a member of the Monitoring and Evaluation team. Researcher acknowledgements The Indigenous STEM Education Project Monitoring and Evaluation team was granted the time and resources to undertake an evaluation of the Science Pathways for Indigenous Communities program. This privileged position allowed us to build new knowledge with a range of people including Aboriginal and/or Torres Strait Islander knowledge holders, rangers, students, families and school teachers about the benefits of the Science Pathways for Indigenous Communities program in some remote Indigenous communities in Western Australia and the Northern Territory. This report identifies the shared outcomes, goals and hopes of the researchers, program staff and communities implementing the Science Pathways for Indigenous Communities program and captures the importance of working together and valuing the contributions, knowledge and skills of all participants involved in Two-way Science. While the CSIRO Monitoring and Evaluation team includes First Nations and non-Indigenous members, it is acknowledged that this evaluation report is written primarily by non-Indigenous researchers who have a Western standpoint and expertise in evaluation methodology. The Aboriginal and/or Torres Strait Islander people who participated in this Project, who are experts in their own lives, cultures and knowledges, guided the evaluation and contributed their time generously by teaching and sharing with the researchers throughout the evaluation. A strengths-based, reflexive methodology was used in this evaluation, and the researchers are accountable to all the people who shared their time and knowledge. It is hoped that the combined knowledge and skills of all involved contribute to the goals of the communities that participated and create positive social change more broadly. Artwork CSIRO gratefully acknowledges the artists who allowed the use of their artwork in this report. The artists are from one of the communities involved in the Science Pathways for Indigenous Communities program, and their artwork is deeply meaningful to the evaluation team and this report. Contents Acknowledgements..................................................................................................................................c Acknowledgement of Country.................................................................................................................................................c Other acknowledgements........................................................................................................................................................c Researcher acknowledgements..............................................................................................................................................d Artwork.....................................................................................................................................................................................d Glossary ......................................................................................................................................................................iii Acronyms ...................................................................................................................................................................iv EXECUTIVE SUMMARY..........................................................................................................................................v INTRODUCTION.........................................................................................................................................................1 History of the Program.............................................................................................................................................................1 The Program design..................................................................................................................................................................3 Program outputs.......................................................................................................................................................................5 EVALUATION..............................................................................................................................................................6 Scope and purpose....................................................................................................................................................................6 Methodology.............................................................................................................................................................................7 Researcher reflection and position........................................................................................................................................10 Framework for analysis...........................................................................................................................................................11 Ethical considerations.............................................................................................................................................................11 Potential limitations of the evaluation..................................................................................................................................12 FINDINGS....................................................................................................................................................................13 Outcome area 1: Strong, effective partnerships established with schools and other stakeholders.................................13 Outcome area 2: Increased community and parental engagement....................................................................................18 Outcome area 3: The Program’s Two-way Science learning approach and resources are culturally responsive and embedded across participating school curriculums.....................................................................................................24 Outcome area 4: Increased teacher capacity in science education using on Country contexts........................................31 Outcome area 5: Increased aspiration (of students), sense of value and school belonging18 .........................................39 Outcome area 6: Increased student engagement and attendance.....................................................................................44 Outcome area 7: Enhanced student results...........................................................................................................................49 FINDINGS: FROM IMPLEMENTATION............................................................................................................56 Contextual factors...................................................................................................................................................................56 Program factors.......................................................................................................................................................................58 DISCUSSION............................................................................................................................................................60 Key themes..............................................................................................................................................................................60 Program logic..........................................................................................................................................................................67 Challenges and barriers..........................................................................................................................................................68 CONCLUSION AND RECOMMENDATIONS.................................................................................................70 Summary of achievements.....................................................................................................................................................70 Recommendations..................................................................................................................................................................73 REFERENCES...........................................................................................................................................................74 Appendix A – Impact Pathways.........................................................................................................................81 Appendix B – Science Pathways for Indigenous Communities Participating Schools................83 Appendix C – Interview Questions..................................................................................................................84 Appendix D – Cultural Standards Mapping..................................................................................................85 Appendix E – Strengths-Based Practice Considerations.........................................................................91 Figures Figure 1. New cluster model for the Program.................................................................................................................................2 Figure 2. Two-way Science teaching and learning cycle.................................................................................................................4 Figure 3. Knowledge Map.................................................................................................................................................................9 Figure 4. Term plan with cross-curriculum integration................................................................................................................28 Figure 5. Pintupi Luritja seasons calendar (with English version)................................................................................................30 Figure 6. Mean student attendance rates: Program schools vs comparison schools................................................................45 Figure 7. Digging for Watja.............................................................................................................................................................51 Figure 8. Student work sample (used with permission)...............................................................................................................52 Figure 9. Land care example...........................................................................................................................................................54 Figure 10. Visual representation of the Program..........................................................................................................................66 Tables Table 1. Summary of achievement of Outcome areas....................................................................................................................vi Table 2. Program variations across jurisdictions.............................................................................................................................4 Table 3. Number of students engaged by year................................................................................................................................5 Table 4. Interview participants and total program participants by stakeholder group.............................................................10 Table 5. Curriculum outcomes for Digging for Watja Two-way Science activity........................................................................51 Table 6. Summary of achievements against outcome areas.........................................................................................................71 Community: the local Aboriginal community where there are close connections with family, culture, and language. Country: the regional lands and waters associated with traditional owners or clan groups that they are responsible for. This term is used at a regional scale rather than meaning Australia as a whole country. The term is capitalised through-out this report to signify its deep significance. Ecology: the branch of biology that deals with the relationships of living things to one another and to their physical surroundings. Elder: someone who has gained recognition as a custodian of knowledge and lore and who has permission to disclose knowledge and beliefs. Indigenous ecological knowledge: cultural understanding of the ecology of the Country. Science Pathways for Indigenous Communities is primarily focused on Indigenous ecological knowledge, but includes a range of knowledges. Learning on Country: an activity outside the classroom where Aboriginal people are teaching aspects of culture, Country and language. ‘Country’ is capitalised throughout this report to signify the importance of Country in the context of Aboriginal and Torres Strait Islander culture, Two-way Science and remote education. Place-based model: a program or initiative that uses methodologies and strategies depending on the local context and aims to address issues that exist at the community level. Science Pathways for Indigenous Communities: a Two-way Science project for selected remote WA and NT desert communities, part of the broader Indigenous STEM Education Project funded by the BHP Foundation and delivered by the CSIRO. Traditional Owner: an Aboriginal and/or Torres Strait Islander person who is recognised by their community as having ownership and knowledge of a particular area or ‘Country’. Two-way Science: a science pedagogy for remote Aboriginal students that links Indigenous knowledge with Western science. In the context of the Science Pathways for Indigenous Communities program, it is primarily Indigenous ecological knowledge. Glossary ACARA Australia Curriculum, Assessment and Reporting Authority ACSHE Australian Curriculum, Science as a Human Endeavour AITSL Australian Institute for Teaching and School Leadership ASTA Australian Science Teachers Association AT Assistant Teacher CONASTA Conference of the Australian Science Teachers Association CSIRO Commonwealth Scientific and Industrial Research Organisation EAL/D English as an additional language or dialect HASS Humanities and Social Sciences ICSEA Index of Community Socio-Educational Advantage IEK Indigenous Ecological Knowledge ILC Indigenous Language and Culture IPA Indigenous Protected Area NAPLAN The National Assessment Program – Literacy and Numeracy NT Northern Territory PP Pre-primary STEM Science, Technology, Engineering and Mathematics TAFE Technical and Further Education TPD Teacher Professional Development VET Vocational Education and Training WA Western Australia Acronyms Science Pathways for Indigenous Communities (the Program) supports Aboriginal communities and schools in working together to develop an integrated Two-way Science learning program that connects Indigenous ecological knowledge with Western science and the Australian Curriculum. The Program aims to develop a culturally responsive model for education to address systemic educational disadvantage faced by Indigenous students whose culture, language and Country have been historically excluded from mainstream education pedagogy and practice. The Program aims to provide a Two-way Science model for remote education; facilitating engagement with Aboriginal leaders in education programs in their local schools, building on students’ strengths to increase engagement at school and increasing the capacity of educators and program partners to deliver Two-way Science in remote schools. The Program utilises a two-way approach to learning. As outlined in the Program resources, the two-way approach is ‘a way of teaching and learning that connects Aboriginal knowledge of the environment to Western science and the school curriculum’ (Deslandes et al., 2019, p. 6). Learning on Country is central to this approach, as Aboriginal knowledge guides learning, and Aboriginal people are involved in all processes of a teaching and learning cycle. A multi-site case study evaluation of the Program was conducted to identify implementation learnings and assess progress towards outcomes, as described in an initial Impact Pathway. The evaluation question was: To what extent did the program produce or contribute to the intended outcomes in the short to medium term? Three remote Aboriginal schools/communities participated in the evaluation from across the Northern Territory and Western Australia. Qualitative data was collected at several points in time during 2018 and 2019. The findings were primarily based on interviews and focus groups with 90 participants, comprising students, families, community members, educators, program partners and program staff. The key evaluation findings include: Strong, effective partnerships were established with schools and other stakeholders, with program staff contributing to increased structure, resourcing and planning; and modelling respectful relationship development skills to support educators in making local connections. Building genuine relationships and developing strong partnerships among the Program, schools, Elders and external organisations were critical to achieving all program outcomes such as improved teacher capacity, student wellbeing and learning outcomes, as well as the achievement of Elders’ and Traditional Owners’ education goals. The Program increased student and community member engagement in education. For students, engagement increased considerably during Two-way Science activities on Country and in the classroom. For the community and family members, engagement increased through Aboriginal leadership in education, incorporating languages and cultures into education and creating opportunities for individuals’ wisdom and strengths to be acknowledged and shared across the broader community. Overall, educator confidence and capacity to plan and undertake Two-way Science (and teaching science in general) using on Country contexts improved as a result of the Program. This was particularly apparent when Aboriginal educators had the opportunity to collaborate in language and when non-Indigenous teachers were supported in connecting with community members to deliver a more culturally responsive curriculum. The Program improved school capacity to embed Aboriginal ecological knowledge and Western science into the curriculum due to a more frequent and effective focus on science and resources that were place-based and easy-to-use, creating cross-curriculum efficiencies for teachers. EXECUTIVE SUMMARY Based on the three case study locations involved in the evaluation research, a summary of the achievements against the Program’s outcome areas is presented in Table 1. Achievements against outcome areas were assessed based on the substantial amount of qualitative data collected, and at this point in the development of the Program, they were judged as either achieved or not achieved. Further evaluation work would need to be conducted to assess achievements in more detail. To help understand the assessments, an estimate of the level of evidence and the type of evidence is also provided. Level of evidence refers to the amount of evidence that supported the assessment. A ‘High’ level of evidence means that a large proportion of evaluation participants provided information that the outcome had been achieved. The ‘type’ of evidence refers to whether the evaluation participants provided ‘direct’ confirmation that the outcome had been achieved, or whether it was based on more anecdotal or second-hand evidence, which would require further data to triangulate and confirm. Table 1. Summary of achievement of Outcome areas OUTCOME AREA ACHIEVED LEVEL OF EVIDENCE* TYPE OF EVIDENCE* 1. Strong, effective partnerships established with schools and other stakeholders 3 High Direct 2. Increased community and parental engagement 3 High Direct 3. The Program’s Two-way Science learning approach and resources are culturally responsive and embedded across participating school curriculums. 3 High Direct 4. Increased teacher capacity in Two-way Science using on Country contexts and cultural competence 3 High Direct 5. Increased aspiration (in students), sense of value and school belonging 3 High Direct 6. Increased student engagement and attendance 3 High Direct 7. Enhanced student results 3 Medium Indirect 8. Centres of excellence in two-way STEM education x** None n/a 9. University teacher training using Teacher Professional Development, Department of Education extending the model to other remote schools x** None n/a *Level of evidence refers to the amount of evidence available to ascertain whether the outcome was achieved (high, medium, low); Type of evidence refers to whether the evidence was direct (directly observed or reported by relevant stakeholders) or indirect (inferred from secondary sources or stakeholders). **A program decision was made not to pursue these outcome areas. The implementation and success of the Program were hindered by several challenges, many that were common to remote education in general, including schools lacking resources, such as appropriately sized vehicles to transport students on Country, staff turnover and the other commitments of community partners that work with schools, such as funerals and cultural duties. INTRODUCTION This evaluation case study (the evaluation) of the Science Pathways for Indigenous Communities program (the Program) reviews the extent to which its intended outcomes were achieved (See Appendix A for the Program Impact Pathway as at 2018) and identifies emerging outcome areas driven by participating schools and communities. The report also reviews implementation learnings that could inform future or similar program activities. The Program is one of several programs that comprise the Indigenous STEM Education Project, which is funded by the BHP Foundation and delivered by CSIRO. The overarching goal of the Indigenous STEM Education Project is to build on the strengths of Aboriginal and Torres Strait Islander2 young people with supported pathways that improve the participation and achievement of Aboriginal and Torres Strait Islander students in STEM subjects. The programs cater to the diversity of Aboriginal and Torres Strait Islander students as they progress through primary, secondary and tertiary education and into employment. At the beginning of this report is an Executive Summary of the Program and the evaluation, focusing on the observable outcomes that were achieved and some that were not achieved. The Introduction of this report outlines the Program and its intended outcomes. The Evaluation section outlines the case study methodology and the research team positioning that informed the assessment of the Program. Findings are separated into Program Outcomes and Implementation Outcomes, as of August 2019. The Discussion section comprises a discussion of the implications of the findings and recommendations to inform future iterations of the Program, similar programs on a broader scale, and policy and practice in remote education for Aboriginal and Torres Strait Islander peoples. History of the Program At the time of the case study evaluation, the Program operated across three communities in the Northern Territory (NT) and six communities in Western Australia (WA), each with unique historical contexts (See Appendix B for a summary description of each school community). Since 1998, Tangentyere Council in central NT has delivered the Land and Learning project, originally as part of its Landcare section, which provided land care services to remote Indigenous communities in central Australia and Indigenous town leases in Alice Springs. The Language and Learning project has supported Two-way Science in 27 Indigenous community schools in central Australia, and this has often been linked to existing NT Department of Education Indigenous Language and Culture programs in schools. From 2011 to 2018, CSIRO partly supported3 Tangentyere’s Land & Learning program through various sources. In February 2016, the BHP Foundation funding delivered through CSIRO supported the continuation of the Tangentyere Council designed and delivered program, 2 To the evaluation team’s knowledge, all student participants in the Science Pathways for Indigenous Communities program as at 2019-2020 were Aboriginal, however, the more inclusive ‘Aboriginal and /or Torres Strait Islander’ has been used in some instances in this report to acknowledge the two distinct cultural groups that comprise Australia’s Indigenous peoples. The term ‘non-Indigenous’ is used to refer to people, such as teachers, who are not Aboriginal or Torres Strait Islander. 3 Bank Australia and others also provided funding for periods of the program’s history. and, as a result, began operating as ‘Science Pathways for Indigenous Communities’ in three communities from 2016. The implementation of the Science Pathways for Indigenous Communities program in WA started in 2016 and, over the course of three years, became established in six communities. The Program was informed by the Land and Learning project in the NT, which also aimed to connect Western science to local language and culture programs in central Australian schools. The Science Pathways for Indigenous Communities program in WA evolved in response to the specific strengths and goals of the participating communities, who have prioritised support to new or existing partnerships between school communities and key stakeholders, such as ranger organisations, research organisations and different levels of government. Throughout the Program, there has been regular communication and collaboration between Science Pathways for Indigenous Communities program staff in WA and NT to support best practice, sharing and continuous improvement. In mid to late 2019, the Program entered a new phase of implementation that involved more schools and communities—increasing in size from nine to a planned 22 communities (and from 400 to over 1000 students) across a broad area of remote Western Australia–with the potential for further expansion if funding became available. This expansion of the Program had also necessitated a shift in the way the program worked with ‘clusters’ of schools. The Program already worked with ‘clusters’ of geographically and culturally aligned schools and communities, but in this new model, CSIRO program staff would deliver Two-way Science workshops to the ‘clusters’ and then seek opportunities to work collaboratively across these clusters by identifying people in each school and community to support the implementation of the Program. These Two-way Science specialists would then take over from the CSIRO program staff and run Two-way Science workshops in their communities and other schools in their cluster. In 2020, program staff were aiming for each cluster to coordinate the Program in their region via a reference group and host an independently funded Two-way Science facilitator (or team) to provide ongoing support after the implementation phase. Figure 1 represents the new model visually. Figure 1. New cluster model for the Program Cluster model Figure 1. New cluster model for the Program The Program design The vision of the Science Pathways for Indigenous Communities program was to support Aboriginal communities and schools to work together to develop an integrated Two-way Science learning program that connects Indigenous ecological knowledge4 with Western science and the Australian Curriculum. The Program aimed to achieve the following goals: • Improve science education outcomes by linking Western science to Indigenous ecological knowledge • Engage Aboriginal people in remote communities in the direction and development of education programs in their local schools. • Strengthen children and young peoples’ cultural knowledge and identity. The connection between Aboriginal knowledge of the natural environment and Western science is used to inform the school-level approach to two-way learning with opportunities to integrate other learning areas, such as mathematics and English. Many disciplines of Western science have parallels in Aboriginal knowledge systems— ecology, astronomy, meteorology, geology, hydrology and more, one being a system of knowledge based on questioning and testing to understand the world, and one taking a holistic view of the way the environment works (Deslandes et al., 2019). A Two-way Science approach uses science as the vehicle for connecting Indigenous knowledge to the wider curriculum. This connection is used to inform the whole-school learning program, with many opportunities to integrate other learning areas, such as mathematics and English. The foundation of Two-way Science is Learning on Country. Learning on Country involves Aboriginal knowledge holders, such as rangers and Elders, teaching cultural knowledge of Country5 to students outside the classroom. These knowledge holders decide what to teach, how to teach and where to teach students from their communities. A Two-way Science program involves Aboriginal people making decisions about the direction and content of the school learning program. Aboriginal leadership of education fosters strong school and community relationships and develops young people’s sense of cultural identity and wellbeing (Deslandes et al., 2019; Guenther, Disbray, & Osborne, 2016). The Program is also place-based and builds on the strengths of Indigenous knowledges and the educational opportunities provided by the rich cultural and environmental landscape of remote Aboriginal communities. The Program follows a teaching and learning cycle, which builds on the educational benefits of Aboriginal and/or Torres Strait Islander knowledges and the engagement of Learning on Country by developing a program of integrated classroom activities that occur before and after the visits to Country (Deslandes et al., 2019; Guenther, Disbray, & Osborne, 2016) (see Figure 2). The Program also involves: Culturally responsive education resources and activities: Program staff worked with remote Aboriginal schools and communities in Western Australia and a team of education and science consultants to create a set of Two- way Science activities tailored to the unique cultural and environmental conditions of the Western and Central Desert regions of Australia. These resources are used by schools and Aboriginal and/or Torres Strait Islander leaders to direct the Two-way education program. Professional development: School staff and community members (including Aboriginal and/or Torres Strait Islander Rangers) participate in professional development training to assist in the development and implementation of the Program. These sessions define clear roles for the community, teachers, students and school leaders in working together to improve student educational outcomes. Partnerships: Facilitated among Aboriginal and/or Torres Strait Islander leaders, schools, community, Aboriginal and/or Torres Strait Islander Ranger programs, scientists and land management organisations based on local opportunities. Pathways: The Program supports local pathways to employment and training. 4 The Science Pathways for Indigenous Communities Program focuses primarily on Indigenous Ecological Knowledge, but other knowledges are involved as well. 5 Country is the regional lands and waters associated with traditional owners or clan groups that they have responsibility for. This term is used at a regional scale rather than meaning Australia as a whole country (Deslandes et al., 2019). Figure 2. Two-way Science teaching and learning cycle Figure 2. Two-way Science teaching and learning cycle The core features of the Program are common to both WA and the NT. However, there are a few differences in terms of stakeholders, resources and policy frameworks, which are summarised in Table 2. Table 2. Program variations across jurisdictions PROGRAM ELEMENTS NORTHERN TERRITORY WESTERN AUSTRALIA Key stakeholders and partners Tangentyere Council, NT Department of Education, Desert Support Services, Desert Wildlife Services, Central Land Council, Kiwirrkurra IPA Rangers, linguist Ken Hansen Desert Support Services, Parks and Wildlife, Department of Biodiversity Conservation and Attractions, Bush Heritage, BirdLife Australia, EON Thriving Communities program, scientific researchers including Simon Cherriman (Wedge- tailed Eagle Research) and Cecil Ellis (Waterbug Blitz) Policy/ Framework Integration with the Northern Territory Indigenous Languages and Cultures Curriculum The Program supports the goals of the Western Australian Aboriginal Cultural Standards Framework (Department of Education, 2015) and School Curriculum and Standards Authority Resources Land and Learning manual created by Tangentyere Council. Bilingual plant, mammal, reptile, bird and invertebrate cards. Planning and class activity resources developed with the Indigenous Assistant Teachers, class teachers and linguists. Two-way Science integrated learning program booklet, which includes unit activities and links to the curriculum, and leadership tools. These resources were developed in collaboration with the Science Pathways for Indigenous Communities program communities. Program outputs As of 2019, the Science Pathways for Indigenous Communities program was operating in nine schools and 11 communities (see Appendix B). Also, in 2018- 19, the Program was working with 42 teachers (34 in Western Australia and eight in the Northern Territory), 49 Aboriginal teacher assistants (35 in Western Australia and 14 in the Northern Territory) and 533 Aboriginal students. Table 3 shows the number of Aboriginal students the Program has worked with each year since 2015–16. The figures for each year (1 September to 31 August) are not unique across years; that is, the same students may be counted across different years if they continued to be part of the Program. The number of students engaged in the Program decreased from 2017–18 to 2018–19 (by 22 per cent in WA and by 9 per cent in NT). This was due to regular variations in attendance at remote schools and one school withdrawing from the Program because of community and school disruptions outside the control of the Program. Table 3. Number of students engaged by year 2015–16 2016–17 2017–18 2018–19 TOTAL Western Australia 82 416 544 426 1468 Northern Territory 73 121 118 107 419 Total 155 537 662 533 1887 EVALUATION Scope and purpose This evaluation used a multi-site case study approach to assess program implementation and progress towards outcomes achieved between October 2017 and August 2019. The purpose of the case study evaluation was to: • identify implementation learnings that support future program activities, such as enablers to successful program delivery in new locations; • highlight progress made against expected outcomes; and • use learnings to better understand program impact and redefine program outcomes in line with remote community goals. The overall evaluation question guiding the research was: To what extent did the program produce or contribute to the intended outcomes in the short to medium term? An Impact Pathway (Appendix A) was developed for the Science Pathways for Indigenous Communities program and outlines the Program’s inputs, intended activities, outputs and expected outcomes and impacts. This Impact Pathway was developed at the beginning of the Indigenous STEM Education Project and reflects an evidence-based, yet largely static and Western-focused logic for the Program. Although the evaluation approach was broadly informed by the achievement of the stated outcomes, through the evaluation, the collection of new knowledge has informed a revised program logic (see Appendix A) to show progress towards outcomes that better reflect the goals of the communities that participated in the case study. The evaluation team was internal to CSIRO, which provided several benefits, including availability, knowledge of the Program and its operations, continuity and access to CSIRO systems and processes (Conley-Taylor, 2005). Two key challenges with internal evaluations were identified and were managed to the extent possible. Specifically, the potential for: • Low perceived objectivity was managed through extensive consultation with stakeholders on the planning and conduct of the evaluation and on drafts of this report, as well as the use of an external peer reviewer who provided an independent assessment of the robustness of the methodology and the validity of the conclusions; and • The potential risk that staff may be less likely to maintain a critical stance in relation to the work was managed through continuous communication with senior leadership in CSIRO, who explicitly supported an environment of transparency and understood the importance of an objective assessment of the Program. Methodology Multiple case-study approach A case study approach was employed as it allows for participants and researchers to explore concepts in greater detail than many other methods, as well as the ability to be a strong conduit for the voices of Program participants (Harrison, Birks, Franklin, & Mills, 2017). Further to this, Johnson (2013) highlighted the value of case studies when working with community-driven programs and understanding how models can be customised to individual communities. This approach relies on replication and theory rather than sampling (Yin, 1994). Three remote schools (all in remote desert regions: two in Western Australia and one in the Northern Territory) were selected by the evaluation team and the program team for potential inclusion in the evaluation.6 The selection was based on several factors: the Program had been fully implemented in the school and community, the principal and teaching staff had been in place for at least a year prior to the evaluation, and there were no major issues occurring in the community that would make the research site visits inappropriate or ineffective. Principals were initially approached to allow the school to take part in the evaluation, and all three agreed after consulting teaching staff and the community. During the research site visits, staff, students, families and local stakeholders (referred to as ‘school communities’ throughout this report) were individually invited and consented to participate in the case studies. Three schools were selected to gain an understanding of a range of experiences and issues and to commit to giving each location in-depth attention and investment of time. Having three locations across two jurisdictions was also considered feasible with a small evaluation team and a finite budget. Fieldwork in the three communities was conducted in 2018 and 2019. All three schools had a high proportion of Aboriginal students. However, it should be noted that each school involved in the Science Pathways for Indigenous Communities program is unique. Generalisations about the entire Program have been made carefully. The three schools/communities were selected based on providing a range of experiences and characteristics to examine, including: school sector (government and non-government), jurisdiction (Western Australia and Northern Territory) and language (primarily English to primarily non-English speaking communities). In addition, schools were selected for the case study that had been involved with the Program for several years and had implemented the Program to the extent that made an evaluation possible and worthwhile from a learning, assessment and strengths-based perspective. It is acknowledged that a few schools involved in the Program had not implemented the Program to as large an extent as the case study schools due to a range of external factors not connected to the Program. It was determined that choosing schools that had fully implemented the Program would provide the best opportunity to assess evidence of emerging outcomes. For each case study site, a local evaluation reference group or less formal group of stakeholders was established well in advance of the fieldwork and comprised local people who were well-connected with the community and the Program and school. For example, members of the school councils were often part of the reference groups. The reference groups for two out of the three schools were more informal, as a formal structure was not recommended by the schools as being effective. These groups were generally convened informally, and one member of the group was identified as the person who could connect with other members to discuss elements of the evaluation approach. This approach was suggested by school communities as the most effective and efficient approach. This enabled informal face-to-face discussions between local members of the group on evaluation-related queries. In order to protect the confidentiality of the individuals, schools and communities that participated in the case study, their names have not been identified in this report. Instead, the report refers only to the two jurisdictions in which the communities were located. However, due to the small numbers of schools involved in the Program as a whole, it is possible that de-identified information used in this report could be recognisable to some individuals. There was a competing goal of naming schools and individuals in order to highlight and celebrate their hard work, achievements and successes. It was decided on balance that it was preferable in this report to maintain confidentiality; however, communities, schools and individuals would be free to self-identify if they chose. 6 One school that was initially selected for the evaluation was subsequently not selected. This was due to the Program team’s judgement that a critical mass of Two-way Science activities had not taken place. Outcomes and indicators The outcomes identified in the Impact Pathway guided the focus of the evaluation. Impacts were not assessed, as the Program was deemed not to have been implemented sufficiently long enough to make a difference at that level. Each outcome area was explored with schools, the community and other stakeholders through interviews and yarning. The indicators of success were identified through this process and are, therefore, largely community-driven. These indicators are discussed in the Findings section at the end of each outcome area. The overall baseline for these indicators was the extensive research literature and community feedback on education models not being effectively designed or implemented in remote Aboriginal communities. Specific and detailed baselines were more difficult to establish and were frequently based on improvements assessed by evaluation participants in the outcome areas. Data collection: Learning on Country, interviews, observations, document analysis The first case study trip in 2018 presented some challenges in relation to language and data collection as English was a second or third language for a majority of students. On the advice of local stakeholders, the researchers had not visited the community prior to data collection, as multiple trips were thought to be substantially disruptive to the school and community. This meant the evaluation team had to learn to work with the language barrier in real-time. An interpreter assisted during the focus groups and/or interviews. The participants were relatively reticent during interviews, and the process of translation made the interviews somewhat more difficult. In no way was this a reflection on the interview participants; rather, interviewing with a translator is inherently more challenging for both interviewer and interviewee (Larkin, Dierckx de Casterle, & Schotsmans, 2012). The evaluation team conducted a visit to the next case study community prior to data collection in order to meet the community and explain the evaluation. During this initial trip, researchers and program staff spoke with the community about how they would like the case study to be conducted and sought their advice on the best way to engage with different groups. For example, talking over lunch, while on Country, or during everyday activities were raised by local reference group members as the best ways to speak with students and community members about their experiences of the Program. The idea of a shared Knowledge Map—that is, a large piece of canvas that case study participants would contribute to with images and words about the impact of Science Pathways for Indigenous Communities—stemmed from this trip. The Knowledge Map (see Figure 3) was a physical example of the reflexivity of the research team, as it transformed from its original purpose into a reflection tool for students who contributed drawings, language, photos and samples found while on Country to the canvas. The Knowledge Map was then shown to the community at a whole-of-school event. The Knowledge Map became not only a record of one Learning on Country trip but also a tangible celebration of local cultural and scientific knowledge. There was planning around this approach prior to the fieldwork. However, the intention was for participants to use this process in a way that suited them. In this community, this approach was favoured by a composite primary class, and they recorded their learning on the calico as they talked about the day’s Two-way Science activities on Country. The Knowledge Map was owned by participants and became part of their classroom learning resources. For the final case study community, the research team took the advice of the school and community members and combined the planning visit with the research data collection visit. For the first few days, processes for engaging with students, teachers, parents and community members were discussed and agreed, and data collection followed later in the week. The approach allowed trust to be built during the initial days shared between the researchers and the community. For all three communities, researchers (in pairs) spent a week at the school during the data collection trip, participating in school-related activities, including Learning on Country trips, classroom activities and community events. During this time, face-to-face interviews and yarning conversations took place with students, parents, Elders, Traditional Owners, family members, educators and other school staff, program staff and program partners. The interviews and yarning were guided by a list of questions (See Appendix C). The flow of conversation and activities being undertaken each day guided how these questions were asked and determined how information was shared by participants. Conversations, observations and being invited to participate in local activities often led to related lines of inquiry that were not anticipated prior to the fieldwork. During the week, a process of reflexivity and development allowed questions to be adapted or given additional focus in responding to what was shared by participants while maintaining the original intention of the research site visit. Participating daily in Learning on Country with the students, Elders and, in some cases, Aboriginal rangers provided many opportunities for the evaluation team to walk-and-listen or sit-and-listen. Aboriginal educator Miriam-Rose Ungunmerr-Baumann believes that ‘By listening, you learn much by not asking questions: watching and listening, waiting then acting’ (as cited in Jackson-Barrett, Price, Stomski, & Walker, 2015, p. 41). For visiting researchers, sitting and listening allows one to gain a sense of awareness of connecting to the Country on which one stands and also to the people one has come to ’yarn’ with (Jackson-Barrett et al., 2015). Semi-structured interviews also occurred by phone with stakeholders that were not in the community and/or available during the case study trips, such as scientists, consultants, ranger coordinators, and program staff. A combination of interviews, informal discussions on Country, and focus groups were conducted, resulting in a total of 91 participants taking part in the case study evaluation (see Table 4). Figure 3. Knowledge Map Figure 3. Knowledge Map Table 4. Interview participants and total program participants by stakeholder group PARTICIPANT GROUP NUMBER PARTICIPATING IN THE CASE STUDY Students 42 Elders and family members 20 Non-Aboriginal teachers 12 Aboriginal teachers 5 Program partners and STEM professionals, including Ranger Coordinators, Consultants and Scientists 5 Program staff 4 Principals 3 Total 91 Informal observation was another method of informing the data collection processes and helping understand and analyse participant responses. Personal observational notes—that is, researcher notes taken after Learning on Country trips at the end of each day—were found by the researchers to be particularly useful during student learning activities, where researchers could note the level of engagement and types of achievement experienced by the students that may not have been easily verbalised by certain groups (e.g., primary school students). While outcome data for this evaluation was collected up until August 2019, an additional visit to one of the communities in WA took place in December 2019, and other methods were used to obtain findings for all three communities.7 These mechanisms were opportunities to share the results of the evaluation and discuss the evaluation team’s interpretation of evidence and how the results could be of use to members of the school communities. The evaluation team provided each case study site with the key findings from their community and the entire Science Pathways for Indigenous Communities evaluation. These community visits have informed sections of the report and are a vital way for the evaluation team to ensure the results of this evaluation, and the time the community spent with the team, are used in meaningful ways for each community. To provide context for the evaluation, and support findings from interviews and observations, the following documents were also examined by the evaluation team: • Science Pathways for Indigenous Communities program planning and implementation documents (e.g., Impact Pathway) and teaching resources • Desert Support Services Ranger reporting documents • Student work samples • Learning on Country reflections from researchers, staff members and students • School and teacher curriculum and lesson planning documents • Program workshop notes • Department of Education guidelines and policy documents Researcher reflection and position The CSIRO evaluation team consisted of five researchers during the period of data collection, analysis and report writing. One team member was a Wiradjuri woman, and the remaining evaluation team members were non- Indigenous. Team member world views, while varying, were privileged in many ways and, through enculturation, were inevitably based on Western modes of theoretical knowledge (Dew, McEntyre, & Vaughan, 2019). Through this case study, the evaluation team had the opportunity to influence and be influenced by the research through processes of reflexivity and development (Attia & Edge, 2017). The evaluation team’s development and methodology were informed by the writings of Aboriginal and/or Torres Strait Islander researchers and their colleagues (Hogarth, 2017; Jackson-Barrett et al., 2015; Martin & Mirraboopa, 2003; Moreton-Robinson, 2000; Nakata, 2002, 2007; Rigney, 2006; Yunkaporta, 2009) and conversations with Aboriginal and Torres Strait Islander CSIRO colleagues with expertise in engaging with Aboriginal and Torres Strait Islander peoples and communities. These perspectives informed the development of a general framework that the evaluation team aimed to apply throughout the project.8 The time spent in the three communities also provided the 7 Stakeholders in the Northern Territory community advised that a visit by the evaluators would be too burdensome on the community. A summary of findings was developed (in English and in language) and provided to the school to present to the community. In the other WA community, a video message was recorded and sent to the community for viewing at an assembly. A face-to-face visit was planned but was cancelled due to the COVID-19 pandemic. 8 The framework comprised a number of considerations including the following approach: recognise complexity, researcher positions and researchers as learners and that knowledge is created through partnerships; be adaptive, place-based, decolonising, relational; and contribute to positive change and focus on utilisation. evaluation team with the opportunity to be an active part of the work and to hear, see and experience what was important to the students, families, Elders and educators who are part of each school community. The evaluation team that conducted this case study was not the same team that developed the original project evaluation approach. The current evaluation team did have the opportunity to continually adapt the qualitative methodology, to meet the unique strengths of each participating community and to acknowledge their existing relationships and explore ways of learning and sharing that encompassed Aboriginal and Torres Strait Islander research approaches. The evaluation team aimed to be as open to learning from the community as possible. For example, the research techniques were adapted to suit the community members’ preferred modes of communicating. This process of researcher reflection and the adaption of research methods are explored further in this section when the data collection processes are discussed. The reflective process involved becoming more cognisant of cultural biases and having the intention of fostering a respectful relationship with participants and promoting open and respectful communication, including difficult dialogues to acknowledge and address power imbalances (Gray & Oprescu, 2016). These relationships were supported by Program staff who were observed to be significantly respected by each community and were skilled at working at the cultural interface (Nakata, 2007). Framework for analysis A thematic analysis was undertaken to synthesise the qualitative and secondary data. The thematic analysis involved three stages of coding, the first resulting in a basic set of distinctive themes, followed by more interpretative codes that represented an understanding of the participants’ experiences (Willig & Rogers, 2017). The interview data was then divided into high-level categories (themes) comprising: • community and culture • relationships • students, family, teaching and school staff • student learning • sustainability These high-level themes were further separated into sub-themes (e.g., student behaviour, student academic achievement), which were reviewed and assigned to the most appropriate high-level theme. Ethical considerations All participants in the evaluation gave their informed consent to be part of the research and have their interviews recorded. Students provided their assent to participate in addition to their parent and/or guardian providing informed consent. Recordings were transcribed and de-identified before analysis commenced. Illustrative quotes have been included in this report and have been de-identified to protect the confidentiality of the participants.9 9 For example, by using singular ‘they’ or ‘their’ in place of gender-specific pronouns where appropriate and by removing references to specific geographic places. Potential limitations of the evaluation Aboriginal and Torres Strait Islander research contexts are multifaceted, with many things that need to be worked through and considered (Jackson Barrett et al., 2015). As discussed earlier, this research project has its own complexities for both the Aboriginal and non-Aboriginal research team members to navigate. Overall, researcher connection with local community members and other stakeholders during community visits represented a positive and relationship-building experience. As researchers, the team were fortunate to listen to local storytelling and create connections; however, as ‘outsiders’, our inability to fully appreciate the knowledge we were being given could translate to a limitation within the findings and discussion. As researchers working across cultural and linguistic contexts, the cultural bias and Western lens applied—even with reflection and reflexivity—means observations, understandings and interpretations of events and evidence will not be as comprehensive and accurate as that of our Aboriginal and/or Torres Strait Islander colleagues or that of local Aboriginal researchers. A proud Bwgcolman woman, Lynore Geia, writes about these nuances, explaining that the use of storying and yarning in everyday interactions with others is a way of making sense of lived experience. It ranges from an informal, brief conversation or a ‘knowing’ look that elicits an emotional response, to more formal ways of storytelling that have prescribed outcomes attached (Geia, Hayes, & Usher, 2013). Another challenge to evaluation that potentially impacts this project is the recognition in Western academia of the legitimacy of qualitative methods such as yarning, storytelling and pathway-mapping (Howard, 2017) as well as other appropriate data collection methods when working in Aboriginal and/or Torres Strait Islander contexts (Jackson-Barrett et al., 2015). While these methods may be the best approach for engaging with Aboriginal and or Torres Strait Islander peoples, the implied or actual pressure to refer to quantitative data such as measures of participation, attendance and performance as a more rigorous form of evidence, and one that speaks most loudly to funding bodies and policymakers, is pervasive (for example, Gray and Bray (2019) focus on randomised control trials and big data). In this evaluation, qualitative data was considered the most effective way to capture learnings, in particular, the somewhat intangible but key outcomes relating to student engagement, wellbeing and community relationships. The case studies used qualitative data from multiple viewpoints across several locations and points in time. The absence of comprehensive and systematically collected program monitoring data is also a limitation of this evaluation report. However, at the time of writing, the Program was establishing program monitoring processes that could be used in future evaluation reports. Data from regular reporting to the funder, the BHP Foundation, covering program outputs has been included in this report. FINDINGS The findings from the case study evaluation are presented in this section and are organised around the outcome areas in the original Impact Pathway. For each program outcome, definitions of key concepts are presented, which were drawn from the research literature and the key indicators used to assess the achievement of outcomes. Relevant research is also cited for each indicator. Findings come from interviews conducted across the three case study communities, as well as from program staff and stakeholders that work across all or a combination of program school communities. Program design and implementation timeframes differed considerably between jurisdictions and across the three case study communities; shaped by different jurisdictional and place-based factors. This variation translated to findings highlighting different stages of program focus and impact. In the two WA communities, the Program was implemented relatively recently (i.e., in the last two to three years) and are in a phase of early establishment and implementation. In these communities, the recent program starting point allowed participants to clearly identify and describe changes since the Program commenced. In the NT community, the pre-existing Land and Learning program (created in 1998) had recently secured funding through the BHP Foundation and became rebranded as Science Pathways for Indigenous Communities in mid-2016. Over its lifetime, the Program’s design remained close to its original format and, at the time of this evaluation, was considered a mature program. In the NT community, the longevity of the Program translated to evidence focused in areas such as teacher capacity, curriculum and student learnings; and less evidence on implementation learnings. This is not a limitation of the study, but worth noting for a clearer understanding of the findings. Outcome area 1: Strong, effective partnerships established with schools and other stakeholders Definitions A partnership can be understood as two or more organisations or groups sharing a sense of purpose and an aligned agenda and participating in collaborative action to achieve joint goals. This is achieved while working together on projects relevant to both organisations or groups (Kirby, Held, Jones, & Lyle, 2018). Partnership development and maintenance This section focuses on partnerships; however, the role and benefit of strong partnerships is a visible theme throughout all the findings. The formation of strong, effective partnerships with schools and stakeholders constitutes a significant program focus and achievement, particularly in the early phase of program implementation. All program staff reported on the importance of building relationships within their school communities and developing a shared understanding between participating stakeholders, especially in remote contexts that program staff understood “can include people coming and going, with varying agendas and a history of cultural impact”. This was particularly necessary where existing relationships were not well-established or had lapsed, and local teachers and other traditional knowledge holders were not employed directly by the school. Program staff reported taking every chance to participate in formal and informal opportunities to get to know people and build trust and understanding, specifically through doing both incidental and planned activities together, such as: sharing car rides, sitting around a campfire and participating in community activities and forums. This was illustrated by a staff member describing a large-scale Ranger Forum by saying: “…just doing things together, and that’s how you get to know people and build that trust and achieve understanding”. This was reflected by a group of Elders who agreed that when they see program staff getting involved in community activities and on Country activities, they feel more trusting of that person: “Because he [program facilitator] rocks. Like, he do all the things, what he do now. Outside. Doing with animals things. We like a person doing that. We trust him, yep.” In the NT, where the Program is well-established, building on and maintaining existing partnerships was a common theme for community participants and program staff. Staff reflected on their early partnership development role, focusing on adding coordination and resourcing to existing relationships within the community. One NT Program staff member explained: “My role here has been to support existing school and community partnerships, so working with what is already there, helping the schools to implement a Learning on Country program.” The staff member went on to further explain their early process of partnership development: “So the Program worked by finding schools that were interested in the project, particularly where there was a principal that supported the idea, and going out and having a meeting with the staff, the assistant teachers and the principal and planning some on Country activity and starting from there.” Program staff highlighted the benefits of engaging with local teachers at schools and recognising their role as important links to the broader community. NT Program staff spoke about forming strong relationships with local teachers and seeking direction from them about how to engage others in the Program: “It’s run through the Indigenous Assistant Teachers, and they will bring in Elders for support, and they will discuss topics with Elders…There’s often at least one strong senior assistant teacher who acts as a bit of a link between the community and the school for those projects.” Recognising pre-existing school partnerships was important to program staff, as these provided strengths to build on. In this community, an effective partnership existed between the school principal, surrounding schools and the ranger program. One program staff member described: “…multi-school bush trips, largely organised by the [school] principal, with two or three other schools…working with ranger groups, going to some site that the ranger is managing the bilbies there or the springs…and then follow up work in the classroom in literacy has been really strong.” Strengthened partnerships Introducing planning, coordination and structure to existing relationships was critical to the program facilitator role in strengthening partnerships. This was particularly relevant where “sporadic and small scale” interactions were already occurring. One program staff member drew attention to the benefits of their planning work and focus on aligning partnership activities with broader learning outcomes: “Now there’s regular contact between the school and the ranger programs and more planned activities happening.” Desert Support Services were a key stakeholder in one community and described a range of specific instances where the Program’s contribution, particularly through structure and resourcing to create stronger partnerships, resulted in a transition from less effective partnerships, where stakeholders were siloed and: “busy doing their own thing”, to becoming much more productive and achieving mutual goals: “We would occasionally drop into the school and do a presentation and tell them about the work we do. I think only a handful of times we’ve had the school kids come out. When I say a handful, probably just one or two, it was difficult.” (Desert Support Services Coordinator) “There’s definitely a closer relationship with the school, [and rangers] and the school itself has that same vision and ideal of being community-focused first, and relationships…in terms of collaboration and networking, and structurally, or institutionally, that has helped. But also personally, and with individuals and family members, it’s—again, I guess it’s just tightened up, or re-invigorated, or reinforced those kinds of things.” (Desert Support Services Coordinator) “[CSIRO’s role is] definitely pretty integral to kickstarting some of this stuff [two-way learning/science]…actually having someone in that role to bring people together, to bring the rangers and the schools together. We were already doing it, but it helps structure it. And we would never have produced all those activities. The school might have done one here or one there, we might have done one here or one there, but the amount of effort and time that has gone into that—yeah, that’s extremely valuable to have someone who is actually on that.” (Desert Support Services Coordinator) Supporting others in building and maintaining partnerships Relationship development support and modelling from program staff have led some participants to feel more confident to engage independently within their partnership networks. One program staff member described their initial role to establish sustainable partnerships: “We led more, we facilitated, not necessarily led particular activities but we certainly helped set things up. And we’d be the people that would talk to Elders.” Program staff working across all communities talked about fostering strong partnerships, consciously working towards building capacity within the school and supporting others in the school community to establish and sustain relationship connections. Program staff observed that “Now you see people like [teacher] setting all that stuff up herself, and having regular meetings with the Ranger Program, and [Elders/Rangers] coming into school talking about what they want.” Artist: Jumarl Title: Pinga ngurrakarti yaninyi (Honey ants going home), 2019 Artist: Jumarl Title: Pinga ngurrakarti yaninyi (Honey ants going home), 2019 In one community, this was particularly evident when several educators talked about their increased confidence to engage with local rangers and Desert Support Services staff. In this case, teachers were reinforcing partnerships when they collaborated to create new Two-way Science learning opportunities. For example, after speaking with a Desert Support Services staff, one teacher discovered that: “If we do things like that [weekly bush walks] she’s happy to organise the Rangers to come out with us if we just email her, which we didn’t know about, so that’s amazing to know because now I feel like we can do…a little bit more now with that.” Another teacher reported feeling confident to independently organise Two-way Science on Country learning with program partners: “…the camp that we did at [location near community] —that was arranged with the scientist and with the ranger. So that was the first one I’ve done…without CSIRO…with external providers. That was really good…so the ranger [Coordinator] and I had a debrief, and there’s definitely things we’d do differently next time, of course. But it was really, really good.” One Desert Support Services staff member commented on the strength of their partnerships, following a period of program-facilitated planning and collaboration between stakeholders over their aligned goals, saying: “It’s definitely grown its own thing or lens. It’s gone beyond one idea. I think it’s probably at a stage where it’s almost self- determined, as in subject to the community, and its ability to respond to particular needs people want to learn from.” Strong partnerships supported outcomes The benefits of strong and effective partnerships were reported by school staff across the three communities. Multiple instances were given by educators who felt that the relationships they had formed with program staff, local Rangers and Elders enabled them to undertake Two- way Science activities more frequently and with increased confidence. One teacher also highlighted that dialogue between local partners creates Two-way Science activities that meet the needs of specific age groups, such as local bushwalks that: “would suit kindy/PP [pre-primary] and my class a lot better than some of these really big trips”. Across two of the participating communities, program partners highlighted the benefits of effective partnerships for their own and other programs. One participant noted that the partnerships developed through the Science Pathways for Indigenous Communities program had reduced duplication and increased coordination. The community member cited an example where local infrastructure management costs were reduced, pointing out that local services: “…ended up saving forty-five thousand dollars off the actual budget of the project because they were working together and they weren’t duplicating stuff. So that’s one example that in terms of education, there’s heaps that we’ve come across just in our four years of doing Science Pathways.” In another community, Desert Support Services staff explained how closely the Program’s goals aligned with their work and that program relationships have supported IPA planning goals: “In the early days, in the inception of talking with [Science Pathways for Indigenous Communities facilitator] and running all these ideas, it really gave us a bit of framework – Martu people Learning on Country, which ties in very closely with the IPA plan – the wishes and vision of the people up that way. So it’s been a—it’s been a tremendous value add to our land management program.” A range of stakeholders acknowledged that within remote community settings, forming genuine and respectful relationships with people often had a flow-on relationship effect. Scientists, educators and program staff gave specific examples of how engaging with even just a few people could be enough to pave the way to forming broader community networks. One program staff member explained that when people appreciated the genuine efforts of program staff, this message is shared with others. In one case, when a program staff member visited a new community to talk about Two-way Science, the community had already heard about their work due to the endorsement of neighbouring community members. Coordination and continuity Some stakeholders, especially those with a coordinating function, reported a complex network of key people and organisations that can lack coordination, therefore creating efficiencies, especially when roles might be unclear. While the program staff have a clearly articulated focus to help improve partnerships; for other stakeholders, such as conservation scientists and various government agency staff, navigating competing parallel strategies (Department of Biodiversity, Conservation and Attractions, Desert Support Services, IPA, CSIRO), governance arrangements and stakeholders with multiple roles can be challenging and can often require additional planning, time and consultation with local community members. Program staff relayed that the concept of Two-way Science is a galvanising approach for multiple stakeholders, emphasising that: “People understand two- way learning first, then to really value it, and integrate it within programs and systems takes time and momentum.” One science professional who participated in the evaluation noted that despite shared goals amongst agencies and organisations, system-level coordination is critical to reducing confusion: “…if we can get people from each of these different parallel programs all on the same field trip then, to the community, it will feel more cohesive. Yes, these are parallel programs, yes these are different strands of funding, but we’re actually all trying to do the same thing…with all these programs running parallel to each other…the community are getting very confused as to what opportunities even exist and who’s actually benefitting from them.” Program staff also see the development of effective, actionable partnerships as a way to mitigate the effects of educator turnover in remote communities. These effects can be most significant when there is a change of principal. One program staff member emphasised this: “So when a new principal or someone takes over they can go okay, well we’ve got his meeting coming up and we’ve got this happening next term…it’s much harder to change direction. And so if you have really engaged local traditional owner groups and really engaged ranger programs and an active plan that’s underway, then I think that leads to sustainable outcomes.” Indicators of success Based on the interviews and focus groups conducted, the evaluation participants identified a number of indicators of success related to Outcome area 1 (Strong, effective partnerships established with schools and other stakeholders) that the evaluation team have also assessed as being achieved. These are in line with indicators identified in the literature (Hunt, 2010; Hunt, 2012; Morley, 2015; Tsou, Green, Gray, & Thompson, 2018; Walter, 2015): • New partnerships were created between schools and community stakeholders, and existing partnerships were strengthened • There was a high level of trust and respect among school staff, community stakeholders and program staff • Strong personal relationships existed among school staff, community stakeholders and program staff, including frequent face-to-face contact • Partnerships focused on achieving shared goals • The value placed on Aboriginal knowledges strengthened partnerships • There was increased coordination and collaboration among stakeholders • Broader community networks were formed • School capacity and confidence in building partnerships with external stakeholders increased Outcome area 2: Increased community and parental engagement Definitions Parents and other adults within a community have an important role to play in engaging with young people to support and influence their academic and career pathways (Gavidia-Payne, Denny, Davis, Francis, & Jackson, 2014; Pendergast, Allen, McGregor, & Ronksley-Pavia, 2018). Broadly, ‘engagement’ can be defined as the time, energy and resources devoted to activities with the intention of achieving an educational goal or outcome (Pittaway, 2012). In recognition of Aboriginal and Torres Strait Islander family and kinship structures, ‘parents’ are defined as students’ primary caregivers, which may include extended family, family clan members and non- biological adults who look after or raise students (Bobongie, 2017; Lohoar, Butera, & Kennedy, 2014). Parental engagement refers to a range of parental practices, both at home and in educational settings, intended to promote children’s academic achievement (Chenhall, Holmes, Lea, Senior, & Wegner, 2011). The term ‘community’ may refer to a place or group of people with something in common. Applying a youth- centred and context-specific lens to the term ‘community’ allows the definition to be specific to each young person and includes a range of possible stakeholders that enable their educational success (Flouris, Crane, & Lindeman, 2016). For the Science Pathways for Indigenous Communities program, ‘community’ could include a range of individuals and organisations such as local ranger groups; Aboriginal and/or Torres Strait Islander and non-Indigenous teachers; peers; cultural leaders, such as Elders and Traditional Owners; and youth and community-based services. ‘Community engagement’ describes a model of authentic collaboration and purposeful interaction between institutions and the communities in which they operate (Lowe, 2017). Building on existing relationships For each school that participated in the study, there was evidence of strong engagement with some families and members of the broader community. In many cases, these families were already engaged with the school in a more formal, paid role (e.g., school staff) outside of the Program. As such, engagement in each community was largely shaped by the collective of Elders, Traditional Owners, local Aboriginal teachers and families who were already actively involved with the school. In WA, program staff acknowledged the existing contribution that the community and family had made in some of the schools, recognising it as a strong platform on which to build the Program. One program staff member explained that: “School and community relations were started…well before we were [running the Program]…They’d [the schools and communities] been doing some great work…they set up the school councils…providing strategic direction. They’d done a preliminary cultural calendar looking at what was happening at different times of the year and how that could inform the school learning program. They had community days where they’d go out bush as a school and do cultural activities, and so all that provided just a fantastic platform. We weren’t starting from scratch.” In the Northern Territory, program staff highlighted the longstanding engagement of various groups within each community, including specific Elder groups, rangers and parents who were employed as teachers. Some of these groups provided knowledge and direction to enable Learning on Country and curriculum planning. This was particularly evident in one community where parents employed as educators were driving the Learning on Country. As one program staff member commented: “The parents I tend to know are the ones who are working at the school as assistant teachers…if it’s a relatively local bush trip, sometimes parents and families will come along. But the trips are usually planned and led by the assistant teachers with my support and the support of the classroom teacher hopefully, and sometimes there are Elders involved.” Community leadership In each community, a visible group of committed family members who may have also been rangers, teachers, Elders or other respected members of the community were significantly engaged with the school and shared their expectations for student learning. In one community, members of the school council, who were also nanas, Elders and rangers, spoke of their hopes for their young ones to, as one Elder put it, “Be strong in their culture…and probably find a job. We want all that.” Many community leaders shared this desire for their young ones to maintain their culture. In one community where the Program had been running for a longer period, the school principal spoke of the concern expressed by family and assistant teachers that the young ones were not maintaining their language. The principal explained that: “[The parents] came to the school and said that they weren’t happy because the kids were losing their language…[the students] just say tree for everything [but] they [the trees] all have different names.” This discussion led to the school’s participation in the Program when they were approached soon afterwards: “I think it was only a couple of weeks later— honest to God, it was serendipity—[program facilitator] walked through the door with [their] plant cards. I just went, well all right…and that’s how the Program started” (principal). Elders and other family members often also spoke about the importance of their kids being able to “operate in both worlds”. That is, to understand and function in the Western education system, whilst having a strong understanding of their culture. This often motivated Elders and family members to be involved with the Program and with the school more broadly. As Elders in one community explained: “Both [two ways] are important to the kids and even to us. We got the kids involved in both ways. We just teaching the white people way of science, like going to school you’re learning to read and write and to do math. Here as Elders, we got them out in the bush teaching them about culture, what our family taught us. I think it helps them a lot because for their children too, they can pass that onto the next generation. So that’s very important. Passing it down.” This sentiment was echoed by family and Elders in other communities where those interviewed talked about wanting to do more (and longer) two-way learning trips out on Country. “What makes it [a] good [school]? Learning on Country. And we work together here. We’re a team. We have good teachers. They learn our kids good. We’re at home. The school is in our home.” (community member) Engagement in curriculum planning WA Program staff were particularly focused on engaging community members and families with the Program. As one staff member explained: “We didn’t have a defined process at the beginning, and it was very much about seeing what [the] community wanted initially, and then what schools wanted from our project. So, I think that was really important and a really useful part of the process. That we didn’t come in with an agenda, particularly when it came to working with Aboriginal people. And from that emerged the themes really, from those initial kind of consultation and discussions.” Program staff also emphasised that Elders “had really clear goals” and that, very quickly, they could work within the existing agenda and planning to respond to the community’s leadership. Similarly, the results showed that community, school staff, rangers and ranger coordinators recognised the importance of community engagement. As such, they sought to maintain and build relationships by seeking out the goals of the community before supporting the achievement of these goals.10 In one community, program staff amended the Program’s goals in line with the broader community goals. One principal emphasised the importance of actively engaging with community and family members: “You need to ask your local communities first what it is they actually want because not everyone wants a language and culture program running like we run one…Our main focus is the kids and the community, so we should always ask. I’m looking after the school, that’s all. It’s their [the communities’] school, and I need to ask them [what they want to achieve]. That’s where I think people should start, they should ask their local communities what they actually want in their schools and how they want their schools to run.” Desert Support Services staff also supported a community-led approach. These coordinators worked across a number of communities in WA and spoke about the Traditional Owners’ clear intentions for student learning: “The [local Indigenous group] have always pushed for education and getting more young people out on Country.” One Desert Support Services staff member reflected on how this long-standing community goal was reflected in their land management and conservation work, as well as their involvement in the Program: “To quote…[the Elders]…they said, ‘we’ve got a classroom too, out there [points to bush]’…It is one of the big aspirations for the [ranger organisation] 10 For example, by using singular ‘they’ or ‘their’ in place of gender-specific pronouns where appropriate and by removing references to specific geographic places. and the [Indigenous Protected Area] itself…To be a Two-way Science hub, to have almost like a bush university out there…That’s been the plan from the start…That’s part of the IPA plan, and then we brought it into this Two-way Science plan.” The program staff encouraged schools to be inclusive of Aboriginal knowledge holders throughout all curriculum and term planning. In most communities, both Elders and local teachers were engaged in some or all stages of the formal curriculum development and/or during the development stage of the on Country learning activities. One program staff member described this collaborative approach: “The trips are usually planned and led by the assistant teachers with my support and the support of the classroom teacher…and sometimes there are Elders involved.” A kindergarten teacher in one community also commented on the involvement of the local community when planning their Two-way Science activities: “At the moment it’s a lot of us coming up with ideas and then talking with the Elders, seeing what they think, and [the Program facilitator is] good. [They] have a lot of good ideas as well.” Overall, the involvement of the community was perceived by the teachers as critical to running the Program, particularly at the initial brainstorming stage. As one teacher explained: “I pretty much am completely led by what they [Elders/rangers] want. Generally, when we do the brainstorm, there’ll be two or three main themes that come up, based on what is in season or what they’re particularly interested in at the moment.” Two teachers from one community reflected that since the implementation of the Program, members of the community have had a stronger influence over the school teaching approach, indicating broader benefits than just how the curriculum is shaped: “They’ve [the community] become more involved in the school, and they’ve had a lot more control and leadership in the Program with teachers. They’ve led the way, and that’s really important, I think. It’s good for self-esteem, identity, trust between the two, between school and them. We’re all in this together.” Overall, the communities demonstrated that ownership at the planning stage correlated to community engagement, which has provided ongoing opportunities to build relationships, understand one another and create a safe space for families and culture within the school environment. In each community, there was evidence of family and community engagement along the Program continuum, and at each stage, community leadership was a strong element of engagement. Engagement in the Teaching and Learning Cycle In addition to curriculum planning, a foundational element of the Program’s model was the teaching and learning cycle. This cycle provided increased opportunities for community and family to engage in learning with the students both before and after Learning on Country trips. Evidence across all communities indicated that bush trips were a popular form of regular parent and community engagement; however, the Program also encouraged local Aboriginal knowledge holders to participate in classroom activities. When local Aboriginal teachers were not available to participate in the classroom, community and family members were sometimes involved in the before and after bush trip activities. In one community, when teachers and program staff facilitated Elders or rangers to visit the classroom following an on Country learning trip, the intention was to talk about what they saw on their trip and more deeply explore Aboriginal and Western knowledges. One program staff member noted how in one community, a group of Elders had a long-standing relationship with the school through art and became involved in Two-way Science both in and out of the classroom: “Yeah, with support [the women’s group] come into the classroom before and after trips. Certain people are really happy to come in and talk with the students and teach them about things before trips and give them some prep and, if necessary, if not all the kids were privy to everything during the trip, she’ll come into the classroom afterwards as well and just make sure everything has been really thorough.” The following is an example of a classroom discussion with some year 2 to 4 students about the items they had collected from a bush trip. The discussion of the honey ant habitat is illustrative of how the program staff and Elders/rangers talk together in the classroom: “When you’re walking around in the bush, yurrkalili [honey ant] time. You see it on the ground, a lot of honey drips onto the ground, that’s how we know that there’s a lot of yurrkalili. Then you see ants who are going up there, taking the honey off the tree. You fellas see that in the bush?” (Elder/ranger) “I found out some interesting things about this yurrkalili and what makes it, there’s a little tiny, tiny insect that goes along on some of these - on the mulga [a small shrub with greyish foliage] and it sucks the sap that’s inside here and chews it.” (program staff) “It’s smaller than the minga ant.” (Elder/ranger) “And turns that into the honey and then the honey goes hard, like that.” (program staff) Engagement during Learning on Country “On Country trips mean grandmother and grandfather are telling them their culture stories.” (teacher/parent) Learning on Country is a central element of the Program, where Aboriginal ecological knowledge and Western science intersected, and the local community worked with the teaching staff to share stories, information and knowledge. Learning on Country trips increased the opportunities for family, community, Elders and others to engage with students, teachers and the school environment. Program staff spoke about this and making trips more accessible: “The biggest [change] is around the frequency of on Country, getting out of the classroom with Elders. That’s increased a lot since we started…it used to be almost once a term, [a] big bush day celebration type thing, whereas what we do [now] is get class groups out with targeted outcomes, and [we have] plans to [do this] on a much more regular basis.” Program staff acknowledged the importance of Learning on Country for ongoing relationship-building: “…the best way to build those relationships is by doing Two-way Science – finding opportunities to get out on country with kids and follow that up in the classroom. But that initial just going out and having a go at something, as messy and imperfect as that may be, the most important outcome from that is that you have developed a relationship because you’ve done something together.” (program staff) In most communities, Learning on Country generated a notable increase in the level of engagement from families and the community. As one teacher recalled: “If they [the community] know we’re going out somewhere, especially fishing—they love fishing —you’ll get even assistant teachers coming out to help out…they’ll come in especially to help out.” Some Elders and parents agreed that they wanted more two-way learning on Country trips, especially overnight: “So you can talk stories, walkabout.” Some teachers across both jurisdictions spoke about the benefits of Learning on Country to provide positive engagement opportunities with families and the broader community. One teacher commented: “these days [being out on country] are so good to get families involved, and it’s not just at the school…we want the families involved in the school as much as possible and I think this is one of the best ways to do it. They’re [Elders/rangers/local teachers] teaching them [students].” The Learning on Country trips were also described by one teacher as facilitating positive and strengths-based interaction between teachers and families, in contrast to behaviour management conversations: “It’s a more positive experience to engage the family instead of a negative one. Instead of…ring[ing] up and [saying]…‘your kid has done this’. [I say] ‘Oh, come out on Country with us. Help us teach the kids to do stuff’… It puts an importance on what they know as well.” A visiting conservation scientist conducting research on an Indigenous Protected Area (IPA) also recognised the importance of connecting with families and the broader community via time spent with students on Country. The scientist commented that: “The kids came out, and from what people are telling me they had a good time. Those same kids are going to go home to their parents and if they’ve had such a good time they’re going to tell their parents that, and it’s a small community, everyone talks.” A principal added that: “…families like to go [on Two-way Science bush trips] because they learn some skills as well from the rangers and the old people as well.” Increasing engagement through a more welcoming space The Program’s strengths-based and ongoing facilitation between the school, families and the broader community can foster a safer space for sharing culture within the physical school environment and Western classroom context. In one community, a group of Elders also employed as rangers spoke about their changing relationship with the school environment as a result of the Program: Elder 1: “I look forward to it [coming to the school]. I’m happy because the kids are there and it makes them happy, and they see parents and guardians and nannas and pops.” Elder 2: “Of course before [Science Pathways for Indigenous Communities] it wasn’t like this, they [Elders/parents] never used to come to the school.” Elder 1: “No, never used to.” Elder 2: “Now it’s open door for us. Parents will come in.” Teachers also reflected on supporting engagement and ownership in the classroom: “I sort of see it as them [Elders and community members] now realising that they can have some ownership of what the students learn and that we’re [the school/teachers] happy to help facilitate that. That helps bring them in, that they know that their knowledge and what they want to teach the students is important to us and… how teachers [can] help them do it.” After participating in a week of Two-way Science activities with Elders/rangers and a program facilitator, a Desert Support Services staff member emphasised the value of a welcoming and respectful space within the school grounds: “It’s really nice, that they [community members] do feel like a part of the Program and they can come in and spend time with the kids on Country…whether or not they’re a ranger. Having that two-way ethos has made people feel comfortable and valued, particularly the Elders, who often do feel isolated at times. As knowledge holders, they feel sad because they know all this stuff, and they haven’t got a way of imparting it because those ways are gone. The way they would usually [pass their knowledge] would be through [Jukurrpa], through the dreaming, through stories, and singing, and walking around and showing each other things. That’s not what happens anymore, but if you have a school that facilities that, then that opens up those doors for them to be able to do that.” In one community, the relationship connections and knowledge developed over time with families and the community was emphasised by a long-term staff member as one of their greatest assets, partly because this was also valued by the community above all else: “But the main reason that they’re happy that I’m here is because—and this is what they say—you know all our stories. It’s not because I’m a good teacher or I run a good school or a good classroom, or we run a good program. I know them. I know their families, and I know their stories. It’s all about I know them, and they know me. We have this relationship.” The creation of safer spaces for culture also supported the new and ongoing engagement of families and the broader community. Across all three communities, instances of engagement outside the Program were Artist: Gurandull Title: Rangkurrji (Bush Turkey), 2019 Artist: Gurandull Title: Rangkurrji (Bush Turkey), 2019 Story: Ngajulu marnipirnini nyungu. Yanayirni kuwikarti malyikarti palanga yirrinirni rangkurrjimili jina. Palaja wirlanirni pala Rangkurrji palangulu wikangajinirni. Nganarnini pala Rangkurrji martumpirrinyi pa julunga. (My painting shows my family hunting for bush turkey. There are two waterholes that the turkeys use for drinking. We track and catch the bush turkeys near Mulyie. We cook them in a campfire. It is nice to eat with salt and hot damper.) noted by school and program staff. All spoke about the Program’s contribution to families and community members feeling more welcome in the school environment, which in turn facilitated interaction and created opportunities for students to learn. One educator commented on the response of a few of the older boys when their families visited the school: “They [the students]…want to sit with their families. They will go and get their food for their families…We’ve got three boys… and [if their] mum’s around [their] mum will just come and sit in the playground for a bit, and the kids will come out and spend time with her.” In another community, the principal reflected on the relaxed nature of the families’ relationship with the school, which they perceived as indicative of being truly comfortable in the school: “They’ve [the parent] come looking for someone…and then they’ll hang about for a bit. So they might be a little bit interested…just to see what’s happening with language or culture or science or English…and then they’ll just wander off again.” Other teachers raised the involvement of Elders in school events as examples of strong parent and community engagement with the school: “Elder [name]…he does a lot with the kids. If there’s an event here, for example, when we had Sorry Day and Reconciliation Day, he came for the big assembly, and he sang to the kids, and he’ll sing in language. He’ll be here next Wednesday when we do the cultural evening. He gets involved as much as he can…[teacher’s name] boys get involved as well…they usually work with the young kids…and get them really involved in hunting and tracking.” Indicators of success Evaluation participants identified a number of indicators of success in relation to Outcome area 2 (Increased community and parental engagement). The evaluation team independently also assessed that they had been achieved: • Parents and other community members felt safer and more welcome at the school • The Program created opportunities for new interactions between the community and school that expanded beyond the Program • The Program created more opportunities for community and family to be involved in student learning • Parents and other community members and groups took part in on Country and classroom Two-way Science activities and often were the driving force behind them • Parents had increased communication with teachers and program staff • Elders, Traditional Owners and other knowledge custodians shared their experiences, language and knowledges with students and teachers in the context of Two-way Science • Aboriginal scientific knowledges held by community members were valued and helped increase community engagement • Elders, Traditional Owners and other knowledge custodians were involved in setting expectations for student learning and in curriculum and term planning • A wider range of community members was engaged with the school. Outcome area 3: The Program’s Two-way Science learning approach and resources are culturally responsive and embedded across participating school curriculums Two program outcomes have been re-interpreted to create a single, over-arching outcome statement that relates more closely to activities and outputs undertaken by the Program and articulated in the Impact Statement. For the purpose of this evaluation, the above outcome replaces these two original outcomes: • Education resources developed into a cohesive community-based curriculum and learning resources (WA) and embedded in ILC delivery/ other curriculum processes (NT). • A school culture of high traditional ecological knowledge and STEM expectations leveraged for improved literacy and other learning areas. Definitions Curriculum is a broad concept that includes knowledge and content, delivery and teaching, assessment and even reporting to parents. The curriculum can include the planned learning for students, subject areas as well as cross-curriculum general capabilities (ACARA, 2020). Mandated values and behaviours are also endorsed through the Australian Curriculum (ACARA, 2020). Curriculum becomes translated into learning materials, syllabuses and textbooks and is used by teachers in designing and planning lessons and units of work (Perso, 2012). Contrasting approaches to education priorities in Australia have been described as either privileging education relevance and the preservation of culture or focusing on English literacy and numeracy attainment with a disregard for context and culture (Perso, 2012). In the context of this continuum, similar to pedagogical approaches, culturally responsive education programs and resources11 are ideally designed to encourage innovation and creativity and find common ground with a spirit of learning that is strongly place-based, ecological and, importantly, built on Indigenous ways of knowing, learning, thinking and communicating (Yunkaporta & McGinty, 2009; Yunkaporta, 2009).12 There is a dearth of discussions in the literature on the term ’two-way’, indicating that it is often defined by the context in which it is adopted, albeit with common characteristics (Douglas 2011; Perso, 2012). In Australia, particularly the NT, “two-way” was historically considered an iteration of bilingual schooling, with a focus on providing education in the language that children speak13 (Douglas, 2011). Resources ready for embedding Educators consistently commented on the Program’s resources as being simple, easy to use and adapt and relevant for their students, which contributed to the ease with each they could be embedded in the curriculum. One teacher explained how the resources created options to work from “this term he [Science Pathways program coordinator] trialled using more of a resource pack, where he’d given us a resource pack of activities and ideas, and we could differentiate it for our group of kids…all teachers have their own creative way of doing things, and if you look at the pack of work, you can sort of see, okay, this will work with my group of kids, or this might not work. So it gives you that flexibility.” This teacher expanded on the value of the resources for understanding and applying Two-way Science learning, in general: “You can see how on Country learning works and what you can produce from it. Just having lots of photographs is always good for a teacher because you can look at something and say, right, I’m going to do it this way. Make your own choices…you can do it your way that’s going to work with your kids.” This intention to create user-friendly resources for educators for ease of embedding and adoption was confirmed by one of the program consultants involved: “I handed out the resources that we’d got and I think when you go to deliver something new, if you say, ‘I’ve got a new idea and by the way, here are the resources and tools to help you,’ I think that has a massive impact on the view of the teachers of how accessible it might be.” One of the principals commented on the efficiencies created through resources, saying: “The teachers would not have the time with all the other learning for designing. So that’s just a step up that helps.” One teacher highlighted that they 11 See Morrison, Rigney, Hattam, and Diplock (2019) for a literature review. 12 In contrast, (Explicit) Direct Instruction has been adopted in some remote schools and is based on a systematic and scripted curriculum developed in the United States. 13 The model of bilingual education used in the NT in the 1970s to 1990s, and still continuing in schools today, is to teach children in their first language in the early years, with a greater focus on English by upper primary. were particularly useful for teachers with limited science backgrounds: “Then you look through the resource books, and they’re collected in themes…it actually flows really well…my background isn’t in science at all. So even with a mainstream science curriculum, I wouldn’t necessarily be really confident teaching that either. So it’s actually a really good way to do it.” Another teacher commented on the simplicity of resources and therefore their appropriateness for people who speak English as a second or third language: “I love the way that [program facilitator] brought through the cards with the names - I plan to sort them out with the mob, get the mob in and say, ‘Let’s all sort them - classify them into what’s local, what’s not quite local’…I love that idea, and it’s that little simple resource that’s not too invasive or not too long-winded, and it’s just short and simple, and the kids understand it, the mob understand it because it is in English. But they would do their little story beside it, and their story would have a different function. It could be an old lady’s story from a long time ago.” For many schools, the additional science equipment and resources meant Two-way Science activities could occur, despite a lack of school-based resources. Without this support, the depth of learning available to students through the Program would likely not have eventuated. Program staff also provided informal training on the use of equipment, such as how to set up and view motion sensor cameras. One teacher commented on how this made it easier for teachers to extend themselves in the area of science: “I think the CSIRO stuff keeps teachers on track with that higher-level learning than we might just fall back into if that wasn’t there. We could have a lovely trip down to [nearby location], but whether we’d get as much out of it as we did with your input, then I don’t know. There would be things we’d be after, but it certainly gives us a higher target I think.” Place-based resources developed with communities to reflect their education goals A number of place-based resources were developed as part of the program, comprising a series of booklets for the original draft Science Pathways for Indigenous Communities resources, resources developed by program staff for (location) specific purposes, and resources developed by teachers. The collaborative processes undertaken by program staff and educators to create place-based resources have been integral to their utilisation by schools. This process created resources that respond to the strengths and needs of each school community. Local language and culture are also key components of resource development, and in each community, the involvement of first-language speakers in drafting resources was essential to ensuring relevance and utilisation. One principal emphasised this point: “You have to have a very clear understanding of the importance of language…and this is certainly evident for where we are today with Science Pathways or two-way science in [this community].” All communities involved in the study considered the continued use of language between generations as important. Many Elders involved in education talked about wanting to see this prioritised within their schools. A Martu Elder spoke about their community’s expectations for education: “Both [two ways] are important to the kids and even to us. We got the kids involved in both ways. We just teaching the white people way of science, like going to school you’re learning to read and write and to do math. Here as Elders, we got them out in the bush teaching them about culture, what our family taught us. I think it helps them a lot because for their children too, they can pass that onto the next generation. So that’s very important. Passing it down.” Educators commented on their use of program resources as a basis to respond to community priorities, as well as curriculum objectives. One of the principals reflected on the role that the science resources played in supporting culture and language to be embedded across the curriculum: “The vision was bi-cultural learners and so to enable that bi-culturalism we needed to connect the country to the learning. We needed to do that in an inclusive fashion, and we had to be very systematic because being the teachers who generally come from a very mono-cultural, mono-lingual background. Science Pathways enabled us to systematise everything better. It provided us the resources and the knowhow to really do what we’re doing, get it tighter, but also go so much deeper with the science which we could never have done.” At the time of the evaluation, a comprehensive program resource in the form of a practical teacher guidebook with links to the Australian science curriculum was in its final stages of development.14 This resource was collaboratively developed by Science Pathway school communities in WA, with input from program staff in the NT, and showcases activities that have already been undertaken with schools. The instructional resource is designed to build the capacity of teachers to undertake Two-way Science inquiries across the teaching and learning cycle. This teaching resource makes reference to a range of curriculum areas, such as maths and technology, and supports the development of general capabilities such as numeracy, literacy and critical thinking. The resource also supports EAL/D and bilingual strategies. The process adopted to develop resources creates additional program benefits, including supporting the sustainability of the Program. The resources acknowledge the Aboriginal ecological and cultural knowledges, leadership and languages shared by communities participating in the Program, creating a record of this. One Program staff member commented: “I think it’s a communication…a moment in time where we say here’s some ideas for what Two-way Science looks like, structured around the idea of Aboriginal leadership really explicitly within all of the activities…it’s a great kind of communication tool, and I think it will be the foundation of our project in many ways in terms of teaching people how to do Two-way Science, but it has all these other benefits as well that are really important. And great with community…people just love it”. Artist: Raul Title: Waterhole, 2019. Artist: Raul Title: Waterhole, 2019. Story: Parrpakarranga ngapakarti yaninayi kuyirrangu. Ngaju wirtijarrinini ngapamartaji pala ngapa yinta. Warnku mankuluman purunypilaman ngalkunyja palanga ngapanga panpilaman palanga ngapanga jurrulu parntilamanta. Palangulu janpuluman (All the animals come to waterhole when it’s hot. I grew up near the fisherman’s spot, and this is deep water for swimming. You use a rock to rub under your arm and throw it in the water to let the snake know that you’re here. That keeps you safe in the water.) 14 The resource was published in December 2019 and is available for purchase or free download of the digital version: www.publish.csiro.au/book/7949/ Cross-curriculum connections Evidence indicates that the Program improves school capacity to embed traditional ecological knowledge and Western science into the curriculum, including across a range of learning areas and general capabilities. This is based on feedback from teachers and observations that the Learning on Country activities were explicitly linked to all curriculum areas. For example, a fish activity was linked to literacy (e.g., individual workbooks), numeracy (e.g., tallying, representing data on graphs), history and social science (e.g., mapping, family stories), arts (e.g., illustrating), technology (e.g., setting up equipment), health and physical education (e.g., cooking). Figure 4 shows an example of a term plan linking the on Country learning to different curriculum areas. A review of the resources used across both jurisdictions highlights the practical integration of traditional ecological knowledge, cultures and languages within the Australian science curriculum. In addition, the integration of literacy, numeracy, ethics and EAL/D strategies and Western science language, such as conditional language structures (such as: “I think if we remove this ant, then the echidna will have no food.”) and literacy and critical thinking used in the field of Western science inquiry and research. A bilingual education approach was also identified in the resources and in some activities, the integration of technology, maths and history. Educators across the three communities spoke extensively about how the support and resources provided by program staff created a more constant and effective focus on science and provided a platform for achieving student outcomes across learning areas and general capabilities.15 Program staff also spoke about their collaborative approach to developing resources, which included supporting local teachers in developing their own science resources. Other benefits of embedding Two-way Science Embedding Two-way Science in the curriculum was seen to partially mitigate the effects of high staff turnover, with a number of educators identifying the: “Process of consolidating the Program so that it’s changeover proof”, ensuring that: “As new people come in, there’s enough old hands who can show them how to do it.” Until this is done, program facilitators were identified by most school community members as the: “Steady constant to make sure that that is kept up”. Another principal noted that when there is a lack of jurisdictional curriculum oversight, the Program supports school accountability to teach as much as possible to the Australian science curriculum. The principal saw the Program as a way to increase the prevalence of science within their curriculum: “That’s where the outside agencies are much more helpful because they’re the ones who come in and actually make sure it’s actually going on in the classroom.” One educator described an embedded collaborative process around curriculum and term planning: “We’ve got the integrated brainstorms and things—at the start of the term or the end of the term— So we’ll have the Martu calendar stuff here, around which everything kind of centres…The two-way comes first, and then we go through the curriculum and work out how we’re going to fit. There’s always a way to link it.” One program facilitator explained how strong partnerships and a program embedded in the school and curriculum could support program longevity despite staff turnover: “If you have a really strong, functioning, planned engagement with a ranger program, then that becomes what you do as a school. And so when a new principal or someone takes over, they can go, okay, well we’ve got his meeting coming up, and we’ve got this happening next term—it’s much harder to change direction. And so, if you have really engaged local traditional owner groups and really engaged ranger programs and an active plan that’s underway, then I think that leads to sustainable outcomes.” 15 Many teachers described how Learning on Country provided experiences to engage students in other learning areas. One teacher provided an example of how two-way Learning on Country assisted student engagement in English and EAL/D: “We pretty much did poetry for the good half of the term and they were all writing eagle poems and fantastic descriptions, lots of adjectives…That’s all EAL/D stuff because they got to touch the dead eagle and all that kind of thing, come up with all their vocab and then get to use that in the poem.” Northern Territory experience – cross-curriculum focus In the Northern Territory, many of the educators and program staff described how the Science Pathways for Indigenous Communities program and resources contributed significantly to implementing Indigenous Language and Culture programs and, in the case of one school, a bilingual program. School administration staff acknowledged the high-level resourcing required to teach language and culture as a central feature of the school curriculum. The principal explained that teaching language and culture is: “…resource-heavy. And you need to have a commitment to get linguists and people, and the schools have to fund and do workshops. And there’s a lot of training involved in getting your staff up to speed.” The principal went on to explain the essential contribution the Program makes to language and culture through science-focused support and resources: “The Program that we run here, the language and culture program, wouldn’t be the language and culture program that we’re running. We can’t run the Program that we run here without [Science Pathways for Indigenous Communities facilitator]. And even though that’s [facilitator’s] sole focus when she comes out here, and that’s all she focuses on is the science aspect, it feeds into the biliteracy and the whole of the language and culture program.” Educators in the NT valued the science resources, not just for Two-way Science learning but to increase student engagement in areas such as numeracy and literacy. One teacher commented about the photobooks that are often used for recalling on Country science learning, Figure 4. Term plan with cross-curriculum integration Science Inquiry Planner CHECK Figure 4. Term plan with cross-curriculum integration saying that: “They’re used all the time, the kids are going through those books all the time. So firstly because they love seeing themselves in in it, but also they’re reiterating that learning over and over. And I think they get used down the track as well.” Another teacher at the school talked about the Pathways’ resources16 supporting literacy development: “Some of the Pathways units are to do with caring for Country and restoring Country. So [we] zeroed in on those. We used those headings, and then we used some of the material that [Science Pathways for Indigenous Communities facilitator] had done out there; we used it back in here. And then they wrote stories about it. If you go into my classroom, there’s a whole pile of prints like this on paper that the students go to all the time… they’re just [a] photographic record of what they did.” Northern Territory experience – collaborative resource development Program staff in NT talked about the process they go through to create resources, emphasising that “as much as possible, it’s done with the community. And drafts are always checked through with staff.” One staff member went on to explain how they facilitate the development of resources with a range of people: “Sometimes it’s with the students… I’ve been able to do one or two books in language where you actually have the student, or an assistant teacher, write to the pictures and put the story together in language, and then it’s translated back into English usually by [the linguist] or someone.” Another forum for resource development is the Aboriginal teacher training and development workshops, co-facilitated by Science Pathways for Indigenous Communities staff. This was identified by staff and educators as a valuable time to create Two- way Science activities and resources. One program staff member described how the process occurred during the workshops: “We’ve sat together and done planning together and developed activities together and prepared resources. And that hands-on activity or whatever has been designed to be translated into other areas once they’re back. They’re easily transferable kinds of skills. So language cards and snap cards and things to support learning in that way; worksheets that might be Western scientific worksheets that they’ve been translated into first language.” Figure 5 shows a Pintupi Luritja seasons calendar developed by the Program’s staff, a linguist, Indigenous assistant teachers, and Elders at Indigenous Language & Culture meetings in the Northern Territory. The calendar was displayed in all classrooms and was used to develop lessons and train staff. Western Australian experience – collaboration and cross-curriculum focus In Western Australia, program staff emphasised their close engagement with educators as they developed curriculum and term teaching plans. This enabled program staff to respond to the needs of schools while they were planning. Educators confirmed that this approach extended their preparedness and goals for Two-way Science lessons; provided a baseline for additional flexibility in response to student needs; and provided content for learning across the curriculum and general capabilities. One school administration staff member commented on what this collaborative approach looks like at their school: “So they, they’ll sit down and do integrated planning which they did two weeks ago, and that’s using all of the templates that [Science Pathways for Indigenous Communities facilitator] gave them. So they’ll do that once a term, if not a couple of times a term. And [facilitator] is sometimes here to do that planning as well. So this was as a consequence of going out on Country, which is amazing. Such a great way—Yeah and the kids will come and talk about it, they’ll come in here and fiddle with a few things and then go and stand and watch.” Educators in the WA communities felt that Two-way Science activities and resources were applicable across the curriculum, encouraging students to achieve a range of learning outcomes. One teacher commented: “Two- way Science is like the crux of engaging curriculum here. It’s from that literacy becomes engaging, history becomes engaging, all that kind of thing. From particularly the bush trips and Learning on Country, that makes everything more engaging.” This teacher went further, saying that: “Two- way comes first, and then we go through the curriculum and work out how we’re going to fit. There’s always a way to link it.” Many educators gave examples of how they draw on Two-way Science to enhance learning in other areas. One teacher commented: “We do different literacy programs, but I always try and tie what we’re writing, the major piece in with whatever theme we’re doing for Two- way Science.” Another teacher spoke about how she used program resources to improve student engagement and learning in maths: “I think if there’s a resource package put together like that, you can just have a look at what you could possibly do. You can edit it yourself. And that’s what I sort of did. I took what was useful, and I made some of it a little bit more difficult. So in maths, we were doing fractions and percentages. I did a pie chart with the kids 16 Pathways is a separate program from Science Pathways. with collecting fish. Because we needed to do something a little bit more advanced. You can see we do graphs quite a bit, looking at lung capacity and different things. But yeah, I think the pack is really good. You can just see, all right, this is a set amount of activities that I can do.” Educators indicated unanimously that the Program provided them with instructional teaching support and structure that was otherwise lacking. Many teachers interviewed gave examples of how Two-way Science content and activities were easily applied across learning areas and general capabilities. One teacher reflected: “As a graduate teacher especially, coming out and having the Two-way Science as a structure for integrated learning, because it’s not just science, it’s HASS [Humanities and Social Sciences], as well and English and maths too, is such a huge benefit. Like, it’s a really good framework to centre your learning around, and so it makes teaching a lot easier to plan engaging stuff. It’s really good.” Figure 5. Pintupi Luritja seasons calendar (with English version) Figure 5. Pintupi Luritja seasons calendar (with English version) Indicators of success Through the exploration of Outcome area 3 (The Program’s Two-way Science learning approach and resources are culturally responsive and embedded across participating school curriculums) with evaluation participants, a number of indicators of success were identified. These indicators align with the research literature (Person, 2012; Yunkaporta, 2009; Yunkaporta & McGinty, 2009) and were endorsed as being achieved by the evaluation team: • Two-way Science learning resources were user- friendly and helped increase teacher knowledge and confidence embedding traditional knowledges across curriculum areas • The Two-way Science approach was evident across multiple learning areas • The Two-way Science learning approach and resources were co-developed with community members and reflected place and local knowledges • The use of Aboriginal scientific knowledges in the classroom increased • Many Two-way Science activities were led by Aboriginal people • Non-Aboriginal teachers were receptive to the Two-way Science approach and resources, including community members leading on Country and classroom activities Outcome area 4: Increased teacher capacity in science education using on Country contexts This outcome has been revised to better reflect the program as implemented. The original outcome statement was “Increased teacher capacity in: Two-way Science using on Country contexts and cultural competence”. Definitions For this outcome, the term ’teacher’ includes non- Indigenous teachers, Aboriginal and/or Torres Strait Islander teachers, Aboriginal and Islander Education Officers (AIEOs), Aboriginal Assistant Teachers, Aboriginal education assistants and Aboriginal languages teachers. Throughout the report, where appropriate, the term “educators” is used to collectively refer to these positions, which are often filled by people from within the local community. The term ‘capacity’ is a broad concept with many applications but is generally defined in the literature as the individual or collective recognition of existing abilities to address obstacles or problems (Taylor & Govan, 2017). Successful capacity improvement initiatives in education describe building on existing strengths in practice, skill areas and relationship development and are best initiated and run by groups with shared goals to achieve mutual benefit (Taylor & Govan, 2017). Increasing teacher capacity against the national professional standards covers improvements in the areas of content and pedagogical knowledge, professional practice within the school setting and professional engagement within professional and school communities (Australian Institute for Teaching and School Leadership (AITSL), 2011; Treagust, Won, Petersen, & Wynne, 2015). For science teachers, national teaching standards outline their expected competencies such as: engaging students in continual exploration in constructive and exciting ways; providing feedback that promotes development; promoting critical thinking and logic; and demonstrating an understanding of relevant science concepts and interrelationships (Australian Science Teachers Association (ASTA), 2009).17 17 For a more comprehensive description of professional science teaching standards, see the Australian Science Teachers (ASTA) website: www.asta.edu.au The term ‘inquiry’ can be defined as a process by which students learn, particularly by actively inquiring through ‘thinking and doing’ related to a phenomenon or problem. From a program perspective, a science inquiry also refers to activity-based science units of teaching and learning. The national science curriculum comprises science inquiry as a key strand (along with science understanding and science as a human endeavour), referencing specific competencies such as questioning and predicting, planning and conducting, processing and analysing data and information, evaluating and communicating. ’On Country’ is a term used by many Aboriginal peoples to describe being on the lands or place, with traditional connections established through family lines (Shay & Wicks, 2017). Debra Bird Rose describes Country as multi- dimensional, including people, animals, plants, earth, soils, minerals, waters, air and Dreamings, and overall, a ’place that gives and receives life’ (Rose, 1996, p.7). Learning and development workshops in the Northern Territory Evidence from interviews and examinations of teaching materials shows that, overall, educator ability to plan and undertake Two-way Science inquiries using on Country contexts improved considerably as a result of the Program. In the NT, evaluation findings emphasise the role that most Aboriginal educators (locally known as assistant teachers) take to lead the planning and delivery of Two-way Science, both in the classroom and on Country. Historically, the employment of Aboriginal teachers and assistant teachers to teach in first language in the NT as part of bilingual education is no longer carried out by most schools and is continued in some schools with Indigenous Language and Culture (ILC) programs. This aspect of the NT school system enables the Science Pathways for Indigenous Communities program to build on the existing capabilities of Aboriginal teachers and assistant teachers to plan and deliver Two-way Science teaching, often in collaboration with non-Indigenous classroom teachers. For Aboriginal teachers and education assistants in the NT, opportunities to collaborate on language resulted in a strengthening of Two-way Science teaching practice and confidence. With Science Pathways for Indigenous Communities’ support, a series of workshops for Aboriginal teachers and assistant teachers were attended by multiple schools from the same or closely related language groups. In March 2017, the case study community hosted the largest gathering of Aboriginal and/or Torres Strait Islander education staff for decades (CSIRO, 2019). The workshops focussed on teaching on Country and in language, and the participation of a linguist meant that much of the discussion could take place in participants’ first language (primarily the closely related languages Pintupi Luritja and Pitjantjatjara). Program staff relayed comments from many of the assistant teachers about how much they learnt from sharing ideas and resources for teaching children about their country and language. “It was clear how supported all these teachers, especially the younger teachers, felt being at this workshop with so many others, several of whom were very experienced” (Report 2019). Another Science Pathways for Indigenous Communities staff member emphasised the value of working and learning in first language: “It’s just a way for people to, as you say, feel empowered and run the workshops themselves. And [the linguist] is so critical in those workshops because he’s supporting people to plan in first language…just giving space for people to work that way is very important.” The workshops have a strong focus on teacher professional development and sharing best-practice teaching approaches within and between schools. As part of their practice contribution to the workshop, teachers modelled a “team-teach with classroom teachers” approach (“I learn from you, you learn from me”), which provided inspiration to several assistant teachers to adopt practices at two other NT schools. The teachers also adopted an approach shared by another school during the 2016 workshop: set up a Luritja word wall in the classroom and test students regularly on how they are improving at reading this word wall. Outside the workshops, Aboriginal teachers and assistant teachers took their learnings back to the school environment to work collaboratively with staff to action two-way learning goals. A program staff member described the assistant teacher role in weekly planning activities with the principal and teachers as “fairly strong”. Building on existing capacity Across both jurisdictions, feedback from teachers, principals and program staff indicated that science teaching capacities varied and that the Program could respond to these varying degrees of science skill and knowledge. Program staff and consultants recognised this diversity and general lack of science expertise amongst the teachers: “…we can’t expect teachers to be biologists, and realistically a teacher coming here, how would they know what species of fish are out there? Or how would they know background about the reptiles that live around here?” (program coordinator). In addition to possessing specific content knowledge, program staff also highlighted the need for teachers to understand the theory and process underlying two- way learning, as exemplified by one staff member: “… when teachers really understand what Two-way Science is about, that’s when teachers value traditional knowledge because they see it and have a bit of an understanding about it, and then also see the connections between that and western science…that becomes fundamental to the whole project, the learning process and the planning process and the adult learning.” In addition, program staff felt that understanding two-way learning can help the often significant learning curve teachers face when starting work in remote schools. This sentiment was confirmed by many of the educators who participated in the evaluation, with one teacher summarising what many others told the evaluation team: “So you’re really not prepared for remote teaching with a university course. Which means that when you come out here, you’re really learning on the job… So much of what you’ve learned and the tools that you’ve developed as a prac student don’t apply.” In one community, the school’s education and training partners observed that despite teachers’ lack of science background, the Program is highly engaging for teachers, improving their science teaching skills and knowledge: “They’re not science people, and [the program facilitator] has got them into this, interested them in science. It makes them better teachers because, all of a sudden, they’re learning words, they’re learning context—that’s really value- adding big time.” This key program partner goes on to emphasise the change in teacher confidence and ability they witnessed among teachers: “They were relieved that there was a way that could interest them in science again.” How the Program builds teacher capacity The findings from interviews with school staff indicate that the Science Pathways for Indigenous Communities program has increased teacher confidence, knowledge and classroom experience in science education using on Country contexts, as well as “fast- tracking” and improving the cultural responsiveness of many non-Indigenous teachers. In particular, face- to-face support, hands-on activities and learning on Country were identified as key methods. Face-to-face Across both jurisdictions, an important component of the Program is face-to-face support and development with educators. Educators of varying experience described the benefits of formal and informal professional development that was contributed to or led by the Program Facilitator. One principal noted that this support for teachers is critical, especially where teachers are starting with more general skills: “Particularly the teachers for the primary school they’re not specialists, they’re not scientists or they’re not trained in science…the best part of it is having somebody like [program staff member] come and spend a week with us, to launch the project but to wrap up the unit is really, really helpful. And it gets the teachers focused in moving on.” Hands-on Many teachers from across the communities reported that the hands-on Two-way Science activities facilitated by program staff greatly influenced their Two-way Science education capacity. Teachers described the experience as being highly relevant and instructive, increasing their confidence and rapport with key community members. Teachers talked about various benefits to learning alongside local partners and science pathways staff and how this impacts their teaching approach: “We actually got to go through the exact process, step by step, [staff member] worked really well alongside me, and then I actually got to see it, so now it’s a lot easier to actually go through it, and then I know, okay we’ve done this bit, we’ve done that bit. I guess when I first came, and I was like, okay, we just go out on Country, and we do this and this, but now there’s actually you who go to contact these people, you think of the activity as prior knowledge…It’s been good.” Science Pathways for Indigenous Communities program staff recognise the value of regular hands-on learning for teachers as a critical element of their professional development, with one program staff member emphasising that: “Going and doing two-way science is the best professional development.” They go on to reflect on longer-term goals for teacher capacity: “You…want teachers to be operating independently and to learn”, giving an example of one teacher who has increased their capacity in Two-way Science teaching: “I’m sure she still appreciates that contact and support as well, but she’s just an example I guess of when a teacher does really take it on and build it into their program without us being there.” Learning on Country: Appreciating the connections Increased opportunities to incorporate on Country learning into the curriculum and the Program’s structured approach to science education on these trips are another reason that educators believe their confidence and capacity to teach science has improved. One principal commented on the importance of the Program’s promotion and modelling of on Country teaching to expand teachers’ experience and skills: “I think we’re learning to get, as we go along—the more you go out bush and the more you look at the Australian Curriculum and the science curriculum…you understand their world view a little bit as well. Then you can appreciate the connections. It’s a lot easier to do what teachers call back mapping.” One teacher illustrated a point made by many educators that on Country teaching and learning was a preferred approach for both teachers and students: “If I could take my classroom and live it outdoors, I would. And this two-way program certainly does that…And when you bring it back, but you just bring in a Western idea. You can give them a certain bit of guidance or perhaps show them something, but they’ll go off and do it their way…They’ve all got their own ideas.” Across both jurisdictions, teachers commented on their increased local knowledge and ability to connect this to Western science and the curriculum. Educators felt that this created efficiencies in their science curriculum and lesson planning. For example, one teacher commented on how this applied in their classroom: “So just going on from the fishing and going to goanna, so you transfer that knowledge that you’ve learned as a teacher from the fishing experience, and you transfer it to the goanna, or the [pampirta]…so what are we going to do to measure? We’re going to go out there and measure it. So what do we take? So they go and get the metre ruler, or they go and get the stretchy ruler and then they go…they’re instantly thinking we’ve used this tool before so we’ll go and use it.” For another WA teacher, the on Country learning represented an appropriate way to learn about local Aboriginal culture and to focus on classroom learning: “I really like the on Country learning, the Two-way Science… they still live on their land with all of their stories still there. For me, I think that’s the only way to do it.” Program documentation also provides evidence that by following on Country learning, teachers were better equipped to facilitate a broad range of inquiry discussions about topics like local birds and plants, making medicines from local plants, simple animal surveys, endangered animals, the witchetty grub life cycle and making a seasons calendar: “Going on bush trips helps to strengthen teachers’ relationships with the community and to spend time with the students.” (educator) Building teacher capacity through relationships Many non-Indigenous teachers spoke about the value of developing relationships with others as beneficial to their confidence and ability to plan and deliver successful Two-way Science activities with their students. The Program encourages school staff to work in a collaborative way, both within the school and with families, community members and key program partners. Evidence indicates that many educators see Science Pathways for Indigenous Communities facilitators as mentors in learning how to engage with others to enhance their teaching approaches and outcomes. One program staff member emphasised that relationship development for teachers can be as simple as having a go at an activity: “That initial just going out and having a go at something, as messy and imperfect as that may be, the most important outcome from that is that you have developed a relationship because you’ve done something together.” Reflecting on the NT experience, one program staff member highlighted that relationships and attitudes are as valuable as teaching skills and experience when it comes to Two-way Science teaching success: “I don’t think it depends on how new the teacher is entirely; I think it depends on their attitude and interests, how they build relationships with the assistant teachers they’re working with and how mutually supportive those relationships are.” Teachers in the NT valued their relationships with Aboriginal assistant teachers, valuing opportunities to collaborate to improve their teaching approach: “…it’s informal chats, and whoever comes to your classroom you know you chat with that AT [assistant teacher]. And they see it as a good thing off you go, kind of thing. But there’s a lot of discussion going on in the classrooms between the ATs and the teachers. Whatever project you’ve got in mind as a teacher, you discuss it with them and see what they want. And that might be something the school could do better. Having meetings with people where we sit down around a cake and a cup of tea and have a really good talk.” Other teachers emphasised that collaborative working relationships with local rangers were required to maximise student learning opportunities. One teacher illustrated the ongoing discussion required to deliver the curriculum, while also being flexible and adaptable within partnerships: “We’ll do like a big, big brainstorm and come up with the ideas…but then it’s actually quite hard to plan it in advance…we were thinking that was going to be lots of country types and things. But then it turned out to be all about echidnas.” Teachers across both jurisdictions gave examples of partnerships with Aboriginal knowledge holders and other science professionals to enhance learning outcomes for students as well as providing classroom support. One teacher gave an example: “One of the best things that’s happened is bringing in scientists, real scientists…we took her [scientist] out on a walk and…she brought a refractometer and—we were doing pollination with my kids—and looking at this thing and looking at it and thinking to themselves, ‘What is this?’ They’d never experienced it…She was able to fine-tune and say all the technical terminology with the kids, and she had those little discussions with them, and that’s what I liked about the scientists is that they do a little—the teacher’s busy running around like a mad chook sometimes and that’s how life is in the classroom for us. It’s very busy.” Additional resourcing The development and use of program resources across the curriculum are highlighted extensively in Outcome area 3. However, in the area of teacher capacity, it is worth noting that the use of program resources was well documented as a significant support for teacher confidence and capacity. School-level resourcing and support enable teachers to undertake more regular on Country learning than they would otherwise. One teacher echoed the sentiments of many, explaining that being interested in two-way learning is not enough to ensure the trips happen and that resourcing is critical: “The resources and the support with organising trips and that kind of thing makes it really accessible. So even teachers who say I’m really into going out bush or things are still able to do it and still able to engage with it and see the importance of it. Yeah, the fact that it’s a whole-school thing makes that a lot easier.” Science Pathways for Indigenous Communities NT has provided resources for classroom teachers and Aboriginal assistant teachers in planning and conducting Two-way Science activities on a range of topics such as vegetation surveys, local plant and animal habitats and bilingual texts based on field trips relating Artist: Tinarra Title: Paddy melons, 2019 Artist: Tinarra Title: Paddy melons, 2019 Story: Jampa nganarna yaninayirni kuyikarti yirrinayirni janinyi pikimalan. Palarrangu pantinyayi jampa nalkuluman wurrkujarruluman. Turlpanyayi karntimarta martaji rurtunga (When we go hunting for bush tucker, we see lots of melons growing. They smell good, but when you eat them, they are poison. They grow next to the road into [the community]. to bilbies and waterholes. One teacher, who has moved on from a Science Pathways for Indigenous Communities school, provided feedback to NT Program staff about their continued use of their Two-way Science skills: “I have certainly been spruiking you and the [Science Pathways for Indigenous Communities] program. We were going on a bush trip looking for tracks, and I used some of the [Science Pathways for Indigenous Communities] tracks worksheets and everybody loved the resources!” (CSIRO, 2019). Improving capacity in the classroom Many teachers commented that their capacity in the classroom had improved as a result of the Program. When Science Pathways for Indigenous Communities staff act as a resource, to provide regular advice and support to teachers about how to approach Two-way Science activities, it builds teacher confidence and ability. Teachers also reported being able to test new ideas and try them in the classroom, therefore creating positive teaching experiences and gaining confidence from this process. One teacher talked about how the program staff member: “gave me ideas of what I could do with the kids”. The teacher explained the process they went through with the program staff member as a new teacher at the school, which included asking the students what they’re interested in: “I showed him my planning for first term, and he went through the planning with me and said what would work, what wouldn’t work. So it is good to have a bit of an idea of what works in the area you’re in, especially when you’ve got no idea. Like, I had no idea myself how it would work, so he was very helpful.” In addition to classroom-specific skills and experiences, several staff members highlighted the valuable engagement, facilitation and negotiation skills demonstrated by program staff. For example, one teacher commented: “We had heaps of rangers, heaps of kids and just seeing, watching how he very skilfully facilitated everyone to come around in a circle so that everyone could listen to the rangers, which the stuff like that, every time I go out there’s something new to observe and be like, okay, that’s how I’m going to do that, that kind of thing.” Another teacher believed that the engagement skills and experiences gained through the Program improved her confidence to engage with local Aboriginal and Torres Strait Islander groups to support classroom work: “Even the confidence to go speak with local community members and whatnot, I feel like I wouldn’t be so nervous, going and speaking to someone in Perth. I’ve had good experiences here.” Having a regular point of contact was also seen to promote program continuity and teacher learning. The program facilitator was described as: “A steady constant to make sure that [the Program] is kept up…I’m really comfortable and really into it, but every time I go out with [Science Pathways for Indigenous Communities facilitator] I’m like okay, this is a really good idea, or he’s got really good ideas for follow up activities or how to use equipment, all that kind of stuff which none of us are experts in by any means at school.” Importance of language Overall, educators felt their science teaching capacity had improved. However, they also talked about better understanding the importance of language and the role they played in supporting first languages in the classroom. While local teachers and community members are acknowledged as being best placed to teach in language, where this is not always possible, the Program improves teacher capacity to incorporate languages into Two-way Science activities through the development of bilingual resources and through prioritising the inclusion of Aboriginal knowledge holders. Participating non-Indigenous teachers unanimously acknowledged that local Aboriginal knowledge and culture should be taught by local language groups and that local teachers and family are critical to this. Across all communities, parents, family and local teachers were observed to teach or provide valuable contributions in and out of the classroom, including when they used first languages to help students understand classwork and instructions better. One teacher in WA commented on the importance of improving their ability to understand and speak local languages in order to improve their teaching ability: “I’m learning from [the students], every day. Even the language words, I’m learning and practising them, and making sure I’m using them enough. With [name] who’s my Aboriginal teacher in here, she is really good with the kids as well, with language. Whenever we can, she’s writing on the board in language trying to get [the students] to be able to recognise the spelling of… Nyangumarta words, and to be using them.” A local teacher in this community went on to illustrate the importance of being aware of language groups and that classroom teachers require the skills to navigate language diversity: “When they come and visit their relatives here—their extended relatives—or, for ceremonies—then, the children come to the school here. So, the teachers have to make sure that they fit into the Program. And, they’re taught the English side of the syllabus, but these teachers teach them, also, the language and the cultural things, as well.” One of the teachers noted that even when local teaching resources have been developed, possessing language and local cultural knowledge is critical and not something that many classroom teachers arrive with: “First of all, my barrier to it is that I don’t speak the language. I know some words all right, but how would I be able to take that book that has that photograph— there’s a story behind it. All those sorts of things you have to think about or process as a teacher before you even think about pulling that book out and using it.” Culturally responsive teachers There is strong evidence from interviews, field notes and reviews of materials to indicate improvements in the cultural responsiveness of many non-Indigenous teachers, including teacher confidence and skills in engaging with Aboriginal knowledge holders to deliver two-way learning activities. While teachers reflected on their personal attitudes and existing cultural awareness process, many teachers also reported how the process of transforming and progressing along the continuum of cultural competence was positive and rewarding. This transformation was largely motivated by the Program and resulted in improved teaching experiences and capacities. Principles of cultural responsiveness among educators are incorporated into the Program in an explicit and informal manner. Program partners such as Desert Support Services acknowledged the Program’s intention of exposing teachers to the strengths of the local culture and was also backed by Elders: “Yeah, get to know about our thing, what we do out bush, you know. The teacher needs to know about it as well.” NT program staff highlighted the role that Science Pathways plays when there is a gap in cultural awareness training for teachers: “A lot of places you don’t [do cultural awareness training]…but it’s possible that there isn’t people to actually facilitate that. There’s so many different language groups and different country…I think the trips probably provide the best level of that that’s available. Science Pathways works.” In some cases, the Program is seen by staff to fast-track teachers’ cultural awareness: “I think it supports them to get on top of cultural knowledge and incorporating community into the classroom and things much faster than if it wasn’t there. Getting that support to help everybody get out on Country together is really critical.” One of the principals involved in the study emphasised the value of teachers learning from Elders and other families, calling it knowledge that: “Cannot be found in a book and that gives an insight into culture and an appreciation of the land around them”. The principal went further, saying that staff connect this knowledge to the Australian Curriculum and identified that it helps students reach their learning outcomes in all areas. Teachers were often observed to take a step back to allow space for local Aboriginal knowledge holders (young and old) to lead the teaching and knowledge transfer. One teacher commented: “Usually the kids are pretty good with just sharing the knowledge…Again, it’s not really me teaching them, it’s them teaching me. Because on the weekly bushwalks, gosh the amount of things we talked about and they’re telling you how the trees change and all this, I’m like okay, I didn’t even know that.” Several teachers spoke about their secondary role to Elders and other community leaders: “I guess [the Elders] passing down their knowledge, it’s something I can’t do…it’s really special to [the Elders], it’s something [the Elders] can share, that cultural understanding keeps growing, you’d hate for some of these things just to be stopped. If [the Elders] are not involved, we can’t teach it all. Yeah, I guess [the Elders are] just vital to it.” Another teacher emphasised their role to support a safe space for culture: “And more importantly, it’s not about us being the people who know about it. You can’t, as a person, be expected to box up a culture and be the person that teaches it. We facilitate the community to come in safely to do that.” The case study interviews suggest that the Program’s staff support and mentor educators to develop attitudes of respect and trust and develop a deeper cultural perspective. One principal emphasised the importance of this: “A lot of people need that kind of mentoring or—and I’m speaking in terms of teachers and principals here— they need to go through that process of letting go and also understanding that life looks different from others’ perspective.” Teachers who demonstrated an attitude of respect and a willingness to learn and who sought to develop trust and understanding were observed by researchers to be champions of Two-way Science. Some of the teachers most engaged in Two-way Science spoke about their approach. For example, in one community, a teacher commented: “It was just a matter of transferring myself into being trusted by the mob, perhaps living some of their experiences…They’ve got all their country here. I’d never experienced the hot desert, ever, ever, so that was a new beginning for me as experience, life in the desert.” In a different community, a teacher reflected on the school’s respectful and reciprocal relationship with families: “We don’t know what’s going on in your business, but we know how pivotal it is. And that’s going to impact on here [school] but guess what, that’s okay, this is [the community’s] culture and we, to the best of our ability, can advocate for that.” Another teacher commented expansively about this topic: “I’ve had conversations with the Elder…and one day we were just sitting on the veranda outside the classroom, and he spoke about—I couldn’t go up to him and ask him, ‘…we’re going to do seasons this term,’ da-da-da-da, ‘This is how we’re going to do it.’ [Instead] I said, ‘What’s this wind that’s just come in?’ Well, he just took off in language, spoke about it. He says, ‘I’m going to go into your classroom now, and that’s what I’m going to talk about, this special wind,’ and that’s how we initiated, how we began seasons. I recorded him. Q: Just that tiny little question just led to all of that. A: Yeah, led to all that and an amazing experience with them. Amazing experience for me because I was able to record what he wrote down and work through with the children about seasons. We had the language names all there, and, basically, at the end of it, all the children were tuned into the different signs that the winds were in, and then we went into things like making vanes, testing it out there in the wind, if it worked, if it didn’t work. A lot of—and then I had a little wee kid come in, and he saw it, and he looked at it, and he blew on it, and then it turned around, but the ATs, they came, and they enjoyed making the wind vanes, so when the mob come in and help in the classroom, we all do it together. Kids and all and adults. And that’s that two-way thing happening. I call it cross-pollination sometimes.” (teacher) Indicators of success The indicators of success raised by evaluation participants, and supported by the literature (Burgess & Cavanagh, 2015) and the evaluation team for Outcome area 4 (Increased teacher capacity in science education using on Country contexts), comprised: • Teachers felt confident to plan and deliver Two-way Science education on Country and in the classroom • On Country learning increased in frequency and focus • Non-Aboriginal teachers effectively facilitated Aboriginal assistant teachers, Elders and other knowledge custodians to lead Two- way Science activities and learning • Aboriginal and non-Aboriginal teachers had overall improved capacity in teaching science • Non-Aboriginal teachers became more culturally responsive, including increased understanding of Indigenous ecological knowledge, local cultural knowledge and language and skills engaging with Aboriginal knowledge holders • For assistant teachers in the NT, opportunities to collaborate in language resulted in a strengthening of Two-way Science teaching practice and confidence. Outcome area 5: Increased aspiration (of students), sense of value and school belonging18 Definitions Student aspiration is defined as the student’s ability to be cognisant of their future educational and career goals. Often, the education and career aspirations of Aboriginal and Torres Strait Islander students directly reflect, promote and validate their culture (Lewthwaite, Osborne, Lloyd, Boon, & Llewellyn, 2015). Some researchers have noted cross-cultural differences in ‘aspirations’, with the Western view of aspirations linked to self-fulfilment and freedom not necessarily prioritised by other cultures, including the wide spectrum of aspirations of Indigenous Australians (Bulloch & Fogarty, 2016; Walker, 2006). The factors supporting the aspirations of Aboriginal and Torres Strait Islander students include: a strong connection to culture, land and community; ensuring students are valued and empowered; early intervention to influence subject choice; wide promotion of professional pathways; positive presentation of images of Aboriginal and Torres Strait Islander students; authentic involvement of families and key community members in building aspirations; innovative and engaging curriculum and pedagogy for young people that are connected to their lived experience; strong peer support networks; and efforts to increase the desirability of higher education (Behrendt, Larkin, Griew, & Kelly, 2012; Gore et al., 2017; Parkinson & Jones, 2018). Pedagogy is defined as “any specific and/ or defined teaching/learning practice or process that focuses on supporting and/or engaging and/or improving student outcomes” (Burgess et al., 2019, p.298). A sense of value, also called “self-worth”, is one element that contributes to how we conceptualise ourselves and is considered a psychological strength, contributing to children’s resilience and wellbeing (Gavidia-Payne et al., 2014). Research indicates that positive self-worth for Aboriginal and/or Torres Strait Islander young people can be built through a strong sense of Indigenous culture and having a sense of pride in one’s Indigenous heritage (Dobia & O’Rourke, 2011); teaching practices that embrace both Western and Indigenous knowledges (Hart, Whatman, McLaughlin, & Sharma-Brymer, 2012); connection to Country and a caring for Country role (Kingsley, Townsend, Henderson- Wilson, & Bolam, 2013); and participation in an intergenerationally connected community. Indicators of positive self-worth for young people includes having pro-social friendships, positive self-esteem, and self-regulation (Hopkins, Zubrick, & Taylor, 2014). School belonging is defined in a range of ways; however, school belonging generally describes to what extent the student feels accepted, included, respected and supported by individuals within the school environment (Allen, Kern, Vella-Brodrick, Hattie, & Waters, 2018). For Aboriginal and Torres Strait Islander students, the school must promote an environment that is engaging, nurturing and focused on achievement. Further, schools must have scope to diligently address racism and engage the student’s family members where appropriate (Riley, 2015; Price, 2012). Identity, wellbeing and self-value within the Program The promotion of student identity, value and wellbeing is a common program theme, reflected in school reporting and program documentation. For example, program documentation included in the review emphasised fostering student cultural identity and wellbeing through the inclusion of Aboriginal leadership in education. An assistant teacher at a school described how the Program supports the connection between culture and a sense of identity: “…if you don’t teach any culture that means children have no identity. It’s better to teach the kids their culture, our culture. Kids can learn both ways.” One of the teachers in a case study community illustrated how their Science Pathways for Indigenous Communities facilitator had a conscious intention to support the wellbeing of students, including young men who are facing the responsibility of being emerging leaders. The teacher observed that he: “… keeps in contact with them. He makes sure that they’re okay and their mental health and wellbeing is cared for.” Another teacher referred to the Program as a catalyst: “…in bringing back that importance of wellbeing, general health, getting the mob out on their country.” A program staff member working across both jurisdictions emphasised that they see Science Pathways for Indigenous Communities as: “A wellbeing program above everything else because it’s a program that engages the community”, noting that in WA, as a result of Science Pathways for Indigenous Communities, they had observed more adults from the community engaged in the schools. The staff member went further, explaining how the Program builds on character strengths of students who have: “got an opportunity to be learning more about their own culture. 18 For Outcome areas 5 and 6, there were fewer quotes from students. Students did not often express concepts of aspiration, school belonging and engagement directly. This could have been due to translation issues or simply that students did not think in these terms. In that process, they are seeing their adults, their family being highly valued by the school because these people have got knowledge that the teachers and the schools can’t have. So the kids are seeing their adults valued, their knowledge valued and the knowledge that the kids have got being valued. It has to have an effect, it has to promote some semblance of pride in the fact that hey, I’m a Martu kid or I’m an [Anangu] kid, and I can be proud of that.” “It [Science Pathways for Indigenous Communities] actually empowers people. It’s not a small thing. It’s a little thing to do, but it’s a massive thing for [the community]…every kid wants to learn about the natural world around them, and it’s enriching to understand the context of the culture, one of the oldest in the world, that we are a part of…one of the reasons why it works is because it’s valuing traditional ecological and cultural knowledge. It’s saying, hey, that’s important and it’s relevant, and it’s scientific, and it’s clever. There’s [maths] and all this stuff involved in it.” (Desert Support Services worker) Connection to Country and culture Participants from all communities associated positive student wellbeing with incorporating language and culture into education; an approach that is integral to Science Pathways activities. Younger students talked primarily about being “happy” and that Two-way Science was “fun”, and that it did not feel like they were learning when they were out bush. Most students also preferred on Country learning to the classroom environment. Older students talked about the importance of learning language and being connected to culture and “the old days”. One student, who did not have a local cultural connection, commented: “I know [other students] are really happy when they go out altogether as a family and cooking food together, and it makes me feel really happy seeing everyone.” This student went on to reflect on what she enjoys about on Country learning: “My favourite thing is all being together and talking about what the old people used to do and make stuff. That’s what I like doing here mostly.” Across communities, a range of adults, such as teachers, assistant teachers and Desert Support Services staff, commented on how they perceived the aspirations and future of their students and young ones, which were often linked to culture. One teacher who had spoken with her students about their futures believed they were motivated by cultural values and a connection to their local community19: “What they would like to do is to be able to stay on their Country and earn money. They don’t want to go off and be a great success anywhere else for any other reason. And if you ask the kids here—you ask them that question generally, they come back with something that’s going to improve the whole community.” Another teacher believed that some of the students aspired to be strong role models for the younger ones. One assistant teacher wanted their boys to be able to read and write, finish school and essentially be able to function and thrive in society, especially within Western systems. The role of Elders and cultural leaders When Science Pathways for Indigenous Communities’ Two-way Science and on Country learning aligned with community goals and there was strong leadership and participation by the community, then the Program had a positive influence on the wellbeing of not only the students but the broader community. Across all three communities, interviews indicated that two-way learning is supported by families, Elders and the broader communities. Elders in one community spoke about their education goals and hopes for their young ones, including to: “Be strong in their culture and probably find a job. Q: Be healthy? A: We want all that.” These Elders, who are also employed as rangers, and lead on Country learning, talked about how program activities out bush have contributed to an increase in showing respectful behaviour: “…sometimes we dig a humpy to make windbreak to show them. Even tracking. Kids are growing strong, coming to school, and it’s really good. Usually, we have a nanna worried, we can’t get this kid involved with other kids, they’re like, ‘Kids are fighting at school’, that’s sort of changed. Yeah, it did. That’s changed, yeah.” The Elders went on to reinforce the wellbeing benefits to students as well as the broader community: “So it takes the trouble out of the school and out of the town as well too if you start taking the kids out to go bush. So if you can arrange an excursion or something at one time to take the kids out, then there’s much more learning. They do leave their problems and worries here and the rest just—and I see them all happy.” Many of the Elders and family interviewed spoke about on Country learning activities as opportunities to pass on their knowledge to the young ones and strengthen intergenerational roles and relationships. In one community, a group of women Elders reflected on the importance of this intergenerational transfer of 19 Guenther, Disbray, and Osborne (2016) also discuss this. knowledge: “So, it’s a chance with us Elders to go out and teach Two-way Science. For us, we learnt a lot with our old people when they was there with us, the bush life. For me now, I’m proud that I did learn from mum, now I want to pass it to these young people.” One of the Elders spoke about how participating in the Two-way Science activities has strengthened her relationship with many of the young students: “I like it [being involved in the school] because I want to learn these young ones—teach them what I know from way back. Q Do they welcome you in? Yeah. They have been.” Not only was the process of passing on knowledge identified as important, on Country learning activities were seen by community leaders as a way to strengthen community knowledge about how to understand and relate to their Country. One Aboriginal educator spoke about this: “Yeah. It’s important. When they go out, tell them more things. When we explain—the trees have bush food, bush medicine, all that. Yeah. About the dreaming. Yeah. That’s what we tell kids.” This was also reflected by parents in another community, with one of the parents stating “Yeah. That’s good for learning culture and always telling story. When she go out bush, and she tell me a lot about animals, bush tucker and culture.” In another community, Elders spoke about a strong connection with their Country as providing support to navigate Western systems: “That’s what we trying to put it to our children, and when they grow up because the world, the thing is changing every time. Your world [Western], that one—because culture, we learn culture straight away from our old people…That’s been keeping me strong, that helped me to think about the Country.” In the WA communities, Elders also spoke about their wellbeing when they’re involved in Two-way Science learning with students. In one community, Elders spoke about how on Country learning keeps them physically and mentally active and connected to their culture: “I think it’s great. It’s good. Keeps people active about our animals, what’s out there and knowing what’s out there. Teaching our kids. Learning. They’re the ones—about the tracks and about bush food. Yeah. Yes. They’re happy when they want to go all the time.” Another Elder spoke about feeling proud at knowing the students were engaging with their knowledge and sharing it with others: “That makes me proud. We don’t take them out on Country for nothing.” Building school belonging One behaviour support teacher spoke of the central role that culture plays in not only curriculum planning but strengths-based engagement with students: “What you’re doing is advocating for a culture in a respectful way, without a doubt, and providing that platform which is so important. This scrappy piece of paper [seasonal curriculum planning map] is the centre of everything. If I need to calm a student down, I can start talking about this. My planner as much as possible involves coming in and going okay, we’re in May, I’ll start talking about that.” Generally, teachers recognised the role that local Aboriginal culture plays in student wellbeing and how important it is to demonstrate that culture is respected and valued by the school. One teacher commented on the importance of bush trips to families: “It’s really nice to see more families out. Even if they’re not with us, they’re just sort of like, oh, we’re going to have a nice day out bush. And it’s one of those things that everywhere you go, you will probably see someone out as well, which is good… I think it’s always been like that. I think at home, for kids, they go out bush a lot.” Some teachers and family commented on the requirement of students to constantly: “walk between two worlds”, learning to be strong in their culture while also navigating Western systems, including education and employment. One teacher commented on the pressure this created for students and how two-way learning helps students to navigate this interface: “[The students] learn so much from their families, and it’s good that it’s at school as well…It brings more consistency in the thought process, knowledge and cultural… I’ve never had to code-switch between school and home, whereas these kids do… but if you can bring another element of their cultural life or their home life into school, then I think that’s really good for them. It’s bilingual, but it’s sort of a— not just a language way, but a cultural way as well”. Strengthening community In some communities, Aboriginal and/or Torres Strait Islander rangers are a key stakeholder and contribute not only to the logistical side of on Country learning but also to improved student wellbeing through their roles as education or cultural leaders or as role models. The Program benefited from ranger groups and also helped strengthen them. One Desert Support Services staff member working within WA described how participating in the Program’s on Country learning builds on the individual strengths of local Aboriginal rangers and reinforces the importance of their knowledge and position as role models for younger students: “And from a land management or ranger sector, you’re seeing individuals grow in that capacity, to lead, and step up, or be that role model, or be that other individual that the students can learn from. Whether it’s their direct family member, or indirect, it’s the local mob. It’s not just another outsider doing it.” The staff member expanded on how the Program creates opportunities for people’s individual strengths to be acknowledged and shared, contributing to broader community building and development: “I think it [sharing knowledge and learning] goes both ways…Seeing some rangers that appear quiet, I guess, but then they come out of their shell and show and display confidence of their knowledge. I guess. I think [ranger’s name] is a good example. [They are] almost too generous with [their] knowledge, which is great, but you talk to [them] and [they] always likes to tell the story that early days and on, [they] didn’t think people wanted to listen to [them], and now [they] a senior ranger, and on the school council and—a lot of people looking up to [them]. But [they] also wanting to learn as well, you know, from the scientists—and these—generally, these are all [their] words. Yes, just having that inter-generational dynamic of learning, and role modelling, is always a great positive. Yes. Just having a positive example, really.” Artist: Jessica Title: Parrjanarni warran (Looking at the land from above), 2019 Artist: Jessica Title: Parrjanarni warran (Looking at the land from above), 2019 Aspiring to be what you can see Across the three communities, student aspirations closely reflected the roles that were visible within their local communities, which included teaching, land management, mining, community and health work and sporting aspirations. Some older students also talked about leaving town and heading to more urban areas. Student aspiration was influenced positively by their engagement with local family and community members in a professional or leadership role. This included increased opportunity to see rangers and assistant teachers leading on Country learning. One Desert Support Services staff member summed this up by saying: “I think it’s wonderful for them to see their family out there doing this awesome job. Yeah, a job that does have them out on Country doing these things that are important to them. It’s a job that makes sense to them. It is something to aspire to.” Across all three communities, participants in the evaluation, especially educators and rangers, commented on the role of Science Pathways for Indigenous Communities in exposing students to role models while implementing Two-way Science on Country. One teacher emphasised how the Program expanded opportunities to links students to relatable aspirational pathways: “The ranger program exists outside of it [Science Pathways for Indigenous Communities program]. But I don’t think that—only the kids that have the family members would get that opportunity to see that and they wouldn’t necessarily see them at work. They would only see them going off and hear about it or be going bush. It wouldn’t be them getting to actually do the stuff themselves. But I also think without [Science Pathways for Indigenous Communities facilitator] it wouldn’t be a sure thing that it would happen.” In communities where the Program operates, students were exposed to additional roles, including ecologists and conservation scientists, and given an increased opportunity to observe and participate in activities where they could see firsthand the knowledge and skills required to work in a range of professional roles. A Desert Support Services staff member described her role in supporting the aspirations of students through Two-way Science activities: “I feel like that’s a big part of my role is to help facilitate that and for me to help the women and the students that have seen a career pathway, to see how they might go about doing that. Things that seem really basic to us are not so easy when you do live regionally and remote. One of the ladies said to me once, ‘How do I do what you’re doing, [ranger name]? How do I be what you are?’ She honestly couldn’t see how she would do that, so I tried to explain I do this training, keep practising, further your studies.” In the two communities where a ranger program operates, becoming a ranger is an aspiration held by many students. Some older students felt that the on Country trips helped them learn about what rangers do in their land management roles. One teacher commented on the role that on Country learning played in developing this aspiration and the education and career pathways available to students in this community: “So when we were on bird week camp, they were like, we want to be rangers like Nanna or that kind of thing. A lot of my class now aspire to being rangers, which is really beautiful…Really their aspirations is what they can see, and so I love that when we go out on Country, there’s a ranger there, they can see what it is to be doing ranger work. I just think that is the most powerful thing, is that in upper primary they’re out learning from rangers, seeing rangers at work. They go into high school and start going into TAFE and getting the qualifications that they know will lead to jobs when they leave, jobs that they want to do and they’ve seen what that is.” In one community, students who aspire to ranger work had access to vocational education and training opportunities to support this. In this community, stakeholders felt they were “producing [the] next generation of good operators for the ranger program”. One Desert Support Services staff member commented on how well this was working: “It makes the ranger program, and the potential employment through the ranger program, much more visible and accessible and aspirational. Many community members agree that young people in this community who are looking for these opportunities have a clearer pathway.” Students who participate in the Program are often exposed to a range of different ecological and conservation science projects, such as the Australian Wildlife Society’s Bilby Project. One of the scientists engaged in working with students talked about the importance of exposing students not only to science but to different career possibilities: “I think a huge part of it is exposing students to science, and I think in many ways it’s showing them what you can do with your life…So, you’ve got to know what you can do with life before you can seek the education required to become that person. So, with regards to students, I think a lot of it is just exposing kids to all the different things in life they could possibly be.” Indicators of success For Outcome area 5 (Increased aspiration (of students), sense of value and school belonging) the indicators of success expressed by evaluation participants, and confirmed by the evaluation team as being achieved, comprised: • Increased focus on student wellbeing • Students aspire to be role models to younger students • Increase in respectful behaviour among students • Strengthened intergenerational roles and knowledge transfer • Elders and older Traditional Owners and other knowledge custodians more physically and mentally active • Students beginning to enhance their ability to negotiate the cultural interface • Students exposed to a more diverse range of career roles, particularly those leading on Country learning such as rangers, ecologists and other scientists • Students’ aspirations for STEM education and careers increased, particularly for STEM roles visible in the community • Reduction in the pressure for students to ’code-switch’ between home and school Outcome area 6: Increased student engagement and attendance Definitions There is a long history of research regarding ’student engagement’, and exactly what constitutes an ’engaged student’ (e.g., Azevedo, 2015; Christenson, Reschly, & Wylie, 2012; Fredricks et al., 2011). While there is scholarly agreement that, at a minimum, student engagement comprises both participatory behaviour and some emotional component, there are numerous conceptual and measurement issues that remain contentious (Reschly & Christenson, 2012). For instance, Hart, Stewart, and Jimerson (2011) argued that the definition of student engagement should be expanded to include the student’s behavioural, emotional and cognitive experiences at school. Behavioural indicators of engagement include student attendance and participation in classroom activities, in addition to the level of concentration and time spent on classroom activities. The emotional component of student engagement can be explained as the student’s emotional response to classroom activities, whilst cognitive indicators of school engagement include the student’s awareness of their learning journey and their willingness to invest in their own learning (Li & Lerner, 2013). It is important to acknowledge that often students can demonstrate any number of these components, however, the student’s motivation could be to simply meet the requirements of the school and the curriculum. The continuing interest in student engagement centres on the student’s connection with other educational outcomes such as student achievement, attendance and behaviour (Hart et al., 2011). Engagement has been repeatedly demonstrated to be a robust predictor of achievement and behaviour in schools. As a result, fostering strong student engagement might actively assist at-risk students in achieving positive outcomes and completing school (Appleton, Christenson, & Furlong, 2008). Along with student engagement, student attendance has been considered a vital and necessary component of quality education, with non-attendance having several negative academic and social impacts (Australian Institute for Teaching and School Leadership, 2019). Student absenteeism has been linked to lower achievement, poor grade retention and an increased chance of dropout (Hancock, Mitrou, Taylor, & Zubrik, 2018). The National School Reform Agreement identifies student attendance as a key performance measure for Australian students, aiming to increase overall attendance as a way to ensure students are engaged in their schooling (Council of Australian Governments, 2018). The relationship between attendance, retention and engagement is complex, with alternative paths for some disengaged Aboriginal and/ or Torres Strait Islander students being more effective than trying to improve attendance (Briggs, 2017). Quantitative indicators of success School attendance data for participating schools revealed no meaningful change in attendance over the time the Program had been fully implemented (from 2017) (see Figure 6). This is based on My School attendance data for seven schools (in 2019, the median number of students was 55, and the median ICSEA was 629) that were taking part in the Program in 2019 and for which there was complete data. Seven schools (in 2019, the median number of students was 103, and the median ICSEA was 738) that were scheduled to join the program in 2020 were used as comparison schools. Taking into consideration the potential influence of a diverse range of factors on student attendance, an aggregated, quantitative attendance indicator may not provide the most robust measure of program success in this case. Several educators interviewed spoke about the positive impact the Program had on student attendance—particularly during Learning on Country trips— however; overall, findings cannot reliably indicate an impact on overall school attendance across the school year. In addition, Learning on Country attendance data was not consistently collated across communities, so monitoring positive improvements in attendance was problematic. Finally, NT schools sometimes only allowed students to come on bush trips if they had been attending school regularly. In relation to engagement, overall, most of the educators and students interviewed reported that students/ they were substantially more engaged during Two-way Science activities, most noticeably during Learning on Country trips and in-classroom follow-up activities. Engagement on Country Students identified factors such as being out of the classroom, hands-on activities, interacting with the Elders and learning in language as reasons for enjoying and engaging with Two-way Science. In one community, an older student explained that it was the hands-on nature of Learning on Country that they enjoyed most: “Yeah, but it’s [Learning on Country] a bit different [than learning in the classroom] because when you go outside…Elders actually show you the things that they do, instead of showing it on the screen. They can show it in real life. That’s why I like going outside very much because you get to learn more things outside, and you get to see the actual things to actually believe it.” Local teachers in another community confirmed that the practical nature of Learning on Country had a positive impact on students, both as a learning approach and as an avenue for more meaningful engagement: “… having a bush trip, it’s exciting…out in the bush, we show all the trees – naming [them] and all that. Better than…[just showing them] the words, you know?” Figure 6. Mean student attendance rates: Program schools vs comparison schools Figure 6. Mean student attendance rates: Program schools vs comparison schools The Learning on Country, two-way approach of the Program provided opportunities for the students to be taught in language. Those interviewed perceived the relevance of the two-way activities—in terms of the focus on local culture and language— as key to increasing student engagement in learning. As one principal explained: “Students are more engaged in learning as a result of the Science Pathways program. The two-way approach means that the students are able to work outside on Country with family and Elders. As the students are learning through family stories and family knowledge, they have a real connection to what is being taught and how they retain this information.” Teachers across all communities agreed that student engagement levels were high when they were taught in language and/or using their first language. One principal reasoned that this might have been because it was easier for students to learn since the students: “Don’t have to code-switch all the time, that’s a really difficult skill to learn, to code-switch between languages… it’s easier for the kids to think and speak in their first language.” This ease of learning, they further explained, went hand in hand with improved engagement and behaviour: “You’ll just see the level of engagement…the kids are different. You can see it. The kids are differently engaged. There’s not a lot of [bad] behaviours.” Another teacher noticed that together the students were “using language more freely” when they were out bush. Elders and family members spoke about Learning on Country as having a positive impact on their young ones’ happiness. One of the Elders recounted the excitement they shared with their young family member following a Learning on Country trip planned as part of the Program: “[The students] say that they’re really happy…to us parents. We can see how happy they are, and excited, when they bring some animals back…They bring a little fish back, a little catfish [and say] ‘Oh we caught this yesterday’. Yeah, he was so happy.” The connection with family that Learning on Country fosters is notable in this excerpt and was also raised by one of the principals as an important way to support engagement across the Program’s teaching and learning cycle: “As a result of the Science Pathways program, students are more engaged and enjoy coming to school—especially when they are doing Two-way Science. It has also meant that families are coming in to help work with the students and therefore provides a sense of belonging at the school.” For some of the students who did not go out bush with their families, going on program trips was particularly engaging, as it was the first time they had been able to visit certain areas around their community, which they found new and exciting. As one teacher explained: “I’ve never seen kids as engaged as they are out here [out bush], and a whole group of them as well, even the ones who aren’t culturally aligned with stuff that we do find it really interesting.” One teacher acknowledged how much students looked forward to bush trips and described how they used the trips as a reward for good behaviour because the trips encouraged the students: “to do better – learn better”. Positive learning experiences During Two-way Science activities, teachers reported that, generally, students displayed positive behaviour, and their involvement in learning activities was often self-directed. One teacher noted that students who were usually quite difficult to engage became actively involved during ‘two-way’ classes: “Especially for high school, it’s really hard to engage them in class, but you see with quite a few of the boys here, they really take to it. They throw themselves right in, and even if they’re not, they want to participate. So, even if they’re not chopping wood, they’re recording or doing something that I’ve asked, and it’s really nice to see them with their families.” (teacher) This positive improvement in student behaviour was explained by one teacher as being directly related to engagement: “[the] kids are just so different when you take them out on Country. Like yesterday [on Country], there was just no behavioural problems, not one” … “you know that they’re engaged. They’re in their element.” One student was notably more engaged out bush, demonstrating an enjoyment to engage in learning that their teacher had not seen before. As the teacher explained: “[student name] who is quite quiet in class, he was just having so much fun tracking, he just absolutely loved it, and he was yelling to us: ‘Come and see what I found, look what I found!’...I saw a different side of him yesterday…and he was really engaging with [Elder and ranger] as well, which was amazing. Every time he found something, he was like ‘come and have a look, come and have a look’!” (teacher) In all communities, teachers commented on how students often exceeded their expectations when participating in Learning on Country in the areas of engagement, inquiry and perseverance. In one community, teachers expressed their surprise at how far the children had walked during the day, as they followed animal tracks and explored different habitats. In another, program staff spoke about the positive learning outcomes the teachers had pointed out to them: “Generally [student] engagement is extremely high out on Country and the teachers comment, but many times they’ve said, ‘Wow, I can’t believe that they did that work out on Country’ or ‘I can’t believe they were so focused on that’ or ‘I can’t believe they were asking questions like that’. It’s really fantastic to see.” Confident engagement in learning Educators observed students were more confidently engaged in learning during bush trips. They commented that students “become the teachers” and frequently deferred to student knowledge, saying “I’m learning from them”. One high school teacher commented on the change in confidence within their class during Two-way Science activities: “They take a lot more initiative out here …especially the boys here, [but also]…the girls. They like to know what they’re doing, and when you’re just in the classroom and just purely doing classroom stuff, it’s all us telling [the students] what to do…you can tell…they don’t feel as confident, and so they just switch off and don’t engage. Whereas, out here, they’re telling me what to do… they’re way more receptive.” Artist: Benjamin Title: Marrngurrangu muwarrpinayi pijunga (Meeting at the creek), 2019 Artist: Benjamin Title: Marrngurrangu muwarrpinayi pijunga (Meeting at the creek), 2019 Another teacher spoke about student engagement being driven by their cultural knowledge and excitement to share this with others: “I’d been here a week before we had any [Two-way Science] classwork, and the kids, they’d run up to the door and bring a baby goanna, or bring different animals to us. You could see that’s what they loved to do, is go out hunting, show you around their place, their land.” Students were observed by the teachers to: “Ask more questions when they’re out on Country”, and older students took on mentoring roles to younger students during bush trips. Many of the educators observed some of their underachieving students transform—demonstrating high- level skills and abilities to learn when engaged in two-way learning on Country. Teachers provided multiple examples of students who had not only become highly engaged but who demonstrated unexpected learning outcomes: “The kids who are really comfortable and have a lot of bush knowledge aren’t always the high achievers in class, so it’s a really good opportunity for that to be flipped and for them to be the experts out there and get that confidence from being able to teach others and teach the teachers as well. Then, of course, if they’ve been able to come out and have this thing where they’re the expert or they’re the most knowledgeable, then when we follow up with that in class. It gives them a sense of achievement, and so they’re more engaged in those follow-up activities in class, and the pre- learning as well, but especially the follow-up.” (teacher) Some teachers noticed how students sharing their knowledge of the land made some students feel proud—proud of their culture, their knowledge of it and their ability to share it with their teachers and peers. As one teacher explained: “they [students] do just Artist: Jyreece Title: Warranpa mujungu - Parlparrja (Land and sky), 2019 Artist: Jyreece Title: Warranpa mujungu - Parlparrja (Land and sky), 2019 really respond so well [to being out bush]…They’re whole new kids when they’re out on Country, and we usually have very minimal behaviour issues. Because they’re telling their knowledge, it’s not me trying to teach them, they’re telling me. A [Kindy/PP teacher] and I usually do a weekly bushwalk. We haven’t got it up and running this term, but the past two years, especially, we were going out every week, and that’s when the kids just shine.” Indicators of success The indicators of success for Outcome area 6 (Increased student engagement and attendance) identified by the community and the evaluation team included: • Increased student attendance and engagement during Two-way Science on Country activities • Increased student positive behaviour and self-directed learning • Increased use of language in learning • Increased student happiness and sharing of learning with family members • Increased student confidence, perseverance and inquisitiveness • Increased student pride in culture • Increased exposure to individuals’ strengths being acknowledged and shared Outcome area 7: Enhanced student results Evidence in the area of student results uncovered that, generally, participants consider success in this area to be less focused on students’ academic results and more focused on how well students apply their knowledge to achieve individual learning goals as well as their contribution to a positive community, family and socio-economic life. Findings across communities highlight the Program’s role in supporting schools and teachers in delivering a culturally responsive curriculum that leads to positive gains in student learning and their connections to the broader community. Definitions The concept ’student results’ is most commonly described in the literature with the terms ’outcomes of schooling’, conceptualised as any positive or negative personal, academic or social product of schooling (Guenther, Lowe, Burgers, Vass, & Moodie, 2019), and ’educational outcomes/goals, such as equity and excellence within schools and for students to be successful learners, confident and creative individuals and active and informed citizens (Council of Australian Governments, 2008). Curriculums cover a broad range of learning areas and general capabilities, with essential minimum outcomes often focusing on foundational areas such as literacy, numeracy, maths and English. Schooling outcomes also support development in areas such as social interaction, cross-disciplinary thinking, national values of democracy, equity and justice, honesty, resilience and respect for others (Council of Australian Governments, 2008) A recent systematic review of educational outcomes for First Nations students in remote schooling in Australia identified a broad range of outcome categories, including academic, wellbeing, aspirations, participation, identities, equity and relational (Guenther et al., 2019). This broad range of potential student outcome categories requires a corresponding range of measurement indicators, well beyond the standardised quantitative achievement measures (such as NAPLAN) noted in the national Measurement Framework for schooling (Australian Curriculum, Assessment and Reporting Authority (ACARA), 2019). Factors supporting student outcomes include community indicators of labour force participation, rates of English language spoken at home and higher proportions of training qualifications in the community (Guenther et al., 2019), learning an Indigenous language (Lonsdale, 2013) and a teaching process informed by Indigenous culture, and when teaching and learning become part of an equal and genuine cultural exchange (Purdie, Milgate, & Bell, 2011). Standardised school reporting In discussions with program and school staff, it was felt that current standardised school assessment data was not a valid or reliable measure to best understand the learning impact of the Program for participating students due to issues regarding equity and culture-fair assessment (Klenowski, 2009). Instead, a more targeted measure, such as progress against individual Two-way Science goals compared to progress against goals in other learning areas for individual students, might provide a more accurate measure of success for student science learning. Teachers across communities talked about the extensive use of individual education planning and goal setting. One teacher interviewed described this approach to measuring student achievement: “We assess the student performance against themselves. How are you improving? We don’t really care how you improve compared to someone else. How are you, what have you learnt, what you’ve achieved.” This detailed level of data was not available to the evaluation team at the time of this report; however, steps are in place to consider this from a monitoring and evaluation perspective. The feasibility of collating data against such a relevant and detailed measure must be considered in terms of resources and cost. Despite the absence of this data, including other forms of assessment evidence in the report, such as standardised testing results, was not considered due to their lack of validity in capturing the individual progress of remote school students, many of whom speak English as another language or dialect. Science curriculum and assessment Across the jurisdictions, the frequency of Two-way Science lessons across the school year varies, and there are differing and sometimes minimal standardised processes in place to assess Two-way Science. Feedback from some educators highlighted that curriculums and formal assessments are significantly modified to respond to the strengths and challenges of their student group. Individual and, sometimes, small group-level plans are developed to align with these. Program staff have supported schools in increasing science in their curriculum, linking it to Science Pathways for Indigenous Communities’ Two- way Science activities. Program staff in the NT have also encouraged schools to consider assessment options that are aligned with the Australian science curriculum content (CSIRO, 2018). At the time of writing, the NT school participating in the evaluation was not required to undertake formal assessment of Two-way Science or student work within the Indigenous Language and Culture program. However, the principal explained that: “Under the Program that [facilitator’s name] has been running for the last few years, we’re coming around to doing a little bit of formalised assessment.” Generally, the bilingual school project school in the NT was considered to have more capacity to teach and formally assess science each term, due to additional teacher resources being created by its bilingual status (CSIRO, 2018). According to one of the principals, new mandatory ILC assessment processes, connected to Australian Curriculum requirements, were being introduced at the time of writing this report. Across jurisdictions, teachers commented on the requirement to think innovatively when assessing students, tending towards oral assessments as opposed to only written ones. A teacher in WA described a major assessment piece that: “Was being able to paint it and explain it orally, rather than something mainstream where it would be very much written work.” The teacher went onto explain that: “Most of the students can’t show the depth of their knowledge through writing, and so you have to look at other ways.” This was reiterated by teachers in the NT, one of whom commented on the range of assessments needed: “You can do some orally, but we do assess writing. But, definitely, their comprehension of what’s happening can be assessed through them retelling, so that’s important.” In the NT, one of the teachers commented on the varying levels of assessment required in a composite class with up to three year-levels. In this case, the teacher highlighted variations within the Kindy to Year 2 class: “So even within that class, we’re going to have a range of different outcomes. For those littlies, all that oral language that they had down there is just what they need. They’re building their oral language, whereas the Year 2 student, she’ll be getting into measuring and she’ll be writing about it and learning more difficult vocabulary.” One indication of student understanding of science and other learning areas is student work samples. One program team member described a Two-way Science activity centred on Digging for Watja (Bush Potato). An Elder took students to dig for Watja (see Figure 7) and explained that the Watja is part of a Tjukurrpa (Dreamtime) story and only found in certain areas. She said that the plant was actively cultivated and translocated to provide a ready source of food as people moved through Country. The Elder explained further the seasonal availability of the tuber and how the vine dies off and becomes dormant during dry times, then grows again with the rain. Students learnt how to locate the Watja by looking for a crack in the ground or listening for the hollow noise when poked with a digging stick. They learnt to dig out the Watja by following the root through the soil and then learnt the correct way to cook it. This activity met curriculum outcomes that required students to demonstrate their knowledge and understanding (see Table 5). Table 5. Curriculum outcomes for Digging for Watja Two-way Science activity KNOWLEDGE OR SKILL AUSTRALIAN CURRICULUM DESCRIPTOR EXAMPLE ASSESSMENT ACTIVITY Learn where the Watja grows and that people planted them for food. Year 1 biological sciences: Living things live in different places where their needs are met (ACSSU211) Oral explanation The Watja is a tuber used for food different from other bushfoods or medicines growing nearby. The Watja is a vine, which is different from a tree and shrub. Year 3 Biological sciences Living things can be grouped on the basis of observable features and can be distinguished from non-living things (ACSSU044) • Complete a plant information label about the Watja. • Draw diagram of the Watja, showing features (leaves, stem, tuber, etc.). • Classify plants according to their Indigenous use. Watja produces new growth with the rain. The vine dries and remains dormant during dry times. • Year 4 biological sciences: Living things have life cycles (ACSSU072) • Year 4 biological sciences: Living things depend on each other and the environment to survive (ACSSU073) • Year 5 biological sciences: Living things have structural features and adaptations that help them to survive in their environment (ACSSU043) • Year 6 biological sciences: The growth and survival of living things are affected by the physical conditions of their environment (ACSSU094) • Draw diagram showing different stages of Watja growth, including vine growth and flower, seed and tuber development. • Indicate how weather affects different stages. • Sprout sweet potato in a jar of water in the classroom to demonstrate stored energy in the tuber. Describe the growth of the sweet potato vine and the way the different parts of the plant are adapted for seasonal growth. People dug up and planted the Watja to provide a source of food when moving from place to place. Science as Human Endeavour Year 5/6 Use and influence of science: Scientific knowledge is used to solve problems and inform personal and community decisions (ACSHE083 ACSHE 100) Write a narrative or draw a sequence of images to show different bushfoods that people use in landscapes as they move from place to place. Identify signs people look for to locate the Watja. Figure 7. Digging for Watja Figure 7. Digging for Watja The work sample in Figure 8 shows a literacy activity (recount) following a Learning on Country field trip to locate and dig Watja. This work sample is indicative of the learning demonstrated by students during the case study fieldwork and shows how the Science Pathways for Indigenous Communities Two-way Science program promotes an integrated model that builds on the engagement of Learning on Country and connects this to science and other learning areas. Supporting academic success While levels of academic success were not directly measured for this report, many participants described elements of the Program that supported learning outcomes. Educators and other stakeholders identified that the approach supported deeper learning about science, integrated language and relevant place-based knowledge and required less code-switching for students. The Program’s focus on education led by local Aboriginal knowledge holders and experts means language is an integral component to teaching. Several educators across communities felt student learning was expedited by the layer of education and language support provided by Aboriginal educators and leaders, particularly for students with a disability or learning difficulty. One teacher talked about the benefits for student learning when family is involved: “I think it’s great because they can relay information and language to the kids if they’re not quite understanding what I’m saying. Sometimes, [the] AT will sit right next to that child and relay the information in a way that [they] would understand, or gives an example, writes it down. So I think if I didn’t have that, considering all the different levels in ages, and in differentiating, and even when we go out on Country, if we didn’t have that, it would be really difficult.” Language, literacy, vocabulary and cultural knowledge Most stakeholders described the positive impact local language, knowledge and natural environment had on student achievement and success. Across both jurisdictions, educators provided a plethora of examples where students demonstrated enhanced vocabulary and literacy skills related to Two-way Science lessons. One teacher illustrated how “place” is integrated into the lesson to enhance the connection to learning: “It is…it absolutely is [connecting to their reality]. In small ways and in deeper ways…What we want to do is we want to go out and be able to do a survey, to do a survey you need to know how to make a grid. We’re going to practice it so in two weeks’ time when you go out to do a grid for wherever the echidna is, you can do it. It’s got to be holistic…So we have the upskilling in the literacy and numeracy areas, and the other areas too. And then the engagement through their interest and their culture.” Other teachers acknowledged the strength of teaching on Country and in language for achieving better student Figure 8. Student work sample (used with permission) Figure 8. Student work sample (used with permission) learning outcomes: “Because the kids here are connected— you know, they’re on Country all the time. And going out—I would say that there’s an increase in development in their vocabulary—a wider vocabulary that they use.” In particular, teachers often talked about expanding Western science and first language vocabularies: “So we talked about what’s nocturnal. What does nocturnal mean? Bilbies are nocturnal. Things like that. So we use—we work with some of the language like that. And we used it for sentences and writing stories. But they discussed all those things quite a lot.” In the NT community, motivation for Two-way Science was driven by local teachers identifying that students were losing their depth of language. The Program supported the school in achieving its learning goal for students—to increase their language vocabulary and cultural knowledge. In this community, one of the senior educators gave an example of how Two-way Science facilitated this: “They’re also learning the plant identification—they’re learning how they categorise the plants and what they use them for. Most plants have a purpose, and that’s how they’re categorised…they take those skills and can use them—because they learn what the plant is, what parts of it they can use and what they can use those parts for. That’s the knowledge that they were losing.” In the NT community, teaching resources were developed in collaboration with the school around Two-way Science activities and were well utilised by the students in preparing and reflecting on learning during bush trips. One NT Program staff described how students participated in developing first-language educational resources, which continue to be integral to the learning process long after the on Country trip. “It [on Country learning] provides something exciting that’s happened that you can get kids to write about. I’ve particularly seen that happen in terms of Indigenous literacy. There’s been some great books that [the school and] I just produced that the kids have written with support from the assistant teachers, and there’s a linguist who the school has paid to go out there occasionally. And the kids can write quite easily in their own language because they know what they want to say.” The program staff member went further, describing the role that literacy in first language plays in building Two-way Science learning activities in achieving broader learning outcomes: “Even if they’re reasonably able to communicate orally, writing about something is still another big step…to write in English about something they struggled to write a few words, but then were asked to write in their language about it, and they were all incredibly eager. Even if they didn’t actually have the literacy entirely, they’d be saying, ‘Help me write this, this is what I want to write, help me’…so I actually think Indigenous literacy is an important part of the Program.” Staying in education longer In one community, the achievement of some high school students to stay in school and complete vocational education (VET) was attributed to the two-way approach taken by the Program. Key vocational education and training partners identified the Program and program staff as critical elements in maintaining student engagement and improving VET outcomes. Due to the success of the Program in this school, the VET educators have re-designed their courses to align with the Program’s two-way model and create consistency for students throughout their education and career pathway as rangers: “We use that exact model. So, every field trip I do has Elders, the same Elders: [names] for the same reasons. So, it’s a consistent model right through from pre-primary to VET…So, they’ve transitioned through and kept learning, and this has never happened before. So, for the first time now, we’re getting a program that keeps kids at school because it’s something to look forward to and it values their knowledge” (TAFE staff member). Applying learning to land management roles Across all communities, students consistently responded positively when asked what they learnt during on Country trips and Two-way Science lessons. Most commonly, students demonstrated strong recall of the ecological highlights from their learning, reciting numbers and types of fauna they recorded, the vast range of tracks they found and what they learned about habitats, natural and built structures and different ecosystems. Students in one community reflected on their on Country learning: “Every type of tree or bird names. That’s what we learnt… different types of trees. We know the trees by the type of leaves.” They discussed learning land care strategies from the rangers, saying that they learn from rangers about “Going out there, clean the waters” and “Keep the clean water”, and a local teacher explained that the: “Camels had made bad and not clean, the rangers showed them that—where the camels had made things messy around the waters.” Figure 9 provides a land care example. Some students applied their learning to improve their nearby waterhole. Students shared this knowledge with family and others to preserve the quality of the waterhole and ensure this was a new priority for the whole community. In communities that have strong connections with ranger programs, many students expressed a strong interest in applying their knowledge as future rangers. Figure 9. Land care example Figure 9. Land care example “One of our activities was water monitoring, and we did that yearly and ended up collecting data and looking at data. The older kids that were involved in that project over the years became proficient in the activity and their understanding of the animals and knowing their names and identifying them. They translated that into making rules around that particular waterhole. The waterhole activity…it’s a creek…with a series of waterholes going down….and we were doing the activity at the top creek and [Aboriginal educator] told me that on the holidays they [students] were up there and they were swimming, and they said, ‘We can’t go there’, they said, ‘We can’t take anything from there and we can’t swim in there, we want to look after that place, so we’ll do all our swimming and fishing and collecting yabbies downstream from that spot’. They’ve developed a real ownership over that space and their knowledge and understanding of that particular area. And then passing that on to all the other kids that haven’t been involved in that activity as well. It would have been the whole community up there” (program staff). Figure 9. Land care example Indicators of success For Outcome area 7 (Increased student results), the indicators of success relayed by evaluation participants included: • Increased school capacity to assess student learning in science • Improvements in student learning, as evidenced by strong recall and work samples • Deeper student learning of science • Increased engagement in other areas of the curriculum, including literacy and numeracy • Enhanced student vocabulary and literacy skills • Increased student understanding of culture and Aboriginal scientific knowledges • Some indication of increased retention due to VET pathways in one community Other outcomes areas There was no evidence of the Program or schools pursuing two other outcome areas (Centres of excellence in two-way STEM education; and university teacher training using Teacher Professional Development or the Department of Education extending the model to other remote schools). Some evaluation participants did comment on the success and future of the Program, as well as its potential for transferability. As mentioned, in one community, the Science Pathways for Indigenous Communities model has been adopted as part of the vocational education program, and classroom educators talked about working collaboratively in a community of practice with other schools to share ideas, similar to the process with Aboriginal Teachers in NT. The Program’s achievements have also been presented at several conferences, such as CONASTA and the Indigenous Desert Alliance Conference in 2019 and the 2018 National Aboriginal and Torres Strait Islander Education Conference. At these conferences, senior assistant teachers delivered workshops on engaging communities in Two-way Science teaching, and senior assistant teachers introduced presentations featuring the ACARA videos about Two-way Science in their communities. One principal talked about the recognition and awards associated with the Program: “It’s programs like Science Pathways that enable people to get a better understanding and the visibility that comes with Science Pathways and those pictures, and what that’s done I think the visibility can really change how Aboriginal education happens and even education as a whole happen across the country, let alone in other indigenous community.” Generally, the case study evidence indicates that these two program outcomes are not the current focus of the Program. These outcomes are also not raised as priorities by school communities that participated in the study. Consideration should be given to revisiting these outcomes and their relevance based on the planned program activities in the next phase of implementation. FINDINGS: FROM IMPLEMENTATION Implementation learnings identify processes and conditions that positively impact program effectiveness. Implementation findings can be used to better understand how outcomes were (or were not) achieved and inform future program design and implementation. In this section, program implementation enablers and barriers are identified that were found across all locations. These implementation outcomes are divided into two categories: program factors—program-driven factors that support success; and contextual factors—existing “place”-related conditions and factors that support the Program in achieving its goals. Contextual factors Endorsement and leadership within the school Many interviewees identified principals as needing to be engaged with the planning and support of teachers and keeping priorities such as Two-way Science on the school agenda: “If we don’t get the leadership...This is a school program...The school creates the time, it creates the facilities, it creates—it manages the resources.” Specific examples were given of longer-term staff, such as principals, being trusted by members of the community to share cultural information with them, including Lore or other cultural business. In these instances, this shared knowledge supported connections and understanding between the school and families. Many people saw the principals’ role as being responsible for the cultural responsiveness of the school and for embedding the Program in the school. A program consultant emphasised this: “It really comes back to the leadership of the school...There has to be adult partnerships where the adults feel as though they are being valued too, where they are being called upon, where it’s being acknowledged that their knowledge is unique and Two-way Science can’t happen without it.” Endorsement and leadership from within the local culture and language groups Across all communities, the local community leaders were engaged in the Program as leaders and experts in the planning and delivery of Two-way Science. The communities’ education goals underpinned the Program’s intent in each location and, with permission, were built upon by the Program. A Desert Support Services staff member noted that: “The [Traditional Owners] have always pushed for education and getting more young people out on Country. We had always heard them.” In all three communities, program staff accessed existing community strengths: “School and community relations were started, probably well before we were there, and so that was the platform on which that kind of interface could happen. They’d been doing some great work, and they set up the school councils, which all those people were on already and providing strategic direction.” Time, knowledge and expertise of local knowledge holders and language speakers Two-way Science activities with integrity rely on the participation of Aboriginal knowledge holders and language speakers. In each community, the Two-way Science curriculums were created with the time and knowledge of local Aboriginal educators and leaders. Non-Indigenous teachers acknowledge that it is not their place to teach local culture, but to create a welcoming space for it. Educators acknowledge that without the time and contribution of local knowledge holders, Two-way Science lessons have to be scaled back or do not go ahead. One teacher emphasised this: “Sometimes, I’ll just happen to know the Martu for something, but I can’t pronounce it properly. So, if a local teacher isn’t there, I’ll try and say it and then usually someone will be like, ‘Oh yeah, that’s right’ and be able to consolidate it. But I think I’m very careful not to be, like, this is how is it as a non-Martu. It wouldn’t work.” Teachers are adaptable and flexible with their lesson planning and delivery One of the strongest messages conveyed by educators was the need to be flexible and adaptable with their lessons. One teacher succinctly summed up her colleagues: “You can plan all you like, but it may not happen like that on the day.” Teachers emphasised the importance of having a respectful and open-minded attitude. One teacher stressed that working with the community is essential: “Don’t go down the path of what you’ve been taught at university or what you’ve learned in some other country. You’ve got to do it this mob’s way and fit in and work a pathway or weave a pathway with this mob.” One program staff member explained the importance to teachers’ success of taking the right attitude approach: “I think it depends on their attitude and interests, how they build relationships with the assistant teachers they’re working with and how mutually supportive those relationships are, and what their interest is in going out on Country with kids to some extent as well.” Curriculum structures that support the integration of Two-way Science In the NT, the Indigenous Language and Culture program (or a bilingual program) provided additional structure that aligned well with the Two-way Science program. One NT Program staff member explained: “Having the structure of the ILC curriculum in place in the Northern Territory schools—it’s been really beneficial to the success of the Program. So having language programs in place supports Learning on Country, which then supports language development. So it’s a nice cycle. Which, of course, supports individual strength.” In WA, EAL/D was identified by some staff as a curriculum adaptation that supported Two-way Science learning. One teacher felt that: “The two-way approach matches really well with the EAL/D learning. So making really explicit what is Martu English and what is standard Australian English. But then there can be a two-way approach to that where it’s like okay, what’s the Martu word, what’s the English word? I think also the philosophy is really good and supports the EAL/D philosophy, which is that one isn’t better than the other, they sit equally, they’re both equally important.” Partners with aligned goals, especially Indigenous ranger programs and science professionals When rangers and others partners, such as Desert Support Services, the Department of Biodiversity, Conservation and Attractions and ecologists, expressed values aligned with the communities, or with Two-way Science learning, opportunities for effective partnerships arose. In most cases, the Program’s role was to coordinate this to fruition with the school students involved: “It really gave us a bit of framework, or conduit…of just making it a nice, tidy package of people –Martu people learning on Country, which ties in very closely with the IPA plan—the wishes and vision of the people up that way. So it’s been a… tremendous value add to our land management program.” These partnerships were observed to be carried out primarily through on Country learning trips, coordinated to include student participation. One of the Science Pathways for Indigenous Communities staff described the value that rangers can bring to the Program in this area: “They are leaders...when [the program facilitator is] there, that’s when they come to the school and work with him...it’s really important to try and get those people on board...Where there’s willing ranger groups, then they’re an ideal situation.” Program factors Program facilitator characteristics, including decolonising20 community development and dialogical developmental approaches The role, skills and attributes of the program facilitators were identified by many case study participants as influential factors in the success of the Program. The following evidence highlights the strongest and most commonly provided perspectives on the role and abilities of the facilitators, collected from a diverse range of participants: • Listening skills and providing support “Listen and provide informal counselling support, provide practical, useful support.” • Judgement and decision-making skills “As a facilitator, you’ve got to make those sort of decisions, I guess, about when you pull back and when you dive in.” • Making time and creating space “Just to make the time and make the place to get everyone together and give them the space to say what they need to say and talk about what it is they really want for their kids. A lot of that is about just providing a space.” • Coordination skills and demonstrating drive “Reinvigorating that drive…to show them the way, to coordinate it—it’s actually having someone in that role to bring people together.” • Engagement and negotiation skills “Making sure all the parties are engaged equally and in the right way; a bit of negotiation at times with staff.” • Listening skills, adaptive and respectful “The Program has been effective, primarily because it has capable, adaptive, individuals within it work[ing] alongside Martu and the school staff and [they are] quiet, respectful, listens well and has other qualities that Martu value.” • Culturally responsive “When you approach Aboriginal education, it has to be from a language perspective and understanding of what it means and for Aboriginal education to be respectful.” • Building trust “Because [the program facilitator] rocks. Like, [they] do all the things, what [they] do now. Outside. Doing with animals things. We like a person doing that. We trust [them], yep.” • Relationship-building “That initial just going out and having a go at something, as messy and imperfect as that may be, the most important outcome from that is that you have developed a relationship because you’ve done something together.” • Building genuine links with community “What I do like about him [program facilitator] that I see that goes beyond just school is that [they have] still got relationships with the mob and [they are] looking after their mental wellbeing because there’s a lot of things that impact on their lives.” One key community stakeholder noted that when trusting, respectful relationships are nurtured, a safe space for sharing is created within that relationship: “[The Science Pathways for Indigenous Communities coordinator has earned] high respect, and you feel that. You see the Elders around him. That’s very, very hard to get. [They] really, really earned that level of respect. [They are] getting knowledge that most people would never get.” Another observation made about facilitators with these skills is the ability to create respectful partnerships, where there were previously failures to connect. Several stakeholders commented on the inability of scientists and rangers to connect positively, despite previous attempts. One of the principals who had observed such an interaction commented that: “The [Science Pathways for Indigenous Communities facilitator] approach and how it’s done I think is enough for people like those scientists to re-think about how they do things, and I do think that that’s happening in this case.” 20 Decolonising in an education context can be defined as a counter-hegemonic framework for contesting colonisation, racialisation, and forced assimilation strategies, and generates empowerment for Indigenous knowledge systems, health, and well-being through education (Battiste, 2011). On Country, co-facilitated teacher professional development with a strong modelling and mentoring component This section focuses on how the teacher professional development was implemented, rather than on the teacher professional development itself, although the two are difficult to separate. Non-Indigenous educators reported that formalised on Country professional development had a significant impact on their ability to understand two-way learning based on local knowledges and languages and translate this learning to the classroom. Being able to see the program facilitator demonstrate good practice two-way teaching with Elders or rangers was also reported to be impactful. Teachers also spoke about the way on Country learning was delivered helped to establish themselves in the community and strengthen their relationships with families. One teacher described this process: “We had heaps of rangers, heaps of kids and just seeing, watching how [program facilitator] very skilfully facilitated everyone to come around in a circle so that everyone could listen to the rangers…every time I go out, there’s something new to observe and be like, okay, that’s how I’m going to do that, that kind of thing.” This was reiterated by another teacher who described their professional development day: “We went to the [campus name] all the [Aboriginal teachers] came with us, so it was sort of getting to know the kids as well, and getting to know the workers, the other teachers here. We sat down with a bush planner, term planner, and the [Aboriginal Teachers] really led that and told us what happens around the bush.” Modelling two-way co-teaching was another element valued by teachers during their professional development. One teacher explained: “I think the biggest thing is the informal stuff around how to facilitate bush trips so that you have Martu sharing their knowledge and it’s organised, but it’s not being teacher-led. That’s done very informally.” DISCUSSION This section comprises a general discussion of the key themes emerging from the evaluation (organised around the most salient findings and not around the Outcome areas described in the Findings section) and a discussion of challenges and barriers, before concluding with a discussion on using the findings. Key themes Learning communities The Science Pathways for Indigenous Communities program provides a successful model to contribute to the development of intergenerational learning communities in remote Australia more broadly. The Program is positioned to facilitate and strengthen progress towards a “sustainable learning community” characterised by ongoing, creative and lifelong learning that adapts and transforms in response to the needs and aspirations of people in the community (Kearney & Zuber-Skerrit, 2012). Through cooperation, open and honest communication and a culture of trust and respect, the school and broader community have already and can continue to engage to achieve shared goals, fostering an environment where people can learn to learn together and achieve individual and broader benefits (Douglas, 2011). The Program was able to support intergenerational knowledge transfer, which is an explicit and critical goal and need that is expressed by Aboriginal and Torres Strait Islander communities across Australia (Haines, Du, Geursen, Gao, & Trevorrow, 2017; Prober, O’Connor, & Walsh, 2011). A significant foundation of this is the program staff’s ability to demonstrate communicative action and respectful dialogue, offering non-Indigenous participants the opportunity and immersive space to more confidently grow their worldview—challenge their assumptions and related behaviours, gradually expand them and reflect more critically on their own teaching practices (Bennet & Moriarty, 2015; Christie, Carey, Robertson, & Grainger, 2015). Viewing the Program from this perspective also increases the focus on the strengths of “place”—the sentient country, knowledge and experience developed in place, as well as “connections”—relationships between the people who have previously or currently reside in or visit each place and other elements of place. Sharing the same conditions, experiences, interactions and connections also contributes towards forming a stronger collective identity shaped by shared interests (Tomlins-Jahnke, 2013). This focus on place and connections can also support awareness of Aboriginal ways of knowing (Harrison & Greenfield, 2011), which can further strengthen the Program’s contribution to an Aboriginal-led and intergenerational learning community. Dynamic interactions and connections within and between human, spiritual and ecological systems are part of this ontology (Lowe & Yunkaporta, 2013) and, in program practice, are underpinned by the strong involvement of Elders, prioritising respect, reciprocity and connection before building new knowledge and partnerships. Indigenous rangers21 enhance Two-way Science programs by providing structured management, venues and increased ownership of an area of land. Where Indigenous rangers are engaged in the Two-way Science learning community, the Program can also enhance the previously demonstrated benefits of Aboriginal ranger programs, including strengthening role models for the community and young people (SVA Consulting, 2016). Stronger exposure of students to Indigenous rangers’ (who may also be Elders) planning and activities ensures education is more responsive to local issues, incorporates local knowledge and integrates student education outcomes with broader community hopes and aspirations. Education strategies that build on community aspirations for education help create connections between home and school. The role of on Country learning is particularly important in providing this, by generating shared experiences across generations and offering stimuli for further inquiry and learning (Douglas, 2011). The importance of significant adults in the lives of young people cannot be underestimated. Nurturing adults in the roles of coaches, teachers, extended family and leaders have the potential to positively influence young peoples’ wellbeing, cognitive change and agency (Rawsthorne, Kinsela, Paxton, & Luscombe, 2019). There is a dearth of evidence detailing an adult perspective on this and the amount of time and commitment dedicated to nurturing young people, including Aboriginal young people, outside their own immediate families. This Program provides a platform for new and more consistent nurturing adult mentors to support young people, especially around on Country learning activities and sharing cultural knowledges, in and out of the classroom. The corollary of this combined benefit is the building of social capital and networks of social support for students, teachers and families, connecting people through interactions to shared values and knowledge and creating opportunities to share beliefs and enhance skills (Zyngier, 2017). The intergenerational learning community lens allows for the development of shared and distributed principles of leadership22. This includes context-specific leadership and knowledge-based organising, meaning everyone in the community has a leadership role in a specific area of knowledge, and the leadership role may shift depending on the context and who, within that context, is the most knowledgeable (Sveilby & Skuthorpe, 2006). In the participating communities, this includes acknowledging students as teachers, classroom teachers as learners, and Aboriginal Knowledge holders as co-educators of students. Program staff also noted that when schools engaged Elders as educators for Learning on Country activities, their relevance and respect increased among young people and the community in general. The Science Pathways for Indigenous Communities program aims to develop relationships and partnerships. Overall, this was achieved through educational opportunities to strengthen community social capital, particularly connections between generations and interculturally. Program implementation also demonstrated that implementing an effective practice model towards sustainable learning communities has numerous benefits such as shared understanding, identity and wellbeing for many members. Future program implementation and practice could consider how additional community members and groups could connect within each learning community to strengthen how people learn together to achieve their aspirations. Strengthening the role of students in the governance and planning of Two-way Science activities, especially in collaboration with Elders, rangers and other nurturing adults, may further enhance student engagement. The role of Aboriginal community members who are “no longer indigenous to place” in each community could also be further respectfully explored to strengthen their connection with the learning community. To reflect the whole-of-community contribution to Two-way Science, the Program name could also be changed to reflect each community’s ownership and leadership, such as Two-way Science with Indigenous Communities. 21 Ranger positions are funded by an array of sources, including Indigenous Protected Area programs, the former Working on Country program, state government ranger programs, fee for service arrangements with mining companies, non-government organisations and one-off grants. Funding is often pooled to create ranger teams that perform similar functions, despite the disparate funding sources. 22 An example in Australia is the Nhunggabarra people, where responsibilities for sustainability are distributed among clans (Sveiby, 2009). Orienting non-Indigenous teachers at the cultural interface The Science Pathways for Indigenous Communities program contributes to conditions for life-long transformative learning, building teacher confidence and capacity to develop a more culturally responsive two-way pedagogy. Underpinning this is the Program’s place-based, immersive approach to teacher education and an emphasis on local Aboriginal leadership, knowledge and influence. These program elements are critical to directing teachers towards making a radical shift in their teaching approach, a process that has shown to be necessary in case study schools. Historically, schools reflect hegemonic forms of knowledge and knowledge-building, often reinforcing existing power structures within the classroom (Hart et al., 2012). This can result in Aboriginal and Torres Strait Islander knowledges and perspectives being relegated to the tokenistic or periphery of the education system. However, teachers have an opportunity to shape and deliver curriculum, pedagogy and assessment in a way that empowers students, partly by acknowledging student cultural, linguistic and intellectual strengths to redefine identities together and create a more inclusive society (Cummins, 2001). The Program’s focus on teacher professional development that is co-led by local Elders and other local Aboriginal cultural educators and that takes place in the community or nearby Country emphasises collaborative two-way learning and demonstrates the centrality of relationships and place-based cultural knowledge as essential elements of every educators’ professional practice within these schools. This process may go some way to reorientating the classroom experience for students in participating schools. For teachers, integrating two systems of knowledge can create tensions, revealing feelings of uncertainty and resistance on a personal and professional level.23 When teachers are receptive to their own critical reflections24, the Program facilitates opportunities for dialogue, modelling, reflection, learning partnerships and, potentially, a successful convergence of Aboriginal and Western knowledges. Resistance to these experiences may result in minimal positive influence, or unintentionally serve to reinforce deficit understandings and problematic dominant behaviours through a narrowed and simplistic view of Two-way Science, aligned with Aboriginal learning style theory (Vass, 2018). The skill and commitment of program facilitators, and the Program’s significant emphasis on strengthening relationships and partnerships, further supports this critical reflection process and should continue to be a primary focus of program staff. This is particularly important considering such personal connections between families and educators can further shift existing assumptions and allow teachers to accept responsibility to critically reflect on their role as teachers (McLaughlin, 2013). This navigation across the cultural interface requires constant negotiation (Nakata, 2002) and benefits greatly from the ongoing dialogue, modelling, guidance and instruction facilitated by program staff, Aboriginal knowledge holders and Aboriginal educators. For those traversing such development, leadership is seen as critical, particularly genuine and engaged support and commitment from the principal. When teachers were supported by program staff and principals in working effectively cross-culturally through reciprocal learning relationships with students and community members, then we observed that teacher capacity to connect more positively with their communities improves. Teachers who experience transformative learning—disrupting their often long- held assumptions to genuinely collaborate—and respond to the cultural context and develop culturally responsive pedagogies aligned with the experiences of their students are more likely to be effective teachers, engaging their students to achieve learning outcomes (Boon & Lewthwaite, 2016; Harrison & Greenfield, 2011; Jackson-Barrett & Lee-Hammond, 2019; Moriarty & Bennet, 2016; Sullivan, Jorgensen, Boaler, & Lerman, 2013). Teachers with the inclination to navigate this discomfort and tension to achieve a pedagogy based on student strengths, trust and respect should be recognised for their skills and qualities and be highly sought after when recruiting teachers to work in remote education. 23 Some teachers fear and resist Aboriginal and Torres Strait Islander education and cultural awareness (Ma Rhea, Anderson, & Atkinson, 2012). 24 Many teachers who work in remote communities are new or recent graduates, and many have had little interaction with people of other ethnicities and social class (Brasche & Harrington, 2012). Supporting student outcomes According to the literature, a diverse range of similar factors influence how all students engage with school and their achievement of educational outcomes (Burgess et al., 2019). The Science Pathways for Indigenous Communities program focuses primarily on influencing school-based factors, specifically engaging teachers, principals and community leaders to influence student outcomes through an enhanced curriculum, teaching approach and school environment. This means that the Program’s contribution to student engagement and learning outcomes is less direct and varied, dependent on a myriad of factors, including the ability of educators and key community members to engage with each other and implement Two-way Science successfully. Program staff have opportunities to engage directly with students particularly during classroom and on Country learning; however, the contribution to student outcomes comes primarily from the development of learning communities framed by collaborative learning partnerships between Aboriginal knowledge holders and culturally responsive teachers to generate positive student learning experiences. In this way, the Program makes a partial but important contribution to student outcomes. This arises from a combination of factors built into the program design and implementation that increases positive learning experiences for students, supports deeper learning in science and contributes to student wellbeing more generally. In the participating communities, regardless of teacher pedagogy, the inclusion of Aboriginal languages, knowledges and perspectives as a core element of the teaching and learning cycle reflected a clear strengths- based, contextualised process to successfully engaging students. It also directly influenced teaching practice. According to participating teachers, the Two-way Science (and linked) activities undertaken during the case-study site visits led to noticeable enhancements in student confidence in their own abilities and self- directed learning using critical inquiry. When students are affectively engaged in their learning, they are happy, feel comfortable and generally identify with what they are learning (Dunstan, Hewitt, & Tomaszewski, 2017). At this point, the onus falls on educators to extend students’ capacities to achieve expected (and aspirational) curriculum learning goals. Leveraging off the affective engagement of students as a result of Two-way Science activities can build further engagement both behaviourally and cognitively, providing the groundwork for overall school engagement and success (Dustan et al., 2017). Strategies to successfully extend and assess students’ learning, once their engagement is heightened, is an area that schools could consider further to potentially increase student academic outcomes and opportunities for educational success. For example, Learning on Country could be documented as a process outcome in the science curriculum, and assessment could be more effectively addressed using a multiliteracy framework (multiple modes of communicating ideas beyond written work). In the case study communities, the involvement of family, community members and other Aboriginal cultural knowledge holders, such as rangers and ranger coordinators, in young peoples’ education was considered valuable in enhancing student learning experiences. Stronger student connections with their communities can result in students feeling more engaged in their learning (Zyngier, 2017). With family and community input, students could be directed to identify real-life social or environmental challenges related to Two-way Science activities in the local community, developing solutions as a collective student group. The process of doing this allows students to appreciate the extent and limits of their own skills and knowledge, as well as their agency to make a change within their community (Bierman, 2008). The Program also has an opportunity to more explicitly focus on developing its position and approach to a “strong and smart” philosophy (Sarra, 2011), more directly articulating to students the message that success in schooling is a journey that can be grounded in a strong Aboriginal and/or Torres Strait Islander identity. Building on existing cultural strengths It is important to acknowledge within this discussion the impacts of colonisation and Western science on Aboriginal peoples that resulted in the suppression of Indigenous knowledge and the attempt to eliminate social structures, culture, sciences and traditions (Rigney, 2003; Fergie, 2018). It also necessary to further acknowledge that many schools, services and organisations still operate within a framework that is embedded in dominant cultural views, perpetuating long-standing privileges and sometimes silencing others (Barrett, 2017; Kearney, 2019). The Science Pathways for Indigenous Communities program operates within this broader context; however, program intention is driven not by a paternalistic or benevolent mission, but recognising the immense strength and value that Aboriginal ecological knowledge and local Aboriginal knowledge have for future generations (Aboriginal and/or Torres Strait Islander and non-Indigenous) in relation to success in life and the ability to sustainably manage the environment. This assumption is evident throughout the Program’s documentation as well as broader organisational partnerships and responses responses (for example, Our Knowledge Our Way in care for Country (Woodward, Hill, Harkness, & Archer, 2020)). Achieving impact through the Program is only possible with a strengths-based lens, which was observed when program staff worked with participants’ collective ambition to respectfully collaborate across differences, share knowledge, show leadership and commit time and resources to improve student learning experiences at their school. The Program utilises community strengths to expand the space and opportunity for existing knowledges to be shared interculturally and intergenerationally to create an enhanced understanding of local ecology, culture and science. Two-way Science effectively celebrates different knowledges using young (and older) inquiring minds to explore how they relate and differ. In this way, Aboriginal knowledges are viewed as valuable in their own right and for their critical contribution to students’ education journey. The Program actively strives to facilitate sustainable community ownership of Two-way Science’s education goals and activities. The Program’s handbook states: “Aboriginal people decide what to teach, how to teach and where to teach students from their communities...The Program involves Aboriginal people making decisions about the direction and content of the school learning program. Aboriginal leadership of education fosters strong school and community relationships and develops young people’s sense of cultural identity and wellbeing.” (Deslandes et al., 2019, p.6). Community ownership and leadership of Two-way Science is seen by program staff as a contribution to community capacity and an important element of program success. In addition, Science Pathways for Indigenous Communities is unique among similar programs in that it includes explicit support, tools and resources for principals and other school leaders, in addition to the support and materials it provides for educators. The inclusion of local Aboriginal languages in Two-way Science resources and facilitating Aboriginal language speakers to educate students and teachers was another cultural strength of the Program. This was particularly evident in schools where Aboriginal languages were commonly spoken by members of the community and students. Elders, local educators and other cultural leaders in each participating community talked about strengthening young peoples’ connections with their first languages and believed that the Program was a positive way to do this. Like those in the community, the importance of reconnecting with learning about language is a perspective shared by other Aboriginal and Torres Strait Islander people, many of whom are actively working to strengthen language knowledge and use, especially in the next generations (Marmion, Obata, & Troy, 2014). Several themes emerged from the community visits around the roles of first languages in education—including the importance of language during classroom learning—to reduce the pressure on students to constantly code-switch between home and school and as a central element of transferring cultural knowledge. A program that recognises the benefits of language reconnection has the potential to improve wellbeing through belonging and empowerment (Doyle & Hill, 2008; Marmion et al., 2014; Sivak et al., 2019). This means building on language strengths in education has the potential to empower students through stronger self-esteem, sense of belonging and sense of pride and improved communication between families and schools. Supporting change at the school and community level The Science Pathways for Indigenous Communities program provides valuable insights into the benefits of a strengths- and place-based model for its ability to build school and community connections and capacities while at the same time deliver more frequent, consistent and effective Australian science curriculum in remote education. In a remote community environment where strong, respectful intercultural relationships and working partnerships with community partners are paramount, the delivery of government policies and standards could be more heavily influenced by each community’s strengths and aspirations towards improving local capacity to meet curriculum requirements in a flexible, culturally responsive way (Parkinson & Jones, 2018). This requires consideration of the role local learning communities could play in developing programs and activities—to deliver on education policy—that are enhanced by the views of local cultural and education leaders, inclusive of cultural and language strengths, and built on existing partnerships. At a system level, the direction and language of policy and recognised standards are generally encouraging. For example, the Alice Springs (Mparntwe) Education Declaration (Education Council, 2019); WA’s Aboriginal Cultural Standards Framework (Department of Education, 2015) and the Australian Institute for Teaching and School Leadership (AITSL) Australian Professional Standards for Teachers (AITSL, 2016) appear to align with the principles underlying the Science Pathways for Indigenous Communities program. However, amongst program schools, the successful translation of policies and standards into practice can differ substantially. When programs, like Science Pathways for Indigenous Communities, are designed to adapt to the strengths and aspirations of each community and respect community protocols and processes, this can result in more successful delivery of policy and quality standards among remote schools. For example, each community involved in the case study shared goals to support student identity, agency and ability to navigate two worlds and to achieve academic and personal success. The Program provided support and resourcing to assist remote school communities in achieving these educational goals in a way that responds to their strengths. This, in turn, helped improve school capacity to achieve quality standards and policy outcomes. The Science Pathways for Indigenous Communities program originated as an Aboriginal community program in the NT, and staff continue to operate under its original principles. This Program provides a successful model to operationalise a range of government education policies and standards, including the WA cultural standards framework (see Appendix D for how the Program delivers against these standards; the standards were developed by the WA Department of Education but the mapping of the Program against these standards apply to the entire Science Pathways for Indigenous Communities program). The Program’s design could also be considered to deliver culturally responsive teaching, including transformative teacher development; the delivery of the Australian curriculum using a two-way pedagogy; strengthening school-community partnerships to improve student learning experiences; reconnection to language where a disconnect has occurred; enhanced teacher practice, including teaching science generally; and science inquiry and cross-curriculum science connections. Learnings from this Program could also be applied to further25 inform the recruitment process for remote teachers, including preferred skill sets and teaching approaches (See Appendix E for a list of strength-based steps to improved remote education). The Program also highlights the critical value that Aboriginal educators, especially those with local cultural knowledge and language, play in their young peoples’ education. Action that supports Aboriginal and/or Torres Strait Islander peoples into education, especially teacher training pathways for community residents, could add great value to all aspects of this and similar programs. Using the findings While the evaluation findings are primarily applicable to three unique remote Aboriginal community schools, many of the learnings can be applied more generally to future iterations of the Program or similar programs. This section frames the evaluation findings to enable a broader application. People and place As with all community work, people and place are at the centre of this Program. The following illustration (Figure 10) developed from the case study highlights the influence people and place have on achieving program outcomes. The case study methodology emphasised that local languages, cultures, country types and residents strongly influenced the design and success of the Program in each school. Where communities and their schools shared strong culture, language, education goals and partnerships, the Program could more seamlessly build on these with support and structure. Across all communities, case study participants described certain individuals as being integral to the success of the Program. One program team member reflected on the challenge inherent to programs that are built on the characteristics of key people: “In some ways, though, it all comes down to people. So if key people leave the community, particularly at the school, then there’s a real risk to the Program not going ahead in the same way that it has been, but what—so the nature of remote education is that there is staff and principal turnover, and that is just the reality of remote education.” Despite this reliance on individuals, building stronger networks and partnerships mitigates the risks of turnover. 25 Program staff report that Science Pathways has already influenced teacher recruitment in Western Australia. Figure 10. Visual representation of the Program Figure 10. Visual representation of the Program School community contribution•Endorsement and active leadership•Sharing time, knowledge•Educators are willing students•Flexible and adaptable lessons•Two-way science is core to•Rangers and scientists withAchievements•Positive student experiences•Deeper student learning•Trust and understanding•Safe spaces for families•Achieving community education goals• •Cross curriculum science connections•Two-way curriculum and lessons•Sharing knowledge and culture•Relationships, coordinationProgram contribution•Build on existing strengths•Modelling and mentoring•Cross-curriculum•Planned, active and visibletwo- •Resources and new ideasScience Pathways for Indigenous CommunitiesHow community, people and place contribute to program successPeople in the community shape the success of the program through their relationships; leadership; being involved by sharing their goals, hopes, time and knowledge; and by being respectful learners and listening to others. Strong partnerships lead to achieving the community’s education goals. Shaped by peopleEach place shapes what the program looks like through language, culture, country, and animals. In turn, the program shapes the place by supporting knowledge, culture and language transfer and increasing opportunities for students to understand and care for their land. Shaped in place Program design learnings The role that individuals, small groups and networks of people played in advancing the Program cannot be underestimated. Evaluation evidence identified that the contribution of individuals was a fundamental yet changing element of the Program. Change in this area was observed through teacher turnover, student attendance and the varying availability, commitment and engagement of educators, community members, families and partner organisations. Frequent changes in these areas were observed to impact levels of leadership, knowledge, individual and group capacity, the frequency of Two-way Science, student learning experiences and the productivity of partnerships. It was also observed that people who contributed to this Program often did so despite experiencing a range of systemic and social barriers. The list of key success factors listed below highlights the extensive contribution that people made to the success of the Program as well as processes that were found to be critical to its success across all three communities. External factors that contribute to outcomes • Endorsement and leadership within the school. • Endorsement and leadership from within the local community and language groups. • Time, knowledge and expertise of local knowledge holders and language speakers. • Teachers have an attitude that says “take a risk, give it a go, step back and learn from students, family and Elders”. • Teachers are willing learners and are adaptable and flexible with their lesson planning and delivery. • Curriculum structures that support the integration of Two-way Science. • Partners with aligned goals, especially Indigenous ranger programs and science professionals. Key program factors that contribute to its success • Program facilitator characteristics—decolonising the community development approach/dialogical developmental approach that recognises power differences but places a central focus on dialogue between people to guide development.26 • On Country, co-facilitated teacher professional development with a strong modelling and mentoring component. • Two-way learning activities and resources that create cross-curriculum efficiencies for teachers. • Two-way Science that is planned and active (embedded) in the school system and community to ensure sustainability, despite remote education barriers (such as teacher turnover) and loss of key individuals. Program logic Evaluation learnings have been used to create a revised Impact Pathway (program logic diagram), representing the Program across the three communities at the time of the evaluation (see Appendix A). The initial Science Pathways Impact Pathway articulates an intended and static approach to program success (see Appendix A). While this pathway captures the original program theory and outcomes, the evidence collected during the evaluation identified very adaptive and place-based models, resulting in responsive and often unique program variations at each location. This suggests that program logic diagrams could be created for each school community to reflect their unique strengths and aspirations. It is acknowledged that this would be resource-intensive, and to partially address this, school leaders would need to be supported by CSIRO staff in the short- to medium-term to build up the capacity to develop program logics. It is also important that location-specific program logics remain dynamic and flexible to adapt to changing needs and aspirations while maintaining some core goals to ensure consistency and clear longer-term direction. 26 Decolonising community development refers to an approach that privileges Aboriginal and Torres Strait Islander leadership, histories, ways of knowing, and ways of being. Challenges and barriers Engagement Evidence suggested that program staff were continually striving to increase engagement or reconnect with parents and community members since this was considered vital to the success of the Program. Alongside the positive stories, some participants highlighted barriers to engagement on an individual and system level, citing challenges common to the remote education sector, such as teacher turnover, lack of resources and local services and navigating the generational impacts of colonisation. Gradual and ongoing relationship development was a common theme highlighted by communities as being vital to engagement. As one teacher recalled: “It took a little while [for parents to engage], I guess to be comfortable with me as maybe they were as comfortable with [other teachers]. It’s getting better, I think. They’ve [community members] got to really trust you, like trust that you’re going to do the right thing by their kids.” One program staff member reflected that it could be difficult to engage parents to participate in Western, school-driven initiatives, such as organising dates for formal activities and ensuring families can attend: “I wouldn’t say that we have family in as frequently as we’d like to, and it’s still a trial and effort to find dates that match and [to] go and pick families up, and they’re busy.” It appeared that informal, or family-led, involvement was the kind of engagement that was generally most common in the communities: “They [families] know they’re welcome. Sometimes, some of the family will come in, they’ll spend the day, and they’ll just have a feed with their kids, which is so lovely. There isn’t that [uncomfortable] element.” Across the two jurisdictions, systemic barriers were raised by educators, such as the need for community members to access and produce government documentation— for example, a working with children card—which limited opportunities for responsive engagement. In one instance, some female Elders were unable to work with female high school students on a specific activity because of a lack of required documentation. Lack of access to services was also noted as a reason for some family members leaving town regularly and, therefore, not engaging with the school and Program: “There’s no store here and no bus service, they have to take the opportunities to go to town. The clinics dropped down. It used to be two or three days a week, now, it’s one day a week.” Access to appropriately sized transport vehicles to take children out bush was also raised as a barrier by educators across the two jurisdictions. Restricted transport options sometimes meant family and community members could not participate in on Country learning. One interviewee advised that these issues were being addressed by their community: “So we used to take more Elders, but hopefully in the future, we can again, now that we’ve got the new vehicle coming and a larger vehicle, we can do more trips, more often.” Several teachers across the communities commented on the impacts of colonisation and how they manifest locally in affecting the community’s resilience. One teacher commented: “There’s still all that disconnection that came from the stolen generation. And the cycle of grief is constant, and there are times when you just wish that the current adults could concentrate more on the kids but…they’re supporting each other to get through this funeral before they have to go to that funeral…it reinforces for me every time how much…people [have gone through].” An example was provided of some local dads being incarcerated, which had necessitated other family members travelling out of the community to assist them, making it more difficult for them to be engaged with the school and the Program. As one teacher noted: “A lot of the kids’ actual parents have social concerns that maybe prevent them [from visiting the school]. So it is normally the older generation that we do see here [at school].” In this community, one of the Elders raised their concern around the impact of introduced drugs on the students’ connection to culture and Country: “And that’s why back in the early days it [drugs] wasn’t known. That’s what we trying to put it to our children, and when they grow up because the world, the thing is changing every time. Your world [Western], that one— because culture, we learn culture straight away from our old people. Q: Does it keep you strong, to know culture? A: That’s been keeping me strong, because, otherwise, I would have been phew—I’m just learning what mum taught me, that helped me to think about the Country. She told me story, way back in the desert story.” Building teacher capacity Across the three communities, the barriers to improving teacher capacity were highlighted by case study participants. These included: • when classroom teachers or non-Indigenous teachers do not value two-way learning or do not participate in on Country learning trips • a lack of available local teachers, Elders, Traditional Owners, family or community members available to lead components of two-way learning • school staff turnover that impacts Two-way Science continuity, consistency and capacity • lack of school resources and having to rely on the use of resources (such as vehicles) from external stakeholders such as Science Pathways for Indigenous Communities Program staff, linguists/translators and rangers Remote education barriers The implementation of the Program was influenced by remote education barriers (such as teacher turnover) and loss of key individuals. When case study participants spoke about the barriers they experienced in delivering Two-way Science, they identified challenges inherent to the remote education sector, such as a lack of school and transport resources, staff turnover and the underemployment of Aboriginal educators. The program staff confirmed that: “Transport is always brought up as a barrier” and gave examples of several schools that were raising money to buy an additional vehicle. A teacher summarised by saying: “Schools always need more funding. It’d be great if we had twice as many local teachers.” CONCLUSION AND RECOMMENDATIONS Summary of achievements The Program met seven out of the nine outcome areas identified in the Impact Pathway. Two outcome areas were not pursued by the program team. A summary of the outcome areas, levels and types of evidence and relevant indicators are outlined in Table 6. Table 6. Summary of achievements against outcome areas OUTCOME AREA ACHIEVED LEVEL OF EVIDENCE* TYPE OF EVIDENCE* 1. Strong, effective partnerships established with schools and other stakeholders 3 High Direct Indicators of success: • New partnerships were created between schools and community stakeholders, and existing partnerships were strengthened • There was a high level of trust and respect among school staff, community stakeholders and program staff • Strong personal relationships existed among school staff, community stakeholders and program staff, including frequent face-to-face contact • Partnerships focused on achieving shared goals • The value placed on Aboriginal knowledges strengthened partnerships • There was increased coordination and collaboration among stakeholders • Broader community networks were formed • School capacity and confidence in building partnerships with external stakeholders increased 2. Increased community and parental engagement 3 High Direct Indicators of success: • Parents and other community members felt safer and more welcome at the school • The Program created opportunities for new interactions between the community and school that expanded beyond the Program • The Program created more opportunities for community and family to be involved in student learning • Parents and other community members and groups took part in on Country and classroom Two-way Science activities and often were the driving force behind them • Parents had increased communication with teachers and program staff • Elders, Traditional Owners and other knowledge custodians shared their experiences, language and knowledges with students and teachers in the context of Two-way Science • Aboriginal scientific knowledges held by community members were valued and helped increase community engagement • Elders, Traditional Owners and other knowledge custodians were involved in setting expectations for student learning and in curriculum and term planning • A wider range of community members was engaged with the school 3. The Program’s Two-way Science learning approach and resources are culturally responsive and embedded across participating school curriculums. 3 High Direct Indicators of success: • Two-way Science learning resources were user-friendly and helped increased teacher knowledge and confidence at embedding traditional knowledges across curriculum areas • The Two-way Science learning approach was evident across multiple learning areas • The Two-way Science learning approach and resources were co-developed with community members and reflected place and local knowledges • The use of Aboriginal scientific knowledges in the classroom increased • Many Two-way Science activities were led by Aboriginal people • Non-Aboriginal teachers were receptive to the Two-way Science approach and resources, including community members leading on Country and classroom activities 4. Increased teacher capacity in Two-way Science using on Country contexts and cultural competence. 3 High Direct Indicators of success: • Teachers felt confident to plan and deliver Two-way Science education on Country and in the classroom • On Country learning increased in frequency and focus • Non-Aboriginal teachers effectively helped Aboriginal assistant teachers, Elders and other knowledge custodians lead Two-way Science activities and learning • Aboriginal and non-Aboriginal teachers had an overall improved capacity to teach science • Non-Aboriginal teachers became more culturally responsive, including increased understanding of Indigenous ecological knowledge, local cultural knowledge and language and skills engaging with Aboriginal knowledge holders • For assistant teachers in the NT, opportunities to collaborate in language resulted in a strengthening of Two-way Science teaching practice and confidence. OUTCOME AREA ACHIEVED LEVEL OF EVIDENCE* TYPE OF EVIDENCE* 5. Increased aspiration, sense of value and school belonging. 3 High Direct Indicators of success: • Increased focus on student wellbeing • Students aspire to be role models to younger students • Increase in respectful behaviour among students • Strengthened intergenerational roles and knowledge transfer • Elders and older Traditional Owners and other knowledge custodians more physically and mentally active • Students beginning to enhance their ability to negotiate the cultural interface • Students exposed to a more diverse range of career roles, particularly those leading on Country learning such as rangers, ecologists and other scientists • Students’ aspirations for STEM education and careers increased, particularly for STEM roles visible in the community • Reduction in the pressure for students to ‘code-switch’ between home and school 6. Increased student engagement and attendance. 3 High Direct Indicators of success: • Increased student attendance and engagement during Two-way Science on Country activities • Increased student positive behaviour and self-directed learning • Increased use of language in learning • Increased student happiness and sharing of learning with family members • Increased student confidence, perseverance and inquisitiveness • Increased student pride in culture • Increased exposure to individuals’ strengths being acknowledged and shared 7. Enhanced student results 3 Medium Indirect Indicators of success: • Increased school capacity to assess student learning in science • Improvements in student learning, as evidenced by strong recall and work samples • Deeper student learning of science • Increased engagement in other areas of the curriculum, including literacy and numeracy • Enhanced student vocabulary and literacy skills • Increased student understanding of culture and Aboriginal scientific knowledges • Some indication of increased retention due to VET pathway in one community 8. Centres of excellence in two-way STEM education x** None n/a 9. University teacher training using teacher professional development, Department of Education extending model to other remote schools x** None n/a *Level of evidence refers to the amount of evidence available to ascertain whether the outcome was achieved (high, medium, low); type of evidence refers to whether the evidence was direct (directly observed or reported by relevant stakeholders) or indirect (inferred from secondary sources or stakeholders). **A program decision was made not to pursue these outcome areas. Recommendations The following recommendations are aimed at building on the identified strengths and successes of the Program and improving its effectiveness and sustainability. The recommendations may have wider implications for Two-way Science and remote education practice and program design. 1. Explore online and virtual methodologies and tools for planning and delivering Two-way Science to a broader range of remote schools, particularly in light of the COVID-19 pandemic. 2. Work with schools to develop assessment approaches derived from Aboriginal pedagogies for assessing Two-way Science learning among students that can be used to effectively capture student progress, especially for students that speak EAL/D. 3. Explore opportunities to help strengthen career pathway support for students in remote areas to become teachers and assistant teachers, as well as to other roles that are aspired to in the community, such as rangers. 4. Further promote Two-way Science pedagogy and resources to universities offering training to pre-service teachers, as well as Departments of Education. 5. Explore opportunities to adapt Two-way Science approaches and resources to non-desert contexts, for example, to the Torres Strait Islands and other tropical contexts. 6. Help build the capacity of schools to develop individualised school and community learning plans that reflect community education goals and that contribute to sustainable learning communities. Plans should include community-driven indicators of success. 7. Explore opportunities for more student involvement in Two-way Science planning and identify opportunities for students to develop and lead two-way projects. 8. Develop sustainability plans with schools that outline how the gains made by the Program and communities working together can be self-sustained by communities. 9. Assist schools in identifying opportunities to collaborate with additional stakeholders where there are shared or aligned goals. 10. Work with Departments of Education to explore how the Program can contribute to teacher recruitment and retention in remote locations. REFERENCES Allen, A., Kern, M.L., Vella-Brodrick, D., Hattie, J., & Waters, L. (2018). What schools need to know about fostering school belonging: A meta- analysis. Educational Psychology Review, 30(1), 1–34. https://doi.org/10.1007/s10648-016-9389-8 Appleton, J.J., Christenson, S.L., & Furlong, M.J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369–386. https://doi.org/10.1002/pits.20303 Attia, M., & Edge, J. (2017). Be(com)ing a reflexive researcher: A developmental approach to research methodology. Open Review of Education Research, 4, 33–45. https://doi.org/10.1080/23265507.2017.1300068 Australian Curriculum Assessment and Reporting Authority (ACARA). (2018). Illustrations of practice. Retrieved from https://www.australiancurriculum.edu. au/resources/aboriginal-and-torres-strait-islander- histories-and-cultures/illustrations-of-practice/ Australian Curriculum Assessment and Reporting Authority (ACARA). (2019). Measurement framework for schooling in Australia. Retrieved from https:// www.acara.edu.au/reporting/measurement- framework-for-schooling-in-australia Australian Curriculum Assessment and Reporting Authority (ACARA). (2020). My school. Retrieved from https://www.myschool.edu.au/ Australian Institute for Teaching and School Leadership (AITSL). (2011). Australian professional standards for teachers. Retrieved from www.aitsl.edu.au/docs/default- source/national-policy-framework/australian-professional- standards-for-teachers.pdf?sfvrsn=5800f33c_64 Australian Institute for Teaching and School Leadership (AITSL). (2016). Professional teacher standards. Retrieved from www.aitsl.edu.au/teach/standards Australian Institute for Teaching and School Leadership (AITSL). (2019). Spotlight: Attendance matters. Retrieved from www.aitsl.edu.au/docs/default-source/ research-evidence/spotlight/11319-aitsl_spotlight_ attendance_web-fa.pdf?sfvrsn=5bb0ff3c_6 Australian Science Teachers Association (ASTA). (2009). National professional standards for highly accomplished teachers of science. Retrieved from https:// asta.edu.au/resources/professional_standards Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Education Psychologist, 50(1), 84–94. https:// doi.org/10.1080/00461520.2015.1004069 Barrett, E. (2017). The wellbeing of Aboriginal children who attend a ‘two-way’ independent separatist school in remote Australia: A post-colonial study (Unpublished Doctoral thesis). Western Sydney University, Sydney, Australia. Retrieved from https://researchdirect.westernsydney.edu. au/islandora/object/uws%3A49905/datastream/PDF/view Battiste, M. (2011). Cognitive imperialism and decolonizing research: Modes of transformation. In C. Reilly, V. Russell, L.K. Chehayl, & M. M. McDermott (Eds.), Surveying borders, boundaries, and contested spaces in curriculum and pedagogy (pp. xv-xxviii). Charlotte, NC: Information Age Publishing. Behrendt, L., Larkin, S., Griew, R., & Kelly, P. (2012). Review of higher education access outcomes for Aboriginal and Torres Strait Islander People: Final report. Department of Education, Skills and Employment report. Retrieved from https://www.education.gov. au/review-higher-education-access-and-outcomes- aboriginal-and-torres-strait-islander-people Bennet, M., & Moriarty, B. (2016). Language, relationships and pedagogical practices: Pre-service teachers in an Indigenous Australian context. International Journal of Pedagogies and Learning, 10(1), 1–12. http:// dx.doi.org/10.1080/22040552.2015.1084672 Bierman, S. (2008). Indigenous pedagogies and environmental education: Starting a conversation. International Journal of Pedagogies and Learning, 4(3), 27–38. https://doi.org/10.5172/ijpl.4.3.27 Bobongie, F. (2017). Family+stories=Research. Qualitative Research Journal, 17(4), 345–355. https://doi.org/10.1108/QRJ-11-2016-0069 Boon, H.J., & Lewthwaite, B.E. (2016). Signatures of quality teaching for Indigenous students. The Australian Educational Researcher, 43(4), 453–471. https://doi.org/10.1007/s13384-016-0209-4 Brasche, I., & Harrington, I. (2012). Promoting teacher quality and continuity: Tackling the disadvantages of remote schools in the Northern Territory. Australian Journal of Education, 56(2), 110–125. https://doi.org/10.1177/000494411205600202 Briggs, A. (2017). Links between senior high school Indigenous attendance, retention and engagement: Observations at two urban high schools. Australian Journal of Indigenous Education, 46(1), 34–43. https://doi.org/10.1017/jie.2016.14 Bulloch, H., & Fogarty, W. (2016). Freeing the ‘Aboriginal individual’: Deconstructing ‘development as freedom’ in remote Indigenous Australia. Social Analysis, 60(3), 76–96. https://doi.org/10.3167/sa.2016.600305 Burgess, C., & Cavanagh, P. (2015). Cultural immersion: Developing a community of practice of teachers and Aboriginal community members. The Australian Journal of Indigenous Education 45(1), 48–55. https://doi.org/10.1017/jie.2015.33 Burgess, C., Tennent, C., Vass, G., Guenther, J., Lowe, K., & Moodie, N. (2019). A systematic review of pedagogies that support, engage and improve the educational outcomes of Aboriginal students. The Australian Association for Research in Education, 46, 297–318. https://doi.org/10.1007/s13384-019-00315-5 Chenhall, R., Holmes, C., Lea, T., Senior, K., & Wegner, A. (2011). Parent-school engagement: Exploring the concept of ‘invisible’ Indigenous parents in three north Australian school communities. Darwin, Australia: The Northern Institute, Charles Darwin University. Retrieved from https://ro.uow. edu.au/cgi/viewcontent.cgi?article=2457&context=sspapers Christenson, S.L., Reschly, A.L., & Wylie, C. (Eds). (2012). Handbook of research on student engagement. Washington, DC, US: Springer Science+Business Media. https://doi.org/10.1007/978-1-4614-2018-7 Christie, M., Carey, M., Robertson, A., & Grainger, P. (2015). Putting transformative learning theory into practice. Australian Journal of Adult Learning, 55(1), 9–30. Retrieved from https://files.eric.ed.gov/fulltext/EJ1059138.pdf Conley-Taylor, M. (2005). A fundamental choice: internal or external evaluation? Evaluation Journal of Australasia, 4(1- 2), 3–11. Retrieved from www.aes.asn.au/images/stories/ files/Publications/Vol4No1-2/fundamental_choice.pdf Council of Australian Governments. (2008). Melbourne Declaration on Educational Goals for Young Australians. Ministerial Council on Education, Employment, Training and Youth Affairs. Retrieved from www.curriculum. edu.au/verve/_resources/National_Declaration_on_ the_Educational_Goals_for_Young_Australians.pdf Council of Australian Governments. (2018). National School Reform Agreement. Retrieved from https://docs.education.gov.au/node/51606 CSIRO. (2018). Science Pathways NT: Progress report December 2018. Unpublished program document. Cummins, J. (2001). Empowering minority students: A framework for intervention. Harvard Educational Review, 71(4), 649–675. https://doi. org/10.17763/haer.56.1.b327234461607787 Department of Education. (2015). Aboriginal cultural standards framework. Retrieved from http://det.wa.edu. au/policies/detcms/policy-planning-and-accountability/ policies-framework/strategic-documents/aboriginal- cultural-standards-framework-.en?cat-id=3457058 Deslandes, C., Deslandes, S., Broun, D., Hugh, C., Walsh, F., Bradshaw, F., & Griffith, J. (2019). Two-way science: An integrated learning program for Aboriginal desert schools. Victoria, Australia: CSIRO Publishing. Dew, A., McEntyre, E., & Vaughan, P. (2019). Taking the research journey together: The insider and outsider experiences of Aboriginal and non-Aboriginal researchers. Forum: Qualitative Social Research, 20(1), Article 18. http://dx.doi.org/10.17169/fqs-20.1.3156 Dobia, B., & O’Rourke, V. (2011). Promoting the mental health and wellbeing of Indigenous children in Australian primary schools. Canberra, Australia: Commonwealth of Australia. Douglas, J. (2011). Learning from country: The value of country visits in remote schools for community engagement and natural and cultural resource management. Desert Knowledge CRC Report No 69. Alice Springs, Australia: Ninti One Limited. Retrieved from www.nintione. com.au/resource/NintiOneResearchReport_69_ Thevalueofcountryvisitsinremoteschools.pdf Doyle, L., & Hill, R. (2008). Our children, our future - achieving improved primary and secondary education outcomes for Indigenous students: An overview of investment opportunities and approaches. Social Ventures Australia report. Retrieved from www.socialventures. com.au/assets/Our_Children_Our_Future.pdf Dunstan, L., Hewitt, B., & Tomaszewski. W. (2017). Indigenous children’s affective engagement with school: The influence of socio-cultural, subjective and relational factors. Australian Journal of Education, 61(1), 250–269. http://dx.doi.org/10.1177/0004944117732637 Education Council. (2019). Alice Springs (Mparntwe) Education Declaration. Retrieved from https:// uploadstorage.blob.core.windows.net/public- assets/education-au/melbdec/ED19-0230%20-%20 SCH%20-%20Alice%20Springs%20(Mparntwe)%20 Education%20Declaration_ACC.pdf Fergie, D. (2018). “I wonder what lies beyond that horizon?” Australian Journal of Education, 62(3), 289– 298. http://dx.doi.org/10.1177/0004944118798659 Flouris, A., Crane, P., & Lindeman, M. (2016). Youth programmes in remote Indigenous communities: Context matters. Rural Society, 25(1), 37–54. http:// dx.doi.org/10.1080/10371656.2016.1150197 Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments. Issues & Answers Report, REL 2011–No. 098. Washington, DC, USA: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from https://files.eric.ed.gov/fulltext/ED514996.pdf Gavidia-Payne, S., Denny, B., Davis, K., Francis, A., & Jackson, M. (2014). Children’s self-concept: Parental school engagement and student-teacher relationships in rural and urban Australia. Social Psychology Education, 18(1), 121–136. http://dx.doi.org/10.1007/s11218-014-9277-3 Geia, L., Hayes, B., & Usher, K. (2013). Yarning/ Aboriginal storytelling: Towards an understanding of an Indigenous perspective and its implications for research practice. Contemporary Nurse, 46(1), 13–17. https://doi.org/10.5172/conu.2013.46.1.13 Gore, J., Patfield, S., Holmes, K., Smith, M., Lloyd, A., Gruppetta, M., Weaver, N., & Fray, L. (2017). When higher education is possible but not desirable: Widening participation and the aspirations of Australian Indigenous school students. Australian Journal of Education, 61(2), 164–183. https://doi.org/10.1177/0004944117710841 Gray, M., & Bray, R. (2019). Evaluation in the Australian public service: Current state of play, some issues and future directions. An ANZOG research paper for the Australian Public Service Review Panel. Retrieved from https://www.apsreview.gov.au/sites/default/ files/resources/appendix-b-evaluation-aps.pdf Gray, M., & Oprescu, F. (2016). Role of non-Indigenous researchers in Indigenous health research in Australia: A review of the literature. Australia Health Review, 40(4), 459–465. http://dx.doi.org/10.1071/AH15103 Guenther, J., Disbray, S., & Osborne, S. (2016). Red dirt education: A compilation of learnings from the Remote Education Systems project. Alice Springs, Australia: Ninto One Limited. Retrieved from http:// www.crc-rep.com.au/resource/RedDirtEducation_ CompilationLearningsRES_EBook.pdf Guenther, J., Lowe, K., Burgers, K., Vass, G., & Moodie, N. (2019). Factors contributing to educational outcomes for First Nations students from remote education communities: A systematic review. The Australian Educational Researcher, 46, 319–340. https://doi.org/10.1007/s13384-019-00308-4 Haines, J., Du, J.T., Geursen, G., Gao, J., & Trevorrow, E. (2017). Understanding Elders’ knowledge creation to strengthen Indigenous ethical knowledge sharing. Paper presented at the Proceedings of RAILS – Research, Applications, Information and Library Studies, 2016, School of Information Management, Victoria University of Wellington, New Zealand. Retrieved from http:// informationr.net/ir/22-4/rails/rails1607.html Hancock, K.J., Mitrou, F., Taylor, C.L., & Zubrik, S.R. (2018). The diverse risk profiles of persistently absent students: Implications for attendance policies in Australia. Journal of Education for Students Places at Risk, 23(1-2), 53–69. https://doi.org/10.1080/10824669.2018.1433536 Harrison, H., Birks, M., Franklin, R., & Mills, L. (2017). Case study research: Foundations and methodological orientations. Forum: Qualitative Social Research, 18(1), Article 19. http://dx.doi.org/10.17169/fqs-18.1.2655 Harrison, N., & Greenfield, M. (2011). Relationship to place: Positioning Aboriginal knowledge and perspectives in classroom pedagogies. Critical Studies in Education, 52(1). Retrieved from https://ir.lib.uwo. ca/cgi/viewcontent.cgi?article=1101&context=aprci Hart, S.R., Stewart, K., & Jimerson, S.R. (2011). The Student Engagement in Schools Questionnaire (SESQ) and the Teacher Engagement Form-New (TERF-N): Examining the preliminary evidence. Contemporary School Psychology, 15, 67–79. https://doi.org/10.1007/BF03340964 Hart, V., Whatman, S., McLaughlin, J., & Sharma-Brymer, V. (2012). Pre-service teachers’ pedagogical relationships and experiences of embedding Indigenous Australian knowledge in teaching practicum. Compare: A Journal of Comparative and International Education, 42(5), 1–21. http://dx.doi.org/10.1080/03057925.2012.706480 Hogarth, M. (2017). Speaking back to the deficit discourses: A theoretical and methodological approach. The Australian Educational Researcher, 44(1), 21–34. https://doi.org/10.1007/s13384-017-0228-9 Hopkins, K., Zubrick, S., & Taylor, C. (2014). Resilience amongst Australian Aboriginal youth: An ecological analysis of factors associated with psychosocial functioning in high and low family risk contexts. Plos One, 9(7), 1–12. https://doi.org/10.1371/journal.pone.0102820 Howard, E. (2017). Context and culture in evaluation: A case study of evaluation anthropology. Evaluation Journal of Australasia, 17(1), 30–38. https://doi.org/10.1177/1035719X1701700105 Hunt, J. (2010). Partnerships for Indigenous development: International development NGOs, Aboriginal organisations and communities. Centre for Aboriginal Economic Policy Research. Australian National University. Working Paper No. 71/2010. Retrieved from https://caepr.cass.anu.edu.au/sites/default/ files/docs/WP71_Hunt_publication_final_0.pdf Hunt, J. (2012). Community development for sustainable early childhood care and development programs: A World Vision Australia and Central Land Council partnership. ANU Centre for Aboriginal Economic Policy Research Working Paper No. 86/2012. Retrieved from https://openresearch- repository.anu.edu.au/bitstream/1885/147835/1/ WP86_-_Hunt_Partnerships_0%20%281%29.pdf Jackson-Barrett, E., Price, A., Stomski, N., & Walker, B.F. (2015). Grounded in country: Perspectives on working within, alongside and for Aboriginal communities. Issues in Educational Research, 25(1), 36–479. Retrieved from http://www.iier.org.au/iier25/jackson-barrett.html Jackson-Barrett, E., & Lee-Hammond, L. (2019). From Pink Floyd to Pink Hill: Transforming education from the bricks in the wall to the connections of country in remote Aboriginal education. Australian Journal of Teacher Education, 44(10). http://dx.doi.org/10.14221/ajte.2019v44n10.3 Johnson, A. (2013). To case study or not to case study: Our experience with the Canadian Government’s evaluation practices and the use of case studies as an evaluation methodology for First Nations programs. The Canadian Journal of Program Evaluation, 28(2), 21–42. Retrieved from https://evaluationcanada. ca/system/files/cjpe-entries/28-2-021.pdf Kearney, A. (2019). Interculturalism and responsive reflexivity in a settler colonial context. Religions, 10(100), 1–18. http://dx.doi.org/10.3390/rel10030199 Kearney, J., & Zuber-Skerrit, O. (2012). From learning organisation to learning community: Sustainability through lifelong learning. The Learning Organisation, 19(5), 400–413. https://doi.org/10.1108/09696471211239703 Kingsley, J., Townsend, M., Henderson-Wilson, C., & Bolam, B. (2013). Developing an exploratory framework linking Australian Aboriginal peoples’ connection to country and concepts of wellbeing. International Journal of Environmental Research and Public Health, 10(2), 678–698. https://doi.org/10.3390/ijerph10020678 Kirby, S., Held, F., Jones, D., & Lyle, D. (2018). Growing health partnerships in rural and remote communities: What drives the joint efforts of primary schools and universities in maintaining service learning partnerships? Primary Health Care Research and Development, 19(5), 503–517. https://doi.org/10.1017/S146342361700086X Klenowski, V. (2009). Australian Indigenous students: Addressing equity issues in assessment. Teaching Education, 20(1), 77–93. https://doi.org/10.1080/10476210802681741 Kline, J., White, S., & Lock, G. (2013). The rural practicum: Preparing a quality teacher workforce for rural and regional Australia. Journal of Research in Rural Education, 28(3), 1–13. Retrieved from http://jrre.psu.edu/articles/28-3.pdf Larkin, P., Dierckx de Casterle, B., & Schotsmans, P. (2012). Multilingual translation issues in qualitative research: Reflections on a metaphorical process. Qualitative Health Research, 17(4), 468–476. https://doi.org/10.1177/1049732307299258 Lewthwaite, B., McMillan, B., Renaud, R., Hainnu, R., & MacDonald, C. (2010). Combining the view of “both worlds”: Science education in Nunavut Piqusiit Tamainik Katisugit. Canadian Journal of Educational Administration and Policy, 98, 1–71. Retrieved from https://pdfs. semanticscholar.org/7a1b/133287c807382a5cecfd2abb72 9f4a77a2de.pdf?_ga=2.9710965.1105536592.1583194808- 1839430586.1582246802 Lewthwaite, B., Osborne, B., Lloyd, N., Boon, H., & Llewellyn, L. (2015). Seeking a pedagogy of difference: What Aboriginal students and their parents in North Queensland say about teaching and their learning. Australian Journal of Teacher Education, 40(5), Article 8. http://dx.doi.org/10.14221/ajte.2015v40n5.8 Li, Y., & Lerner, R. M. (2013). Interrelations of behavioral, emotional, and cognitive school engagement in high school students. Journal of Youth and Adolescence, 42(1), 20–32. https://doi.org/10.1007/s10964-012-9857-5 Lohoar, S., Butera, N., & Kennedy, E. (2014). Strengths of Australian Aboriginal cultural practices in family life and child rearing. Child Family Community Australia, Paper No. 25, Australian Institute of Family Studies. Retrieved from https://aifs.gov.au/cfca/sites/ default/files/publication-documents/cfca25.pdf Lonsdale, M. (2013). Making a difference: Improving outcomes for Indigenous learners. Australian Council for Education Research report. Retrieved from https:// research.acer.edu.au/indigenous_education/29/ Lowe K., & Yunkaporta, T. (2013). The inclusion of Aboriginal and Torres Strait Islander content in the Australian national curriculum: A cultural, cognitive and socio-political evaluation. Curriculum Perspectives, 33(1), 1–15. Lowe, K. (2017). Walanbaa warramildanha: The impact of authentic Aboriginal community and school engagement on teachers’ professional knowledge. The Australian Education Researcher, 44(1), 35–54. https://doi.org/10.1007/s13384-017-0229-8 Ma Rhea, Z., Anderson, P., & Atkinson, B. (2012). Improving teaching in Aboriginal and Torres Strait Islander education: Australian professional standards for teachers. Melbourne, Australia: Australian Institute for Teaching and School Leadership. Retrieved from www.aitsl.edu.au/docs/ default-source/default-document-library/improving- teaching-in-aboriginal-and-torres-strait-islander- education-professional-development-and-the-australian- professional-standards-for-teachers-monash-university.pdf Marmion, D., Obata, K., & Troy, J. (2014). Community, identity, wellbeing: The report of the second national Indigenous languages survey. Retrieved from https://aiatsis.gov.au/publications/products/ community-identity-wellbeing-report-second- national-indigenous-languages-survey Martin, K. L., & Mirraboopa, B. (2003). Ways of knowing, ways of being and ways of doing: A theoretical framework and methods for Indigenous research and Indigenist re-search. Journal of Australian Studies, 27(76), 203–214. http://dx.doi.org/10.1080/14443050309387838 McLaughlin, J. (2013). ‘Crack in the pavement’: Pedagogy as political and moral practice for educating culturally competent professionals. The International Education Journal: Comparative Perspectives, 12(1), 249–265. Retrieved from https://files.eric.ed.gov/fulltext/EJ1017678.pdf Moreton-Robinson, A. (2000). ‘Talkin’ up to the white woman: Indigenous women and feminism. St Lucia, Australia: University of Queensland Press. Moriarty, B., & Bennet, M. (2016). Practicing teachers’ reflections: Indigenous Australian student mobility and implications for teacher education. Social Inclusion, 4(1), 32–41. http://dx.doi.org/10.17645/si.v4i1.373 Morley, S. (2015). What works in effective Indigenous community-managed programs and organisations. Child Family Community Australia Paper No. 32. Melbourne, Australia: Australian Institute of Family Studies. Retrieved from https://aifs.gov.au/cfca/ publications/what-works-effective-indigenous- community-managed-programs-and-organisations Morrison, A., Rigney, L-I., Hattam, R., & Diplock, A. (2019). Toward an Australian culturally responsive pedagogy: A narrative review of the literature. Adelaide, Australia: University of South Australia. Retrieved from https://apo.org.au/sites/default/files/ resource-files/2019/08/apo-nid262951-1392016.pdf Nakata, M. (2002). Indigenous knowledge and the cultural interface: Underlying issues at the intersection of knowledge and information systems. IFLA Journal, 28(5/6), 281–291. https://doi.org/10.1177/034003520202800513 Nakata, M. (2007). The cultural interface. The Australian Journal of Indigenous Education, 36(S1), 7–14. https://doi.org/10.1017/S1326011100004646 Parkinson C., & Jones, T. (2018). Aboriginal peoples’ aspirations and the Australian curriculum: A critical analysis. Education Research for Policy and Practice, 18(1), 75–97. https://doi.org/10.1007/s10671-018-9228-4. Patton, M.Q. (2018). Principles-focused evaluation: The GUIDE. New York, US: Guilford Press. Pendergast, D., Allen, J., McGregor, G., & Ronksley- Pavia, M. (2018). Engaging marginalised, “at-risk” middle-level students: A focus on the importance of a sense of belonging at school. Education Sciences, 8(138), 1–19. http://dx.doi.org/10.3390/educsci8030138 Perso, T. (2012). Cultural responsiveness and school education: With a particular focus on Australian’s First Peoples: A review & synthesis of the literature. Darwin, NT: Menzies School of Health Research, Centre for Child Development and Education. Retrieved from https:// www.menzies.edu.au/icms_docs/312407_Cultural_ Responsiveness_and_School_Education.pdf Pittaway, S. (2012). Student and staff engagement: Developing an engagement framework in a faculty of education. Australian Journal of Teacher Education, 37(4), Article 3. http://dx.doi.org/10.14221/ajte.2012v37n4.8 Price, K. (2012). A brief history of Aboriginal and Torres Strait Islander education in Australia. In K. Price (Ed.), Aboriginal and Torres Strait Islander education: An introduction for the teaching profession. New York, USA: Cambridge University Press. Prober, S.M., O’Connor, M.H., & Walsh, F.J. (2011). Australian Aboriginal peoples’ seasonal knowledge: A potential basis for shared understanding in environmental management. Ecology and Society, 16(2). Retrieved from http://www.ecologyandsociety.org/vol16/iss2/art12/ Purdie, N., Milgate, G., & Bell, H. (Eds.) (2011). Two way teaching and learning: Toward culturally reflective and relevant education. Melbourne, Australia: ACER Press. Retrieved from https:// research.acer.edu.au/indigenous_education/38 Rawsthorne, M., Kinsela, G., Paxton, K., & Luscombe, G. (2019). Young person’s well-being: Exploring material, subjective and relational factors. Children Australia, 44, 13–22. https://doi.org/10.1017/cha.2018.53 Reschly, A.L., & Christenson, S.L. (2012). Jingle, jangle, and conceptual haziness: Evolution and future directions of the engagement construct. In S.L. Christenson et al. (Eds.), Handbook of Research on Student Engagement (pp. 3–19). Berlin, Germany: Springer Science+Business Media. https://doi.org/10.1007/978-1-4614-2018-7 Rigney, L.I. (2003). Indigenous Australian views on knowledge production and Indigenist research. Studies (AIATSIS), 1(3), 32–49. Retrieved from https://pdfs.semanticscholar.org/97aa/6 12e543fdc6a311141ace5e1b37a5b2f91bf.pdf Rigney, L. I. (2006). Indigenist research and Aboriginal Australia. In J. Kunnie and I. Goduka (Eds.). Indigenous peoples’ wisdom and power: Affirming our knowledge through narratives (Chapter 3). London, UK: Routledge. https://doi.org/10.4324/9781315252414 Riley, L. (2015). Conditions of academic success for Aboriginal students in school (Unpublished Doctoral dissertation). Australian Catholic University, Sydney, Australia. Retrieved from https://doi.org/10.4226/66/5a9cd5d5b0bd5 Rose, D.B. (1996). Nourishing terrains: Australian Aboriginal views of landscapes and wilderness. Canberra, Australia: Australian Heritage Commission. Sarra, C. (2011). Strong and smart – towards a pedagogy for emancipation: Education for First Peoples. London, UK: Routledge. Shay, M., & Wicks, J. (2017). Aboriginal identity in education settings: Privileging our stories as a way of deconstructing the past and re-imagining the future. The Australian Educational Researcher, 44(1), 107–122. https://doi.org/10.1007/s13384-017-0232-0 Sivak, L., Westhead, S., Richards, E., Atkinson, S., Richards, J., Dare, H., Zuckerman, G., Gee, G., Wright, M., Rosen, A., Walsh, M., Brown, N., & Brown, A. (2019). “Language breathes life” – Barngarla community perspectives on the wellbeing impacts of reclaiming a dormant Australian Aboriginal language. International Journal of Environmental Research and Public Health, 16(20), 3918. http://dx.doi.org/10.3390/ijerph16203918 Sullivan, P., Jorgensen, R., Boaler, J., & Lerman, S. (2013). Transposing reform pedagogy into new contexts: Complex instruction in remote Australia. Mathematics Education Research Journal, 25(1), 173– 184. https://doi.org/10.1007/s13394-013-0069-4 SVA Consulting. (2016). Department of the Prime Minister & Cabinet: Consolidated report on Indigenous Protected Areas following Social Return on Investment analyses. Retrieved from https://www.socialventures.com.au/ assets/Consolidated-SROI-Report-on-IPA-WoC.pdf Sveilby, K-E. (2009). Aboriginal principles for sustainable development as told in traditional law stories. Sustainable Development, 17(6), 341–356. https://doi.org/10.1002/sd.389 Sveilby, K-E., & Skuthorpe, T. (2006). Treading lightly: The hidden wisdom of the world’s oldest people. Sydney, Australia: Allen and Unwin. Taylor, D., & Govan, B. (2017). STEM outreach in Northern Queensland: The importance of providing professional development and networking opportunities to educators. International Journal of Innovation in Science and Mathematics Education, 25(5), 57–68. Retrieved from https://openjournals.library.sydney. edu.au/index.php/CAL/article/view/11727/11450 Tomlins-Jahnke, H. (2013). The task of an educator is supporting “communities of learners” as transformative practice. Education Perspectives, 45(1,2), 65–73. Retrieved from https://files.eric.ed.gov/fulltext/EJ1024840.pdf Treagust, D., Won, M., Petersen, J., & Wynne, G. (2015). Science teacher education in Australia: Initiatives and challenges to improve the quality of teaching. The Journal of Science Teacher Education, 26, 81–98. https://doi.org/10.1007/s10972-014-9410-3 Tsou, C., Green, C., Gray, G., & Thompson, S.C. (2018). Using the Healthy Community Assessment Tool: Applicability and adaptation in the mid-west of Western Australia. International Journal of Environmental Research and Public Health, 15(6), 1159. https://doi.org/10.3390/ijerph15061159 Vass, G. (2018). ‘Aboriginal learning style’ and culturally responsive schooling: Entangled, entangling and the possibilities of getting disentangled. Australian Journal of Teacher Education, 43(8), 89–104. Retrieved from http://ro.ecu.edu.au/ajte/vol43/iss8/6 Walker, K. (2006). ‘Aiming high’: Australian school leavers’ career aspirations and the implications for career development practice. Australian Journal for Career Development, 15(2), 53–60. https:// doi.org/10.1177/103841620601500209 Walter, M. (2015). The vexed link between social capital and social mobility for Aboriginal and Torres Strait Islander people. The Australian Journal of Social Issues, 50(1), 69–88. https://doi.org/10.1002/j.1839-4655.2015.tb00335.x Willig, C., & Rogers, W.S. (2017) The SAGE handbook of qualitative research in psychology. London, UK: SAGE. http://dx.doi.org/10.4135/9781526405555 Woodward, E., Hill, R., Harkness, P., & Archer, R. (Eds.) (2020). Our knowledge our way in caring for country: Indigenous-led approaches to strengthening and sharing our knowledge for land and sea management: Best practice guidelines from Australian experiences. NAILSMA and CSIRO. Retrieved from https://www.csiro.au/en/ Research/LWF/Areas/Pathways/Sustainable-Indigenous/ Our-Knowledge-Our-Way/OKOW-resources Yin, R.K. (1994). Case study research design and methods: Applied social research and methods series. Second Edition. Thousand Oaks, USA: Sage Publications Inc. Yunkaporta, T. (2009). Aboriginal pedagogies at the cultural interface (Unpublished Doctoral thesis). James Cook University, Townsville, Australia. Retrieved from http://eprints.jcu.edu.au/10974 Yunkaporta, T., & McGinty, S. (2009). Reclaiming Aboriginal knowledge at the cultural interface. The Australian Educational Researcher, 36(2), 55–72. Retrieved from https://researchonline.jcu.edu.au/8651/ Zyngier, D. (2017). How experiential learning in an informal setting promotes class equity and social and economic justice for children from “communities at promise”: An Australian perspective. International Review of Education, 63(1), 9–28. https://doi.org/10.1007/s11159-017-9621-x Appendix A – Impact Pathways Impact Pathways !"#$%&!!!$!$!$!!!'(((!!$!!!!& &A&./:,&BBC& Development of agreements with schools and key stakeholders including agreed monitoring framework Schools using Science Pathways resources in the classroom and on- country and these resources clearly identify ‘two-way’ science practice (where western STEM knowledge and practice complements traditional cultural knowledge and practice) Increased student aspiration, sense of value and school belonging Greater understanding and care of environment Facilitate design and delivery of on Country activities Increased community, parental engagement More, higher quality & greater workforce diversity of STEM professionals. Increased student results University teacher training using TPD; Departments of Education extending model to other remote schools Monitoring data Education resources developed into a cohesive community-based curriculum and learning resources (WA) and embedded in ILC delivery/ other curriculum processes (NT) Indigenous knowledge & culture valued: complementarity to western science and math’s demonstrated Identification and development of tools and learning resources for on- country science activities (e.g., bush foods & medicines, animal survey, waterhole monitoring, App/i- book template for collecting local data) Increased student engagement and attendance School culture of high TEK &STEM expectations, leveraged for literacy & other subjects ‘ Social cohesion/ reconciliation Development of activity plans and on Country activity registers for schools Centres of excellence in two- way science education Alternative STEM career pathways such as rangers, Parks and Wildlife, CSIRO cadet as well as university pathways become expected pathways in remote communities On the job Teacher Professional Development (TPD) / development of formal TPD TPD program/activities Increased teacher capacity in science inquiry using on- country contexts Development of program monitoring processes OUTCOMES The uptake, adoption or consumption of our work IMPACTS Benefits to eco, environ, soc INPUTS What we invest $3.05m Staff Existing relationships with remote communities Tangentyere Council Land & Learning program and resources BHP relationships with communities I2S2 curriculum resources/procedures OUTPUTS Our deliverables ACTIVITIES What we do Recruitment of schools and engagement of key stakeholders Participation: CSIRO, BHPF SP Team Schools, Elders, Stakeholders Teachers, Stakeholders, Jurisdictions Universities, Jurisdictions Tangentyere, I2S2 Resources Schools, Stakeholders Principals, Teachers, Students Students, Elders, Family, Community Community, Schools, Students Impact Pathway Statement for Science Pathways for Indigenous Communities (Original) Strong effective partnerships established with schools and other stakeholders Assumptions • Local communities have rich cultural practices that can provide the content for STEM education. • The diversity of communities and jurisdictional differences mean that approaches will be customised to the specific circumstances of the communities. They include different histories and resources, such as: the existence of local Indigenous ranger groups; presence of nearby national parks, reserves and IPAs; and levels of involvement of Elders and other community members in teaching students about Country. • There are many intricacies to consider when engaging with communities and IEK. A substantial investment in partnership development is essential. A key aim of the Program is to identify localised pathways to support students in staying on Country. External factors • NT has a strong emphasis on the Indigenous Land and Culture Program. • WA has a strong focus on partnering with ranger groups and other organisations (e.g., Parks and Wildlife, Greening Australia, EON Foundation). • In some communities, literacy and numeracy are very low. Many communities are committed to Direct Instruction. • High mobility is a particular challenge in terms of transience of some students and staff turn-over. Revised Program Logic At the centre of the logic is the teaching and learning cycle, which is critical to the pedagogical approach taken by educators as part of the Program. They cycle should remain fixed as much as possible, as it provides structure for teachers and students. • The outer circle comprises several principles that can be used to guide the implementation and direction of the Program in any community. Principles act as a guide, providing direction, but not a detailed prescription, therefore offering opportunities to adapt to different contexts, changing goals and challenges (Patton, 2018). At a later date, principles could be evaluated for both the Program’s process (implementation) and results so that their hypothetical effectiveness can be tested. • The starting box highlights the Program’s inputs and how the delivery of the program is place-based and contributes to knowledge development in that place. • The outcomes box: Starting at the bottom is a list of identified outcomes, achievable from the short term to the longer term. • Science Pathways for Indigenous Communities revised program logic based on evaluation findings Appendix B – Science Pathways for Indigenous Communities Participating Schools The following table summarises information on the schools involved in the Science Pathways for Indigenous Communities program as at 2018 (Australian Curriculum, Assessment and Reporting Authority (ACARA), 2020). All schools were located in “very remote” areas according to the ABS remoteness classification. SCHOOL LOCATION(S) STUDENT ENROLMENTS (FTE) TEACHING STAFF (FTE) NON- TEACHING STAFF (FTE) SECTOR YEARS* ABORIGINAL AND/OR TORRES STRAIT ISLANDER STUDENTS (PER CENT) LANGUAGE OTHER THAN ENGLISH (PER CENT) Western Australia Parnngurr Community School Parnngurr 30.0 4.0 6.0 Non- government PP-10 97 n/a Rawa Community School Punmu and Kunawarritji 60.0 8.4 4.2 Non- government PP-12 90 100 Strelley Community School Warralong, Strelley 75.0 7.0 16.5 Non- government PP-12 100 100 Wiluna Remote Community School Wiluna 76.8 8.0 6.8 Government K-12 91 85 Leonora District High School Leonora 96.0 15.0 10.5 Government K-12 69 77 Tjuntuntjara Remote Community School Tjuntjuntjara 33.6 4.0 1.1 Government K-12 100 90 Northern Territory Watiyawanu (Mt Liebig) School Amunturrngu (Mt Liebig) 58.0 4.3 1.8 Government T-12 97 100 Haasts Bluff School Ikuntji (Haasts Bluff) 28.0 2.0 2.3 Government P-6 96 94 Areyonga School Utju (Areyonga) 29.0 4.1 2.0 Government P-9 100 100 Note: PP = Pre-primary; K = Kindergarten; T = Transition; P = Preparatory. *The “years” refers to the school years that the school can offer; some schools may not have students at every year level, particularly above Year 10. Appendix C – Interview Questions The questions outlined below provided a starting point for conversations with interview and focus group participants, as well as during yarning sessions. Close-ended questions were proceeded by prompts and follow-up questions seeking elaboration from participants. TEACHERS / PRINCIPALS FOCUS GROUP / INTERVIEW QUESTIONS How long have you taught at this school? How long have you been teaching at remote Aboriginal community schools? Student related questions Do the students find on Country activities engaging? Prompt: Is it relevant to their everyday lives? Do they find the on Country-related classroom content engaging? Does this contrast with other science curriculum? How do you measure student success? Do you provide personalised student support? (Hint: What does this look like?) Individual (teacher) related questions Are you confident in organising and delivering: Support for on Country Two-way Science? Developing class lessons pre and post on Country activities? What would help with this? Has the Science Pathways for Indigenous Communities support and TPD been important in assisting you with this? How important has the role of the program coordinator [name] been to you? School related questions Does the program integrate well with your school’s approach to curriculum and pedagogy? If no, why not? If yes, how? Does your school have a culture of high expectations? Does this apply to both students and teachers? Are you supported in your professional development to enact new programs and ideas such as Science Pathways for Indigenous Communities? Do you feel you belong to a community of practice with this (or other) programs? Family/community related questions Does your school have strong relationships with Indigenous families? Has the Science Pathways for Indigenous Communities program helped in building relationships with families? Other community members/ organisations such as ranger groups? STUDENT FOCUS GROUP QUESTIONS Student related questions What do you like about Learning on Country? What’s good about it? What’s not as good about it? Does Learning on Country help you learn about science? (Hint: what is science? ranger work, looking for tracks, etc) Is this different from what you’ve done before in school? How? What are some things about country you have learned from elders (on Country and during school time)? From assistant teachers? From rangers or other community partners? Does your teacher talk about Learning on Country in the classroom? (Hint: Can you show me something in your journal, word walls, classroom/learning walk) What work do you do when on a field/bush trip?' Do you have Martu [insert relevant language group] teaching you? Individual (teacher) related questions Do you ask for help when you don’t understand something? How do you feel asking for help? Does your teacher help you when you ask? (Hint: What does this look like?) Does your teacher relate science to your everyday life? School related questions Does your school expect you to do well? Do you learn about your local culture at school? Does your school teach about your culture in other subjects? How does this feel? Family/community related questions Is your school part of the community? (Hint do parents and families feel welcome? Do they get engaged?) Has Science Pathways helped in building relationships with families? Other community members/ organisations? Appendix D – Cultural Standards Mapping A Science Pathways for Indigenous Communities Two-way Science approach aligns directly with the Standards, Performance Descriptors and Indicators in the WA Department of Education Aboriginal Cultural Standards Framework (Department of Education, 2015). A description of how Two-way Science meets each Performance Descriptors and Indicators is provided in the table below. The case study evidence suggests that schools, with the help of the Program, are progressing from Cultural Awareness to Cultural Responsiveness on the Framework’s continuum. Although the Cultural Standards Framework were developed in WA, the principles apply to the entire Science Pathways for Indigenous Communities program and relevant communities. Standard: RELATIONSHIPS Culturally responsive schools foster positive participation, communication and interaction between staff, Aboriginal students, their parents and families and the local Aboriginal community. PERFORMANCE DESCRIPTOR INDICATOR MEETS Staff establish and maintain positive relationships with Aboriginal students, their parents and families. Staff build respectful working relationships with Aboriginal students. Students language and cultural knowledge is expressed and valued through a Two-way Science program. Students have opporutnities to lead Learning on Country. Staff provide information to parents of Aboriginal students about their children’s progress and support families to participate in all aspects of their children’s education. The Two-way Science teaching and learning cycle promotes sharing and reflection of student learning with the community. Parents participate as knowledge experts as teachers of language and cultural knowledge on Country and in the classroom. Staff work with Aboriginal students, their parents and families to support students at key transition stages of schooling. Teachers communicate regularly with local Aboriginal staff and their families in the development of a Two- way Science learning program. This can incorporate knowledge and sensitivity to cultural transitions in student life, as well as the interaction between students and their families who are working as rangers or in related roles in the community. Staff engage professionally with local Aboriginal community members and organisations. Staff, in partnership with the local Aboriginal community, understand and accept processes and protocols for respectful interaction. Two-way Science provides a model for respectful interaction through the planning of a learning program based on cultural knowledge. Staff draw on the expertise of Aboriginal staff (including Aboriginal and Islander education officers, and regional Aboriginal education teams) and local Aboriginal community members and organisations to enrich learning experiences for students. Two-way Science asks the community, including Indigenous school staff, Rangers and Elders to direct cultural and scientific learning and work with teachers to connect this to the Australian Curriculum in a learning program. Staff and students participate, as appropriate, in local Aboriginal community events and activities. Students join with real authentic learning projects with the local community in a multigenerational learning environment. Staff and local Aboriginal community members share experiences and knowledge to support student learning. Two-way Science asks the community, including Indigenous school staff, rangers and Elders, to direct cultural and scientific learning and work with teachers to connect this to the Australian Curriculum in a learning program. Staff provide Aboriginal students, their parents and families, and local Aboriginal community members with leadership opportunities. Staff provide Aboriginal students with opportunities to take on leadership roles at school. Aboriginal student cultural knowledge is valued in a Two-way Science program, leading to increased confidence and leadership opportunities. School leaders seek feedback from Aboriginal students, their parents and families on education delivery to inform decision-making. Two-way Science asks school and community to meet to plan and review the learning program with the school board and Two-way Science reference group. School leaders provide opportunities for local Aboriginal community representation on the school council/board. Two-way Science asks school and community to meet to plan and review the learning program with the school board and Two-way Science reference group. Staff broaden their knowledge and improve practices in Aboriginal education. Staff participate in local Aboriginal community networks to understand the perspectives of Aboriginal people on education. Staff work alongside students and community to use student cultural knowledge as a springboard for a culturally responsive learning program. Staff engage with local Aboriginal community members to identify opportunities for including cultural events at the school. A Two-way Science learning program is designed around cultural education and events. Staff incorporate local Aboriginal community members’ cultural and linguistic knowledge and expertise to build contextual relevance to the education of Aboriginal students. Local language and culture are the foundation of a Two-way Science learning program. Standard: LEADERSHIP Culturally responsive schools have leaders who develop and sustain an individual and school-wide focus on improving education outcomes for Aboriginal students PERFORMANCE DESCRIPTOR INDICATOR MEETS School leaders develop a clear vision for the teaching and learning of Aboriginal students. School leaders ensure school values are underpinned by high expectations for Aboriginal students. Two-way Science provides specific leadership training for a culturally responsive learning program. School leaders involve parents and families and local Aboriginal community members in the development of the school vision and ethos. Two-way Science asks schools leaders to ask parents, families and community members the question ‘what do you want your kids to learn about Country and culture?’ as the basis of the learning program. School leaders ensure Aboriginal education outcomes are defined in the school plan and reported in the annual school report. Two-way Science uses the ‘Leading by Design’ process to identify key outcomes for Aboriginal student learning. School leaders establish a culture of continuous improvement and academic optimism for Aboriginal students. Two-way Science provides simple effective assessment methodologies to track student learning and value cultural knowledge in achievement in science and other learning areas. School leaders build staff capability for effective teaching of Aboriginal students. School leaders expand their own cultural knowledge relevant to the local context. School leaders work closely with local elders to connect cultural learning goals to the learning program. School leaders support staff to build individual capability for developing their knowledge of local Aboriginal histories, peoples, cultures and languages. A Two-way Science program immerses staff in the culture of their communities. School leaders support staff to teach in ways that are responsive to the learning needs of individual Aboriginal students. A Two-way Science program provides a culturally responsive framework for remote community education. School leaders develop and sustain a school-wide focus on monitoring the progress of Aboriginal students. Two-way Science promotes a whole-school approach to teaching, learning and assessment. School leaders and staff are cognisant of their attitudes, behaviours and performance in teaching Aboriginal students. Two-way Science provides teachers with a clear idea of their students’ strengths and connects this to the Australian Curriculum, leading to opportunities for reflection and change in attitude as their cultural understanding grows. School leaders support innovation and change in Aboriginal education. School leaders encourage flexible and adaptive approaches to engaging and teaching Aboriginal students. Two-way Science and Learning on Country provide a diversity of opportunities for students to learn in ways that are culturally appropriate and cater for different strengths and learning styles. School leaders share best practice and innovation in Aboriginal education with colleagues and across education networks. Science Pathways shares knowledge between practitioners in a community of practice using digital and face-to-face methods and has an established community of practice and alumni. School leaders embed collaborative practices that promote school-wide contributions to improvement and innovation in Aboriginal education. Two-way Science promotes a whole-of-school approach to the development of a learning program, with regular opportunities for reflection as a staff group and with the community. School leaders recognise the capability of Aboriginal staff to lead innovative approaches in engaging and teaching Aboriginal students. Aboriginal staff lead the planning and facilitation of cultural learning in a Two-way Science learning program. School leaders demonstrate transparency and accountability to Aboriginal students, their parents and families. School leaders provide comprehensive reports about Aboriginal student performance as part of the annual school report. Science Pathways has developed assessment strategies and tools to track student learning. School leaders share education success with the local Aboriginal community and the broader community. The Two-way Science teaching and learning cycle includes a reflection and review process. All activities in education resources suggest ways of sharing the learning with community. Community and students work together on learning projects. School leaders report on the progress and outcomes of targeted Aboriginal programs and initiatives and seek feedback from the local Aboriginal community on their value and success. Two-way Science integrates all stages of the teaching and learning cycle with the community, including planning and review of cultural and curriculum learning goals with the community. Standard: TEACHING Culturally responsive schools have high expectations for Aboriginal students and teach in ways that enable them to better reach their full education potential. PERFORMANCE DESCRIPTOR INDICATOR MEETS Teachers know how culture and experiences shape the learning of each Aboriginal student. Teachers know of the cultural, language and family connections of Aboriginal students. Teachers learn about students’ story, culture, language and Country through Two-way Science program. Local Aboriginal knowledge holders teach cultural content to students and teachers. Teachers know relevant background experiences that Aboriginal students bring to school. Teachers connect with students and their families through on Country experiences and planning the program together. Teachers take into account the preferred learning style of each Aboriginal student. Two-way Science provides cultural content taught by elders alongside hands-on, authentic science projects that cater to a wide range of learning styles. Teachers support Aboriginal students to identify and achieve their aspirations. Two-way Science connects schools with community organisations, such as Aboriginal ranger programs, land management organisations and scientific researchers, providing opportunities for career development and work experience. Teachers know the curriculum content and how best to teach it to Aboriginal students. Teachers recognise the diverse perspectives that Aboriginal students bring to the content being taught. Two-way Science connects cultural knowledge to the Australian Curriculum, building on the strengths of students. Teachers incorporate into learning experiences the knowledge and experiences that Aboriginal students possess. Two-way Science allows students to lead teachers and demonstrate their knowledge of culture and Country. Teachers use resources developed for Aboriginal students to address specific learning needs. Science Pathways for Indigenous Communities two- way Science education resources create the space for local Aboriginal people to lead the cultural learning alongside authentic STEM education activities. Teachers incorporate Aboriginal histories, cultures and languages into learning activities. Two-way Science allows local culture, language and histories to be taught by elders in a school learning program. Teachers plan for and implement effective teaching practices for Aboriginal students. Teachers set learning goals that reflect high expectations for each Aboriginal student. Two-way Science builds on the engagement and student strengths on Country and connects this to a rigorous integrated learning program at school. Teachers seek background information about Aboriginal students from previous schools to improve success in transition. Two-way Science strengthens school knowledge and understanding of Aboriginal families and student aspirations and fosters connections to careers in local industries such as Aboriginal ranger programs. Teachers assess the learning needs of each Aboriginal student and differentiate their teaching practices accordingly. Student culture, language and family background are integrated into a Two-way Science learning program through strong family and community partnerships. Two-way Science provides tools to assess and track science learning. Teachers identify and implement teaching and learning approaches suitable for Aboriginal students as English as an additional language/dialect learners. Student language and culture are taught by local Aboriginal people in a Two-way Science program. This builds confidence and success for EAL/D students through connections to other learning areas. Teachers provide a range of learning opportunities for Aboriginal students. Two-way Science promotes an integrated program that connects to science, STEM, humanities and social sciences, art, technology, numeracy and literacy in authentic learning activities. PERFORMANCE DESCRIPTOR INDICATOR MEETS Teachers assess, provide feedback and report on the progress of Aboriginal students. Teachers take into account the cultural and linguistic backgrounds and proficiency of Aboriginal students when planning their assessments. Two-way Science adult learning program provides guidance on connecting cultural learning to the Australian Curriculum, including assessment strategies. Teachers use formative and summative assessment data to inform planning and to identify explicit achievement targets for Aboriginal students. Two-way Science provides a model for connecting student cultural learning to the Australian Curriculum and tracking achievement through formative and summative assessments in science. Teachers analyse the progress of each Aboriginal student and adjust their teaching practices as required. Two-way Science promotes a whole-of- school tracking of science learning using the Australian Curriculum’s scope and sequence. Teachers report the progress of Aboriginal students in formats appropriate to the cultural and linguistic backgrounds of students and their parents. Two-way Science connects cultural learning goals to science and other curriculum areas and communicates this to students and their families. Standard: LEARNING ENVIRONMENT Culturally responsive schools build an environment that is welcoming for Aboriginal students and reflects community aspirations for their children. PERFORMANCE DESCRIPTOR INDICATOR MEETS Staff support Aboriginal students to feel a sense of belonging and connection to the school. Aboriginal students demonstrate pride in, and ownership of, the school. A Two-way Science school celebrates local Aboriginal culture and language in the physical school environment through integrated science and art projects, such as maps, murals, animal and plant images and language. Staff have positive interactions with Aboriginal students. Two-way Science strengthens teachers’ relationships with students and their families through a deeper understanding of the local language and culture. Staff welcome parents and families of Aboriginal students at the school. A Two-way Science school welcomes family through a celebration of local language and culture and by engaging families as cultural educators on Country and in the classroom. All students and staff are aware of, and demonstrate respect for, the cultural and linguistic diversity of the school community. A Two-way Science school welcomes family through a celebration of local language and culture and by engaging families as cultural educators on Country and in the classroom. Staff promote positive wellbeing for Aboriginal students. Students’ language and culture are valued and respected in a Two-way Science learning program, laying the foundation for the rest of the learning program. Staff involve Aboriginal students, their parents and families to establish a physical environment that is welcoming for Aboriginal students. Staff engage Aboriginal students, their parents and families, and community members in developing an environment which displays and respects their histories, cultures and languages. The local community is engaged in all aspects of a Two-way Science learning program, including leading the Learning on Country and participating as learners with students in science and other learning activities. This includes creating artworks at the school that share and celebrate the local culture. The parents and families of Aboriginal students and local Aboriginal community members use facilities at the school where appropriate. The local community is welcomed at the school as a participant in the development and implementation of a Two-way Science learning program, joins in meetings and assemblies and leads education activities. Staff and students use local Aboriginal community facilities and sites to enable students to learn in settings connected to local histories, cultures and languages where appropriate. Two-way Science connects students with local elders, Aboriginal rangers and the local Aboriginal organisation in education activities. PERFORMANCE DESCRIPTOR INDICATOR MEETS Staff work with Aboriginal students, their parents and families to establish shared expectations and responsibility for attendance and behaviour. Staff work with Aboriginal students, their parents and families to ensure an understanding of expectations and legal obligations under the School Education Act (1999) for attendance. Two-way Science improves attendance outcomes for students through an engaging learning program inside and outside the school grounds. Staff work with Aboriginal students, their parents and families to ensure an understanding of expectations related to behaviour, engagement and learning. Teachers, AIEOs and students work with the local community to identify expectations of behaviour during a Two-way Science program. An engaging learning program reduces behavioural issues at school. Staff support and encourage local Aboriginal community members to promote consistent and clear messages about student attendance and behaviour. Two-way Science fosters school and community partnerships in the education of students, leading to improved communication about all school matters. Parents reinforce school attendance and behaviour messages when they can see that the school values the local language and culture of their children. Staff identify and implement strategies to engage hard-to-reach Aboriginal students, their parents and families. Parental engagement with the school increases when the local language and culture are valued. Staff are aware of factors that may be affecting the behaviour of individual Aboriginal students and implement appropriate school and classroom management strategies. Learning on Country and engaging in hands-on education activities reduce behaviour management issues for some students. School leaders collaborate with other school leaders to ensure the transition of Aboriginal students is well- defined and enacted. Two-way Science fosters communication and collaboration between schools through a community of practice and opportunities to share experiences, challenges and successes. Staff establish a supportive and safe learning environment for Aboriginal students. Staff know about factors in the local Aboriginal community that may have an impact on student engagement. Two-way Science improves school and community communication, leading to increased understanding of potential impacts on student engagement. Staff work productively with other agencies to support the health, wellbeing and safety of Aboriginal students, their parents and families. Two-way Science connects schools with local ranger groups, Aboriginal corporations and youth agencies, leading to coordination and collaboration with students and their families. Staff, in collaboration with parents and families, develop plans to support Aboriginal students at educational risk. School and community communication and partnerships are strengthened through a Two-way Science program. Staff put in place appropriate steps to support the social and emotional wellbeing of Aboriginal students. Students’ language and culture are valued in a Two- way Science program, leading to improved social and emotional wellbeing. Standard: RESOURCES Culturally responsive schools target resourcing to optimise the education outcomes for Aboriginal schools. PERFORMANCE DESCRIPTOR INDICATOR MEETS Staff acknowledge and value the expertise of Aboriginal staff. Staff draw on the knowledge and expertise of Aboriginal staff (including Aboriginal and Islander education officers, and regional Aboriginal education teams) in connecting the school with the local Aboriginal community. Aboriginal staff lead cultural learning or facilitate community connections to identify cultural knowledge experts who can teach students on Country and in the classroom. School leaders support Aboriginal staff to identify and pursue leadership and aspirational opportunities. Two-way Science fosters Aboriginal leadership of the education program. School leaders and staff use flexible recruitment practices to encourage applications from Aboriginal people. Two-way Science improves community engagement with the school, leading to career opportunities for local people. School leaders allocate staff to support the learning needs of individual Aboriginal students. School leaders, in planning the staffing profile, give specific consideration to the learning needs of Aboriginal students. School leaders foster local staff and community leadership of a Two-way Science learning program tailored specifically for the needs of the students in the community. School leaders consider the expertise and experiences required for staff to work with Aboriginal students. Two-way Science provide continuous opportunities for school staff to build cultural competence through direct engagement with community and culture. School leaders target the learning needs of individual Aboriginal students when allocating financial resources. School leaders use student characteristics funding to implement appropriate teaching and learning adjustments for Aboriginal students. School leaders resource school staff to plan and direct a Two-way Science program based on the language and culture of local students. School leaders plan for improved Aboriginal student outcomes and budget accordingly. Two-way Science supports schools to structure staffing around the language and culture of local students, as well as the opportunities for community partnerships. Staff use culturally appropriate education resources to strengthen Aboriginal student engagement and learning. Staff select a range of evidence-informed resources to support the learning needs of individual Aboriginal students. Science Pathways for Indigenous Communities Two-way Science education resources create a space for local Aboriginal leadership and cultural content to connect to the Australian Curriculum and an integrated learning program. Staff use technology to connect Aboriginal students’ local perspectives with national and global perspectives. Students build on their local knowledge to connect to national and global perspectives. Staff incorporate the Aboriginal and Torres Strait Islander histories and cultures cross-curriculum priority in all learning areas. A Two-way Science program connects to all elements of Aboriginal and Torres Strait Islander Histories and Cultures cross-curriculum priorities. Science Pathways for Indigenous Communities has developed four ’Illustrations of Practice’ Two-way Science videos that are hosted on the ACARA Aboriginal and Torres Strait Islander Histories and Cultures website. Staff liaise with Aboriginal staff, including Aboriginal and Islander education officers, and Aboriginal community members to determine the appropriateness of cultural resources and materials. Aboriginal staff and community are included in the planning and delivery of a Two-way Science learning program. Staff understand, respect and act in accordance with the cultural and intellectual property rights of Aboriginal people. Science Pathways for Indigenous Communities provides guidance to schools around Aboriginal Intellectual Property rights during the adult learning program. Appendix E – Strengths-Based Practice Considerations The following table summarises key strengths-based steps towards improved future Two-way Science practice in remote education for different sectors. SECTOR CONSIDERATIONS Policy and program design • Value remote education programs that include success factors such as: facilitation of genuine relationship development between stakeholders; identify and build on existing community strengths; target resources and interventions to achieve community education goals, include immersive and interactive cultural learning led by community leaders, integrating local languages and cultural knowledges into learning. • Funding that allows for: time spent developing genuine partnerships and undertaking program monitoring and evaluation. • Improve career pathway support for local Aboriginal students in remote areas to become teachers and assistant teachers. • Promote the Two-way Science pedagogy and resources to pre-service teachers , as well as teachers and principals on a national level. • Support and resource remote schools and their educators to undertake regular and consistent Two-way Science, including on Country learning and in the classroom. • Support and resource Aboriginal roles in communities and schools to guide two-way learning activities and share Aboriginal ecological knowledge with schools. • Better utilisation of existing Indigenous research and practice evidence to improve education outcomes in remote communities. Program Implementation • Program staff with strong community development and interpersonal skills and the knowledge and skills to facilitate Two-way Science inquiries with multiple stakeholders in an education setting. • Develop unique school and community plans with indicators of success that reflect each community and school’s education goals. Plans are visible and actively embedded in school practices. • School community-specific program monitoring to measure progress against student, school and community goals and contribute to a useful evaluation. • Place-based learning that supports students’ contribution to local social change. • Increase student involvement in Two-way Science planning and identify opportunities for students to develop and lead projects related to their learning activities and outcomes. • Where families from different Aboriginal language groups contribute to one school community, respectfully support discussions to encourage all families to play a role in Two-way Science. Schools • Recognise the importance of soft skills and attitude considerations for non-Indigenous educators in remote education, including: a “give it a go” attitude, valuing connections with the local community, being flexible and adaptable when teaching and possessing a growth mindset in relation to their cultural capacity. • Identify opportunities for educator development (preferably, in first languages) to share teaching experiences, approaches and resources. • Provide structure and ongoing support for Aboriginal and non-Indigenous teachers to consistently include Aboriginal ecological knowledge holders in student learning. • Create opportunities for community leaders and others to contribute to student learning through leadership, knowledge-sharing, being present during learning, and other ways within the school. • Support local staff to gain formalised cultural, leadership, mentoring and teaching roles within the school. • Identify a range of assessment approaches derived from Aboriginal pedagogies that can be used to effectively capture student progress, especially for students that speak EAL/D. Aligned partners • Identify existing goals and performance indicators that align with Two-way Science approaches and connect with school communities over these. • Prioritise time spent developing effective partnerships with schools and other stakeholders • Recognise that forming genuine partnerships takes time and commitment, which is often in addition to planned activities. Remote community governance groups • Share education goals and aspirations to support schools and others to contribute to student learning experiences. • Share the education commitment with a broad range of community members and identify emerging leaders to support Elders in sharing knowledge within schools. • Develop agreements with organisations to protect Aboriginal knowledges and uphold others’ obligations of respectful conduct. • Identify opportunities for those who collaborate with communities to contribute to community goals. For further information Business Unit/Area Name Jonathan Bates t +61 3 9123 4567 e jonathan.bates@csiro.au w www.csiro.au/loremipsum Business Unit/Area Name Bianca Chen t +61 3 9234 5678 e bianca.chen@csiro.au w www.csiro.au/loborero Business Unit/Area Name Alexander Milovich t +61 3 9345 6789 e alexander.milovich@csiro.au w www.csiro.au/quisieriure Business Unit/Area Name Takahiro Kojima t +61 3 9456 7890 e takahiro.kojima@csiro.au w www.csiro.au/blaoreet Contact us t 1300 363 400 +61 3 9545 2176 e csiroenquiries@csiro.au w www.csiro.au we do the extraordinary every day We innovate for tomorrow and help improve today – for our customers, all Australians and the world. Our innovations contribute billions of dollars to the Australian economy every year. As the largest patent holder in the nation, our vast wealth of intellectual property has led to more than 150 spin-off companies. With more than 5,000 experts and a burning desire to get things done, we are Australia’s catalyst for innovation. We imagine We collaborate We innovate