What’s the Social Impact of your Programmes?
Developing outreach programmes that engage the community is critical – only if it meets community needs and organisational goals. However, setting clear objectives and success indicators for evaluation is challenging. Limited knowledge on which measurement tools to use is an issue. Developing transparent reports for stakeholders and justifying to management is another hurdle to cross.
Acquire Techniques & Tools to Evaluate your Community Engagement Efforts
Led by Matt Healey, a social research and evaluation practicing consultant from Australia, join this 1-day practical workshop to learn the different forms and approaches in community engagement measurement – formative, summative, process and outcomes. Discover how to set objectives for evaluation based on your programme goals. Find out how to identify indicators to determine programme success. Acquire a data collection and analysis framework including quantitative, qualitative and mixed methods. Examine how to develop effective reports for different stakeholders.
Gain Answers to Questions asked of Engagement:
- Was the engagement approach the right one?
- Were the right people engaged sufficiently?
- Did engaging people result in the right outcomes?
- As a result, did the programme achieve its goals?
Practical Learning and Hands-on Exercises:
- Build a measurement framework for your organisation
- Develop a programme logic to outline programme outcomes and goals
- Stakeholder mapping and information needs template to set evaluation objectives
- Drafting evaluation questions and indicators for programme outcomes
- Take away data collection templates (e.g. surveys, interview guides)
- Receive a guide on ‘rapid data collection for events’
- Consolidate your evaluation plan and build your reporting framework
Benefits of Attending
- Identify what your programmes are seeking to achieve
- Gain insight into the resources available for evaluation
- Determine what ‘success’ looks like to stakeholders
- Review evaluation questions, including programme sustainability
- Classify data sources and role in the evaluation
- Examine different types of data collection tools and approaches
- Assess strengths and weaknesses of different data types
- Learn how to best present data to various stakeholders
- Outline roles and responsibilities for implementation and reporting
- Analyse formats for reporting depending on different stakeholders
- Translate findings into lessons for future programmes
Matt comes from a diverse background, having worked in consulting, the not-for-profit sector and higher and international education. Co-founder of First Person Consulting, a social research and evaluation consultancy established in early 2015, he has provided services to all types of organisations, including community organisations, large government agencies and international not-for-profits. He is recognised for his adaptable and valued services which support his clients in delivering projects and programmes to improve social and environmental outcomes.
Engaging the community is a key element of many of the projects and programmes that Matt has evaluated and these projects cover a range of content areas, including health promotion, mental health, education, financial literacy, sustainability and climate change.
Recent evaluation experience includes:
• Evaluation of the Ride or Walk to School Programme, ACT Health. A programme focused on increasing rates of active travel (riding or walking) among primary school aged children to school.
• World Suicide Prevention Day Awareness Campaign Analysis, Movember Foundation, an online awareness campaign to prevent male suicide.
• Evaluation of the Local Prevention Programme, Victorian Responsible Gambling Foundation. This was a grant programme which engaged a range of at-risk communities and populations in gambling harm reduction and prevention projects.
• You’re the Boss Financial Literacy Programme Evaluation, The Salvation Army. A programme which engaged with community members from low socio-economic backgrounds to build their skills in financial literacy.
• Democracy in Geelong project evaluation, Local Government Victoria. A project engaging citizens of Geelong City Council in a citizen’s jury to inform the restructuring of the government leadership.
• Co-Design Learning Project, Australian Red Cross. A co-design programme to design three new services for indigenous and other groups that access Red Cross services.
Matt is Secretary of the Australasian Evaluation Society (AES) Special Interest Group (SIG) on Human Centred Design and Evaluation. He also presents regularly at the annual AES conference on a range of topics suitable for attendees of all types. Matt is on the convening committee for the 2018 annual conference for the AES.
Who Should Attend
Senior level executives responsible for Community Engagement, Public Outreach, Programme Development
Session 1: What are we evaluating? Understanding your programme
Evaluations are about understanding the outcomes and achievements of programmes and learning how those programmes can be improved into the future. As such, the first step is identifying what the programmes are hoping to achieve.
- What is the problem or need being addressed by the programme?
- What are the outcomes and goals of the programme?
- Who in the community will be engaged through the programme?
- What resources are available for the programme and the evaluation?
Activity: Develop a programme logic to describe the programme activities, outcomes and goals
Session 2: Why are we evaluating? Understand your audience and define objectives
Once you understand your programme with a programme logic it is then important to set objectives for the evaluation. These can relate to a range of priorities, with one of the most common being the ‘audience’ – usually sponsors / funders, the community and other organisations. Having clear objectives and understanding the needs of your audience allow the evaluation to directly inform future decision making, as well as inform how the results will be reported.
- Who are the key stakeholders in the programme?
- What do they need to know, and when?
- What does ‘success’ look like to these stakeholders?
Activity: Stakeholder mapping and information needs template; develop evaluation objectives
Session 3: What do we want to find out? Focusing the measurement effort
With a programme logic, an understanding of stakeholders and objectives, the next step is to focus our measurement efforts. This stage helps to identify the specific areas of interest for our measurement efforts, as well as the indicators that will determine if our programme is successful.
- Reviewing domains of evaluation questions, including programme sustainability
- Developing different types of evaluation and measurement questions for the case study
- Identifying indicators for outcomes and programme success
Activity: Drafting evaluation questions and indicators for programme outcomes
Session 4: It’s all about the data
The next stage in the evaluation planning phase requires us to think about data. Community engagement programmes produce a range of data, and understanding what data is available and where it will come from is important to effective evaluation and measurement. Another consideration is prioritising the types of data you will need to avoid burdening community members, as well as the various approaches to data analysis and data visualisation.
- Identify data sources and role in the evaluation
- Review different types of data collection tools and approaches, such as surveys (in-person and online), interviews, vox pops, rapid data collection methods and observations.
- Strengths and weaknesses of different data types, including quantitative, qualitative and mixed methods
- Data analysis and data visualisation, including how to best present data to a variety of audiences (including internal and external stakeholders)
Activity: Completion of data collection and analysis framework
Session 5: Implementation and reporting
Once the evaluation and measurement plan has been completed, the last step is to outline roles and responsibilities related to implementation. This includes the responsibilities for collecting and analysing data, governance and reporting. The task of reporting is often the most difficult, but starting with a plan for how the report will be written, and who it is for, streamlines this process.
- Identifying roles and responsibilities for different team members
- Formats for reporting depending on different stakeholders
- Planning for sharing results (internal and external publications)
- Learning from evaluation, particularly in identifying the lessons for future programmes
Activity: Consolidation of the evaluation plan; reporting framework