Outcome-Based Evaluation
The design, management and implementation of a wide range of evaluation research studies. My approach is based on the principles of empowerment evaluation, a process where evaluation strategies are developed in collaboration with those individuals who are invested in the data.
Client: WorkForce Central – Tacoma and Pierce County
This project involved working with three Workforce Development Councils in Washington State and Building Changes in evaluating the impact of the Housing and Employment Navigator program, an innovative approach helping homeless families progress in their career development and employment goals. The project is supported by the Department of Labor Workforce Innovation Fund, and the research involved random assignment of eligible homeless families in four counties to the Navigator and Control group conditions. Data from client surveys, DSHS and Employment Security and interviews with clients and other stakeholders was used to assess long term differences in employment, housing and public assistance outcomes. The final report and executive summary identified important impacts of the model and lessons for future program implementation. More information about the project is available at the WorkforceNavigator website
Client: Washington Service Corps
From 2018-2020 I worked in collaboration with Bee’s Knees Consulting on the evaluation of a capacity building intervention implemented by the Washington Service Corps (WSC) with its AmeriCorps member sites. We employed a random assignment comparison group methodology to help assess the impact of training and technical assistance in performance measurement and evaluation for these sites on participant knowledge of, attitudes about and capacity to utilize evaluation tasks and concepts. The final report included quantitative data from pre and post program surveys along with interviews with participants and observations of the intervention activities to help show the efficacy of the intervention. The organization has used to results to hone their capacity building strategies and demonstrate organizational impacts as required for their CNCS evaluation.
Client: Puget Sound Partnership – Stewardship Program Evaluation
Working in conjunction with ORS Impact, I managed the comprehensive evaluation of the Puget Sound Partnership Stewardship Program. In this project we worked with staff and stakeholders on the development a theory of change model and overall evaluation plan, on the development of data collection tools, and on the provision of evaluation technical assistance to program grantees implementing local environmental social marketing efforts. The data for the report was gathered through program and stakeholder surveys and from local program evaluation summary reports. The report highlighted the important role the program played in building the capacity of local organizations that influence the stewardship behaviors of residents across Puget Sound.
Outcome-Based Strategic Planning
Using the tools of outcome-based evaluation to help organizations and foundations with strategic planning efforts. This work involves collaboration with clients to develop theory models and evaluation frameworks showing the connections between the programs, their organizational strategies, and the potential impacts on both.
Client: King County Conservation District
I worked with the King Conservation District on the development of a theory of change model to support their strategic planning efforts. This work has involved engaging staff and advisory committee members in the development of models for six program areas working with urban and rural residents and landowners, and reviewing the current data being collected by the organization. The organization has used the model to help shape 2020 to 2025 strategic action plan and in the formulation of an evaluation plan to capture ongoing data on program implementation and impacts.
Client: Facing Homelessness
I am currently working with Facing Homelessness in the development of a theory of change model to help the organization better understand how its community focused and client service activities are contributing to enhanced community awareness and understanding of issues of homelessness in Seattle. The process has involved working with strategy area teams in building outcome pathways that reflect the organization’s intended impacts and creating a comprehensive model that sets the framework for ongoing service delivery and program evaluation. A next phase of the work will involve identification of priority outcomes and creating an evaluation plan that supports the organization’s ability to gather implementation and impact data.
Client: Alaska Conservation Foundation
I worked with the Alaska Conservation Foundation staff and board to develop a theory of change model to support the Foundation’s decisions about grantmaking practices. This model drew from strategic planning efforts initiated by the Foundation to articulate the connections between their core strategies of building conservation capacity and leadership, anticipated short term organizational changes, and longer term impacts on the conservation movement in Alaska. The foundation has used this model to make more strategic decisions about their grantmaking and program planning.
Evaluation Coaching and Technical Assistance
Working one-on-one with non-profit organizations, public sector agencies, and foundations to address specific needs and build local capacity for evaluation. I work with clients to help build skills in the various facets of the evaluation process, from formative theory development to data collection, analysis, and reporting.
Client: City Year – Seattle-King County
I worked with City Year to develop a comprehensive evaluation plan to assess impacts of local services for “at-risk” middle school youth. In this project, we have developed a theory of change model identifying the connections between program strategies and the intended outcomes to align with the City Year national Whole School Whole Child initiative. This work focused on the development of instruments to collect data from youth and Corps members, as well as development of a system to access school-based performance and behavior data.
Client: Northwest Indian Treatment Center
For the past eight years I have worked with the Northwest Indian Treatment Center (managed by the Squaxin Island Tribe) on their ongoing evaluation of SAMHSA and Indian Health Service funded treatment programs. These programs have focused on the provision of inpatient alcohol and drug treatment services and aftercare recovery coaching support services. Our evaluation work has included the development of new tools to gather data from clients, staff and recovery coaches, the development of program database for tracking, managing and analysis of the data, and the completion of summary reports. We have further captured data regarding program implementation through client interviews and observations of agency sponsored Recovery Coach Academy training sessions.
Client: South Puget Intertribal Planning Agency (SPIPA)
I am currently providing technical assistance to the SPIPA Healthy Families program staff working on a home visitation grant funded through the Administration of Children and Families in six tribal and urban communities. The work has involved assistance and training on data collection, data management and data analysis required for grant and parenting model reporting. In the work we have developed systems to help staff monitor when and how to collect required assessments and screenings and provision of guidance on strategies for how to keep up with data reporting requirements in the Parents as Teachers Penelope system.
Analytic Database Systems
The planning and development of systems to support organization’s needs for capturing, maintaining, monitoring and reporting data. In this work we have developed online systems that help the program/organization capture data on activities and outcomes at the client and/or program level and have customized reports to support program planning, development and evaluation efforts.
Client: Port Gamble S’Klallam Tribe: Tribal Database
The Port Gamble S’Klallam Tribe needed a better way to capture and manage data on different outcomes and performance measures across various programs and departments. In collaboration with Geo-Light, we worked with programs to identify appropriate performance measures and in the development of a user-friendly web based system where programs could enter quarterly data into a common platform. The system was expanded to allow programs that did not have local data systems to use this platform to manage data on individual clients. From there we designed several reports that programs use to monitor the measures and summarize key client outcomes. The system has been in use for two years and provides more consistent data for grant reporting, program planning and for decision making through the Tribal Council.
Customized Evaluation Training Workshops
Development of evaluation training workshops to help organizations, foundations, and practitioners build their own evaluation capacity. I have developed training workshops on topics, ranging from an overview of outcome-based evaluation to more specific topic areas such as logic models and the theory of change model development, data collection tool development, and using Microsoft Access and Microsoft Excel for data analysis and reporting. I also conduct customized “Train the Trainer” workshops with small groups that are targeted to your specific evaluation and data collection topics.
Client: State of Alaska Department of Health and Social Services
In collaboration with the Foraker Group in Alaska, I facilitated a series of evaluation workshops for non-profit organizations funded through the Department of Health and Social Services. These workshops included presentations on introductory evaluation principles, the use and development of logic models, and the identification of program outcomes. Many of the participants included staff working with local Native Alaskan communities and so the workshops addressed some of the unique aspects of working with, and collecting evaluation data from, these diverse populations. I also conducted more specialized “train-the-trainer” workshops with a smaller group of local non-profit professionals that were targeted on building their technical assistance capacity to work directly with local agencies and communities.
Needs Assessment and Gaps Analysis
The use of quantitative and qualitative data collection approaches to gather data about community and participant needs, and to assess potential gaps in community services. This work often includes helping clients align the identified needs and services gaps with the identification of evidence based practices and strategies to use with the participant population.
Client: Port Gamble S’Klallam Tribe: Tribal Home Visitation Program
I worked with the Child and Family Services Department for the Port Gamble S’Klallam Tribe on the completion of a needs assessment to determine local practices for home visitation services. Drawing on a grant from the Administration of Children and Families we gathered quantitative and qualitative data from local providers, community elders, expectant families and families with children 0-5 on current needs, challenges, and interest in home visiting services. We coupled this information with data from Tribal Health and Wellness centers and from the Kitsap County Health Department to help identify key community needs, and select an evidence-based program to implement over the next four years. Subsequent phases of the project will involve a comprehensive evaluation of the implementation and impact of the services in the community.
Client: City of Tacoma Human Services: Domestic Violence Services
I worked with the City of Tacoma Human Rights and Human Services department to complete an assessment of the needs, services and gaps in the provision of services to individuals affected by Domestic Violence in Tacoma and Pierce County. The objectives of this assessment were to gather information from local service providers and other key community stakeholders to better understand the needs of the client and victim population, to establish the range of direct and indirect services currently being provided, to identify potential shortcomings and gaps in the provision of services to the client population, and to present any opportunities for building the local capacity to serve this population. The subsequent report informs the City’s strategic planning and funding decisions in support of service provision for the affected population.