Background

More and more legal aid offices and support centers are learning new ways to look at what they do and how it works. They are taking a good, hard look at how they can improve their programs and services to get the best results for clients and client communities. They are finding new and better ways to ask "How well did we do?" and "Could we do better?," and then to communicate about the value of the work they do.

In California, there is an extra incentive to increase the evaluation of legal services programs. Along with a $10 million annual line in the state budget for free legal assistance to low-income individuals and communities comes a requirement for a report to the state Legislature on the "efficiency and effectiveness" of funded projects.

To help prepare the report, the Administrative Office of the Courts, the Legal Services Trust Fund Program, and the Legal Aid Association of California are inviting grant recipients to participate in an ambitious program-owned evaluation of the projects funded with Equal Access Fund grants. This project builds on existing evaluation work already ongoing within legal services programs, but creates an ongoing statewide evaluation process that is more formal and structured.

The Evaluation Tool

The first step is to think about the planning that went into creating the project to be evaluated. What need was identified, what were the strategies for addressing it, and what results are being sought? The underpinning of a good evaluation is a clear sense of the goals and objectives of the project. The next question is, "How will we know whether we are achieving the desired results?" And then, "How do we gather information that will help us answer that question?"

The tools contained on this webpage demonstrate ways that legal service programs have collected evaluation information.

Each tool has three main elements:

1. A two-page "Summary" explains what the methods measure and how, and lists legal services programs that have already used it.
2. "Guidelines" for each suggest potential uses, share some cautions, and offer a variety of helpful hints.
3. Sample materials from other programs, including the "Instruments" used to gather evaluation data, as well as examples from evaluation reports. Please note that there is a range of materials, and samples include both first efforts and very sophisticated examples. Use these forms in combination, or customize the "Tool" to meet the needs of your program.

Evaluation Tools

• Client/Customer Satisfaction Survey
• Client/Customer Follow-Up Interview
• Community Partner Survey
• In-Court Observation Process
• Court Case File Review
• Client Outcomes Captured at Case Closure
• Focus Groups

Three sections that follow the seven tools address special evaluation challenges:

• Evaluating Support & Impact Projects
• Evaluating Technology Projects
• Partnership Grants Evaluation Protocol

Examples of comprehensive evaluation reports

• SHAC: The First Six Months. The Self Help Access Center is Filling a Critical Niche in the Sonoma County Justice System
• HCA Evaluation Six-Month Progress Report IV, The Health Consumer Alliance and the Health Rights Hotline
• Lifting Welfare Employment Sanctions in the CalWORKS Program: The Experience of the Family Advocacy and Services Team (FAST)

Other useful materials

• A Summary of Ethical Guidelines
• Making Self-Evaluation Part of Program Management
• Case Service Reports: How to Use Them and How Not To
• Making "Measurable Outcomes" Work
• GIS Tutorial (Legal Services of Northern California)

  Champions and Other Resources

  • Click above for a list of people who can provide further help, and information about how to contact them. The "champions" whose examples are in the toolkits have agreed to provide advice and technical assistance to others. Research staff from the Center for Families, Children & the Courts of Administrative Office of the Courts can help in many ways, including advice about research techniques and practical help getting started. The staff of the Legal Aid Association of California will connect programs with others who can help them, and will coordinate a schedule of "working groups" to talk about successes and share challenges. The list also includes web-base resources, both general advice and directions for conducting effective research and specific suggestions about evaluating legal services.

  Authors
  

  Good luck with your evaluations. We hope this toolkit will help strengthen your program and help you continually improve your critically important services to clients. Comments and other feedback about the tools, and suggestions for their improvement, will be much appreciated.