Evaluation
Working in the Schools (WITS) has a long standing commitment to conduct comprehensive and well-crafted annual evaluations of its reading programs. In particular, Power Lunch and Workplace Mentoring are the most extensively and consistently evaluated programs.
The evaluations of the Power Lunch and Workplace Mentoring programs are comprehensive and seek to compile quantitative and qualitative data information from students, teachers, and mentors. Based on a six-step outcome framework, these evaluations involve pre-tests and post-tests. Comparing student groups that participate in our programs (treatment) with student groups that do not participate in our programs (control), information for the survey is compiled throughout the year from teachers, students, reading mentors and Illinois Standards Achievement Test (ISAT) scores. Pre-Test measures are usually taken within the first month of each school year (Fall) and Post-Test measures are taken during the last month of each school year (Spring). All instruments included in the evaluations are already developed and have been tested and checked against other known measures of performance to insure their validity.
Teachers are expected to complete two different types of surveys during the Pre-Test and Post-Test stages of the evaluation. The first survey will ask for background information (e.g., number of days they read aloud to their students), ratings of students' reactions to Power Lunch, personal satisfaction with the program, feedback on the benefits of the program and suggestions to improve it. The second survey asks teachers to provide reading and academic information about each Power Lunch student (e.g., report grade card for reading, reading grade level, reading fluency). Students' reading motivation is also obtained based on two types of intrinsic motivation: topic and individual.
Mentors also complete a short survey with open and close-ended items during the Post-Test stage of the evaluation. The survey is administered primarily in an online format through an external service site linked to WITS' website. For mentors without access to the Internet, the survey is also presented as a paper questionnaire. Mentors provide background information about themselves such as workplace, ethnicity/race, gender and age, as well as their student's name (used only for case matching purposes). The rest of questions assess perceived reading aspects about their assigned students (e.g., reading confidence, reading fluency) and a series of process-related aspects such as satisfaction (e.g., transportation, working environment), program attendance. In the last section of the survey, mentors are encouraged to share personal stories about their experience and any suggestions that could help WITS staff improve the quality of the Power Lunch for both students and teachers. All comments and suggestions offered during the Pre-Test stages are shared and discussed among WITS staff members in order to address any issues that may affect the effectiveness of the program or the joy of volunteering as a mentor.
A major report containing the results and findings of each evaluation will be produced annually, at the beginning of the fall. WITS will be in the capacity to produce smaller or individualized reports if they are requested. As usual, all evaluations reports are uploaded to WITS' website in order to disseminate and make public the findings, regardless of their positive or negative nature.
For specific results, please see 2006-2007 and 2007-2008 results posted on our website.
