TeachTown Basics Evidence of Effectiveness
Systemwide Data
To download a copy of TeachTown Evidence of Effectiveness report, please click here.

Background

During the 2011-2012 and 2012-2013 school years, 355 school districts in 41 states and 4 provinces across the United States and Canada, respectively, used TeachTown: Basics with 8,991 students who had disabilities or who were identified as at-risk for school failure in preschool through grade 8. This evaluation examined student usage and progress on the computer-assisted instruction portion of the curriculum during the implementation of TeachTown: Basics in districts using the curriculum’s online reporting system. Results are reported for students who were at least 2 years old and who had used the program for at least eight months, which included 2,018 students who had used the program anywhere between 8 to 22 months.
In addition to usage and progress data, the data set contained students’ chronological age and gender. The mean age was 6.7 years old (range: 2.2 to 20.9 years old) with a standard deviation of 2.8. Approximately 76 percent of the sample was male.

This evaluation addressed the following questions:
(1) How much do students use TeachTown: Basics and does usage change over time?
Figure 1 shows the average total amount of time students spent on the computer-based lessons over monthly intervals. Across all age subgroups, program usage peaked in month two and then stabilized across months four through six.1 Average daily usage in months three through eight ranged from 7 to 13 minutes; the minimum recommended usage is 15 minutes daily.
Figure 1. Total minutes spent on computer-based program over eight months of use disaggregated by students’ chronological age.
(2) How much progress did students make over eight months of program use?

Approximately 90 percent of students who used TeachTown: Basics (or 9 out of 10 students) showed growth in the curriculum as defined by mastering anywhere between 5 and 206 lessons. Figure 2 shows the average number of lessons mastered for students broken out by the amount of time spent on the program.2 The findings indicated that after investing 14 to 65 minutes daily on the program over eight months, students mastered 27 lessons which represent an average of 81 concepts or skills learned.3 After investing 5 to 8 minutes daily on the program over eight months, students mastered 10 lessons which represent an average of 30 concepts or skills. After investing less than 5 minutes daily on the program over eight months, students mastered 5 lessons which represent on average 15 concepts or skills.
Figure 1. Total minutes spent on computer-based program over eight months of use disaggregated by students’ chronological age.
Figure 2. Cumulative number of lessons mastered over eight months of use disaggregated by amount of program use.
Figure 2. Cumulative number of lessons mastered over eight months of use disaggregated by amount of program use.
2The standard deviation for cumulative lessons mastered was high across months one through eight (range: 3.8 to 13.3 lessons mastered), indicating great variability among students in the number of lessons mastered.

3Each lesson in TeachTown: Basics covers anywhere between 1 and 4 concepts with the majority covering 4 concepts.


(3) How many training exercises do students complete before passing a lesson’s posttest and does the number of exercises to mastery vary across learning domains?
Figure 3 displays the gain in percent correct from pretest to posttest for mastered lessons and the average number of training exercises taken to master the lessons. Each lesson begins with a pretest. If the student scores less than 80% on the pretest, he or she proceeds to the training exercise. Once the student answers 80% or more of the questions on the training exercises correctly the student continues on to the next exercise or posttest. The student must answer 80% or more of the questions correctly on the posttest in order to master a lesson. In every learning domain, students showed statistically significant gain in percent correct from pretest to posttest. The average number of exercises taken to master a lesson varied across learning domains ranging from 11 exercises in Language Arts to 19 exercises in Adaptive Skills and Social and Emotional.4
Figure 3. Gain in average percent correct on pretests and posttest and average number of training exercises completed for mastered lessons disaggregated by learning domain.
Conclusion
The purpose of this evaluation was to examine program usage and progress data of students with disabilities or who were identified as at-risk for school failure in preschool through middle school after receiving eight months of instruction in TeachTown: Basics. The results indicated that although average usage was lower than the minimum amount of usage recommended by the publisher, usage was consistent over time.Further, students who spent more time on the program mastered substantially more lessons and material than students who spent little time on the program. More specifically, students who spent a median of 18 minutes daily on the program—which is just over the minimum amount of time recommended by the publisher, mastered 27 lessons representing on average 81 concepts or skills over eight months. By
contrast students who spent a median of 4 minutes daily on the program mastered five lessons representing 15 concepts or skills over eight months.5

4The standard deviation for average exercises to mastery was high across domains (range: 13.0 to 21.8 exercises), indicating great

variability among students in the average number of exercises to mastery.

5Usage in the high program usage subgroup ranged from 13.5 to 65 minutes daily and 18 minutes daily represents the middle of the distribution within the high usage subgroup. Usage in the low program usage subgroup ranged from virtually no program usage to just over 5 minutes daily and 4 minutes daily represents the middle of the distribution within the low usage subgroup.