Who has participated in the Ultimate TV Turnoff Challenge?

 

In the 2001-2002 academic year, 44 first through fifth grade students from Tuscan Elementary School in Maplewood, NJ participated.  In the academic year 2002-2003, 143 students from kindergarten through fifth grade participated.  In the academic year 2003-2004, 243 kindergarten through fifth grade students completed the challenge and 312 kindergarten through fifth grade students participated in the challenge.  Data was analyzed for four years:  2000-2001 ~ the baseline year before the innovation, 2001-2002 ~ the first year of the innovation, 2002-2003 ~ the second year of the innovation, and 2003-2004 ~ the third year of the innovation.  

 

How was student achievement measured?

 

During the spring of the 2000-2001, 2001-2002 and 2002-2003 academic years, Tuscan Elementary students in second through fifth grade were given the Terra Nova Achievement Tests.  During the spring of the 2003-2004 academic year, only students in third and fifth grade were given the Terra Nova Achievement Tests.  The Terra Nova is one of the district’s accountability measures that are used to track student progress.  The test includes sub-tests in reading, language arts, and mathematics; a total composite score for all three sub-tests is also calculated.  Kindergarten and first grade students are not given achievement tests because they have not mastered the skills which are necessary to complete a complex paper-pencil test.  Therefore, teacher administered reading scores are used to track the students’ progress. 

Does TV Turnoff make a difference in students’ achievement test scores?

 

YES!  For Ultimate TV Turnoff participants, their average Terra Nova test results for the reading, language, and math subtests and the total composite were consistently higher than the students who did not participate.  This statistically significant result did not happen by chance alone; the TV turnoff initiative provided an environment that was conducive to increased student achievement. 

 

The Terra Nova scores are designed to allow comparison across grades.  The analyses compared the participating students to the non-participating students with and without regard to grade level.  Table 1 displays the results.  In the baseline year, there were no significant differences in test scores, indicating that all students at Tuscan Elementary School were similar (or not statistically different) before the Ultimate TV Turnoff challenge began.  For the students who participated in the first year, their Terra Nova test scores were very significantly higher in the language subtest, significantly higher in reading subtest and the total composite, and higher in the math subtest.  For students who participated in the second year, their Terra Nova scores were very significantly higher in all subtests (reading, language, and math) and in the total composite score.  For the third grade and fifth grade students who participated in the third year, their Terra Nova scores were very significantly higher in the subtests for reading, language and in the total composite scores and significantly higher for the math subtest. 

 

Table 1.  All students – an overall comparison of participating and non-participating students.

Year

Number of Students Participating

Year of Participation

Reading

Language

Math

Total - Composite

2000-2001

None – baseline year

2001-2003

No significant differences

No significant differences

No significant differences

No significant differences

2001-2002

41

(3 students too young for test)

2001-2002

Sig. higher  *

Very Sig. higher  **

Higher  +

Sig. higher  *

2002-2003

104

(39 students too young for test)

2002-2003

Very Sig. higher  **

Very Sig. higher  **

Very Sig. higher  **

Very Sig. higher  **

2003-2004

72

(171 students not tested)

2003-2004

Very Sig. higher  **

Very Sig. higher  **

Sig. higher  *

Very Sig. higher  **

+ p ≤ .10 (Higher); *  p ≤ .05 (significantly higher); ** p ≤ .01 (very significantly higher).

 

Table 2 presents the grade level results.  For the majority of the grades, the average score of the Ultimate TV Turnoff Challenge participants was significantly higher than the average score of the non-participants.  Note:  in the first year of the innovation 44 students participated, which resulted in some grade levels having only a few participating students.  Because there were few participants, the grade-level results for 2001-2002 should be used in combination with the overall student analysis displayed in Table 1.  The level of significance in the subtests increased from the first year to the second year for all students.  The level of significance increased again from the second year to the third year for the third grade students.  The reader should notice that as the number of participating students increased the level of significance for the test scores were higher for participants.  For example, the class of 2011 had 11 students participating in 2001-2002; these students had higher test scores in language.  In 2002-2003, 27 students participated; their test scores were significantly higher in reading and language and the total composite was very significantly higher.  The grade level results for 2002-2003 are representative for all grades except the class of 2010, which still had a small number of students participating.  The grade level results for 2003-2004 are representative for all grades except for the class of 2011, which still had a small number of students participating. 

 

Table 2.  Grade level results – a comparison of participating and non-participating students.

Class

Year

Grade –  # Participating

Reading

Language

Math

Total - Composite

2010

00-01

3rd - zero

01-02

4th

6 out of 112

02-03

5th

14 out of 115

Sig. higher  *

Higher  +

Sig. higher  *

2011

00-01

2nd - zero

Sig. higher  *

Higher  +

01-02

3rd 

11 out of 97

Higher  +

02-03

4th

27 out of  91

Sig. higher  *

Very Sig. higher  **

Sig. higher  *

03-04

5th

24 out of 81

2012

00-01

1st – zero

Not tested

Not tested

Not tested

01-02

2nd

14 out of 93

Higher  +

Higher  +

Higher  +

02-03

3rd

32 out of 94

Sig. higher  *

Sig. higher  *

Very Sig. higher **

Very Sig. higher  **

03-04

4th

41 out of 84

Terra Nova not administered

Terra Nova not administered

Terra Nova not administered

Terra Nova not administered

2013

00-01

K – zero

Not tested

Not tested

Not tested

Not tested

01-02

1st

10 out of 114

Not tested

Not tested

Not tested

02-03

2nd

29 out of 111

Sig. higher  *

Sig. higher  *

Sig. higher  *

Very Sig. higher  **

03-04

3rd

48 out of 107

Very Sig. higher **

Very Sig. higher **

Sig. higher  *

Very Sig. higher **

2014

00-02

PreK/K - zero

Not tested

Not tested

Not tested

Not tested

02-03

1st

33 out of 106

Sig. higher  *

n/a

n/a

n/a

03-04

2nd

44 out of 94

Terra Nova not administered

Terra Nova not administered

Terra Nova not administered

Terra Nova not administered

2015

02-03

Kindergarten

3 out of 104

Not tested

Not tested

Not tested

Not tested

 

03-04

1st

46 out of 100

Sig. higher  *

Not tested

Not tested

Not tested

2016

03-04

Kindergarten

39 out of 96

Not tested

Not tested

Not tested

Not tested

– no difference; + p ≤ .10 (Higher); *  p ≤ .05 (significantly higher); ** p ≤ .01 (very significantly higher).

2010 Graduating Class

 

In 2003-2004, these students were no longer at Tuscan Elementary School.  The previous year results remain in this report for comparison purposes. 

 

In 2000-2003, there were no differences in participation based on demographics.  The students in the 2010 graduating class were very similar when they were in third grade; there were no differences between the participants and non-participants at that time. 

 

Six of 112 fourth grade students participated in 2001-2002; there were no significant differences for the participants in comparison to the non-participants.  When all the students who participated in 2001-2002 were analyzed as a group, the overall participants’ scores for reading, language, math, and total composite ranged from higher to very significantly higher than the non-participants.  Why are the gains not showing up for this class?  A very small number of students (6) from this class participated the first year.  When the number of participants is less than 30, the changes in test scores must be extremely large for difference to be noticeable.  As the number of participants and non-participants becomes more equal, the score differences between groups are more recognizable because they are not masked by the group size differences.

 

Fourteen of 115 fifth grade students participated during the second year, 2002-2003.  During their fifth grade year, the Ultimate TV Turnoff Challenge participants had significantly higher reading scores and higher language scores which translated into a significantly higher total composite score for these students.  Table 3 displays the results of the statistical analysis for 2010 graduating class.

 

Table 3.  2010 Graduating Class – Ultimate TV Turnoff Challenge Results.

Achievement Test Results

3rd grade

2000-2001

4th grade

2001-2002

6 of 112 participated

5th grade

2002-2003

14 of 115 participated

Reading

No differences between groups

No differences between groups

TV Turnoff students had significantly (*)  higher scores than non-participants

Language

No differences between groups

No differences between groups

TV Turnoff students had higher (+) scores than non-participants

Math

No differences between groups

No differences between groups

No differences between groups

Total Composite

No differences between groups

No differences between groups

TV Turnoff students had significantly (*)  higher scores than non-participants

+ p ≤ .10; * p ≤ .05; ** p ≤ .01.

 


2011 Graduating Class

 

In 2000-2003, there were no differences in participation based demographics.  In 2003-2004, significantly more girls (66%) participated in TV Turnoff than was found in the class (44%).  The students who participated in the Ultimate TV Turnoff Challenge had significantly higher reading scores in second grade than the non-participants.  Four students participated all three years, eleven of the 2003-2004 students had participated in at least one of the previous years, 13 students were new to TV Turnoff, and 16 students who participated in previous years chose not to participate in 2003-2004.

 

Eleven of 97 third grade students participated in 2001-2002; these students had higher language scores than the non-participants.  When all the students who participated in 2001-2002 were analyzed as a group, the overall participants’ scores for reading, language, math, and total composite ranged from higher to very significantly higher than the non-participants.  Why are the gains not showing up in all subject areas for this class?  A small number of students (11) from this class participated the first year.  When the number of participants is less than 30, the changes in test scores must be extremely large for difference to be noticeable; as the number of participants and number of non-participants becomes more equal, the differences in the groups are more recognizable because they are not masked by the differences in group size.

 

Twenty-seven of 91 fourth grade students participated in 2002-2003; the students who participated had very significantly higher scores in language than the non-participants.  In addition, the students who participated had significantly higher scores in reading and the total composite score. 

 

Twenty-four of 81 fifth grade students participated in 2003-2004; the students who participated had higher scores than the non-participants, but not statistically higher scores.  The higher, but not significantly higher scores were found for reading, language, math, and the total composite score.  The gains were not significantly different between participants and non-participants partly because of the small number of students who participated at the 5th grade level.  Table 4 displays the results of the statistical analysis for the 2011 graduating class.

 

Table 4.  2011 Graduating Class – Ultimate TV Turnoff Challenge Results.

Achievement Test Results

2nd grade

2000-2001

3rd grade

2001-2002

11 of 97 participated

4th grade

2002-2003

27 of 91 participated

5th grade

2003-2004

24 of 81 participated

Reading

TV Turnoff students had significantly (*)  higher scores than non-participants

No differences between groups

 

TV Turnoff students had significantly (*)  higher scores than non-participants

No differences between groups

 

Language

No differences between groups

TV Turnoff students had higher (+) scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

No differences between groups

 

Math

No differences between groups

No differences between groups

No differences between groups

No differences between groups

Total Composite

TV Turnoff students had higher (+) scores than non-participants

No differences between groups

TV Turnoff students had significantly (*)  higher scores than non-participants

No differences between groups

 

+ p ≤ .10; * p ≤ .05; ** p ≤ .01.

 


2012 Graduating Class

 

In 2003-2004, these students were not given the Terra Nova Achievement Test; 41 of 84 fourth grade students participated in the Ultimate TV Turnoff Challenge.  The previous analyses were left in the report for comparison purposes. 

 

In 2001-2003, there were several significant differences in the demographics of the participants and non-participants.  Significantly more girls (66%) participated than was found in the class (46%).  The ethnicity of the students who participated was:  78% White, 19% Black, and 3% Asian.  The ethnicity of the participants was significantly different from the proportions of the class from 2000‑2003:  56 – 61% White, 32 – 38% Black, 1% Hispanic, and 4 – 5 % Asian/Pacific Islander.  There were no differences in participation based on education disabilities or participation in the free/reduced lunch program.  The students in the 2012 graduating class who participated in the Ultimate TV Turnoff Challenger were very similar when they were in first grade; there were no differences between the participants and non-participants in reading scores as measured by their teachers in fall 2000 or spring 2001.

 

Fourteen of 93 second grade students participated in 2001-2002; the participating students had higher scores in reading, math, and total composite than the non-participants.  When all the students who participated in 2001-2002 were analyzed as a group, the overall participants’ scores for reading, language, math, and total composite ranged from higher to very significantly higher than the non-participants.  Why are the gains not as high this year?  The small number of students (14) from this class participated the first year.  When the number of participants is less than 30, the changes in test scores must be extremely large for difference to be noticeable.  As the number of participants and number of non-participants becomes more equal, the differences in the groups are more recognizable because they are not masked by the differences in group size.

 

Thirty-two of 94 third grade students participated in 2002-2003; the participating students had significantly higher scores in reading and language than the non-participating students.  In addition, the participating students had very significantly higher scores in math and total composite scores.  Table 5 displays the results of the statistical analysis for the 2012 graduating class.

 

Table 5.  2012 Graduating Class – Ultimate TV Turnoff Challenge Results. 

Achievement Test Results

1st grade

2000-2001

2ndd grade

2001-2002

14 of 93 participated

3rd grade

2002-2003

32 of 94 participated

4th grade

2003-2004

41 of 84 participated

Reading

No differences between groups - teacher admin reading tests

TV Turnoff students had higher (+) scores than non-participants

TV Turnoff students had significantly (*)  higher scores than non-participants

Not tested

Language

Not tested

No differences between groups

 

TV Turnoff students had significantly (*)  higher scores than non-participants

Not tested

Math

Not tested

TV Turnoff students had higher (+) scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

Not tested

Total Composite

Not tested

TV Turnoff students had higher (+) scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

Not tested

+ p ≤ .10; * p ≤ .05; ** p ≤ .01.

 


2013 Graduating Class

 

In the 2003-2004, there were no differences in the demographics of the participants and non-participants.  In 2001-2003, there were two differences in the demographics of participants and non-participants:  2001-2002, one student with disabilities participated and for 2002-2003, two students with disabilities participated.  The ethnicity of the students who participated during 2001-2003 was different from the ethnicity of the class.  The ethnicity of the students who participated was:  70-79% White, 17-30% Black, and 0-3% Asian.  The ethnicity of the class was 52-59% White, 38-45% Black, 2% Hispanic, and 1% Asian.  There were no differences in participation based on gender or participation in the free/reduced lunch program. 

 

Ten of 114 first grade students participated in 2001-2002; there were no significant differences for the participants in comparison to the non-participants.  The teacher-administered reading tests from both the fall and spring were used.  Why were the gains not noticeable in the first year?  The small number of students (10) from this class participated the first year.  When the number of participants is less than 30, the changes in test scores must be extremely large for difference to be noticeable.  As the number of participants and number of non-participants becomes more equal, the differences in the groups are more recognizable because they are not masked by the differences in group size.

 

Twenty-nine of 111 second grade students participated during the second year, 2002-2003.  During their second grade year, the Ultimate TV Turnoff Challenge participants had significantly higher reading, language, and math scores compared to the non-participants.  These higher reading, language, and math scores then translated into very significant differences in the total composite score. 

 

Forty-eight of 107 third grade students participated during the third year, 2003-2004.  During their third grade year, the Ultimate TV Turnoff Challenge participants had very significantly higher reading, very significantly higher language, and significantly math scores compared to the non-participants.  These higher reading, language, and math scores translated into very significant differences in the total composite scores.  Table 6 displays the results of the statistical analysis for the 2013 graduating class. 

 

Table 6.  2013 Graduating Class – Ultimate TV Turnoff Challenge Results.

Achievement Test Results

Kindergarten

2000-2001

1st grade

2001-2002

10 of 114 participated

2nd grade

2002-2003

29 of 111 participated

3rd grade

2003-2004

48 of 107 participated

Reading

Not tested

No differences between groups

TV Turnoff students had significantly (*)  higher scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

Language

Not tested

Not tested

TV Turnoff students had significantly (*)  higher scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

Math

Not tested

Not tested

TV Turnoff students had significantly (*)  higher scores than non-participants

TV Turnoff students had significantly (*)  higher scores than non-participants

Total Composite

Not tested

Not tested

TV Turnoff students had very significantly (**)  higher scores than non-participants

TV Turnoff students had very significantly (**)  higher scores than non-participants

+ p ≤ .10; * p ≤ .05; ** p ≤ .01.

 


2014 Graduating Class

 

In 2003-2004, these students were not given the Terra Nova Achievement Test; 44 of 94 second grade students participated in the Ultimate TV Turnoff Challenge.  The previous analyses were left in the report for comparison purposes. 

 

In 2002-2003, there was a significant difference in participation based on gender:  78% of the participants were girls while the proportion of girls in that class was 62%.  There were no differences in participation based on ethnicity, education disabilities, or participation in the free/reduced lunch program.  During 2000-2001, these students were in pre-Kindergarten and did not participate in the program.  During 2001-2002, these students were in Kindergarten and did not participate in the program.

 

Thirty-three of 106 first grade students participated in the Ultimate TV Turnoff Challenge.  The teachers measured the students reading scores in the fall of 2002 and in the spring of 2003.  There were no significant differences in the students’ reading scores in the fall of 2002.  In the spring of 2003, the students who participated had significantly higher reading scores than the students who did not participate.  Table 7 displays the results of the statistical analysis for the Class of 2014.

 

Table 7.  2014 Graduating Class – Ultimate TV Turnoff Challenge Results. 

Teacher  Rating

Pre-K

2000-2001

Kindergarten

2001-2002

1st grade

2002-2003

33 of 106 participated

2nd grade

2003-2004

44 of 94 participated

Reading

Not Applicable

Not Applicable

TV Turnoff students had significantly (*)  higher scores than non-participants

Not Tested

+ p ≤ .10.

 

 


2015 Graduating Class

 

In 2002-2003, three students of the 104 Kindergarten students were permitted to participate because their older siblings were participating in the challenge; the parents wanted all the children in the family to participate.  Achievement data or teacher reading test data was not available for this class. 

 

In 2003-2004, 46 of 100 first grade students participated in the Ultimate TV Turnoff Challenge in 2003‑2004.  The teachers measured the students reading scores in October, 2003 and in June, 2004.  There were no significant differences in the students’ reading scores in October, 2003; the average score of the class was a 3.55 which is the middle of the “Advanced ~ Beginning of 1st Grade” category.  In June, 2004, the students who participated had significantly higher reading scores than the students who did not participate.  Table 8 displays the results of the statistical analysis for the Class of 2014.

 

Table 8.  2015 Graduating Class – Ultimate TV Turnoff Challenge Results. 

Teacher  Rating

Pre-K

2001-2002

Kindergarten

2002-2003

3 of 104 participated

1st grade

2003-2004

46 of 100 participated

Reading

Not Applicable

Not Applicable

TV Turnoff students had significantly (*)  higher scores than non-participants

+ p ≤ .10.

 

 

2016 Graduating Class

 

In 2003-2004, 39 of the 96 Kindergarten students participated in the Ultimate TV Turnoff Challenge.  Achievement data or teacher reading test data was not available for this class. 

 

Table 8.  2015 Graduating Class – Ultimate TV Turnoff Challenge Results. 

Teacher  Rating

Kindergarten

2003-2004

39 of 96 participated

Reading

Not Applicable