Project Title: PIE Study - Cross-sectional and Longitudinal Study of Component Processes for Comprehension
Lead Investigators: Christopher Lonigan and Richard Wagner
Timeline: 2010 – 2012 extended to 2015
Research Design: The PIE study employed a longitudinal design. Students in grades Pre-kindergarten through 5th grade were tested on 18 cognitive and linguistic factors at multiple time points across several years. During the first year of the study, students completed a large battery of assessments. The students in preschool, first grade, and third grade were followed each year for two years to assess the latent constructs the study was attempting to measure. Based on the results from the first wave of assessment, fifth grade students were included in the follow-up phases of assessment. Furthermore, based on the increasing complexity of reading dimensions revealed in the preliminary analysis, two waves of follow-up testing were added thus providing five longitudinal data points.
Setting and Participants: Students were recruited from a diverse group of school districts in five north Florida Counties. During the two recruitment years, 1,839 students were consented and assessed (Pre-K n = 343, Kindergarten n = 236, 1st grade n = 295, 2nd grade n = 228, 3rd grade n = 280, 4th grade n = 231, 5th grade n = 226). Of the 1,144 students eligible to participate in the follow-up assessments (preschool, 1st, 3rd and 5th graders), 89% were located and assessed in the first follow-up year and 83% in the second follow-up year.
Measures: For each of the 18 constructs, the measurement battery included at least three representative measures (either standardized or researcher constructed). To address the concern of the size of the assessment battery for each student, a missing-by-design approach was used to reduce the testing time and time away from the classroom. In short, not every student completed every measure. Children were randomized to one of four conditions, created by randomizing construct clusters, which varied in the number of measures that were administered. Within each of the four assessment conditions, the order of tests were also varied to reduce the effect of any possible testing-order effects in the final analysis. Finally, for each of the follow-up administrations, the students were randomized to an assessment condition.
Key Outcomes: One of the outcomes of the PIE study demonstrated that the distinctness of different components of reading (i.e., decoding vs. comprehension) is a developmental process. Findings indicated that a clear component representing comprehension does not emerge until third grade. Another finding was that relatively few children have substantial difficulties with comprehension that can be attributed to a single skill domain. Thus, the majority of children who struggle to understand what they are reading will need instructional support for both decoding and comprehension skills in order to increase their capacity to read with understanding.
The Basic Science Studies focused on understanding critical elements of cognitive and linguistic processing at the word, sentence and text level. Methods used in these studies included both traditional and novel approaches (such as tracking of eye movements) that was used to study both conscious and ‘under the hood' processes. Researchers examined how information was processed during reading; the role of inhibitory processes, attention regulation, and sensorimotor activity during comprehension; understanding in relation to contextualization; dialect awareness; meta-cognitive processes; and semantic connections in relation to comprehension monitoring.
Project Title: PIE – Tracking; Eye tracking Study
Lead Investigators: Ralph Radach and Christopher Lonigan
Timeline: 2010 – 2014
Setting and Participants: A sub section of students from the PIE study; approximately 50% of the 1st through 5th graders (1st graders n = 147, 2nd graders n = 113, 3rd graders n = 137, 4th graders n = 112, 5th graders n = 112) for a total of 620 students.Research Design: The eye tracking component was focused on examining online information processing during reading for comprehension. It was administered in three separate sessions. Session one consisted of 50 trials of silent sentence reading and sentence scanning. Session 2 consisted of silent paragraph reading and paragraph scanning. Finally, session three consisted of 50 trials of reading sentences aloud and vision screening.In the sentence reading tasks, students were directed to read single-line sentences with controlled target word material. The exact same sentences were used across all grades in order to track developmental changes independent of grade differences in reading material. The reading-like scanning task served to approximate the attentional and visual-motor demands of reading without adding an additional linguistic workload.
Measures: The eye tracking measures included paragraph reading, sentence reading (silent and aloud) and the Landolt scanning task. The paragraphs were selected from the Florida Assessment for Instruction in Reading (FAIR) passages selected from the FAIR Oral Reading Fluency (ORF) toolkit. The extracted eye movement parameters were correlated with the off-line assessment data (listed under the PIE study) to evaluate performance of struggling readers as well as their response to specific interventions.
Key Outcomes: Preliminary analysis of the sentence level reading tasks revealed significant influences on reading time of word length and word frequency whose effects were moderated by the students’ level of decoding skills. Also, the part of the moment-by-moment of reading that is associated with comprehension; i.e., reading time, is primarily predicted by the students’ reading comprehension skill and not by the size and depth of their vocabulary.
Project Title: Semantic Relations & Comprehension Monitoring – Project 8
Lead Investigator: Ralph Radach
Setting and Participants: A subgroup of students from the PIE study during the follow–up phase of the study.
Measures: Using the eye-tracking machines, four studies of 5th and 7th graders (5th grade students from the second year of follow-up from the original PIE study) manipulated parts of text to yield changes and challenges in students on-line processing of text. The four parts of text were a) verb-agent inferences, b) semantic plausibility, c) passage topic monitoring, and d) complex causal relations. Students in the sample also completed a battery of standardized reading and vocabulary tests.
Key Outcomes: Results indicated that comprehension monitoring strategies can be observed within a single sentence. Causal inconsistencies were detected and lead to increased refixation durations, which resulted in large increases in re-reading of the initial parts of sentences. This indicated attempts by the reader to establish coherence. Another finding was that text difficulty influences students’ moment-by-moment reading of text.
Project Title: Inhibitory Control and Comprehension
Lead Investigators: Michael Kaschak and Christopher Lonigan
Setting and Participants: A subgroup of students from the original PIE study (Pre-K n = 146, Kindergarten n= 113, 1st grade n = 138, 2nd grade n = 112, 3rd grade n = 139, 4th grade n = 113, and 5th grade n = 115 for a total of 876).
Research Design: The construct of interest in this project was inhibitory control. The element of this construct, along with executive function, that was of most concern for this study was children’s abilities to attend selectively to specific elements of a situation, (while ignoring others) to perform a task. The participating students were administered six computer-based tasks designed to measure different aspects of inhibitory control.
Key Outcomes: Despite extensive pilot testing, some tasks yielded inconsistent data quality during the initial data collection phase, which has made it difficult to complete data analysis for this component of the project. Because of these technical problems, a number of modifications were made to the tasks for the follow-up data collection phase. Analysis and interpretation of these data is ongoing.
Project Title: Implicit Learning & Comprehension
Lead Investigator: Michael Kaschak
Setting and Participants: The same subgroup of students from the original PIE study who were part of the inhibitory control and comprehension study (Pre-K n = 146, Kindergarten n= 113, 1st grade n = 138, 2nd grade n = 112, 3rd grade n = 139, 4th grade n = 113, and 5th grade n = 115 for a total of 876).
Research Design: The participating students were administered three computer-based tasks designed to measure aspects of implicit language learning.
Measures: 3 Computer-based tasks:
Key Outcomes: Analysis and interpretation of these data is ongoing.
Project Title: Inference Study
Lead Investigator: Christopher Lonigan
Participants and Setting:
Research Design: Students were administered two computer-based tasks that consisted of reading or listening to two- to three-sentence paragraphs describing the relations between a set of real and made-up terms. The students were then asked a series of true-false statements to assess the degree to which they were able to 1) access and integrate information from their existing knowledge base with the information provided in the sentences, 2) use their knowledge base along with the provided information to make inferences, and 3) recall the information presented in the sentences.
Key Outcomes: Preliminary analysis indicated a complex pattern of relations between types of inferencing, with limited evidence that responses to the different types of questions coalesce into the categories outlined by Hannon and Daneman (2001) and Hannon and Frias (2012). These analyses also provided little evidence that there was cross-modality consistency of knowledge integration and access across oral and text-based presentations.
Project Title: English-learner Study: RFUEL
Lead Investigator: Christopher Lonigan
Project Purpose/Goals: The goals of this project were to develop and validate both a comprehensive assessment instrument and a screening measure for Spanish-speaking ELL preschool children’s early literacy skills (SPELA: Spanish Preschool Early Literacy Assessment). A primary design goal of this measure was that it would be appropriate for Spanish-speaking ELL children regardless of Spanish language dialect variations to which they are exposed. The domains included in the measures included: Definitional Vocabulary, Phonological Awareness, Print Knowledge, and Invented Spelling.
Setting and Participants: The RFUEL study drew its population from two completed studies. One was a large-scale measurement development study in which children’s oral language, early literacy, and reading skills were measured in both English and Spanish. These were preschool through first grade Spanish or English-learner students. The second study was a large-scale intervention study that examined the efficacy of a preschool curricula on the pre-academic skills of Spanish-speaking English-learner students also in preschool, Kindergarten and 1st grade. Students were in grades two through fourth when they were located and re-consented for this study.
Measures: A reduced PIE- assessment battery focusing on: decoding, reading comprehension, and oral comprehension
Key Outcomes: Analyses and interpretations of findings for these data are ongoing.
Lead Investigators: Most of the project's key personnel were involved in phases of these studies. The key personnel most associated with these studies included: Kenn Apel, Stephanie Al'Otaiba, Carol Connor, Young-Suk Kim, Christopher Lonigan, Beth Phillips, and Shurita Thomas-Tate.
Timeline: 2010 – 2012
Purpose: To accelerate the development of promising interventions, 10 design studies and, for some of the interventions, small-scale randomized trials, were conducted over the first two years of the project. The content for the developed interventions was based on several cognitive and linguistic components for which there was already substantial evidence of their role in supporting comprehension: semantic knowledge, syntax, comprehension monitoring, background knowledge, strategy use, text structure, and morphological awareness.
Across the two-year period seven distinct interventions were developed and field-tested in these design studies. Each included primary and secondary instructional targets (e.g., vocabulary, morphological awareness, expository text structure). All but two of these interventions (Content Area Literacy Instruction, CALI and Dialect Awareness Study, DAWS), were designed as small-group, pull-out interventions for children demonstrating risk of, or already displaying, reading difficulties. The five other new small group interventions are described in detail under Comparative Efficacy I.
Lead Investigators: Kenn Apel, Stephanie Al'Otaiba, Carol Connor, Young-Suk Kim, Christopher Lonigan, Beth Phillips, Chris Schatschneider, and Shurita Thomas-Tate.
Timeline: 2012 – 2015
Purpose: Given preliminary results from the multivariate study of components of reading comprehension, which indicated that there was less distinction between potential component language skills that could contribute to reading comprehension, the component interventions developed to promote foundational skills (i.e., syntax, narrative, morphological awareness, text structure) were tested in a comparative efficacy study to identify which interventions produced large effects, effects that generalized across multiple domains of language skills, or both. Children who were identified as at risk for comprehension difficulties in grades Pre-K to 4 were randomly assigned to one of the interventions developed for their grade level or to a business-as-usual control group. Specific grades and eligibility criteria were applied for each study.
Project Title: Reading for Understanding, Comparative Efficacy I
Lead Investigators: Christopher Lonigan, Beth Phillips, Carol Connor, & Young-Suk Kim.
Type of Study: Efficacy
Timeline: Year 3 - 2012-2013
Setting and Participants: The sample consisted of 3,702 students in Pre-kindergarten through fourth grade with vocabulary scores below the 48th percentile according to pre-intervention screening. Students were matched within school and randomly assigned to interventions (up to 4 possibilities per grade) or a business-as-usual control group within their grade.
Research Design: Five new interventions were designed and implemented in Pre-kindergarten through fourth grade. A slightly expanded version of Dialogic Reading, DR-Extended (DR-E) was added to the preschool interventions because it is a vocabulary intervention with known positive effects for this age group, and it provided a useful counterfactual against which to compare the other two preschool interventions. Each intervention was intentionally designed to be modular – focusing on just one or a few primary language skills (vocabulary, grammar, narrative). During Comparative Efficacy I, 3 to 5 interventions were directly competed against one another to examine which particular one would be most effective in each grade. Interventions were implemented by trained research personnel in small groups of 3 – 5 students.
Measures: The following are the assessments used in the study:
Intervention Description: There were 6 interventions: COMPASS, ERC, LIM, MAT, TEXTS, and DR-E (adapted from existing Dialogic Reading interventions). Different interventions were administered in different grades: in pre-K – COMPASS, LIM, DR-E; in K, G1 and G2 – COMPASS, LIM, MAT, TEXTS; in G3 – COMPASS, LIM, ERC; in G4 – ERC, TEXTS. Interventions lasted 8 to 12 weeks, each for 25 minutes 4 days per week, conducted in the form of pull-out small group instruction, by trained project personnel with range of knowledge and credentials.
Key Outcomes: Despite general unity of linguistic skills, effects were specific – DR-E led to impacts on vocabulary measures, LIM led to impacts in syntax/grammar measures, COMPASS led to impacts on narrative comprehension and so on. Also, students differently benefited from the intervention depending on the strength of their initial language skills.
Project Title: Reading for Understanding, Comparative Efficacy II: Testing Combinations and Contrasts in Targeted Small Group Interventions in Early Grade
Type of Study: Efficacy
Lead Investigators: Christopher Lonigan, Beth Phillips, & Young Suk Kim.
Timeline: Year 4 - 2013 – 2014
Setting and Participants: Based on the findings of the CE I Study, a second comparative efficacy study was conducted in pre-k, kindergarten and fourth grade. This study took place in 61 schools representing nine public school districts, plus Head Start and subsidized child care centers. Schools were recruited that served a moderate to high proportion of students receiving free or reduced price lunch. All grade-eligible general education classrooms were invited to participate. Consent was received from 1334 and 1718 prekindergarten and kindergarten students, respectively, who were screened on two standardized measures of listening comprehension to determine their qualification. Of these, 739 (~55%) students in prekindergarten and 871 (~51%) in kindergarten met the qualification criteria of being at or below a standard score of 97 on two measures in listening comprehension. These students all (excepting small percentages who left their schools during the study) were pre- and post-tested on the measures listed below and randomized within school and grade after being matched on the screening measures assigned to a block.
Research Design: Three of the five interventions from Comparative Efficacy I were implemented in Pre-Kindergarten ad Kindergarten: Dialogic Reading - Extended (DR-E; vocabulary focus), Language in Motion (LIM; language/syntax focus), and Comprehension Monitoring and Story Structure (COMPASS; narrative text structure and oral comprehension monitoring focus) since each of these interventions was found to be effective for improving one or more language- or comprehension-related skill during the Comparative Efficacy I Study. This study was an incomplete-blocks randomized efficacy trial of four experimental conditions: (1) LIM/COMPASS, (2) LIM/DR, (3) DR-E/COMPASS, (4) Business-as-usual. In each active condition students received 18 weeks of small group intervention lessons approximately 20 minutes in length; they received nine weeks of each of the two included interventions. Within assigned active condition, students were also randomly assigned to the order in which they received the two interventions (i.e., DR-then-LIM or LIM-then-DR). All intervention sessions were conducted in small groups of no more than five students by trained interventionists supervised by the intervention developers and the Investigators.
Screening Listening Comprehension Measures (also administered at post-test)
Additional Pretest-Post-test Measures
Key Outcomes: Preliminary analyses have been conducted on the proximal measures and on key standardized assessments of oral language. All analyses were conducted for each grade separately. To account for the nesting of children in schools and assignment blocks, all analyses were multilevel models using school and block as random factors and controlling for age and for pretest score on the relevant measure.
Preliminary results suggest broader impacts from the paired interventions than were previously indicated for each intervention delivered singly. However, results continue to suggest that impacts are modular such that significant effects only accrue to measures of constructs explicitly targeted by the interventions received.
Project Title: Reading for Understanding, Comparative Efficacy II: Testing Static and Dynamic Versions of a Text Structure Intervention
More information coming soon.
Project Title: LIM/CALI Study - Integrating language, text, and academic knowledge interventions to promote second grade reading for understanding
Lead Investigators: Christopher Lonigan, Beth Phillips, Carol Connor
Timeline: Year 5- 2014-2015
Type of Study: Efficacy
Project Purpose/Goals: Two efficacious interventions, Language in Motion (LIM) and Content Area Literacy Instruction (CALI) were combined to create a hybrid intervention that provided oral language intervention (i.e., LIM) combined with CALI design features (i.e., appropriate informational text, specific comprehension strategies of connecting, visualizing and generating questions, written activities, discussion strategies to build word and content knowledge), which was titled LIM/CALI. The intervention also included an augmented focus on lexical quality through exploration of targeted vocabulary words embedded in texts and instruction. Key goals were
Setting and Participants: This study took place in 42 schools (195 classrooms) in 8 school districts in North Florida and South Florida. Schools were recruited that serve a moderate to high proportion of students receiving free or reduced price lunch. All second grade general education classrooms were invited to participate. Consent was received from 2,285 second-grade students. OF these, 2,256 were screened on reading comprehension to determine their qualification. Of these, 1,148 (~50%) students met the qualification criteria of being at or below the 40th percentile in reading comprehension. 1,123 of these students were pre-tested on the measures listed below and randomized within school after being matched on the screening measures. Once randomized, students assigned to the LIM/CALI Plus condition were evaluated based on two standardized decoding measures administered at pretest to determine eligibility to receive the augmentative decoding intervention. Eligibility was set at an average standard score on the decoding measures below 100.
Research Design: There were three experimental conditions: (1) LIM/CALI Only: 12 weeks of LIM-CALI Intervention, which includes 48 30-minute lessons, (2) LIM/CALI Plus: The identical LIM CALI intervention plus the decoding intervention (for decoding-qualified students only). The decoding intervention includes 48 10-minute lessons delivered in 12 30-minute lessons over three week and then weekly 20-minute booster-sessions during the first six weeks of the LIM/CALI intervention, (3) Business-as-usual.
All intervention sessions were conducted in small groups of no more than five students by trained interventionists supervised by the intervention developers and the Investigators. All analyses took school and assignment block into consideration using multi-level modeling. Analyses for hypothesis one compared each active condition to BAU. Analyses for hypothesis two compared the LIM/CALI Plus condition to the LIM/CALI only condition.
Additional Pretest-Post-test Measures
*Used to determine eligibility for decoding intervention for students randomized to LIM/CALI Plus condition.
Intervention-Aligned Assessment for LIM/CALI. This 40-item assessment includes items addressing intervention components inclusive of syntax, vocabulary, and listening comprehension. Item formats include multiple choice, short answer and cloze items. Intervention Aligned Assessment for Decoding. This measure includes single-, and multi-syllable individual nonwords and real words to be read aloud by students for a total of 80 items that represent a wide variety of phonemes, digraphs, and blends.Intervention-Aligned Assessment for Reading Comprehension. This measure includes two subtests, inclusive of brief researcher-developed passages embedded with the syntax targets for which students complete multiple choice items and a researcher-developed maze task requiring students to select the appropriate syntactically-correct word to complete the connected text sentences.
Key Outcomes: Analyses revealed significant positive effects on the proximal and ‘distimal’ (i.e., near-transfer) measures of targeted vocabulary, targeted syntax, and reading comprehension (but not listening comprehension). LIM-CALI Plus had a significant positive effect on the proximal measure of decoding. However, there were few significant effects on the standardized measures of language, decoding, and reading comprehension. There was also a significant interaction that revealed that the intervention was more effective for children who had higher scores initially on the proximal measures.
Project Title: Professional Development Study of LIM/CALI
Lead Investigator: Beth Phillips
Type of Study: Efficacy
Project Purpose/Goals: To investigate the intensity level of professional development support required by educators to achieve high levels of fidelity in their implementation of a language intervention.
Setting and Participants: Forty coached and 40 workshop-only second grade teachers representing 23 schools in seven school districts participated. Within these schools, 1184 students were screened, and 385 qualified and were disproportionately randomized within schools to receive (n =268) or not receive the intervention (n =112).
Research Design: The central question was regarding differential implementation by teachers in two randomly assigned conditions. Teachers assigned to the workshop only condition received a four-hour face to face professional development that included didactic, demonstration and video, and hands-on practice components. Coached teachers participated in this workshop and also received three coaching visits that included modeling and observations with reflective feedback sessions in which teachers and coaches jointly developed implementation goals. All lessons were audiotaped; coached teachers also received four rounds of written feedback on recorded lessons. All teachers were observed live three times during the eight week intervention period and rated on five other audio-recordings by personnel blind to their coaching condition.
Measures: Participating children received a limited battery of language and comprehension assessments (both proximal and distal) before and after the 8 weeks of intervention or BAU experiences. Hierarchical linear modeling (HLM) was used for group comparisons among the teacher fidelity and the child assessment data. Descriptive, correlational additional HLM models were used to analyze the fidelity data.
Key Outcomes: Preliminary results indicated that coached teachers implemented a greater number of lessons, and those lessons were rated as more engaging for students. Some general aspects of implementation fidelity, such as adherence to lesson plans, appear comparable between groups. Analyses are in progress to explore whether students receiving intervention from coached or uncoached teachers experienced differential skill gains. Furthermore, detailed exploration of scaffolding and engagement techniques used within the intervention is underway with both this sample and the earlier LIM/CALI Efficacy sample.
More information coming soon.