The presenter in the fourth session of TextProject’s Virtual Institute series is David Francis. Dr. Francis is internationally known for bringing expertise in psychometrics and statistics to empirical investigations of critical problems in education, including the assessment of students with disabilities and English Language Learners (ELLs).
In this column, I am going to focus on Dr. Francis’s comments on what the new-generation of assessments means for students with disabilities and ELLs. But the presentation also includes a comprehensive review of the PARCC and Smarter Balanced assessments. Since Dr. Francis is a member of the both the Independent Review Panel for the National Assessment of Title I and of Technical Advisory Group of the What Works Clearing House, his review merits attention. I strongly recommend that educators study his “list of things to like” about the new assessments (on Slides 7-12, which correspond to minutes 5:26 to 13:50 on the video).
Over the remainder of the presentation, Dr. Francis makes numerous critical points about the new assessments in relation to the strengths and needs of students with disabilities and ELLs. Beginning around the 14th minute of the video, he describes the need to attend to variations in text difficulty as a function of reader and text characteristics. Dr. Francis and his colleagues found that effects of text difficulty can be student specific, even after reader characteristics are controlled (e.g., gender, proficiency).
Dr. Francis then goes on to describe the plans of the two consortia for accommodations for students with disabilities. The PARCC accessibility policies have three levels: embedded supports, accessibility features, and accommodations. Embedded supports (e.g., audio amplification, highlighting) can be activated by any student at his or her own discretion. Accessibility features (e.g., background/font color) are also available for all students but these features need to be activated by a school-based educator prior to the assessment based on students’ personal needs profile. For the final level of accommodations for students with disabilities, four types of accommodations have been identified: (a) presentation (e.g., assistive technology), (b) response (e.g., Braille note-taker), (c) timing and scheduling (e.g., extended time), and (d) setting (alternative location). The information for accommodations on the Smarter Balanced has been more limited and continues to be developed (at the time of the presentation on April 19, 2013). But the Smarter Balanced consortium has stated that there will be support for ELLs, students with disabilities, and other students with special needs, including visual, auditory, and physical supports. The aim, according to the Smarter Balanced consortium, is to ensure that all students can demonstrate what they know and can do.
After a comprehensive review of assessment features for students with disabilities, Dr. Francis turns to issues involved in assessments of ELLs, especially the limitations of research on accommodations for ELLs. A study conducted by Dr. Francis and his colleagues examined the available research (approximately 20 studies with 65 effect sizes). This study showed that three accommodations significantly influenced the performances of ELLs: English dictionaries/glossaries, simplified English, and extra time.
The subsequent slides of this presentation are available on the video as well as the PowerPoint but, with time limitations, Dr. Francis could not complete the narrative. The final eight slides of this presentation are important to study, however, in that they give Dr. Francis’s conclusions about the new generation of assessments. Most importantly, he makes an important proposal regarding online assessments (which will be part of the assessments of both consortia). His proposal is that online assessments could provide added value by giving students three scores for one testing event: (a) one estimating ability if all items had been administered without accommodations, (b) one estimating ability as if all items had been administered with accommodations, and (c) one estimating ability under the tested conditions. Such a strategy, Dr. Francis states, would provide different information about students, mastery, and language ability in relation to content and yield information on the impact of accommodations for individuals and for given groups.
The concluding statement of the PowerPoint presentation is worthy of note, from a scholar with the international stature in psychometrics and statistics of Dr. Francis: “I am optimistic because I have great confidence in the teams and in the value of setting high expectations for all students.”