Evaluation of a Speech-Language Outcome Measure for Preschool Children

Last modified by Support on 2012/03/01 11:09


Research Recap

Evaluation of a Speech-Language Outcome Measure for Preschool Children

Summary prepared by:
Thomas-Stonell, N., McConney-Ellis, S., Oddson, B., Robertson, B., Rosenbaum, P.

Why did we do this study?
Speech language pathologists need valid and reliable outcome measures to learn how well therapy works and to get information that will help make treatments better. Before an outcome measure is used in clinics, it must be tested to see how well it measures change. A poorly developed measure might give invalid answers that in turn causes clinicians to draw wrong conclusions. This study tested how well the American Speech-Language-Hearing Association Pre-Kindergarten National Outcomes Measure System (Pre-K NOMS) worked with preschool children in Ontario.

Who participated in this study?
Six Ontario organizations (listed below) partnered to collect data on 213 preschool children who were receiving speech and language therapy. Most of the children were 4 -5 years old and about 67% were boys. The children had a variety of communication disorders. About 33% of the children had communication disorders associated with a medical condition (e.g., cerebral palsy, developmental delay, autism). 25% of the families reported that they spoke more than one language at home. Treatment was provided in either English (95%) or French (5%). The most common areas of treatment were articulation (oral speaking ability) and language production (expression of ideas and thoughts).

How did we do the research?
At the start of treatment, the clinician rated the child’s communication skills using the Pre-K NOMS. Then the clinician and the child’s parent both completed a questionnaire, developed by the research team, to rate the child’s communication skills. The questions focused on the same communications areas as the Pre-K NOMS and asked participants to compare the child’s skills to their peers.

At the end of treatment, the clinician completed the Pre-K NOMS a second time. The child’s parent and clinician once again rated the child’s communication skills using the same questionnaire they completed at the start of treatment.

After completing the second ratings, parents and clinicians looked at the questionnaires they completed at the start of treatment to remind them of their initial ratings. They used this information to help them judge how much the child’s communication skills had changed and whether these changes resulted in “a difference in a child’s ability to communicate independently in ‘real life’ situations.” Our research team compared the changes noted by the parents and clinicians to change measured by the Pre-K NOMS.

What did we find?
The researchers divided the children into ‘change’ and ‘no change’ groups based upon agreement between parents and clinicians. We examined the Pre-K NOMS ratings for each child in these groups to see if their ratings matched what parents and clinicians observed. The Pre-K NOMS missed about 33% of the children who showed ‘real life’ communication changes as judged by both parents and clinicians. In all cases where the ratings did not match, parents and clinicians agreed that the child had made ‘real life’ improvement, but the Pre-K NOMS did not measure any change.

What do these findings mean?
The Pre-K NOMS does not do a good job of measuring functional communication changes for this population. It underestimated the number of children who made important ‘real life’ changes. In addition to better communication skills, parents and clinicians noted positive changes in these children’s social skills, play, attention, and frustration. The Pre-K NOMS could not measure these changes.

What’s next?
Members of our team continue to conduct research in speech and language outcome measurement. We will use the data collected in this study to create a new outcome measure designed to capture ‘real life’ communication changes for preschool children at home and in their communities.

For more information contact
Prof. Nancy Thomas-Stonell, Scientist, Bloorview Research Institute, Bloorview MacMillan
Children’s Centre, Toronto, Canada
Email: nthomasstonell@bloorviewmacmillan.on.ca
Telephone: 416-425-6220 ext. 3425

How was the research funded?
The Hospital for Sick Children Foundation and the Bloorview Children’s Hospital Foundation.

The researchers wish to thank the staff at our partner organizations: The Children’s Hospital of Eastern Ontario, The Ottawa Children’s Treatment Centre, Pathways Health Centre for Children, Porcupine Health Unit and The York Region Preschool Speech and Language Program and the Bloorview MacMillan Children’s Centre.

Created by CRRN CRRN on 2010/03/08 19:32