Teaching and Learning Forum 2005 Home Page
Category: Professional practice
Teaching and Learning Forum 2005 [ Refereed papers ]
Structured experiential learning: The Ages and Stages Questionnaire is effective as a teaching tool for medical students

Pam Nicol
School of Paediatrics and Child Health
The University of Western Australia

A survey of final year undergraduate medical students found that only 45% felt confident in performing a developmental assessment on a child. Yet, this is a core curriculum outcome. The objective was to plan, implement and evaluate a student centred program that will increase undergraduate medical students participation in learning, in particular, the confidence of fifth year medical students to assess the growth and development of a child. A 'Parents-as-Teachers' learner centred program was implemented using a parent completed, child development early detection system, the Ages and Stages Questionnaire (ASQ). The program was designed using a social constructivist epistemology. It involved the students and the parents completing an assessment of the child's growth and development, and the student's reflection on the two methods and the process. Descriptive and interpretative analysis was made of the reports subsequently written by the students. There was a 70% agreement between student assessment and the parent's assessment. Key themes to emerge from the interpretative analysis demonstrated students benefited from this program. Students valued the experiential structured learning; they increased their confidence in developmental assessment and their awareness of the value of a partnership with parents. However, some students continued to doubt the reliability of parental report.


Introduction

A survey of final year undergraduate medical students from the University of Western Australia found that only 45% felt confident in performing a developmental assessment on a child. Yet this skill is stated as an important outcome of the paediatric and child health curriculum. In child health, knowledge of growth and development is recognised as a core curriculum outcome for medical programs (Paton & Cockburn, 1995).

It is essential that primary care practitioners are competent in monitoring developmental progress as developmental and behavioural problems are commonly first identified by them (American Academy of Pediatrics, 2001). Surveillance of developmental progress refers to a process of eliciting and attending to parents concerns, making accurate and informative longitudinal observations on children, obtaining a relevant developmental history, and promoting development (Centre for Community Child Health & Royal Children's Hospital Melbourne, 2002). It is recommended that community screening for development should not be limited to inquiry at one point in time and should use tools that have been demonstrated to have adequate psychometric properties (Child and Youth Health Intergovernmental Partnership, 2002).

The Ages and Stages Questionnaire (ASQ) has been well validated, the sensitivity ranges from 76-91% and specificity from 81-92% (Squires, Potter,& Bricker, 1990). There is 94% agreement between parent's assessment using the ASQ and expert clinician assessment. It is a parent completed, child development early detection system. Items in the questionnaire were originally derived from a range of standardised developmental and curriculum based assessment tools in response for the need for less expensive, well developed tests that recognised the role of parents in monitoring their child's development and resulted in consistency of monitoring over time with timely intervention. (Squires et al, 1990).

This tool is specifically designed to be part of a child health-monitoring program. The Department of Community and Child Health are currently implementing the ASQ at pilot sites in Western Australia. Therefore it is an ideal teaching and learning tool for implementation in the local undergraduate medical program.

It is hoped the program would have the additional benefit of promoting an attitude of respect and partnership between the student and the parent. Collaboration of parents and clinicians in teaching has been successful in other medical programs in improving mutual understanding and effecting attitudinal change in the students (Blasco, Kohen & Shapland, 1999). A study by Stacy and Spencer (1999) reported that patients saw themselves in active roles as teachers and as facilitators of the development of students' professional skills and attitudes (Stacy & Spencer, 1999). Owen and Reay's (2004) study found that mental health consumers successfully sustained delivery of tutorials for 4th year medical students in psychiatry over a 4 year period and that there was a general trend towards improved attitudes across all measures (Owen& Reay, 2004).

Attitudes are important because they are viewed as a mediating link between clinical competence (knowledge and skills) and clinical performance; that is, attitudes influence what the doctor does in actual practice (Newble, 1992). One study that measured attitudes found a persistent decline in several attitude scores as students progressed through their medical educational program (Woloschuk, Harasym & Temple 2004). A decline was noted (among others) in the doctor-patient relationship, communication and preventative medicine sub-scales. It may be that these areas were not formally taught in the clinical years and therefore formed part of a hidden curriculum. By including these as an assessable part of the unit, student awareness of their importance will be raised.

Clinical teaching in medicine is influenced by the historical "apprenticeship style" approach to teaching. It is based on a passive, static view of knowledge that holds the structure can be modelled for the learner. However, the expert clinician uses intuition when making clinical decisions (Benner, 1982). The steps involved are not necessarily made clear to the novice learner leading to confusion and "shallow" learning. This style of teaching does not encourage reflection by the student on the complexities of the task and their individual learning needs.

In contrast, a social constructivist epistemology that was used to develop this program, argues towards a more adaptive and active view of knowledge. The student utilises a naturally complex real world situation in order to construct a personal knowledge of this skill. Structure is provided in the form of "scaffolding" that guides the student through the task to a new level of understanding (von Glasersfeld, 1995). The parents are the 'expert' coaches by providing assistance to the student in the best way to approach their child. They are able to fill in the gaps for the student in the behaviour of their child that cannot be elicited in the assessment. The parent also benefits by being provided with an insight into their child's development.

There is an increasing trend toward involving consumers in health education. Collaborative patient-centred practice is emerging as a framework for interdisciplinary education (D'Armour & Oandasan, 2004). Interpersonal competence, that is, an appreciation of the skill and uniqueness of all individuals involved, is required for this framework to be successful (The American College of Nurse-Midwives, 1998). The development of this competence in health care practitioners is one of the challenges for health educators interested in patient-centred practice.

Program context

The students are in the fifth year of their medical undergraduate program at the University of Western Australia. The program is undertaken whilst they are completing an 8 week unit in paediatrics and child health. Approximately 125 students complete the unit each year.

The students have prior experience in adult clinical examination and assessment. This is based on clearly stated instructional sequences. They have begun to develop a personal attitude to patient-family relationships. However, the paediatric environment requires a more flexible and intuitive approach and often challenges their prior experiences.

The program

The outcomes state that the student will be able to: The program involves a one hour lecture on child health and development, a 30 minute tutorial on practical and theoretical aspects of screening for growth and development, a practical assessment which the student organises themselves and a written report. The tutorial provides an opportunity for the students to ask questions and clarify concerns.

For the practical assessment the Stycar developmental milestones, appropriate toys, growth charts, and ASQ forms are provided. The students are required to weigh and measure the child, to complete a developmental assessment and to request the parent to complete the ASQ. The short reflective report is a summary of their findings, and a comparison between their assessment and the parents. They are asked to reflect on any observed differences and their experience of the process. The students requested the report be marked as a pass/fail and feedback given.

Evaluation

A descriptive and an interpretative analysis of the reports were completed to evaluate the program. The current Students Perception of Teaching Surveys (SPOTS) was compared with the previous year's SPOTS to see if there was a change in the mean scores. The 6th year survey will be compared for improvement in the area of confidence in the performing assessment of a child's development when the next survey becomes available.

Results

Sixty three percent of the growth measurements were charted accurately (< 25th centile discrepancy). One student commented "the most difficult thing was measuring, I had to get help from a nurse, I can see results vary on who is measuring". Thirty five percent made appropriate comments regarding growth parameters. Some extreme measures were not commented on, for example, a length far greater than the 97th centile, head circumference <2nd centile, some were typing errors (905cms, 807cms!). Only half the students recorded the head circumference.

The actual ASQ score was recorded by 72% of students. Agreement between student assessment and ASQ report was recorded in 70% of the assessments. In the reports where a discrepancy was noted, parent confirmation or disagreement was discussed and explained in 75% of the reports. If the parent was not aware of any problems identified by the students, and the ASQ score was below the cut-off score, then the child's doctor was requested to review the child at the next appointment.

Reasons for the discrepancy between the student's assessment and the parent's report varied. The parent often reported the child's communication was not their usual behaviour. The ASQ score usually supported this. At other times, the discrepancy revealed parental concern rather than actual delay. One child's parents were concerned about the child's poor pronunciation of words (she had recently had grommets inserted), however the child scored 60/60 on the communication sub domain of the ASQ and the student did not assess any language or communication deficit.

Key themes

Increased knowledge of milestones

The students comments demonstrated that the exercise increased their awareness of the range of normal development and what was considered an acceptable cut off score. The interaction with the parents also increased the students understanding of the parent's perceptions or feelings, for example, the parents were happy to know their child's development was similar to their peers, and what parents think are normal milestones. By using the ASQ, there was increased reporting of play, socio-emotional and problem solving. Students often prefaced assessment with good observational descriptors of the child's temperament and interaction.
A happy and interactive child who initially engaged with me, got tired and bored and turned to mother for a hug.

Increased knowledge of assessment tools

The tools were generally found to be useful but limitations were discovered.
Now understand I can do a valid and valuable appraisal using simple tools.
For example, some tools depended on operator skills and the child's cooperation. The ASQ was reported as more helpful, clearly set out, age-specific, allowed for graded responses, gave specific examples, easy to undertake, and simple to interpret. Overall students commented it gave similar results to observer report.
The structured approach of ASQ enabled me to direct my attention.

The ASQ gave me a better mental picture.

Awareness of techniques and engagement of the child

The student comments revealed they gained a lot from this interaction with the child. They learnt about observing the child at play, the challenge of creating rapport before attempting the assessment, about the need for flexibility of tasks, the importance of preparation, using play as a test tool and the realisation that how you say things can influence the reply.
They will naturally do a lot of tasks you want them to do.

I was interested to see trust develop in the child, happy to be weighed without mother despite very shy at first.

I took non-structured approach as I soon realised that would be impossible.

I learnt not to play ball games before less exciting activities.

No chance to stop and read through milestones while a very active toddler is constantly running away to go and play with other kids.

Easy to structure the play to test specific components.

Increased confidence in recognition of normal development and problems

All students found the exercise increased their confidence in assessment of development.
Despite initial hesitations, this has been useful exercise; actually doing has cemented the ideas more firmly in my mind and increased my confidence in assessing and interpretation.

I can now explain results more succinctly.

Parents as reliable and valid reporters

Two sub themes emerged from the data. Firstly, recognition of the parent as expert in their child as they spend the most time with their child.
I found the child hard to engage verbally, he rarely followed instructions and generally ignored the test administrator. (ASQ indicated poor verbal comprehension), I felt comfortable accepting the parent's input, as they were more used to his speech and mannerisms, in this way, I felt I gained a more accurate picture of his communicative development.
Secondly, the students reflected on potential parental bias. Some of the students commented on the parent's ability to complete a reliable and valid assessment of their child. In some cases this was appropriate, for example when the parents expressed concern about their child's diminished visual acuity following cataract surgery, however no gross visual deficit was observed and the ASQ was well above cut off for vision. However, some comments revealed the student's perception that parents would over-estimate their child's ability rather then give an objective assessment. This was even the case when the ASQ demonstrated good concordance with the student's assessment.
(The) mother seemed to have a good grasp on the developmental level, I understand this may not be so in many other mothers, especially those with developmentally delayed children.

Even though ASQ more specific, I still think observer's examination more useful, mother may have bias or mislead the examiner, whilst physical examination allows clinicians to identify problems without mother's bias affecting the result. ASQ can fill in the gaps, which may be true at home.

One child had a speech and language delay on ASQ (the child's problem solving was borderline, there were no intelligible words, no recognition of pictures, tended to ignore the mother) but the student found the child's hearing normal.
The mother may have overemphasised her child's communication ability and seemed intent on making him do what was requested, even interrupting his play, perhaps I needed to clarify any concerns the mother had.

Enjoyment of the process

I enjoyed the opportunity to perform this assessment, because in many ways it relies on actually joining in with the child's play, something that I always enjoy given the chance!

I enjoyed completing the assessment; it provided an opportunity to develop skills with interacting with children and a better understanding of developmental age of a 3 year old.

Evaluation

The SPOT feedback showed a significant increase in the mean for students' assessment of the relationship between teaching and learning methods and unit outcomes. Although not specifically related to this program, it does suggest that this type of teaching makes the learning process more transparent to the students. There was no significant increase in the mean for the outcome of "I now feel confident in assessing a child's growth and development". This is of interest in that the 45% who felt confident in assessing the growth and development of a child in 6th year, reported a mean confidence of around 4.00 (range of 1-5, 1= not confident, 5=very confident) for this skill in year five SPOTS last year. It is unclear whether their confidence has decreased over time, or that increased exposure to children has decreased their confidence as they become aware of the complexities of the skill. A study by Fadlon et al (2004) on teaching communication skills to medical students using a formal structured approach found a decrease in students' self confidence following the program. The authors attributed this to reflexivity and introspection in the program caused over-confident students to reassess their skills. Under confident students were offered a structured lifeline in the form of a model, which they can practice (Fadlon, Pessach & Toker, 2004).

Discussion

The program has expanded the students' knowledge of developmental milestones and the limitations and benefits of two tools. It has increased the student's ability to play with children and interact with their families. There has been an increased recognition of parents as partners in the assessment process. The students have reported enjoying this experiential learning despite some students initially expressing doubts as to its value.

The lack of precision in documenting and reporting the measurements is of concern. It is unclear whether this represents a lack of concern on the students' behalf or difficulty in the plotting on the graph. In the case of the ASQ scores, which require no graphing, it most likely indicates that the students do not consider it important to document the actual scores. However in medicine concise and objective documentation is essential, both for interpretation by other professionals and for legal purposes. This needs to be made clear to the students. Feedback on their submitted work is essential and in this case was student driven.

The students generally preferred using the more structured ASQ tool as it provided better guidance, which is appropriate for their level of competence as advanced beginners. It also reflects the superiority of the tool as compared with milestones. It is important to note that experiential learning requires structure, and that the complexities of the skill being learnt require the components being made clear to the learner (Kolb, 1984). This is sometimes overlooked in apprenticeship style learning in clinical practice. The tool itself became part of the learning and the parent acted as a coach for the student in relating to the child. They were able to help the student complete a skill that may be slightly beyond their ability without support. This is the process of scaffolding that is central to constructivist theory (Vygotsky L, 1978).

However, when implementing a program that is student centred and addresses attitudes, one of the disadvantages is the teacher may have to accept some loss of control on the outcome. The results showed a 70% agreement between student and parent. This is much lower than the 94% reported in ASQ reliability studies. The most likely explanation is the student's lack of experience and is demonstrated by the students reporting that ASQ scores supported the parent's assessment in many (but not all) cases.

Moreover, attitudes are divergent knowledge; there is not a clear right or wrong outcome. Being challenged on and subsequent reflection on personal experiences creates attitudes (Kolb, 1984). In this case, the student's perception of the parent's ability to assess their child's development using a reliable tool that asks specific behavioural measures cannot be marked 'wrong'. Expert agreement of 94% for the ASQ does mean that some parents will come to a different conclusion to an expert. It will also vary with the parent's ability and attitude, which in turn is influenced by past experiences. Whether the results are as anticipated, this method of teaching increases interaction between the student and the family, raises students self awareness and demonstrates to the students the need for flexibility and observation in child health practice.

The comments reflect this to be a more powerful learning experience than being told the maxims of clinical practice with children and their family, namely, "parents know their child the best" and "most of your assessment will be done by observing the child". Thus the process of learning and concept development can be powerful outcomes in themselves. It illustrates 'double loop learning', where the whole activity is part of a larger cycle, in which the reflection takes place by engaging in the activity and the assumptions implicit in it (Boud, Keogh & Walker, 1985). Of note, when there was a difference in assessment results, the students reflected on the situation and used a problem solving approach to identify what may have been real concerns for the mother. This is an example of the student using what Schon (1983) has described as reflection-in-action (Schon, 1983). Students reflecting-on-their-action when writing the report often concluded that a proper explanation of the assessment not being a pass-fail test and variability of reaching milestones should reduce any perceived tendency of parent to overestimate abilities of the child.

It appears that at this stage of evaluation, the program's effectiveness is increased knowledge of the process, in clarity of teaching and learning methods, concept development and increased enjoyment of interacting with children and their parents. These are likely to have more long term benefits than confidence in the actual skill at this stage of their careers. Could this model of teaching and learning be explored as 'relationship-centred' teaching?

Future research should explore the parents' perceptions of their role in teaching the students, their perceptions of the students learning in this type of program and a reliable measure of the students' competence in the skill. This knowledge would help direct future development of the program.

References

American Academy of Pediatrics (2001). Developmental surveillance and screening of infants and young children. Pediatrics, 108(1), 192-195.

Benner, P. (1982). From novice to expert. American Journal of Nursing, 402-207.

Blasco, P., Kohen, H. & Shapland, C. (1999). Parents-as-teachers: Design and establishment of a training program for paediatric residents. Medical Education, 33(9), 695-704.

Boud, D., Keogh R. & Walker, D. (1985). Reflection: Turning experience into learning. London: Croom Helm.

Centre for Community Child Health, Royal Children's Hospital Melbourne (2002). Child health screening and surveillance: A critical review of the evidence. National Health and Medical Research Council. [verified 26 Jan 2005] http://www.nhmrc.gov.au/publications/pdf/ch42.pdf

Child and Youth Health Intergovernmental Partnership (2002). Child health screening and surveillance: Supplementary document - context and next steps. National Public Health Partnership. [verified 26 Jan 2005] http://www.dhs.vic.gov.au/nphp/publications/chip/screening.pdf

D'Armour, D. & Oandasan, I. (2004). Interprofessional education for collaborative patient-centred practice: an evolving framework. [retrieved 7 Oct 2004] http://www.hc-sc.gc.ca/english/hhr/chapter10.html

Fadlon, J., Pessach, I. & Toker, A. (2004). Teaching medical students what they think they already know. Education for Health, 17(1), 35-41.

Kolb, D. A. (1984). Experiential Learning: Experience as the source of learning and development. New Jersey: Prentice-Hall.

Newble, D. I. (1992). Assessing clinical competence at the undergraduate level. Medical Education, 26, 504-511.

Owen, C. & Reay R. E. (2004). Consumers as tutors - legitimate teachers? BMC Medical Education, 4(14).

Paton, J. Y. & Cockburn, F. (1995). Core knowledge, skills and attitudes in child health for undergraduates. Archives of Disease in Childhood, 73(3), 263-265.

Schon, D. (1983). The reflective practitioner. How professionals think in action. London: Temple Smith.

Squires, J., Potter, L. W. & Bricker, D. (1990). The ASQ User's Guide for the Ages and Stages Questionnaire: A parent-completed, child monitoring system. (2nd ed.). Maryland, Baltimore: Paul Brookes Publishing Comp.

Stacy, R. & Spencer, J. (1999). Patients as teachers: A qualitative study of patients' views on their role in a community-based undergraduate project. Medical Education, 33, 688-694.

The American College of Nurse-Midwives (1998). Building community: Developing skills for interprofessional health professions education and relationship-centred care. Journal of Nurse-Midwifery, 43(1), 61-65.

von Glasersfeld, E. (1995). A constructivist approach to teaching. In Steff L & Gale J (Eds.), Constructivism in eduction (pp. 3-16). New Jersey: Lawrence Erlbaum Associates, Inc.

a Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. MA: Harvard University Press.

Woloschuk, W., Harasym, P. H. & Temple W. (2004). Attitude change during medical school: A cohort study. Medical Education, 38, 522-534.

Author: Pam is currently a Senior Lecturer and Clinical Educator for undergraduate medical students in the School of Paediatrics and Child Health at the University of Western Australia Faculty of Medicine and Dentistry. She is a registered nurse and educator who has worked with children and their families since 1978 and has a Master of Public Health.

Ms Pam Nicol, Clinical Educator
School of Paediatrics and Child Health
The University of Western Australia
Princess Margaret Hospital, PO Box D184, Perth WA 6840
Tel: 08 9340 8943 Fax: 08 9388 2097 Email: pamelan@ichr.uwa.edu.au

Please cite as: Nicol, P. (2005). Structured experiential learning: The Ages and Stages Questionnaire is effective as a teaching tool for medical students. In The Reflective Practitioner. Proceedings of the 14th Annual Teaching Learning Forum, 3-4 February 2005. Perth: Murdoch University. http://lsn.curtin.edu.au/tlf/tlf2005/refereed/nicol.html

Copyright 2005 Pam Nicol. The author assigns to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format (including website mirrors), provided that the article is used and cited in accordance with the usual academic conventions.


[ Refereed papers ] [ Contents - All Presentations ] [ Home Page ]
This URL: http://lsn.curtin.edu.au/tlf/tlf2005/refereed/nicol.html
Created 26 Jan 2005. Last revision: 26 Jan 2005.