Teaching and Learning Forum 99 [ Contents ]

Measuring the social responsiveness of medical schools and health professional institutions

Max Kamien
Department of General Practice
The University of Western Australia
Society is demanding more social responsiveness and accountability from all its institutions including medical schools. This requires medical schools to accept social responsiveness as a fourth obligation, alongside the traditional three tasks of teaching, research and clinical service. We maintain that medical schools should accept some responsibility for shortcomings in the quality of their graduates, their distribution and their impact on the burden of ill-health in the community.

This paper builds on previous work by WHO on defining and measuring the social responsiveness of medical schools with regard to relevance, quality, cost effectiveness and equity. We suggest a number of relatively simple measures by which a school could evaluate itself or measure change in its own level of social responsiveness to the priority health care needs of its constituent community.


Measuring the social responsiveness of medical schools

Medical schools and academic health centres throughout the world take their traditional tasks of teaching, research and clinical service seriously. But some have been less willing to accept a fourth obligation which is to be socially responsive to the society in which they are situated and which provides most of their funds (Edinburgh Declaration, 1988; Petersdorf and Turner, 1995). Critics of medical education have long described this as a serious disjunction between doctors' education and societal need (Association of American Colleges, 1932 and 1984).

Society is increasingly demanding more responsiveness and accountability from all of its institutions and is asking medical schools to be part of that process. Society wants accessible, competent doctors who are more knowledgeable about common problems than esoteric ones. It also wants enough doctors to serve the needs of special groups such as the elderly, the mentally ill, the deprived and the geographically isolated and to do so in the most cost effective manner (Ewan, 1985; EFPO, 1992).

There have been notable attempts at fulfilling these needs such as the creation of a global network of community based institutions (Network of Community Oriented Educational Institutions for Health Sciences, 1986) as well as blueprints for future educational action (General Medical Council, 1993; Association of American Medical Colleges and American Medical Association, 1993; Pew Report 1993; Australian Medical Council, 1998). Many other medical schools are also beginning to devote some of their energies in education, research and service not only towards improving quality of care but also to the issues of relevance, cost effectiveness and equity in health care for the community, region and nation they have a mandate to serve (Boelen and Heck, 1995). Indeed the World Health Assembly - the governing body of the World Health Organisation - unanimously affirmed this direction in its 1995 resolution, WHA48.8, "Reorientation of medical education and medical practice for all" (WHO, 1995).

It is not the intention of this paper to impose on the diversity or academic autonomy of a medical school by giving a detailed view of what or how something should be taught. The aim is to inform the existing positive mission of a medical school and enable it to assess to what degree it is exposing students to important topics and issues which affect patients, taxpayers, health departments and which are necessary in providing good, equitable, affordable health care for most of the people most of the time.

This paper builds on work already done by WHO on defining and measuring the social responsiveness of medical schools (Boelen and Heck, 1995). It produces a number of relatively simple measures by which a school could evaluate itself and measure change in its own level of social responsiveness or through which an accrediting body could compare one institution with another or against some desired norms.

Method

The WHO internal monograph on defining and measuring the social accountability of medical schools was used as a starting point (Boelen and Heck, 1995). It proposes a social accountability grid which measures the four values of relevance, quality, cost effectiveness and equity through the domains of education, research and service and for each domain focuses on the phases of planning, doing and impacting (see figure 1).

Figure 1: A social accountability grid

Values Domains and phases
EducationResearchService
Planning DoingImpacting PlanningDoing Impacting Planning DoingImpacting
Relevance








Quality








Cost effectiveness








Equity








The grid is a strategy by which a medical or health professional school could examine its level of social accountability
This document was followed by the production of a number of social responsiveness questionnaires for use with faculty, students, administrators and community health care providers and receivers. These too were culled and further items added which we think can be used to efficiently measure the level of social responsiveness of a medical school. Although tables of suggested measurements are available for each of the values and domains this paper concentrates on the domain of education since it is (or should be) the main mission of a medical school.

Relevance

This has been defined as the degree to which the most important health care problems are tackled first. Thus a priority is placed on the most important local and national health issues, significant and modifiable ailments and on people with the highest burden of ill-health.

Table 1 describes measures of relevance which should either be readily available or which, with no more than moderate effort, can usually be obtained.

Table 1: Relevance in education

The relevant
curriculum
How can it be measured?
PlanningDoingImpacting
Places a priority on:

the 10 most important national and local health priorities

the most modifiable significant ailments in the community

those with the highest burden of ill-health

intent for relevance in mission statement and curricular objectives

demonstrates that national or local data on priority health needs are used to update curriculum at least every 10 years

proportion of pass/fail examinations devoted to priority issues

amount, adequacy and funding for clinical teaching in ambulatory and community settings

proportion of randomly selected students who can identify priority problems and % reporting competence and confidence in dealing with them

provincial data on changes in health indicators about the 10 priority areas

The most controversial measure is likely to be about the impact of education, research and service on outcome measures and changes in local and provincial health statistics. Of course there are many variables and confounding factors beside education which influence outcomes. But if, for example, immunisation rates in a country are poor, medical education should accept some responsibility. It should be a priority topic in undergraduate and postgraduate education and research and, where a university runs a service practice, it should demonstrate how it may be done better.

Quality of care

Correct diagnosis and treatment is the sine qua non of medical care. But quality of care extends beyond this to the ability to think critically, practise evidence based medicine, communicate efficiently and empathically, and behave ethically. Table 2 describes measures by which the influence of education on quality of care can be assessed.

Table 2: Influence of education on delivery of quality care

Education for quality careHow can it be measured?
PlanningDoingImpacting
Places a priority on:

preparing graduates to assess and provide quality of care

mission statement which includes desired curricular outcomes in terms of quality of care competencies eg, ability to think critically, communicate efficiently, behave ethically and continue self directed learning evidence of consistency in teaching across disciplines based on:

evidence based medicine and/or

inter-departmental consenses for 10 most common conditions found in primary medical care

interview randomly selected students on their awareness, knowledge, attitudes and skills about evidence based medicine

feedback on performance of graduates from hospital administrators who employ them and from professional bodies which provide vocational training and education

medical audit of a random sample of graduates in primary medical care on management of 10 most common conditions and provincial data on the application of effective preventative medicine, eg. immunisation, cervical smear rates

In the domain of education a common finding, especially in traditionally organised medical schools, is for students to have to learn different answers to the same question in order to satisfy teachers from different medical disciplines. One common example is the diagnosis and management of the sore throat. Students can receive different teaching from departments of ear, nose and throat, paediatrics, microbiology and family medicine. Therefore one measure of education for quality care would be evidence of teaching across disciplines preferably based on research evidence or if that is lacking on interdisciplinary consensus (Table 2).

Cost effectiveness

Cost effectiveness is getting the maximum benefit from the use of scarce resources. It is a topic that occupies much time and effort in the health departments of all developed and developing countries. Yet of all the four values being examined it is our observation that cost effectiveness is neglected in most courses for future doctors and health care professionals. A medical school's commitment to this issue can be obtained by applying the measures in Table 3.

Table 3: Education for cost effectiveness

Education for cost effectivenessHow can it be measured?
PlanningDoingImpacting
the teaching of all pertinent departments in the medical school places emphasis on cost effective care is cost effectiveness a topic on curriculum?

is evidence based medicine used to inform this teaching?

which departments teach it?

ongoing topic in clinical training?

a topic in final year written and clinical examinations?

5-10% random selection of medical students: are they aware of it having been taught? Does it influence the way in which they practise medicine?

where available, comparative cost data on medical school's graduates for:

investigations
prescribing

In some countries, such as Australia, comparative data on the investigation, costs and prescribing habits of the graduates from the 10 medical schools are available. One school produces graduates who generate greater costs for drugs. The reasons for this are still under investigation. Such data have a major potential to influence the teaching of medicine.

Equity

Equity focuses on the delivery of high quality health care to all people, especially those medically underserved and economically and/or geographically deprived. It also means equity of access to a medical education for members of those deprived groups (Table 4).

Table 4: Equity in education

Equity in educationHow can it be measured?
PlanningDoingImpacting
The medical school is concerned with equity of access to a medical education for all population groups affirmative entry policies for members of under-represented population groups

marketing of career in health services to potential entrants from under-represented communities

availability of financial assistance to help prevent exclusion of meritorious students from deprived backgrounds

% of students admitted:

female
rural
minority groups

compared to % in provincial population

% of all minority groups in graduating classes
challenges students to provide care to deprived and medically underserved populations programs to encourage students to learn about continuity of care through care of deprived patients over time are students exposed to deprived and/or underserved groups?

% students who are regular members of student societies eg, rural, overseas

medical school provides information on physician supply and societal need to help career choice

what proportion of doctors providing designated care to underserved populations in the province, eg,. indigenous, rural, elderly, mental health, are graduates of the medical school

fulfilment of need for various types of doctors in province eg, 1 surgeon/25,000 people; 1 primary care doctor/2,500 people; 1 psychiatrist/20,000 people

A socially responsive medical school should also have a place at the leading edge of health care delivery, both as a model for demonstrating care for the future rather than the past, and as a means of challenging future graduates to provide appropriate medical services to deprived populations.

Use of social responsiveness indicators

To illustrate the use of a social responsiveness indicator, we focus on the value of equity. We suggest that a Likert scale be used to measure each cell in Table 4, where:

4meets all criteria and is a model of success
3meets most criteria but some refining of programs is necessary
2meets some criteria but much still needs to be done
1attempts at planning are underway but nothing concrete has yet begun
0nothing planned, nothing started

Thus a school could have a maximum equity score of 24. We maintain that a responsible and responsive medical school with an equity in education score of less than 50% would, at the very least, review and debate its performance and would hopefully make some necessary changes.

The domains of research and service

A similar set of indicators to those used for Education are available for Research and Service. Their contents are summarised in the following two paragraphs. Their full contents are available on request.

Research
Since there is already a lot known about the most common conditions and health priorities, research in these areas is more likely to be applied rather than basic. The level and amount can be measured by asking a faculty whether it has a research plan to encourage some members to do research in the four values of relevance, quality of care, cost effectiveness and equity and whether national health priorities and the views of the broad community are taken into account in identifying applied research priorities. Similarly process measures would include the number of refereed research papers produced in the last five years which address priority areas. Outcome measures would include any evidence that such research impacts on satisfaction with access to and quality of care, local priorities, priority health programmes and priority health status.

Service
Social responsiveness in service can be measured in a similar manner to that of research. For example, what plans and processes does the faculty have to assist in providing services to people and areas of greatest need and to educational and locum support for isolated doctors? How many ambulatory clinics does it support in providing care to the deprived and to special groups with a high burden of poor health such as those with AIDS, the elderly and the mentally ill? What proportion of staff contribute their expertise to national and provincial health authorities? Outcome measures would include the degree to which these clinics meet cost effective targets such as a 90% rate for cervical cancer smears, skin cancer examinations, blood pressure control and full immunisation. Measures of impact would also include any evidence of the effect of role modelling on other health care providers and documentation of change in access to health care by underserved populations.

Discussion

The social responsiveness grids and the accompanying measurement tools can be used in several ways. Firstly they are useful for a school which wants to examine its performance in these areas. The information may be useful to a wise Dean in shifting the direction of the school in which some departments are inflexible or unresponsive. Secondly, they can be used, with care, as a basis of accreditation and quality assurance and thirdly, and of most potential use, to document change in a medical school's performance over time.

It is recognised that some of the measures may not be readily available, or if they are available may not be accurate, for example, the proportion of graduates of a school providing care in underserved geographical or disciplinary areas. In such cases a desirable implication of using these measures will be the necessity to improve the amount and quality of information that is available.

Social responsiveness must also be measured within the socio-cultural context of a country and of a medical school. Some measures may not be appropriate or applicable to all countries or all medical schools. For example, the needs of poor countries may be better served by epidemiological studies to define health care priorities or to monitor health programs rather than focussing on issues of quality of care or cost effectiveness.

The aim of this paper is to produce measurement tools which can help medical schools to measure their levels of social responsiveness in meeting the particular health care priorities of their constituent communities. Clearly the tools used here to measure social responsiveness need refinement and since the main mission of the medical school is education, the items ultimately need to be weighted in terms of their importance, validity and reliability. Attention must also be paid to the sustainability of the measures to reflect a commitment from most departments and not just those with an orientation to community based primary health care. The intention of this paper is to start discussion and action in such a process.

References

Association of American Medical Colleges (Rappleye, W.C., director) (1932). Medical Education. Final Report of the Commission on Medical Education, New York: Association of American Medical Colleges.

Association of American Medical Colleges (1984). Physicians for the twenty-first Century. Report of the Project Panel on the General Professional Education of the Physician and College Preparation for Medicine (The GPEP Report). Washington, Association of American Medical Colleges, 1984. Journal of Medical Education, 59, 1-208.

Australian Medical Council (1998). Guidelines for the Assessment and Accreditation of Medical Schools. Canberra: Australian Medical Council.

Boelen, C. and Heck, J. E. (1995). Defining and measuring the social accountability of medical schools. WHO/HRH/95.7. Geneva, World Health Organisation.

World Conference on Medical Education (1988). The Edinburgh Declaration. Lancet, 2, 464.

EFPO Coordinating Centre (1992). Educating future physicians for Ontario. Executive summary of the component interim report. Hamilton, Ontario: McMaster University.

Ewan C. (1985). Objectives for medical education: expectations of society. Medical Education, 19,101-102.

General Medical Council, United Kingdom (1993). Tomorrow's doctors: Recommendations on Undergraduate Medical Education. London: General Medical Council.

Health Professions Education for the Future: Schools in Services to the Nations (1993). Report of the Pew Health Professions Commission. San Francisco: The Pew Health Professions Commission.

Liaison Committee on Medical Education (1993). Functions and structures of a Medical School: Standards for Accreditation of Medical Education Programmes leading to the MD Degree. The Association of American Medical Colleges and the American Medical Association.

Petersdorf, R. G. and Turner, K. S. (1995). Medical Education in the 1990s and beyond: A view from the United States. Academic Medicine, 70(7 Suppl ), S41-S47.

The Network of Community-Oriented Educational Institutions for Health Sciences (1986). The Network of Community-Oriented Educations Institutions for Health Sciences. A Short Description of Its Aims and Activities. Maastricht: The Network of Community-Oriented Educational Institutions for Health Sciences.

World Health Organisation (1995). Reorientation of medical education and medical practice for health for all. World Health Assembly Resolution, WHA48.8. Geneva, World Health Organisation.

Professor Max Kamien, Department of General Practice, The University of Western Australia, 328 Stirling Highway, Western Australia 6010

Please cite as: Kamien, M. (1999). Measuring the social responsiveness of medical schools and health professional institutions. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context, 184-192. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/kamien.html


[ TL Forum 1999 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1999/kamien.html
Last revision: 26 Feb 2002. The University of Western Australia
Previous URL 17 Jan 1999 to 26 Feb 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/km/kamien.html