Teaching and Learning Forum 99 [ Contents ]

An investigation into the predictive validity of the IELTS Test as an indicator of future academic success

Patricia Dooey
Centre for International English
Curtin University of Technology
In recent times there has been an increase in the number of overseas students choosing to study in universities throughout Australia. As a result, the issue of eligibility criteria has become an important one, with the assessment of proficiency in the English language being a key aspect. At present one of the most widely-used tests for this purpose, is the International English Language Testing System (IELTS).

The current study seeks to answer two questions:

  1. Is IELTS an accurate predictor of performance and success in Business, Science and Engineering?
  2. Is the current minimum of Band 6.0 overall on the IELTS test, sufficient to ensure success in these schools?
It was conducted as a small-scale quantitative study carried out among first-year undergraduate students at Curtin University of Technology, Western Australia. The students were from diverse non-native English speaking backgrounds, and were admitted to the Schools of Business, Science and Engineering in Semester 1, 1997, on the basis of their IELTS score.

While the findings do not offer conclusive evidence about the validity of IELTS as a predictor of academic success, it would appear that, of the four modules, Reading shows the strongest correlation. Furthermore, while language is but one of many important factors contributing to academic success, it plays a less significant role at the higher levels of proficiency.

The paper concludes with a discussion of the limitations of strictly applying quantitative measures when making decisions regarding readiness to study in an English medium university. The recommendations offer suggestions for further research, and for practical applications which would facilitate student support services in this area.


Introduction

Curtin University of Technology currently has the highest proportion (15%) of overseas students in Western Australia, with a plan to increase this figure to 25% as part of its internationalisation strategy. It is Australia's third largest enrolment institution for overseas students, and was recently voted the number one university of technology in Australia by Asiaweek, one of the region's premier business magazines. Because of its reputation, many students from South-East Asia are attracted to Curtin, particularly its Business school. The Centre for International English is the Western Australian IELTS Administration Centre.

The IELTS test is used as an indicator of English proficiency, both for overseas (international) students, and those who come through the Tertiary Institutions Service Centre (TISC) system. The latter are mostly school leavers who have failed to meet English language requirements in the Tertiary Entrance Examination. Although IELTS is intended for non-native speakers, TISC has also chosen to accept an IELTS Test Report Form from native speakers under these circumstances.

Prospective students must obtain an overall band score of 6.0 on IELTS, in order to be admitted into Curtin. This is in contrast to the other Western Australian public universities, which require an overall IELTS band score of 6.5. Curtin, along with the other tertiary institutions, sets down further minimum requirements in each of the four macro skills: listening, reading, writing and speaking.

The receiving institutions nominate the period of validity for an IELTS Test Report Form. For most, this is either one or two years. Curtin University, however will accept a student's IELTS score for an indefinite period. This means that a prospective student may meet English proficiency requirements on the basis of an IELTS Test Report Form which is more than two years old.

In the current economic climate, there is increasing pressure on universities to accept more overseas full fee-paying students. This requires some flexibility with regard to minimum entry requirements. If students who are borderline, or even below the recommended level in terms of English proficiency, are accepted, the onus falls upon the university to provide the necessary English support in order to augment these students' chance of success. As there is no fixed 'pass' or 'fail' in IELTS, it is suggested that each receiving institution make its own informed decision regarding the appropriate level of English required for the different courses it offers.

An earlier study at Curtin University investigated the possible correlation between IELTS scores and academic success among 61 NESB (Non English speaking background) students studying in the disciplines of Business/Social Science and Science, respectively. Qualitative analysis of the data indicated that a majority of students attributed their success to the fact that in Maths orientated units, less language was required. Conversely, low achievement in a particular unit was considered to be primarily due to the fact that the unit was 'difficult to understand because of too much language/writing' (Fiocco, 1992, p.12).

Therefore, for purposes of comparison in this study, it would seem appropriate to select courses which are deemed 'linguistically demanding', for example, Business; and 'less linguistically demanding', for example, Science and Engineering, and those which have a substantial proportion of overseas students.

Predictive validity of language tests

Much of the literature available in the area of the predictive validity of language tests in general seems to share certain common conclusions: that 'English proficiency is only one among many factors that affect academic success', (Graham, 1987). Further common conclusions are that the relationship between English proficiency, as measured by a standard test, varies when students are grouped according to academic discipline; and that students who score higher on such a test have a greater chance of future academic success.

Several predictive validity studies have been carried out in relation to IELTS, and while some indicated that there was some correlation in the case of individual modules, most did not find a strong overall positive correlation between IELTS scores and subsequent academic success.

Notwithstanding the results of previous studies, it is pertinent for each institution to set down its own admissions criteria, in relation to the amount of English support available on campus, and the English language requirements may need to be judged in relation to the strength of other qualifications, for example, maths scores or portfolios.

For the purpose of this investigation, then, two major questions emerge:

  1. Is IELTS an accurate predictor of first year performance and success in the Schools of Business, Science and Engineering?
  2. Is the current minimum of Band 6.0 overall on the IELTS test, sufficient to ensure success in these schools?

Participants

The entire population of this study comprised a total of 89 students who entered first year of an undergraduate course at Curtin University in Semester 1, 1997 on the basis of an IELTS score.

They had been enrolled in the disciplines of Business, Science and Engineering. Details of students' nationality and first language were noted from the information they gave when applying for the IELTS test. In all, 15 nationalities, and 13 first languages, were represented. A total of 23 students indicated that their nationality was Australian, and their first language English. For the purpose of this study, these 23 students are hereafter referred to as 'native speakers', although in terms of eligibility to sit for IELTS, the definition of 'native speaker' remains unclear. Since IELTS is intended to test for readiness of non-native speakers to train or study in the medium of English, it was decided to eliminate this group from the correlations, and to record their scores separately. One other student who scored well on IELTS, appeared to have withdrawn from her course early in first semester, and was eliminated from the study.

Most of the 30 Business students did a 'common core' first year, but a variety of courses was represented among the Science (21 students) and Engineering (14 students) faculties.

Procedure

The students in the study fell broadly into two categories; those who applied via the International Office (overseas students), and those who applied via the Tertiary Institutions Service Centre (TISC), who were school leavers and permanent residents. The English language proficiency criteria presented on admission, in this case IELTS scores, therefore had to be obtained from two separate sources according to the point of entry of the students.

Once the students could be identified by their alpha-numeric number, it was possible to obtain their semester-weighted averages (SWAs) from the university's statistics office. Since most of the candidates involved had taken IELTS at the Curtin centre, it was then possible to obtain further information from the IELTS database.

IELTS scores were recorded for the entire group first, then with the native speakers excluded. Details of the IELTS scores of the native speakers (all of whom met minimum English language entry requirements) and academic results were recorded on a separate table. A further table was generated to show the academic results and IELTS scores of those (n=20) who did not reach Curtin's English proficiency cut-off points. All correlations were calculated using the data from non-native English speaking students only.

Scatterplots were generated to show possible relationships between the following:

Frequency tables were also calculated for: Finally, correlations between the SWAs and IELTS scores (by subtest and overall), were calculated for the entire group, and by discipline.

Results/discussion

The study found no evidence to suggest that those students who did not meet Curtin's minimum English proficiency criteria, were destined to fail. In terms of the IELTS overall band score required, only two students failed to meet admissions requirements, and both passed their first year. It would appear, at least from this study, that Curtin's screening procedures are working, since the majority of the 'at risk' cases eventually passed in their first undergraduate year. Nonetheless, for those who are identified as needing English support, the procedure varies from one discipline to another. In other words, the admitting school may recommend that a student enrol in a concurrent English support unit, but not make it compulsory. Furthermore, some schools allocate credit points to such units, while others do not, in which case, students perceive them as an added burden on top of an already demanding workload.

Conversely, most of the clear failures were among the native speakers who came in with high IELTS scores. This would seem to indicate that, for this group at least, language was not a key factor contributing to their academic success. Since all of these students entered the disciplines of Science and Engineering (previously identified as 'less linguistically demanding'), it would seem that skills other than language proficiency are needed to ensure academic success in these disciplines. In fact, at these higher levels of proficiency in English, several other factors must come into play. This concurs with the findings of previous studies which found that high levels of English proficiency, as measured by the IELTS test, do not necessarily lead to academic success.

The only consistently positive correlation between IELTS scores and academic results showed in the case of the reading subtest, and this correlation was moderately significant (.396) in the second semester of Business, a discipline regarded as 'more linguistically demanding'.

However, no single group of students (Business, Science or Engineering), fared significantly better than another, in terms of overall achievement as measured by semester-weighted averages.

Conclusions

This study was initiated by the need to answer two questions; firstly whether IELTS is an accurate predictor of first year performance and success in the Schools of Business, Science and Engineering; and secondly whether the current minimum of Band 6.0 overall on the IELTS test is sufficient to ensure success in these schools. The issue of whether a high level of English proficiency can give any indication of future academic success is a very complex one.

The findings of this study suggest that overseas students who do not fully meet admissions criteria in terms of their language, may well have the potential to succeed academically. This factor strengthens the case for examining special cases separately. All students who came in with IELTS scores below the cut-off point, would have been assessed individually by the Matriculation Committee prior to being granted admission, and despite being considered 'at risk', they generally succeeded.

On the other hand, a different picture emerged in the case of the 23 native English speakers who are admitted on the basis of an IELTS score. All entered the 'less linguistically demanding' disciplines of Science and Engineering, but 15 of them failed to achieve a pass mark, despite their apparent lack of difficulty with the English language. It seems evident that in their case, more than a high level of English proficiency would be needed in order to pass. In other words, high IELTS scores alone did not guarantee success.

Finally, the academic success of the four students who commenced their courses more than two years after taking the IELTS test, was not in question.

Recommendations

It is not clear whether those students who had low English scores on entry enlisted the help of English support tutors either on campus or privately, in a bid to pass their first year. This factor must be taken into account when using this data in the future to confirm or change minimum English language entry requirements, as measured by IELTS.

It is therefore recommended that the following measures be implemented:

That all students' records, from admission to graduation, be recorded on a central university database. This would not only facilitate early intervention in the case of 'at risk' students, but would help future researchers, courses tutors and members of the Matriculation Committee.

That qualitative research be carried out to investigate the case of those students who did not meet minimum English entry requirements, yet succeeded academically; in particular, to establish the levels of English support obtained.

That data be collected relating to the factors other than English language proficiency, which affect academic outcome, as in the case of those students who easily met English criteria, but failed in their first year of undergraduate study.

That clearer guidelines be established whereby non-native English speaking students can be directed to the appropriate English language support units, and where possible, academic incentives be provided to ensure successful completion of these units.

References

Fiocco, M. (1992). English proficiency levels of students from a non-English speaking background: a study of IELTS as an indicator of tertiary success. Unpublished research report. Perth: Curtin University of Technology.

Graham, J. G. (1987). English Language Proficiency and the Prediction of Academic Success. TESOL Quarterly, 21(3), 505-521.

Please cite as: Dooey, P. (1999). An investigation into the predictive validity of the IELTS Test as an indicator of future academic success. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context, 114-118. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/dooey.html


[ TL Forum 1999 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1999/.html
Last revision: 25 Feb 2002. The University of Western Australia
Previous URL 14 Jan 1999 to 25 Feb 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/dj/dooey.html