|Perceptions of learning and assessment in beginners and intermediate level Italian Studies
Rebekah Sturniolo-Baker and Rocco Loiacono
[ Refereed papers ]
Much research has been conducted into devising assessment that is FOR learning, rather than OF learning. Such research suggests this can be best achieved by giving students regular, low-stakes assessment, which aids both the student and teacher in measuring progress and gives an opportunity for more feedback to be provided to the student, thereby creating a more inclusive learning environment. In our discipline, Italian Studies, there have been attempts to give practical application to this research by giving students assessment on a more regular basis. It was decided, therefore, to assign students regular in-class tests of 20 to 30 minutes duration.
In taking this step, it was believed that regular in-class testing can provide students with smaller milestones to work towards, as well as giving teachers a clearer idea as to student progress, so any weaknesses can be identified and worked on. While regarded in the literature as best practice, we wanted to determine both staff and, more importantly, student perceptions as to whether they believed it creates a more inclusive environment which is conducive to learning for all students. Our study suggests that students (as well as staff) very strongly believe that their needs are better served by having regular, low-stakes in-class assessment. This result lends ongoing support to the ideal of assessment FOR learning across disciplines.
Therefore, the challenge of devising assessment that is inclusive and engages students (Cooper 2006) is imperative. The bulk of recent research conducted suggests that an assessment for learning approach, which encompasses regular, low-stakes assessment, is better suited to achieving this goal. In the discipline of Italian Studies at the University of Western Australia, attempts have been made to put this theory into the practice of language teaching. By language teaching we mean students learning not only about all the various grammatical aspects of the written language, but also developing the necessary communicative skills (listening and oral comprehension and written and oral expression). A large amount of content is covered in language programmes. Therefore, any assessment format must take into consideration time constraints, as contact hours are less than those that have been in place in previous years. At a beginners' level, it was not uncommon in many Australian Universities for there to be around six contact hours a week, whereas these days there are only four at the most. This reduced time impacts on the ability of staff to review the course content in a more complete way in contact time. Students therefore must somehow make up for this lost time in their own study time by completing grammar exercises and the like. However, members of staff interviewed for this project have advised us that, in their view, more often than not, students do not put in the required extra time that is required when learning a language.
In light of the research which suggests that regular assessment is more desirable, which will be discussed further below, the first attempt at trying to adopt a model based on more regular assessment saw students at first year beginners and intermediate (ex-TEE/WACE) level being given a series of take home assignments to complete which would then be marked and returned to students in class. Whilst giving students scope to apply language competencies in a more creative way, the viability of these assignments as an assessment and learning tool was questioned by students and staff on the basis that it may not be an accurate measure of student progress, as take home assignments can lend themselves to overuse of dictionaries and internet resources, as well as student collusion. Further, given increasing time-constraints, regular in-class testing can provide students with smaller milestones to work towards, as well as giving teachers a clearer idea as to student progress, so any weaknesses can be identified and worked on. Thus, at both levels, take home assignments have been eliminated, or their weighting has been significantly reduced, with far greater emphasis placed on regular in-class tests of 20 to 30 minutes duration. In this way it was hoped that regular assessment would provide students with an opportunity to: 1) better understand the learning goals and the criteria that will be applied to judge the quality of their achievement; and 2) receive feedback that helps them make further progress.
To study perceptions of this approach, that is, whether regular in-class assessment is preferable to one longer assessment at the end of the unit or take-home assignments, we have therefore:
the term 'assessment' refers to all those activities undertaken by teachers and by their students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged (p. 2).From this definition the concept of formative assessment is made clear. Formative assessment refers to assessment that it specifically intended to generate feedback on performance to improve and accelerate learning (Sadler 1998), whilst summative assessment refers to an assessment that is designed purely to contribute to an overall grade at the end of the study unit. More emphasis has been place on formative assessment as it is argued that it helps students to appreciate the standards that are expected from them (Yorke 2003). In particular, learning is now more commonly conceptualised as a process whereby students actively construct their own knowledge and skills (Barr and Tagg 1995; DeCorte 1996; Nicol 1997). Further, Nicol and Macfarlane-Dick (2006) suggest that:
motivation and self-esteem are more likely to be enhanced when a course has many low-stakes assessment tasks, with feedback geared to providing information about progress and achievement, rather than high-stakes summative assessment tasks where information is only about success or failure, or about how students compare with their peers (e.g. grades) (p. 212).Gibbs and Simpson (2005) make the point that the assumption of most universities in the United Kingdom is that "frequent assignments and detailed written feedback are central to student learning" (p. 8). They go on to discuss the point raised by Vos (1991) that in those subject areas which have less frequent assessment tasks, students tend to study fewer hours (p. 13). In other words, students consciously study more often when they know that they are going to be assessed more often. In regards to the issue of on-going assessment, Gibbs and Simpson (p. 14) point out that:
Exams can have the effect of concentrating study into a short intense period at the end of the course with little study of, for example, lecture notes, until many weeks after the lecture. Frequent assignments .... or tests ..... can distribute student effort across the course, often on a weekly basis, ...This formative assessment approach has also been explored by Yorke (2003). He argues that some assessments are deliberately designed to be simultaneously formative and summative - formative because the student is expected to learn from whatever feedback is provided and summative because the grade awarded contributes to the overall grade at the end of the study unit. In his view, summative assessments in relation to a curricular component (the student passes or fails a module, for example) can act formatively if a student learns from them. Yorke (2003) also suggests that:
The act of assessing (formally and informally; formatively and summatively) has an effect on assessors as well as on students. Assessors learn about the extent to which students have developed expertise, and can tailor their teaching accordingly (p. 482).Another related concept to the issue of regular in-class assessment is the notion of feedback. Sadler (1989) describes feedback as "a key element in formative assessment, and is usually defined in terms of information about how successfully something has been or is being done" (p. 120). He maintains that students can "use it to monitor the strengths and weaknesses of their performances, so that aspects associated with success or high quality can be recognised and reinforced, and unsatisfactory aspects modified or improved" (p. 121). The benefits of regular feedback are crucial to students' positive progress through a course; and more frequent assessment means that there are more opportunities for teachers to provide the students with feedback. With the use of regular tests in the Italian units discussed in this paper, students can be given feedback from as early as week four in the semester, and then every three weeks after that, which means that they have a significant amount of time to reinforce their strengths and work on any weaknesses before the final exam.
This correlates with Gibbs and Simpson (2005), who state that "feedback may need to be quite regular, and on relatively small chunks of course content, to be useful" (p. 17). Students may find it easier to take on board what has been said if it refers to smaller sections of learning and not the sum total of everything they have been taught during the semester. Brown (2004-5) also echoes these sentiments regarding the timing of assessment: "timing of assessment is also a key issue, since the responses given to assessed work need to allow opportunities for amendment and remediation of errors" (p. 83). As Gibbs and Simpson (2005) affirm, the more specific the feedback is, the more useful it is to the student.
Gibbs (2010) discusses various ways in which assessment and feedback can help to support student learning, and illustrates this through the use of case studies at Leeds Metropolitan University. One example he gives is of a unit dealing with animation concepts, which previously used formative assessment at the end of the semester and consequently had quite a high attrition rate. Subsequently changes were made to make the assessments not only more engaging but also have them earlier on in the semester so that students would receive more timely feedback. The result was that "there has been a reduction in non-submissions and an improvement in grades" (p. 33).
Our pilot study, in looking at perceptions of the effectiveness of regular in-class tests as both an assessment and as a learning tool, addresses some of the points noted above by the scholars cited, in particular: formative assessment and the relationship between regular assessment and useful feedback.
More specifically, the unit programs are as follows.
Assessment in these units is therefore aimed at ascertaining if students have successfully mastered fluency in the language. Any exercises completed in class are specifically chosen to enable students to practise these concepts as well as their vocabulary and communicative skills. In order to assess the students' ability in this regard, as well as taking into account the reticence on the part of students to put in the extra study time outside of contact hours, it was decided that the students should be given a short twenty minute test (in the case of beginners students) and a twenty to thirty minute test (in the case of intermediate students) test during a tutorial every three weeks. These tests would assess them on the linguistic competencies that they had covered in class during those previous three weeks.
A total of 53 students completed the questionnaire out of 74 students enrolled in both units. Students were also given the opportunity to write any additional comments they might have had on the questionnaire form. Several students took the opportunity to provide written comments on the form, while others provided more detailed feedback via email. We have included these students' comments in this paper. The comments we received were similar across both learning streams, so we have grouped them together. Where a comment is specific to a particular unit, we have mentioned in which unit the student who commented is enrolled. Finally, by way of definition, it must be specified at this point that references to "on-going" assessment relate to the in-class tests that students must complete, given the significant weighting they have in both courses, as outlined above.
A total of four staff-members in Italian Studies, School of European Languages at UWA, provided written responses via e-mail to the following questions:
|Q 1 - The extra grammar exercises I have been given this semester have been:|
|Very helpful||Helpful||Somewhat helpful||Not helpful|
|Q 2 - I find having on-going assessment throughout the semester to be:|
|Very helpful||Helpful||Not helpful||Not sure||Don't care|
|Q 3 - I would rather have one longer assessment at the end of the semester:|
|Yes||No||Not sure||Don't care|
Useful in preparing for exams
Useful as revision tool/promotion of consistent study
Utility of revision exercises
Importance to student learning
The responses above also appear to confirm the results of a report of a survey undertaken of staff teaching languages at beginners' level at UWA, namely, that regular in-class assessment is extremely important to students' learning. The student perspective on regular in-class assessment is also very favourable to regular in-class testing. Regular in-class tests are not only seen to be an invaluable assessment tool, but an invaluable learning tool as well. This seems to concord with Yorke (2003, cfr above) and Nicol and Macfarlane-Dick (2006), who conclude that:
In order to produce feedback that is relevant and informative and meets students' needs, teachers themselves need good data about how students are progressing. They also need to be involved in reviewing and reflecting on this data, and in taking action to help support the development of self-regulation of their students (p. 214).Nicol and Macfarlane-Dick go on to add:
Frequent assessment tasks, especially diagnostic tests, can help teachers generate cumulative information about student's levels of understanding and skill, so that they can adapt their teaching accordingly (p. 214).From the students' perspective, the method of shorter, on-going assessment was found to have widespread approval. They believe that, not only is on-going assessment a beneficial way for them to gauge how well they are doing in the unit, but it also encourages and motivates them to study regularly, and go beyond simply what is done during class time. This correlates with the findings of Gibbs and Simpson (2005), referred to above. The questionnaires administered to the students clearly support these observations. Overall, 90% of the students surveyed stated that they found on-going assessment helpful or very helpful; and 79% stated that they would not prefer to have one longer assessment at the end of semester. This overwhelming preference that students have towards on-going assessment would indicate that students also believe that this approach is more engaging as a learning tool. This preference appears to be further underlined by the statistical difference between cohorts in the response to question 2. In Italian 1102, the beginners' unit, a total of 96% believed on-going assessment was either very helpful or helpful, versus 81% in Italian 1104. Such statistics would seem to highlight that the need for regular feedback is even greater for students enrolled in beginners units.
The views of staff canvassed as part of this study seem to accord with the student view. The staff we questioned noted a strong preference for regular in-class testing as a valuable learning tool, which accords with Yorke's (2003) view on the subject. Often, the tests will include exercises of the same type that have been completed and reviewed in class as part of the learning programme, which greatly assists in identifying if various "threshold" concepts have been reached, and lend themselves to more effective feedback for students. For example, if students were found to have not understood a particular point of grammar assessed in a given test, this could be revised in class and then re-presented in a subsequent test to ascertain if the concept had been successfully understood. It is hoped that regular in-class assessment will give the student the incentive to complete the exercises for classwork during the language tutorials, in the knowledge that they may be revisited in an upcoming test. Such tests are obviously not able to be manipulated as easily as take-home assignments; students must rely completely on their own knowledge.
For teachers, this means that they can see relatively quickly if the majority of students are keeping up and have achieved the required level of language competency, rather than waiting until the end of semester and potentially discovering that the students are not coping. As there is a large amount of course content, breaking it down makes it more manageable and somewhat easier to assess if the students have developed the necessary language skills in order to pass each milestone. This confirms Brown (2004-5) and Gibbs and Simpson's (2005) viewpoints that assessment and subsequent feedback needs to be regular to be effective. The comments collected from teachers in Italian Studies corroborate this view. As stated in the analysis section, the teachers felt that regular in-class assessment is very useful as a diagnostic tool to see if the students are actually keeping up with the language program, and believed that it gave the students an incentive to complete their tutorial exercises. Interestingly, while the questionnaire showed that overall 83% of the students find extra grammar exercises to be helpful or very helpful, the written responses from several students indicated that they did not see any point doing extra exercises if they were not able to then go over them in class. As mentioned in quote 3 under "Staff perspectives", unless a task is assessable students will not complete it. This ignores the reality that language acquisition requires regular practice, and completing extra exercises is a way for students to do that. This dovetails quite neatly with the teachers' comments about time constraints, which mean that it is physically impossible for them to check that the students are completing all of the assigned activities within class time. With less time being allocated to language courses in recent years, the suggestion here may be that languages actually need a bigger time commitment from students because of the large amount of content that is expected to be covered by staff.
Given the overwhelming preference of both students and staff for regular in-class assessment that we have found in this pilot study, allied with the reality that the challenges of devising assessment that is inclusive and engages students is not merely restricted to teaching one language (i.e. Italian), or, for that matter, language teaching generally, we therefore believe the results of this project can have application across many learning disciplines. In devising assessment structures, student perspectives must be borne in mind. An assessment structure may be devised because it is believed to have greater benefit for students; however, students should be included in this process to ascertain if THEY believe a particular assessment structure enhances their learning. In addition to the standard student evaluation forms, we suggest a short questionnaire of the type devised for this study could be given to students to gauge their response regarding assessment methods.
In this case study we investigated the views of staff and students, with a view to encouraging best practice. It is apparent that both staff and students in Italian Studies value regular in-class assessment with feedback, which further validates the current views of the scholars in the field, which we have discussed in this paper. We hope that by highlighting these issues we can encourage the application of this practice on a wider scale. This study was conducted via the use of short questionnaires, although there was an opportunity given to both students and teachers to write comments and give their personal views on the subject. This is a baseline study which could easily be further extended by:
Black, P. J. and Wiliam, D. (1998). Inside the Black Box: Raising standards through classroom assessment. Kings College, London [viewed 14 October 2011] http://weaeducation.typepad.co.uk/files/blackbox-1.pdf
Brown, S. (2004-5) Assessment for learning. Learning and Teaching in Higher Education, issue 1, pp. 81-89.
Cooper, D. (2006). Talk About Assessment: Strategies and Tools to Improve Learning. Toronto, ON: Thomas Nelson.
DeCorte, E. (1996). New perspectives on learning and teaching in higher education. In A. Burgen (ed.), Goals and purposes of higher education in the 21st century. London, Jessica Kingsley.
Gibbs, G. and Simpson, C. (2004-5). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, issue 1, pp. 3-31.
Gibbs, G. (2010). Using assessment to support student learning. Leeds, Leeds Metropolitan Press.
Nicol, D. J. (1997). Research on learning and higher education teaching. UCoDSA Briefing Paper 45 Sheffield, Universities and Colleges Staff Development Agency.
Nicol, D. J. and Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, vol. 31, no. 2, pp. 199-218.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, pp. 119-144.
Sadler, D. R. (1998). Formative assessment: revisiting the territory. Assessment in Education, 5(1), 77-84.
Vos, P. (1991). Curriculum control of learning processes in higher education. 13th International forum on higher education of the European association for institutional research, Edinburgh.
Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, vol. 45, no. 4, pp. 477-501.
|Please cite as: Sturniolo-Baker, R. & Loiacono, R. (2012). Perceptions of learning and assessment in beginners and intermediate level Italian Studies. In Creating an inclusive learning environment: Engagement, equity, and retention. Proceedings of the 21st Annual Teaching Learning Forum, 2-3 February 2012. Perth: Murdoch University. http://otl.curtin.edu.au/tlf/tlf2012/refereed/sturniolo-baker.html|
Copyright 2011 Rebekah Sturniolo-Baker and Rocco Loiacono. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.