Teaching and Learning Forum 2000 [ Proceedings Contents ]

The right-mix approach to teaching project evaluation

Paul Worthington
School of Information Systems
Curtin University of Technology
    This paper discusses the use of various approaches to teaching project evaluation units. As projects involving industry collaboration are highly contextual, it is becoming more appropriate to use non-traditional teaching methods. It is also relevant to set up student groups as though they were consultants, thereby employing the use of role-playing, real-life simulation, and case study scenarios.

    The author uses his best-practice consulting experiences to develop a course structure where the students are first guided through project evaluation roles, policies and practices, but where their acquired business acumen, recently learned, is challenged. The paper defines the processes involved in setting up a learning environment where students are expected to become members of an effective project evaluation team. Group dynamics, communication skills, skills with using appropriate risk analysis methods, leadership, supervision skills, people skills and technical skills are all taught and learned, but reflective skills and evaluation are also considered to be significant ingredients when teaching project units effectively. The course has been designed to ensure that students' expectations are directly related to their actual performance contributed through participation and involvement in the unit. This paper investigates a "right-mix" approach to teaching and learning within project-based undergraduate units. The "right-mix approach also ensures that student expectations are related to performance.

    The dynamic nature of project evaluation has made it a requirement that current project evaluation methodology and practices are followed. Students are encouraged to facilitate the process by collecting artifacts from online resources for inclusion within the content of the course itself. In order to aid the course, teaching and learning materials have been delivered by the use of WebCT (a tool that facilitates the creation of integrated World Wide Web-based educational environments).

Teaching and Learning Forum 2000 Home Page

Introduction

The task of developing an undergraduate unit, which aimed to teach business students how to reduce the incidence of project fatality provided a challenging experience. It is imperative that students confront the issue of project fatality given that according to Lewis (1999) "by most estimates, more than 70 percent of all IS projects fail". This has been attributed to issues such as resistance to change, limitations of existing systems, lack of executive consensus, lack of a senior-executive "champion", unrealistic expectations, lack of cross-functional project teams, Lack of team skills and late staff involvement (Reid Moomaugh & Associates (1995). With these issues acknowledged, to create an appropriate learning environment for project evaluation a "right-mix" approach, combining good teaching practice and good business practice was, considered essential.

In this paper I present findings from ongoing evaluation of my "right-mix" approach. I outline the theoretical perspective that directed my planning, disclose the general structure of the project evaluation unit, and the assessment approach adopted employed. I conclude with the feedback obtained from the students about the "right-mix" approach used.

Theoretical perspective

The proposed unit outline for Project Evaluation 202 introduced in second semester 1999 - School of Information Systems) was to teach theoretical concepts of evaluating small to mid-range information systems projects and the principles of software acquisition. Topics included evaluation of software, hardware, telecommunications, business objectives, vendor proposals and project risk. A "constructivist" approach to the method of assessment and learning theory was considered to be applicable to the unit. According to Funderstanding (1998), using the "constructivism" approach ensures that students don't simply regurgitating someone else's "right" meaning but allow students to construct their own meaning of the evaluation process and to make assessment part of the learning process.

Implementation of the "right-mix" approach

The seven principles for good teaching practice as described by Chickering and Gamson (1987) were used as a guideline to implement the "right-mix" approach. In order to enhance this learning approach, identified best practices relating to project evaluation in industry were incorporated into unit construction.

The seven principles were encompassed in the unit components and are stated below:

  1. Good Practice Encourages Student - Faculty Contact
  2. Good Practice Encourages Cooperation Among Students
  3. Good Practice Encourages Active Learning
  4. Good Practice Gives Prompt Feedback
  5. Good Practice Emphasises Time on task
  6. Good Practice Communicates High Expectations
  7. Good Practice Respects Diverse Talents and Ways of Learning
To achieve good teaching practice the following unit components were included:

2 hour tutorial / laboratory sessionThe sessions were facilitated by a tutor. Students working in groups of 3-6 completed cases with specific tasks. The students had the opportunity to ask and obtain assistance from their tutor when completing the assigned work. Working in groups students shared their experiences and gained an understanding of the tasks that were required of them.
WebCT discussion bulletin boardPrompt feedback was given to students through the use of this online medium.
A 1 hour lectureA formal lecture format was used to cover the unit content including theories and practical project evaluation examples.
Weekly student presentations Students were asked to research via the Internet and online databases information regarding the weekly lecture topic. A short presentation by students was required each week. Feedback on the topic was encouraged.

Assessment approach adopted

Desired outcomes of student learning that were relevant to the project evaluation unit as described by Pearson et al (1999) included the development of students' Students needed to demonstrate outcomes as detailed above. Due to the contextual nature of project evaluation case studies were developed to allow students to demonstrate mastery of a desired real-life task. According to Sweeny (1993) a preferred method to assess these skills is authentic performance assessment. An extended form of authentic project assessment, as described by the Miami Museum of Science, was selected to measure the students' ability in real-life tasks and situations. The components of this assessment included:

A student portfolio Students were asked to develop a directed student portfolio to demonstrate their learning and achievements. The portfolio included their case study work, annotated lecture notes, tutorial question and a 3 weekly self-reflection on personal learning achievements during the semester.
A group/ team assignment Students were required to work in a cooperative group on an extended case study simulating a real world challenge. This form of assessment offered the opportunity to assess creativity, planning skills, ability to integrate knowledge and to work with others. The assignment also aimed to demonstrate the students' ability in investigative research by incorporating the task to navigation of on-line resources.
A group presentationStudents were required to do a presentation in groups of 3-6. The group presentation allowed the students to develop their presentation skills which are often required by consultants in industry.

Formal examination consisting of multiple choice questions and a case study (4 compulsory questions covering the breadth of the project evaluation unit) was intended to examine the lecture material component of the unit.

Validation of the "right-mix" approach

Student reflections, a formal unit evaluation (using Student Evaluation of Educational Quality, SEEQ questionnaire) consisting of 15 questions were used to assess the quality and validate the use of the "right-mix" approach. The student evaluation questionnaire was used to validate the use of the student portfolio; as a means of assessing students learning outcomes.

Student reflections

Typical students comments on the "right-mix" approach to teaching and learning include:
'The Weekly lecture notes provide a good understanding of the topics covered in the unit, The laboratory tutor encourages everyone to participate and motivate every student to talk and give his or her opinion, the portfolio is considered high quality work because students need to organize their work every week'

'So far in this unit I have found the content and course material to be very interesting, the lectures are very informative and well prepared. Working in groups has been very beneficial as it allows the group members to exchange their ideas on the relevant material to be researched ... and concludes upon concluding this unit I can now look back and reflect on the unit and the outcomes that I have learned from the unit as a whole. I found the major assignment to be very beneficial to understanding the concepts in the lectures leading up to the assignment. Working on the assignment as a group made it easy to understand how a project team would operate in the real world ...'

As mentioned, remarks of this kind were typical of the generally positive reactions to the unit and the "right-mix approach". Moreover, they point to the genuine educational advantages associated with the project.

Student Evaluation of Educational Quality (SEEQ) Instrument

A voluntary "Student Evaluation of Educational Quality" (SEEQ) instrument (35 question, 9 point Likert scale [http://cea.curtin.edu/seeq/index.html] was administered to 112 of the 256 students who completed the project evaluation unit for the first time. This was used as a means of gauging students' views on teaching or evidencing teaching abilities.

A summary of the students surveyed revealed that students found the class intellectually challenging and stimulating and that their interest in the subject had increased as a consequence of this class. The survey also revealed that the methods of evaluating student work were fair and appropriate, and that the proposed objectives agreed with those actually taught, indicating that students knew where the class was going.

Student evaluation survey

The online student evaluation survey was made available through WebCT. From the 252 students enrolled 182 responses were obtained (72% response rate).

Responses showed that 75% of students either strongly agreed or agreed that doing the portfolio was a good way to assess their learning in the unit. 20% of students were neutral, while 5% of the students either disagreed or strongly disagreed.

Responses showed that 89% of students either strongly agreed or agreed that the use of a portfolio was a good way to gather unit materials. 8% of students were neutral, while 4 % either disagreed or strongly disagreed.

Responses showed that 52% of students either strongly agreed or agreed that the feedback about their portfolio usage was appropriate. 29% of students were neutral, while 17 % either disagreed or strongly disagreed.

Conclusion

Marsh and Roche (1999) state that "Traditional lecturing methods are often used by default, in situations where the possibility of more effective alternatives either have not been considered, appear to be 'too radical', or involve too much time and effort to establish". The "right-mix" approach, as described in this paper, uses radical ideas, effective alternatives and involves time and effort to establish. Notwithstanding, this approach aimed to satisfy pedagogical standards of good practice, and to ensure authentic outcomes of student learning were achieved.

References

Chickering, A. W. and Gamson, Z.F. (1987). Seven Principles of good practice in undergraduate education. AAHE Bulletin, 39(July), 3-7.

Funderstanding (1998). About Learning/Theories, How do people learn? Constructivism. http://www.funderstanding.com/learning_theory_how1.html [17 December 1999]

Lewis, B. (1999). IS Survival Guide. http://www.idg.net/crd_project_70702.html [17 December 1999]

Marsh, H. and Roche, L .(1999). Improving Academic Teaching Project - SEEQ Factor 1, http://lsn.curtin.edu.au/seeq/fact1.html [17 December 1999]

Miami Museum of Science (1999). http://www.miamisci.org/ph/lpdefine.html#AA [17 December 1999]

Pearson, M., Barlowe, C. and Price, A. (1999). Project based learning: Not just another constructivist environment. Proceedings HERDSA Annual International Conference, Melbourne, 12-15 July. http://herdsa.org.au/vic/cornerstones/pdf/PearsonM.PDF [43 kB, 17 Dec 1999]

Reid Moomaugh & Associates (1995). Re-engineering why it so often fails. http://www.improve.org/reengfl.html [17 December 1999]

Sweeny, B. (1993). Forced-Choice, Performance and Authentic Assessments. http://www.teachermentors.com/RSOD%20Site/PerfAssmt/DefineFCAuthPA.html [17 December 1999]

Please cite as: Worthington, P. (2000). The right-mix approach to teaching project evaluation. In A. Herrmann and M.M. Kulski (Eds), Flexible Futures in Tertiary Teaching. Proceedings of the 9th Annual Teaching Learning Forum, 2-4 February 2000. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2000/worthington.html


[ TL Forum 2000 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf2000/worthington.html
Last revision: 20 Feb 2002. Curtin University of Technology
Previous URL 17 Dec 1999 to 20 Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2000/worthington.html