At the last forum I reported on the use of GroupSystems to collect student feedback on a large first year unit. Since then I have collected data from two further cohorts of students. The most recent cohort includes feedback specifically about lectures and tutorials. In particular there is analysis of the effects of anonymity and breakdown into national culture groupings including Australian, Singaporean, Malaysian and Indonesian students.
GroupSystems is an example of software that supports face-to-face meetings. Key features include anonymity, parallel input, recording and structure. The structured modules provide support for idea generation, idea organisation and idea evaluation. Whilst there are numerous business applications the software can be useful in the classroom for structured discussions including collecting student feedback.
The decrease in government funding for higher education has also seen a commensurate rise in the intake of international students particularly from the South-East Asian region such as Singapore, Malaysia and Indonesia. The increasing diversity of the classroom increases the need for teachers to be aware of cultural differences. Examining student feedback is one way of identifying differences in student experiences.
In the last forum (Atkinson, 1998) I reported on an alternative approach to traditional questionairre methods. This was based on a focus group approach supported by software generically known as Groupware. A series of pilot sessions were held for small groups of a first year information systems unit known as IS100. In the past semester this pilot project was extended to a larger cohort of students. The cohort was divided into national culture groups including Australian, Singaporean, Malaysian and Indonesian. Furthermore an investigation was carried out as to the effects of anonymity on the productivity of each cultural group.
In the first section of this paper groupware is briefly reviewed. Research on national culture is then discussed. The research design is described followed by a discussion of findings and implications for future research.
The Groupsystems software is based around the familiar meeting paradigm. The software enables an agenda to be developed consisting of modules that support brainstorming, categorisation and voting. A meeting room environment consists of participant workstations in a U shaped network with a workstation that supports the agenda creation and the running of the agenda. A data projector displays the public aggregate data and a printer is available for the timely creation of reports. Either one or two persons help facilitate the meeting taking the participants through the agenda and the supporting software modules.
The main features of the Groupsystems environment are anonymity, simultaneous input, structured modules and reporting. Anonymity allows individuals freedom through alleviating fear of evaluation by peers or superiors. This can result in richer and more honest ideas. Simultaneous or parallel input allows many persons to enter ideas at the same time thus promoting productivity - more ideas generated in a given time. The structured modules of Groupsystems help to keep group members focussed on task and provide support for activities such as multi-criteria decision making which are not commonly undertaken at traditional meetings because of the difficulty of structuring and summarising inputs.
The GroupSystems software was developed in a North American cultural context and as such reflects values such as technology, democracy, anonymity, structure, productivity and recording. Given the globalisation of business it is of interest to see how other cultures appropriate the software. First a framework for comparing cultures is required.
In a series of studies at IBM, Hofstede (1983; 1985) identified five attitude or value dimensions differentiating national cultures and then positioned particular cultures on those dimensions. These five dimensions are labelled individualism/collectivism, power distance, uncertainty avoidance, masculinity/femininity, and time orientation.
Individualism refers to a preference for a loosely-knit social network in which individuals are expected to take care of themselves and their immediate families, whereas collectivism refers to a preference for a tightly-knit social framework in which individuals can expect their relatives or other closely-related groups to look after them (usually in exchange for unquestioning loyalty).
Power distance is the extent to which a culture accepts the fact that power in organisations and families is distributed unequally. In a culture with high power distance (such as most Asian cultures), subordinates defer to superiors and do not question their authority, whereas in a culture with low power distance (such as in most Western countries), subordinates are relatively free to disagree with their superiors.
Uncertainty avoidance refers to the extent to which people within a culture are uncomfortable with situations they perceive as unstructured, unclear, or unpredictable. It indicates the extent to which members of a culture feel threatened by uncertain and ambiguous situations and try to avoid those situations by adopting strategies such as seeking greater career stability, establishing more formal rules, not tolerating deviant ideas and behaviours, and believing in absolute truths.
Cultures that exhibit masculinity show a preference for achievement, heroism, assertiveness and material success, as opposed to a culture exhibiting femininity which shows a preference for relationships, modesty, caring for the weak, and quality of life.
Time orientation is considered to be of importance to decision-making situations and lies on a "continuum with long-term orientation at one pole and short-term orientation at the other" (Watson and Raman, 1994). Cultures with a short-term orientation are characterised by people who expect quick results, and in group decision making in this culture problems are considered independently and quick solutions sought. Cultures with long-term orientation (such as Japanese) consider the long-term future more important and are more likely to accept that benefits may take longer to be achieved.
Western and Asian societies differ on a number of these cultural dimensions, but especially in relation to power distance. This cultural characteristic can be observed through the concept of face where in Asian cultures (which have high power distance) 'loss of face' would occur for both superior and subordinate if open disagreement occurs between them. This would not be an issue in Western cultures where power distance is low and 'face-saving' is less important.
Hu (1944) identified two dimensions of the concept of 'face', namely, lien (the confidence others have in your integrity) and mien-tzu (outside face or reputation based on one's personal efforts). Maintaining face is seen as being of paramount importance in business dealings with Asian cultures (Redding and Ng, 1982).
Do groups from high power distance (and correspondingly high concern for 'saving face') national cultures derive greater productivity from Groupware anonymity than low power distance national cultures?To account for differences in group abilities it was decided to measure productivity as the productivity gain between an identified and an anonymous treatment. Hofstede's power distance index orders the national cultures as Malaysian > Indonesian > Singaporean > Australian (Hofstede, 1985). Hence the expectation is that productivity gains from identified to anonymous treatment would be in this order. Thus the hypothesis to be explored is:
|Ha :||The productivity gain from identified (I) to anonymous (A) treatment is in the order Malaysian > Indonesian > Singaporean > Australian.|
The availability of at least four national cultures amongst the Curtin University student population represented am opportunity for implementing an experimental design. In addition to the research objective the experiment provided an opportunity to gain student feedback on a core first year subject, and to involve students in Groupware as part of their curriculum.
The key feature of the design was that each national culture group would undertake two Groupware sessions, one under the anonymous treatment and the other under an identified treatment. This repeated measures design would allow comparisons of productivity to be made whilst attempting to control for a key variable - that of group composition. The details of the design are described below.
To control for natural group variability the productivity was measured as a gain between the identified and anonymous treatments. It was expected that productivity would be greater under the anonymous condition. However, if it wasn't, a negative gain would be recorded. The productivity measurement was conducted via the Groupware system electronic record from each session and the use of a word processor word count feature.
Perceptions of participants with regard to the anonymous and identified treatments were collected via a questionnaire which included a scale and an opportunity for open comment. Given the repeated measures design where each participant had experienced both conditions it was possible to ask participants directly to compare the treatments. This questionnaire was administered at the end of the second Groupware session.
Students were invited to participate in two Groupware sessions in lieu of their normal tutorial/laboratory sessions. Course credit of 5% was given for those students who participated in both sessions. In addition to the course credit incentive, Groupware was being discussed in lectures and thus was potentially of interest. A further incentive was that students were to be given the opportunity to provide feedback on the Information Systems course to date.
Four national cultures were identified from participant supplied information. These were Australian, Singaporean, Malaysian and Indonesian. Based on the relatively low enrolment numbers in this semester it was not possible to form a statistical sample size of groups, rather there was enough for a pilot study. The actual number of groups was Australian (7), Singaporean (3), Malaysian (1) and Indonesian (1). Unfortunately the group size for the Malaysian group was only three and was thus omitted from the analysis. The intention was to form group sizes of ten commensurate with the seating capacity of the Curtin Groupware Facility. Due to varying availability of participants the actual group sizes varied. Furthermore participants were allocated to groups based on timetabling information supplied.
The Groupware process design for each task was similar consisting of idea generation, idea organisation and discussion, and finally idea evaluation. The key measurement of productivity (words generated per participant) was based on the ten minute idea generation step. It was felt that measurements thereafter would be less likely due to culture and anonymity and more to do with group dynamics and facilitation. Each session was designed to be completed in one hour.
To control for facilitation variation, the four facilitators involved had all undergone the same amount of education and training and were given a script to follow for all the sessions. The participants from Singapore, Malaysia and Indonesia were in their first year in Australia and thus could be considered relatively free of Australian cultural influences. Additionally given that they were at least of eighteen years of age their native cultural influences would have been well developed. Thus the national culture identification was considered to be quite strong. It was noted that there are sub cultures based on race (for example Chinese Malays) or geographic location (Country Malay), that could be the subject of further research.
Further controls included the room environment, Groupware technology (GroupSystems for Windows) and the GroupSystems tools used (topic commenter, categoriser and vote). All the participants were GroupSystems novices.
|National culture||Treatment||#Words||# Participants||#Words per|
gain (A - I)
The hypothesised order based on power distance of Indonesian > Singaporean > Australian, is supported for two of the three national cultures. Surprisingly the Singaporean groups have derived greater productivity than the Indonesian group. This may be because of the small sample size we are yet to observe the natural variability in group productivity. This caution could be applied to all of the results indicating the tentative nature of this pilot study.
The implicit expectation of a gain in productivity for all cultures is supported for the Singaporean and Indonesian groups- indicating a generally positive effect of anonymity over the identified condition. However, suprisingly the Australian groups were more productive in the identified condition. Aside from the issue of small sample sizes a possible explanation is that learning effects from one session to the next had a more positive effect on productivity for Australian groups compared to the othe cultures.
Open ended comments generally supported the preference for anonymity. A sample of comments is given in Table 2 below. These comments illustrate the diversity of individual views on anonymity both for and against. There was no obvious differences in the content of the comments across cultural groups. A tentative proposition is that the benefits and drawbacks of anonymity are perceived similarly across these cultural groups. Field study research could be used to investigate this proposition.
With regard to the content of the feedback received on lectures and tutorials the results were presented in a written report to the unit controller. Whilst the Groupsystems process allowed a large number of comments to be efficiently collected a comment from the unit controller was that "we have only scratched the surface of the issues here". This indicated the need to gain a deeper understanding of student issues possibly through one-to-one interviewing. The process however was perceived as a useful supplement to the quantitative surveys which are currently used.
The students were generally positive about the Groupware experience in that it gave them a hands on experience of technology they were reading about and an opportunity to provide feedback on the course. Additionally students were able to see what their colleagues views were giving them a better understanding of the variety of feedback and whether their own views were shared or idiosyncratic.
To complement the experimental work aimed at measuring productivity, field studies with different culture groups are necessary to find out how Groupware in general (and anonymity in particular) is perceived by business. It may be that despite demonstrated productivity in experimental groups, perceptions of Groupware and anonymity are such that traditions of identified contribution without Groupware technology prevail.
The larger question is how Groupware values and technology are appropriated by different cultures and also by groups of mixed cultures. With increasing globalisation and internationalisation of business the interaction of mixed cultural groups may be based on Groupware models that have not yet been considered.
DeSanctis, G. and Gallupe, R.B. (1987). A Foundation for the Study of Group Decision Support Systems. Management Science, 33, 5, May, pp589-609.
Connolly, T., Jessup, L.M. and Valacich, J.S. (1990). Effects of Anonymity and Evaluative Tone on Idea Generation in Computer-mediated Groups. Management Science, 36, 6, June, pp689-703.
Gallupe, R.B., Bastianutti, L.M. and Cooper, W.H. (1991). Unblocking Brainstorms. Journal of Applied Psychology, 76, 1, pp137-142.
Gray, P. and Olfman, L. (1989). The User Interface in Group Decision Support Systems. Decision Support Systems, 5, pp119-137.
Ho, T.H., Raman, K.S. and Watson, R.T. (1989). Group Decision Support Systems: The Cultural Factor. Proceeding of the 10th International Conference on Information Systems, Boston, Massachusetts, December 4-6, pp119-129.
Hofstede, G. (1978). Culture and Organisation - A Literature Review Study. Journal of Enterprise Management, 1.
Hofstede, G. (1983). The Cultural Relativity of Organizational Practices and Theories. Journal of International Business Studies, Fall, pp75-89.
Hofstede, G. (1985). The Interaction between National and Organisational Value Systems. Journal of Management Studies, 22, 4, pp347-356.
Hu, H.C. (1944). The Chinese Concept of Face. American Anthropologist, 46, pp45-64.
Jessup, L.M., Connolly, T. and Galegher, J. (1990). The Effects of Anonymity on GDSS Group Process with an Idea-generating Task. MIS Quarterly, 14, 3, September, pp312-321.
Nunamaker, J.F., Jr., Applegate, L.M. and Konsynski, B.R. (1987). Facilitating Group Creativity: Experience with a Group Decision Support System. Journal of Management Information Systems, 3, 4, Spring, pp5-19.
Rawlinson, J.G. (1981). Creative Thinking and Brainstorming. Gower Publishing Limited, Westmead.
Redding, S.G. and Ng, M. (1982). The Role of 'Face' in the Organizational Perceptions of Chinese Managers. Organizational Studies, 3, 3, pp201-219.
Tan, B.C.Y., Watson, R.T., Wei, K.K., Raman, K.S. and Kerola, P.K. (1993). National Culture and Group Support Systems: Examining the Situation where Some People Are More Equal Than Others. Proceedings of the 26th Annual Hawaii International Conference on the Systems Sciences, Hawaii, January, Vol. IV, pp132-141.
Valacich, J.S., Jessup, L.M., Dennis, A.R. and Nunamaker, J.F., Jr. (1992). A Conceptual Framework of Anonymity in Group Support Systems. Group Decision and Negotiation, 1, pp219-241.
Vogel, D.R. (1992). Group Support Systems in International Contexts. Proceedings of the 2nd International Organisational Behaviour Teaching Conference, Curtin University, Perth, Western Australia, December 16-19.
Watson, R.T. and Raman, K.S. (1994). Culture: A Fourth Dimension of Group Support Systems. Communications of the ACM, 37, 10, pp45-55.
|Please cite as: Atkinson, D. (1999). Student feedback collected using groupware. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. http://lsn.curtin.edu.au/tlf/tlf1999/atkinson-d.html|