|[ Teaching and Learning Forum 2001 ] [ Proceedings Contents ]|
Despite few studies, a number of studies in EUC training studies have ignored these variables in a training environment. Suggestions by scholars such as Robey (1982) have invoked little response in EUC community to investigate these variables. The Bostrom's (1990) framework was an attempt to take the suggestions and despite such attempts only few studies have investigated this aspect of individual differences. Since 1997 only Bohlen (1997) has considered this aspect in his study. Even he has failed to explain the theoretical background in reporting his outcomes. This study examines user individual differences of learning in a training environment.
Studies in EUC training have established that learning style variable is an important predictor of performance, both by itself and in interaction with training methods (Bostrom, Olfman, & Sein, 1990, p. 106). The learning style is defined as the knowledge of skills and cognitive factors that individuals possess during a learning sequence (Presland, 1994). It has been an accepted fact that individuals change their learning styles during a course of a learning sequence (Honey & Mumford, 1992). The dimensions along which such changes occur have been studied in learning theory. In training domain, to study such changes, one should first identify learner characteristics and then plan their training settings to accommodate changes. Studies in instructional psychology have demonstrated that it is necessary to adapt instructional methods and teaching strategies to accommodate key individual differences (Bostrom, Olfman, & Sein, 1990). EUC studies (Bohlen & Ferrat, 1997; Davis & Bostrom, 1993) have recommended that individual learning styles be determined before training is provided in order to measure outcomes.
There are multiple competing learning style theories available. The single learning style continuum argues that each individual can be placed somewhere on a bipolar scale. Examples of such fields are field independent/dependent scales (Witkin & Goodenough, 1988). The definite learning style model proposes that each person has one of finite number of learning styles. Examples of this are serialist/holist classification (Pask & Scott, 1972). The situational learning style model postulates that individuals are able to select from a number of possible learning styles, depending on the learning task at hand. Examples are surface/deep processing tasks (Marton, 1976). The multidimensional learning style model specifies that each person has a different degree of combination of styles. Examples are analytic/intuitive dichotomy (Pask & Scott, 1972). The current state of this theoretical development suggests that there is no clear agreement on a universal learning style theory or measurement.
In order to measure learning styles in EUC training studies, two instruments have been widely used. The first instrument is Kolb's Learning Style Inventory (KLSI). The instrument is based on experiential learning. The theory views learning as a discovery process that incorporates the characteristics of problem solving and learning. Ruble and Stout (1993) criticised Kolb's instrument for its validity in EUC training studies. The criticism was laid on the poor psychometric properties of KLSI. In answering to the criticisms, Bostrom et al. (1990) have accepted this fact. Further, it appears that many studies that have used KLSI were conducted in a tertiary setting where there is time for change in learning styles. However, in short training studies, such changes may not happen. So, it is possible to assume that learning style is stable for the duration of study in a short training program.
The other instrument used is Honey and Mumford's Learning Cycle. Honey & Mumford (1986) modified Kolb's approach and produced a model called the learning cycle. In this model, the learners are classified according to their strengths and weaknesses compared to their preferences. The model suggests four contrasting stages of a learning cycle.
Honey and Mumford (1986; 1992) modified Kolb's approach into learning cycle and classified learners in terms of their strengths and weaknesses for each stage of the cycle. They suggest four contrasting stages of a learning cycle. Activist are people who involve themselves in new experiences, tackling problems by brainstorming, and moving from one task to the next as the excitement fades. Reflectors are cautious and thoughtful people who like to consider all the possible angles before making any decisions and whose actions are based on observations and reflections. Theorists are people who integrate their observations into logical models based on analysis and objectivity. Pragmatists are practical people who like to apply new ideas immediately, and get impatient with an over emphasis on reflection. A wholly effective learner has the abilities characteristic of all four stages. However, such ideal learners are rare. It is mentioned that no one particular style is better than the other.
These two training approaches represent radically different users. While the exploration oriented training approach facilitates users to trial and error features, the instruction oriented approach provides little user control. The two approaches also feature deductive and inductive orientations respectively. Previous studies clearly indicated that the issue of suitability of training approaches for EUC training is yet to be resolved. While certain studies advocate the supremacy of the exploration oriented approach (Black, Carrol, & McGuigan, 1987; Carrol, Smith-Kerker, Ford, & Mazur-Rimetz, 1987; Kamouri, Kamouri, & Smith, 1986), other studies have established that the instruction oriented approach is effective in EUC training (Davies, Bagozzi, & Warshaw, 1989; Gomez, Egan, & Bowers, 1986). These studies reveal that there is no agreement regarding training outcomes.
Despite the disagreements in EUC training outcomes agreement, some studies have agreed that the primary role of training approaches in EUC training should be to provide meaningful learning through the integration or assimilation of new information in short term memory and knowledge from long term memory (Davis & Bostrom, 1993). However, in order for this process to occur, learners must actively work with both prior knowledge and new information. So, training materials, which support this process should be considered in training approaches. Studies have emphasised the importance of training materials in defining and deciding upon which training approaches to use.
To assist training approaches, preparation of training materials should be considered in terms of three components: concepts, procedures and usage of a given software application. The preparation of training materials should focus on the features of application software in EUC training (Gentry, 1994). The training material features considered in the previous studies can be classified under two categories: process features and structural features. The above discussion highlights the need for proper construction of training materials. The process and structural features will elaborate three further components: concepts, procedures and usage. In this study, the usage component refers to both the functional elements of software packages as well as the interfaces for measurement purposes. Hence, these features will be provided in terms of instruction orientation and exploration orientation. These two orientations will constitute the training approaches variable. Studies conducted by Bostrom (1990), Davis (1993), Bohlen (1997), Davies (1985) and Olfman (1994) have clearly indicated that there is a link between training approaches, user differences and training outcomes.
Are end user training outcomes affected by different user learning styles when users learn software (using short training programs)?The literature review indicates that the learning style preferences affect outcomes. However, there is very little evidence available as to which learning style is superior in a given situation. Very little experimental research has been done to verify that end user training outcomes are related to method of instruction and learning styles. If training outcome are affected by method of instruction and learning style as suggested by Bostrom (1990); Davis (1993) and Sein (1999), then these factors should be considered while training end users. Therefore, to address the first research question, the following null form of hypotheses are generated and stated in the following hypotheses.
Are end user training outcomes affected by training approaches when users learn software (using short training programs)?As indicated earlier, the two training approaches - instruction and exploration - accommodate radically different styles. The instruction approach supports inductive approach and the exploration supports deductive approach. The learners will be expected to possess different styles for these two approaches. While the instruction approach supports learners who depend upon complete set of instructions, exploration supports learners who would like to experiment with the available functions in a software application. Therefore, to address the second research question, the following hypotheses are stated:
The participants of the research were tertiary end user computing students enrolled in a computer science program. The participants possess limited IT knowledge. They range from 18 years to 40 years in age. Participants were drawn from Computer Science, Information Technology, Mathematics, Food Science, Aviation, Software Engineering and Sports Science courses. Participants have sufficient knowledge of PC operations. The participants were administered with Honey and Mumford's Learning Style Questionnaire (LSQ) to categorise them into learning style groups. The grouping is to establish a relationship between types of learning preferences. The participants filled in a set of questionnaires to determine their level of knowledge and experience prior to the LSQ.
Based on previous studies, effectiveness is defined for this experiment in terms of "score" gained by the number of steps used to conduct a task, number of errors committed and the number of backtracks in completing a step (Davis, 1993, p34; Bohlen, 1997 , p17; Olfman, 1995 p.344). To be effective, participants would use minimum number of steps with precision. It is difficult to predefine the minimum and maximum scores for given tasks as participants may opt to conduct a step in a task in any arbitrary manner leading to a varying combination of keystrokes. Therefore, the measure is mentioned as a function of various types of strokes. The effectiveness could be mathematically defined as:
Effectiveness = function (correct strokes, icon access, menu access, dialogue box interaction, errors, backtracks)
This could be mathematically shown as
Effectiveness = f(CS, IA, MA, DB, BTRK, ERR)
Based on previous studies, efficiency is defined for this experiment in terms of "time" taken to a complete a task (Bohlen and Ferrat, 1997, p17; Bostrom, 1990, p19; Sein, 1993, p343). The factor time is directly proportional to the number of keystrokes. It could be mathematically defined as:
Efficiency = function (keystrokes, time)
This could be shown mathematically as Efficiency = f(KS, T)
The experiment was organised into 4 sessions of about 30 minutes each. The first session was a briefing session and the Learning style preference questionnaire was filled in by users. The second session was used for training. The third session was used for a 12 task hands on exercise. The fourth session was used for filling in the satisfaction questionnaire.
The students were provided with training manuals. The training manuals were prepared based on Wood's (1990, p164) task complexity model. The training manual was examined by two independent judges for suitability and approved for the purpose of this research. The training manual consisted of actions for both icon and menu operations. So, students were able to choose either one of the styles. To guide students to follow steps either with icons or with menus, a number of verbal and imagery type of clues were provided.
In addition to various guiding instructions, the training manuals provided a number of information cues to students. Whenever students committed an error, a provision to recover from the error was given in the manuals.
During the training phase, subjects were allowed to work on the training manual for 45 minutes. The time restriction was to comply with various administrative procedures. In addition to this (45 minutes), subjects were given with another 45 minutes to work with various examples. These two sessions were held on different days in order to meet administrative procedures in booking computer laboratories.
Once the training and the example exercises were completed, subjects were administered with a hands on exercise. The hands on exercise consisted of 12 tasks of a project management schedule. Solutions to the tasks were recorded using Lotus ScreenCam program for playback and recording. The hands on task was recorded using a Lotus ScreenCam software. The entire hands on task was recorded and the average size of the file was about 4 MB. Replaying the file collected the responses. This operation took about 45 minutes per participant.
Subjects were asked to playback their solutions and record the number of accesses to menus, icons, number of keystrokes activated, correct keystrokes, backtracks, erroneous strokes and any interaction with dialogue boxes. These were used to compute the effectiveness. The time was recorded using the computer clock and was used to compute efficiency.
When the data was initially analysed, it was found that some students had failed to complete the tasks or failed to save the files properly. This has resulted in an elimination of 20 students from the data analysis.
Initially the data was tested for normal distribution and was found to be normal. When the regression analysis was performed, it was found that he variables also correlated well. This was established by performing a regression analysis with training type and learning style preferences as two variables (this is not shown in this paper). The data was then analysed to examine various trends.
Figure 1: Box plot for efficiency
Figure 1: Box plot for effectiveness
Table 1 shows the mean and standard deviation values for the training outcome efficiency and effectiveness respectively. The mean values for the outcomes efficiency and effectiveness are comparable for training orientations and learning styles. In addition to these, the variances are also comparable. When, a regression analysis was performed for both effectiveness and efficiency, a normal curve was yielded asserting the data is normal (this is not shown in the paper). This collective trend was interpreted as the existence of strong evidence for a univariate analysis.
The results of analysis of variance (shown in Tables 2 and 3) indicate that none of the effects (training type, learning style preference and the interaction) were significant at 0.10 level. The analysis performed reveals the nature of the main effects and the interaction effects. The analysis shows relatively small R-square values for efficiency and effectiveness. These small R-square values indicate that the models did not account for a good deal of variation in these dependent variables.
|TRGTYPE * LSTYL||3||.764||.516|
Table 3: Effectiveness
|TRGTYPE * LSTYL||3||.434||.729|
|a. R squared = .023 (adjusted R squared = -.022)|
The F test performed indicates that the main effects and interaction effects are not significant. For efficiency outcome, the F-values are F(Training approach, 1) = 0.691; F(Learning style, 3) = 0.671. For effectiveness outcome, the F-values are F(Training approach, 1) = 0.888; F(Learning style, 3) = 0.425. All the values are well over the significant levels and this is an indication that the null hypotheses cannot be rejected. This is confirmed by the p-values over the level of significance for every hypothesis.
In terms of effectiveness, the exploration groups have scored significantly higher means for activists, reflectors and theorists groups. Pragmatists have scored lower average. This can be translated as the exploration training treatment yielding significantly better results compared to the instruction group in terms of scores.
This also supports the proposition of Assimilation Theory. Subjects used their previous knowledge to derive new knowledge in order to achieve meaningful learning. Theorists who have undergone training can recall their conceptual knowledge to reduce their time in performing given tasks. On the other hand, people who have explored the application, found it difficult to complete the tasks in short time duration because of the lack of previous knowledge. The exploration group found it difficult to arrive at a meaningful learning which was essential to conduct the tasks quickly. This is shown in the outcome efficiency. Subjects have taken considerable time to absorb the new knowledge when it comes to exploration. The instruction approach has provided conceptual models to subjects. The conceptual models provided a context in which thinking is facilitated for reasoning purposes. In the case of instruction training type, subjects are provided with assimilative contexts with set of instructions and step by step procedures reflecting the functions of the application. Previous studies have confirmed this trend.
The exploration, on the other hand, allowed users to carry out a task based on the semantic distance. In other words, the semantic distance, which is relationship between a user's conceptualisation of an operation and the mechanisms that the training type provides to carry it out, is facilitated through the deduction process. In this study, the instruction training closely represents the user's conceptual model and hence semantically direct. Exploration training based subject were not able to do this because of the number of steps involved and the complex conceptual model provided by the deductive process.
It should be noted that the study supports the concept of using instructions to train end users when the application software is difficult to learn. A number of previous studies have supported this concept. However, there are studies, which have shown that this is not the case. The differences could be attributed to the lack of classification followed in this study. Further, the study did not categorise the tasks into simple and complex as defined by Mayer and hence this could have an impact in the disparity of the result.
Another aspect that is worth noting is the task itself. Despite the fact that the tasks are evaluated for appropriateness, and the model followed to create the tasks was Wood and Campbell's model, the tasks were not evaluated for their complexity. It appears that there are no universal guidelines available for such a purpose. This could have influenced the outcome of training to some extent.
Bohlen, G., & Ferrat, T. (1997). End user training: An experimental comparison of lecture versus computer based training. Journal of End User Computing, 9(3), 4-27.
Borgman, C. L. (1986). The user's mental model of an information retrieval system. International Journal of Man Machine Studies, 24(1), 47-64.
Bostrom, R. P., Olfman, L., & Sein, M. K. (1990). The importance of learning style in End User Training. MIS Quarterly, 14(1), 101-119.
Carnevale, A., & Carnevale, E. (1994). Growth pattern in workplace training. Training and development, 48, 22-28.
Carrol, J. M., Smith-Kerker, P. L., Ford, J. R., & Mazur-Rimetz, S. A. (1987). The minimal manual. Human Computer Interaction, 3(2), 123-153.
Davies, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Communications of the ACM, 35(8), 982-1003.
Davis, S., & Bostrom, R. (1993). Training end users: An experimental investigation of the roles of the computer interface and training methods. MIS Quarterly, (March), 61-79.
Gentry, C. G. (1994). Introduction to instructional development: Process and technique. Belmont, CA: Wadsworth Publishing Company.
Gomez, L. M., Egan, D. E., & Bowers, C. (1986). Learning to use a text editor: Some learner characteristics that predict success. Human Computer Interaction, 2(1), 1-23.
Honey, P., & Mumford, A. (1986). Using your learning styles. Maidenhead-Berkshire: Peter Honey.
Honey, P., & Mumford, A. (1992). The manual of learning styles. Maidenhead-Berkshire: Peter Honey.
Kamouri, A. L., Kamouri, J., & Smith, K. (1986). Training by exploration: Facilitating the transfer of procedural knowledge through analogical reasoning. International Journal of Man-Machine Studies, 24(2), 171-192.
Marton, F. (1976). What does it take to learn? Some implications on an alternative view of learning. In N. J. Entwistle (Ed), Strategies for Research and Development in Higher Education, 200-222.
Pask, G., & Scott, B. C. E. (1972). Learning strategies and individual competence. International Journal of Man-Machine Studies, 4, 217-253.
Presland, J. (1994). Learning Styles and Continuous Professional Development. Educational Psychology in Practice, 10, 179-184.
Robey, D., & Taggart, W. (1982). Human information processing in information and decision support systems. MIS Quarterly, 6(1), 61-73.
Ruble, T. L., & Stout, D. E. (1993). Factors of success for end user computing: An unwanted leap in faith. MIS Quarterly, 17(1), 115-118.
Sein, M. K., Bostrom, R. P., & Olfman, L. (1999). Rethinking end user training strategy: applying a hierarchical knowledge level model. Journal of End User Computing, 11(1), 32-39.
Sein, M. K., Olfman, L., Bostrom, R., & Davis, S. (1993). Visualization ability as a predictor of user learning success. International Journal of Man-Machine Studies, 39, 599-620.
Witkin, H. A., & Goodenough, D. (1988). Cognitive Styles, Essence and Origins: Field Dependence and Field Independence. New York: International University Press.
|Author: Raj Gururajan, School of Computer and Information Science, Edith Cowan University. email@example.com
Please cite as: Gururajan, R. (2001). End user computing: Learning style differences as a predictor of training outcomes. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2001/gururajan.html