Teaching and Learning Forum 99 [ Contents ]

Cognitive learning theory and the development of information literacy

Karen Macpherson
Faculty of Communication
University of Canberra
A current focus in teaching and learning at the University of Canberra is the importance of developing two generic competencies to equip students for the workforce: cognitive skills, including problem solving and analysis; and information literacy.

In 1997 the researcher conducted an experimental study with 255 first year undergraduate students. The aim of that study was to determine the efficacy of a short teaching module based on learning theory paradigms in developing students' ability to retrieve information from electronic databases.

Teaching strategies emphasised the development of multiple concepts of the information retrieval process, the acquisition of realistic expectations of electronic database capabilities, and the modelling of a problem-solving heuristic.

Results suggested that a conceptual approach to teaching information retrieval increased subjects' likelihood of adopting the search methods employed by expert searchers.


Introduction

It is axiomatic that information is not useful unless it can be retrieved. As little as five years ago, information retrieval was regarded as the domain of the expert search intermediary. Technology, however, has brought information to the end-user, and it is for this large group, characterised by a broad range of individual differences and variability in performance, that teaching strategies need to be designed. Information retrieval is one of the components of information literacy, which is an essential competency to enable students, as future professionals, to function as lifelong learners in the information society (Bruce, 1998; Yerbury & Parker, 1998; Julien, 1998; Halpern, 1998). The research reported in this paper investigated issues in bibliographic instruction, and teaching strategies, grounded in learning theory, which may assist undergraduates to achieve some aspects of information literacy.

A small number of studies has suggested that information retrieval requires conceptual knowledge and problem-solving ability, as well as skills training in the mechanics of database operation. As Balaraman (1991) observed, a majority of researchers has found an interaction between some form of conceptual training and the task performed. However, although there is a strong theoretical basis for using a concept-based approach in the teaching of more effective information retrieval strategies, there is little empirical evidence of their efficacy (Oberman, 1991).

The researcher combined elements of three cognitive learning theories - transforming mental models, cognitive flexibility theory, and situated cognition - to design the teaching module used in this study. These theories suggest that learning will be improved through the teaching of new information grounded in existing knowledge; the modelling of procedures; the development of complex constructs through analogy and concept-based principles; and an emphasis on the web-like nature of knowledge.

The purpose of the teaching module was to develop a conceptual knowledge of, and to demonstrate a problem-solving heuristic for, use by novice end-users in information retrieval. Penhale & Taylor (1986, in Jacobson & Jacobson, 1993), observed that expert searchers display more citations, review more search sets, enter more search terms, adjust their search strategies more often, and change databases more often, than do novices. The teaching module was designed to encourage the development of some of these skills, such as using more search terms and adjusting search strategies more often, in novice end-users.

Methodology

Research design

An experimental study was conducted with 255 first-year undergraduate students at the University of Canberra. Eighteen tutorial groups were involved, and randomly assigned to nine experimental, and nine control groups. All participants were given a 45 minute, skills-based, on-line demonstration of information retrieval from electronic databases. All participants completed a pre-test (Survey 1) which gathered demographic data, and measured knowledge of electronic databases and problem-solving ability. Subjects also completed a post-test (Survey 2), which measured any changes in electronic database knowledge; and an Information Retrieval Assignment, which enabled data on search strategy formulation to be gathered. The experimental groups received a 30 minute teaching module, based on learning theory paradigms.

Variables

The independent variable in the study was the 30 minute teaching module designed to determine whether concept-based instruction will influence understanding of electronic databases, search behaviour and search outcomes. The dependent variables were performance on the two post-tests. Intervening variables included previous knowledge of electronic databases, age, and human factors such as academic major, computer enjoyment and computer anxiety. Data on these variables was gathered on the pre-test.

Limitations

The researcher acknowledges the difficulties inherent in conducting an experiment in a natural setting. However, several features of the experimental design were incorporated in order to minimise the impact of intervening variables.

Firstly, a large sample size was chosen to enable control of intervening variables such as age, gender, and faculty of study. Secondly, the experiment was administered early in first semester, to a first-year cohort, in order to reduce the impact of knowledge of electronic databases. Thirdly, the study followed a pre-test, post-test, experimental/control group design, in order to ensure as accurate a measure as possible of any influence of the independent variable.

Hypotheses

A number of hypotheses were tested. The first two (hypotheses 1a and 1b) concerned the relationship (if any) between type of instruction and acquisition of knowledge of electronic database searching. All hypotheses are stated in the null form.

Hypothesis 1aThere is no difference between experimental and control groups on pre-test (Survey 1, Section 2) scores.

If this hypothesis is accepted, the conclusion that the experimental and control groups are equivalent in terms of their existing knowledge of electronic databases can be drawn.

Hypothesis 1bThere is no difference between experimental and control groups on post-test (Survey 2, Section 2) scores.

If this hypothesis is rejected, it may be concluded that the teaching module influenced the development of electronic database knowledge, as intervening variables were controlled.

Hypothesis 2There is no difference in search outcome (information retrieval assignment) scores depending on type of instruction.

If this hypothesis is rejected, it may be possible to conclude that type of instruction influenced information retrieval assignment performance.

Hypothesis  3There is no difference on information retrieval scores depending on problem-solving ability as measured by Survey 1, Section 3.

If this hypothesis is rejected, it may be that problem-solving ability and information retrieval scores are related.

Results and discussion

Demographic data gathered in Survey 1, Section 1 indicated that 67.6% of the 255 subjects were female; 32.4% were male. Forty-two per cent of the participants were under the age of 20; 18.1% were aged between 20 and 22; 20.1% were aged between 23 and 30; 11% were aged between 31 and 40; the remaining subjects were aged over 40.

With regard to education, 71.1% of subjects had completed Year 12 as their highest level of academic achievement. Eighteen per cent had completed a TAFE course, and 5% held a degree. The remaining subjects held an overseas Year 12 equivalent qualification.

Thirty three per cent of subjects had never used an electronic database before; another 26% had used them "once or twice"; 24% had used them only several times a year.

Hypotheses 1a and 1b

A t-test indicated that there was no significant difference between experimental and control groups on the pre-test questions on electronic database knowledge (Survey 1). There was, however, a significant difference in the post-test (Survey 2) on electronic database knowledge between the groups (t = 3.174; df = 170; p < .005).

These results support a decision to accept hypothesis 1a, and to reject hypothesis 1b, leading to the conclusion that the teaching module may have assisted subjects to acquire electronic database knowledge.

Hypothesis 2

Hypothesis 2 postulated no difference in search outcome (Information Retrieval Assignment) scores depending on type of instruction.

A Ratings Sheet was designed to enable the Information Retrieval Assignment data to be coded and analysed. Variables included: number of concepts identified; number of databases accessed; self-evaluation of search success; number of synonyms used; use of truncation; number of search strategy reformulations; use of Boolean operators; overall suitability of search strategies; and overall search success.

Results suggested that there were significant differences between experimental and control groups on a number of variables for each of the three research questions that comprised the Information Retrieval Assignment. Significant results are summarised in Table 1 below.

Table 1: Significant results for Hypothesis 2 variables

Variable Experimental group (%)Control group (%)Statistical test
Topic 1
No. of concepts 1812z= -2.0898; p=.0366
Self-evaluation of search success "quite useful"/ "very useful" 1950Pearson chi-square 13.388; df 2; p=.00124
Topic 2
No. of concepts 7854 z = -3.3337; p=.0009
"and/or" Boolean operators 13.26.7 Pearson chi-square 6.46643; df2; p=.03943
Topic 3
No. inappropriate concepts 6448 t = 2.15; df 161; p=.033
"and/or" Boolean operators 20.35.4 Pearson chi-square 29.85; df 3; p=.000

For research topic 1, the experimental group used significantly more concepts than did the control group. They also used significantly more truncation, either correct or incorrect (Pearson chi-square value 15.418; df 3; p = .00149). Seventy-five per cent of control subjects used no truncation at all, compared with 58% of the experimental group.

For self-evaluation of search success on topic 1, the control group rated their search success higher than did the experimental group. This finding provides some support for the phenomenon of the "false positive" reported by Applegate (1993), which describes the tendency of novice end-users to report satisfaction with the results of poor searches. For topic 1, there was greater agreement between researcher ratings and experimental group self-ratings, than between researcher ratings and control group self-ratings. It is possible that experimental group self-ratings were more realistic than control group ratings because the teaching module had emphasised the importance of evaluating the relevance of sources located.

For research topic 2, the experimental group again used significantly more concepts than did the control group. Again, they used significantly more truncation, either correct or incorrect (Pearson chi-square value 15.806; df 3; p = .00124). Sixty per cent of the control group made no attempt to use truncation, compared with 45% of the experimental group. The experimental group used significantly more compound ("and/or") Boolean operators. This topic was rated by the researcher as the most straightforward of the three search topics.

For research topic 3, the experimental group used significantly more inappropriate concepts than did the control group, which may have reflected the emphasis of the experimental module on the importance of identifying as many terms as possible to use in a search. The experimental group used significantly more databases in their searches (z = -2.3554; p = .0185). As with topic 2, the experimental group used significantly more compound Boolean operators, compared with the control group.

Results regarding search strategy formulation as measured by use of truncation, number of concepts, and use of compound Boolean operators, support a decision to reject the null hypothesis, and lead to a tentative conclusion that the teaching module assisted subjects to develop appropriate search strategies.

Hypothesis 3

Hypothesis 3 suggested that there is no difference in information retrieval scores depending on problem-solving ability as measured by Survey 1, Section 3.

Log cross ratio analysis indicated that participants who scored above the mean on problem-solving questions were more likely to use more Boolean "and/or" operators, to use more truncation, and to design more successful search strategies, than participants who scored below the mean on problem-solving questions. Good problem-solvers were 90% more likely to score above the mean on electronic database knowledge than participants who scored below the mean on problem solving.

Further, significant correlations were found between problem-solving ability and number of synonyms (r = .2153; p = .026), number of concepts (r = .2132; p = .020), and number of reformulations (r = .1872; p = .056; Spearman's rank order correlation p = .046) on topic 2. There were significant correlations between problem-solving ability and number of concepts (r = .2071; p = .207) and number of reformulations (r = .2320; p = .028) on topic 3.

These results suggested that hypothesis 3 should be rejected, and supported a tentative conclusion that good problem solvers were more likely to formulate appropriate search strategies than poor problem solvers.

Conclusions

Results of the study suggested that: search strategy formulation is significantly correlated with electronic database knowledge; search strategy formulation is significantly correlated with problem-solving ability; concept-based instruction may influence the use of compound Boolean operators, truncation and number of concepts identified; and concept-based instruction may increase subjects' likelihood of adopting search methods of expert searchers.

Previous research (Borgman, 1986) has suggested the phenomenon of the "permanent novice" in search behaviour, as end-users do not use electronic databases with sufficient frequency to develop expertise. The researcher's results to date, however, suggest that it is not the "novice" status of the searcher per se, but their problem-solving ability, which affects the formulation of good search strategies. This finding is important, as domain-specific problem-solving heuristics can be learned. Other correlates appear to be knowledge of electronic databases, and level of education.

Concept-based teaching was effective in assisting novice end-users to develop a realistic understanding of the capabilities of electronic databases. Modelling of a problem-solving heuristic aided the design of search strategies. However, it should be kept in mind that the experimental treatment was of short duration. The researcher is currently conducting further research to determine whether similar results will be achieved with a series of concept-based modules.

References

Applegate, R. (1993). Models of user satisfaction: Understanding false positives. RQ, 32(4), 525-539.

Balaraman, K. (1991). End user studies in CD-ROM environment. Proceedings of the ASIS Annual Meeting, 28, 283-294.

Borgman, C. (1986). Why are online catalogs hard to use? Lessons learned from information retrieval studies. Journal of the American Society for Information Science, 37(6), 387-400.

Bruce, C. (1998). The phenomenon of information literacy. Higher Education Research & Development, 17(1), 25-43.

Halpern, D.F. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53(4), 449-455.

Jacobson, F. & Jacobson, M. (1993). Representative cognitive learning theories and BI: A case study of end user searching. Research Strategies, 11, 124-137.

Julien, H. (1998). User education in New Zealand tertiary libraries: An international comparison. Journal of Academic Librarianship, 24(4), 304-313.

Oberman, C. (1991). Avoiding the cereal syndrome, or critical thinking in the electronic environment. Library Trends, 39, 189-202.

Yerbury, H. & Parker, J. (1998). Novice searchers' use of familiar structures in searching bibliographic information retrieval systems. Journal of Information Science, 24(4), 207-214.

Please cite as: Macpherson, K. (1999). Cognitive learning theory and the development of information literacy. In K. Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/ Learning in Context, 236-242. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/macpherson.html


[ TL Forum 1999 Proceedings Contents ] [ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [rjatkinson@bigpond.com]
This URL: http://lsn.curtin.edu.au/tlf/tlf1999/macpherson.html
Last revision: 26 Feb 2002. The University of Western Australia
Previous URL 18 Jan 1999 to 26 Feb 2002 http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/km/macpherson.html