Make a conclusion. Article 2. Integrating a webquest in the primary school curriculum using Anchored instruction: effect on learning outcomes*

ARTICLE 2. INTEGRATING A WEBQUEST IN THE PRIMARY SCHOOL CURRICULUM USING ANCHORED INSTRUCTION: EFFECT ON LEARNING OUTCOMES *

Eliane Segers¹, Mienke Droop², and Ludo Verhoeven¹ ²

¹Behavioural Science Institute

²Expertisecentrum Nederlands (Expert Centre for Language Education)

Radboud University Nijmegen

The Netherlands

Abstract. Based on a literature review, a means for integrating WebQuests in day-to-day school activities is introduced using principles of Anchored Instruction. Following these ideas in an effect study, including 109 children in 4th, 5th and 6th grade, significant learning gains were found, with a large effect size. Differences in learning gain were found between groups of children from different teachers, and reading comprehension was an important predictor of learning gain. The study gives strong support for a more integrated approach in which learning to search for meaningful information and processing this information is part of the more general educational curriculum.

Keywords. Applications in subject areas, elementary education, improving classroom teaching, teaching/learning strategies, evaluation methodologies.

1. Introduction

Computers have a place in education nowadays. In some schools more than in other schools and in some schools more integrated into the day-to-day activities than in other schools. Many schools have computers with an Internet connection, and thus the computer is often used as a tool to gather information about all different kinds of topics. Children in the upper grades of primary school seem to prefer to use the Internet, and not books, when they have to write a text (Segers and Verhoeven, 2003).

However, finding relevant information on the Internet is not an easy task for them. A review study by Kuiper, Volman and Terwel (2005) revealed several problems for children when searching the web for information. They have difficulties with the large amounts of texts, and focus on collecting factual knowledge. They furthermore pay little attention to reading and processing the information, and have difficulty in assessing the relevance of the information found and the reliability of the website.

A possible means to overcome these problems is to offer a layer of structure between the child and the Internet (Segers and Verhoeven, 2009). This allows the children to focus on the learning task and less on the searching task. The concept of a WebQuest is promising in this line of thought. A WebQuest is an inquiry-oriented activity in which the information that one needs can be found on pages on the Internet that are linked to in the WebQuest (Dodge, 1995).

A WebQuest consists of a series of web pages, and starts with an introduction to a particular topic and some background knowledge on the topic (Dodge, 1995). The next page describes the task to be done. Sources of information are then listed, which are most often links to web pages needed to perform the task. The fourth page describes the process to be gone through to accomplish the task. The description of the process is broken down into individual steps and followed by a guidance page that often presents questions about the information presented in the different web pages. The WebQuest ends with a conclusion, which usually summarizes the learning experience. Also a rubric is provided at the end, that helps children to evaluate their own work and often a page with guidelines for the teacher is included.

The concept of a WebQuest is based on well-established educational theories such as those of Marzano (1992) and Bloom (in Marzano and Kendall, 2007), in which the development of higher order thinking skills (analysis, synthesis and evaluation) is stressed. The idea also leans to the constructivist ideas of children constructing their own knowledge. The instructional ideas behind the concept of the WebQuest fit in the most established theories of instructional design as summarized by Merrill (2003) in the ‘first principles of instruction’: learning is enhanced when the learner is involved in solving real problems, when prior knowledge is activated, when new knowledge is demonstrated to the learner and is being used by the learner, and finally, when the new knowledge is being integrated into the world of the learner.

Even though the use of WebQuest is widespread throughout the world, empirical research on the impact on learning and the cognitive requirements is scarce (Abbitt and Ophus, 2008) and gives little information about how a WebQuest could be integrated into the schools.

In the present study, we will therefore first present a literature review of the empirical research that was published on the use of WebQuest, resulting in a way to integrate WebQuests in school. Second, a study will be described, in which a WebQuest was introduced in a primary school in The Netherlands, following the lines of the literature review. The study aimed at unraveling child characteristics that contribute to the learning effects of doing a WebQuest, taking into account the differences between teachers and grades.

2. Empirical research on WebQuests

A first relevant study on the learning impact of WebQuests was performed by Milson (2001). In a qualitative case study, 23 6th graders did a WebQuest on Ancient Egypt. It was noted that, even though the WebQuest provided the relevant Internet links, children were still inclined to search for answers themselves via Internet search engines, thinking this would be the easier way to find an answer to a question, since in some engines; one can type in an entire question. Milson was optimistic (though with reservations) at the end of the study, because teachers can guide the children to show behavior that is asked for in a WebQuest. This is clarified by stating that the teacher in this study made sure the children indeed read all questions and web pages and engaged them in higher level thinking.

Another qualitative study was conducted by Wagman (2005), who gave a very thorough description on the implementation of a WebQuest in a high school in a Latin course. Forty-six pupils, about 15 years of age, took part in this project. They were motivated and showed learning gains in the course of the project. Unfortunately, no control group was enrolled.

Wagman observed that children with weak reading skills had difficulties in going through the amounts of text. In interviews, the pupils indicated that they would prefer a combination of learning by own experiences and by teacher guidance.

A third qualitative study involved six fifth-grade students (i.e., five girls and one boy). Ikpeze and Boyd (2007) addressed the overload of information that children can encounter when working with the Internet in general, but also in WebQuests. One advantage of the WebQuest detected by the authors was that it appeared to help bridge the gap between content literacy and technological literacy. Children were not just learning to deal with technology, but were using the technology to enhance their learning. The importance of the teacher was again highlighted.

King (2003) performed the first study on WebQuests with a more experimental design. An experimental group (n=30) and a control group (n=30) of students in teacher education were divided in groups of 5 to 6 persons. Their assignment was to construct a WebQuest for children in grade 6. The experimental group had an extra day to visit the group the WebQuest was designed for and instruct them on the use of the WebQuest. Before and after the assignment, the expectations of the students about the effectiveness of computer-based instruction were measured with a questionnaire. It turned out that the expectations of the experimental group remained the same, whereas the control group had higher expectations in the end. Working with the children thus diminished the expectations. However, students in the experimental group did design a better WebQuest according to the researchers. Knowing the focus group thus leads to the design of better material. Stinson (2003) emphasized the same thing in her article on experiences of making WebQuests by students in teacher education.

MacGregor and Lou (2004) added something to the traditional WebQuest. They compared two groups of fifth graders (n=26 in each class) who did a WebQuest on endangered species.

One of the groups received additional support in the form of a concept map that needed to be filled with information they found on the Internet. This group showed the highest learning gains; giving this extra form of structure proved to be successful.

Gaskill, McNulty, and Brooks (2006), compared the use of WebQuests to conventional instruction in two intervention studies. In the first study, WebQuest and conventional instruction conditions in the study of science were compared in a high school history setting and the conventional instruction condition was found to produce higher learning gains than the WebQuest condition. In the second study, university students in a WebQuest condition were found to gain just as much knowledge as university students in a conventional instruction condition. Differences between the conventional instruction conditions in the two studies may, however, explain the discrepant results. In the conventional intervention in the first study, for example, a movie was shown, a very experienced teacher told stories and there were group discussions. In the WebQuest condition, the children worked individually at their computer and there were no interactions.

In a multiple case study, Kanuka, Rourke, and Laflamme (2007) examined five different communication methods, which included the use of WebQuests. The participants were university students, and it was found that in both the WebQuest and debate methods of communication, the students posted more messages, reflecting the highest level of cognitive involvement. Unfortunately, the learning gains demonstrated by the students were not assessed.

Allan and Street (2007) studied the use of a WebQuest in 87 university students by using a questionnaire. Sixty percent of them believed the WebQuest indeed promoted higher order thinking, while 40% felt their learning was at a lower-order level. The authors drew attention to the work of Young and Wilson (2002) who proposed to add two stages to the WebQuest, based on the theory of anchored instruction (Bransford, Sherwood, Hasselbring, Kinzer, and Williams, 1990). In anchored instruction, a common knowledge base is introduced and discussed in class, e.g. a movie that helps getting children intrinsically motivated. Then, the learner is invited to help solve a ‘real world’ problem. Several studies proved the effectiveness of this principle, e.g. Kinzer, Gabella, and Rieth (1994), and Glaser, Rieth, Kinzer, Prestidge, and Peters (1999). An informal review by Ruzic and O’Connell (2003) found in general positive effects of anchored instruction, especially for children with learning problems.

The first of the two stages Young and Wilson (2002) proposed to add focuses more on the introduction and pays more attention to activate prior knowledge and the second is to enhance the conclusion part by having children share and compare their work. Allan and Street (2007) concluded that the addition of these two levels led to major shifts in learning level, as reported by two different groups of students (one who had a WebQuest with these extra levels and one who did not).

Segers and Verhoeven (2009) compared learning gains of 229 6th graders after doing a WebQuest on Ancient Rome in either a condition in which the Internet sources were provided or a condition in which they had to ‘Google’ to find answers to the questions in the WebQuest or their own questions. It turned out that girls had similar learning gains in both conditions, but that boys only showed significant learning gains in the WebQuest with Internet sources condition.

The above presented studies did not focus on the use of the rubric at the end of a WebQuest.

By using this rubric, children can evaluate their own work. Via the rubric, the child is asked to revise his or her own work. Rewriting one’s work is not that common in primary schools, and especially not based on the child’s own comments. Allal and Chanquoy (2004) showed that direct instruction is needed for children to be able to give feedback on their own work or that of peers. Rijlaarsdam, Couzijn and Van den Bergh (2004) argued that children should review the work of peers, need specific guidelines (these could be in the form of a rubric) and preferably should have more than one reviewer.

Most WebQuests, finally, also include a teacher page. The role of the teacher when children are doing a WebQuest is complex. Children need to become ‘web-literate’. Ikpeze and Boyd (2007) finally argued that a WebQuest should not be seen as an isolated activity, but should be integrated in what goes on in the class. The teacher often has the role of a coach in these situations (Leu, Kinzer, Coiro, and Cammack, 2004).

3. The present study: combining WebQuest and Anchored Instruction

Integrating the knowledge from the literature described above, mainly using the ideas of WebQuests and Anchored Instruction leads to a means of integrating a WebQuest in the day-to-day school activities. It builds on the principles of learning put forward by Merrill (2003), including theories of Anchored Instruction (Kinzer et al. 1994; Glaser et al. 1996) but extending the ideas of Young and Wilson (2002), and strongly emphasizes the idea that a computer-based activity should not be an isolated activity (Ikpeze and Boyd, 2007).

Furthermore, emphasis is being laid on working in small groups (Glaser et al. 1996), the use of concept maps (MacGregor and Lou, 2004), interaction with the teacher (Leu et al. 2004), reviewing the own work and/or that of peers (Allal and Chanquoy, 2004), and presentation of the work to the class and others (Merrill, 2003).

In an effect study, we put this integrated approach to the test. Research questions were:

Which child characteristics are related to these learning effects?

What are the learning gains of a WebQuest when integrated into the curriculum, taking into account grade and teacher, and controlling for child characteristics?

4. Method

4.1. Participants

Children from one school situated in a city in the south of the Netherlands participated. The school population mainly consisted of children speaking Dutch at home and with at least one parent born in The Netherlands. Four groups with a total of 109 children were involved. Each group consisted of children in 4th, 5th and 6th grade, which is quite common in a so-called Jenaplan school. The group consisted of 54 boys and 55 girls. The average age in 4th grade was 9;7, in 5th grade 10;8 and in 6th grade 11;9. Fifteen to 20% of the children had immigrant roots. The children had a normal IQ, as controlled for by the Standard Progressive Matrices (Raven, 1965).

The children were familiar with the computer, because of a lesson series at school aimed at learning children to use the computer. They indicated to use the computer between 30 minutes to an hour per week in school, and the same amount of time outside school for school purposes. Only 1 child reported not to have a computer at home, 27% had their own computer and 72% could use a family computer. Over 90% considered themselves ‘quite good’ to ‘very good’ (4-point scale) in the use of MS Word and the Internet, and 87% surfed the Internet at least a few times per month.

4.2. Materials

In order to gain information about the language skills of the participants, the Taaltoets Allochtone Kinderen (TAK; Language test for immigrant children, Verhoeven and Vermeer, 1993) for children in grade three to six was used. The TAK is a diagnostic instrument to measure Dutch language skills from immigrant and non-immigrant children. The TAK consists of seven subtests that can be used to assess different language skills. In the present study, four subtests were used: (1) definition task; (2) receptive vocabulary task; (3) text reading task one and two. These four subtests were chosen because they assess the language skills that are expected to be important in understanding written texts, draw conclusions about the text or summarize the text, and orally present this to another person.

The definition task was assessed orally. The participant was asked to give a definition for 25 words. It was taken into account that children learn to define words in two phases. During the first phase, a word is characterised (e.g., ‘when you are blind, you walk with a stick like this’) and in the second phase a word is defined (e.g., ‘blind means you cannot see’) (Verhoeven and Vermeer, 1993). The receptive vocabulary task assessed the understanding of words in written language, which is an important component of reading comprehension. The participant was asked to choose the meaning of a word out of four choices. In total, the task contained fifty multiple-choice questions (Verhoeven and Vermeer, 1993). Text reading task one assessed the understanding of coherence between sentences as expressed by connection words such as because, then, or after. The participant was presented with two texts in which these connection words are deleted. In forty multiple-choice questions the participant was asked to choose the correct connection words. Text reading task two assessed the understanding of meaningful relationships between sentences in a written text as expressed through the central elements of the sentence, such as nouns, verbs, and adjectives. The participant was presented with two texts in which these central elements are deleted. In forty multiple-choice questions the participant was asked to choose the correct central element (Verhoeven and Vermeer, 1993).

A Knowledge Test with 17 open questions was developed to gain insight in the children’s knowledge of the Middle Ages. The first question was to indicate the time period, then there were seven ‘What …’questions (e.g. ‘What is a serf?’), four ‘How..’questions (‘How did you become a knight?’), and five ‘Why..’questions (e.g. ‘Why did many people die of the plague in the middle ages?’).

The child received 2 points for a completely correct answer, 1 point for a partly correct answer and 0 points for a wrong answer. The knowledge test was scored by the first author of this article, kappa interrater reliability between first and second author was.818, indicating a good interrater reliability.

4.3. WebQuest

The WebQuest (http://www.webquests.nl/matrix/wq092/) used was designed by the authors for the present study. The WebQuest consisted of the following sections: introduction, task, process, guidance and resources, evaluation of task performance, and brief conclusion.

In the Introduction, the children were told they were real writers now and were to write a follow-up story for the movie they just had seen. As a real writer, their Task was first to do some research about the Middle Ages. In order to do so, they performed the WebQuest and had to choose at least three out of the following eight topics: from kings to farmers, the plague, the monastery, witches, knights and castles, education, food and drinking, cities and living, and could add their own questions as well.

The Process section described how they had to search for information, write a piece of work about their research, evaluate that, write a story, etc. The guidance and resources section provided example questions (‘how…’, ‘what…’) as well as links to webpages. The questions were open-ended where possible and prompted the children to higher order thinking in that for example the past was taken into the present (e.g. ‘Why is it not like that nowadays?’). In the evaluation section, children were asked to evaluate their own work, by looking again at the content, the design, their writing style, spelling and grammar and the length of the text. The conclusion repeated the learning goal.

4.4. Procedure

The whole project took place in a period of 5 weeks. Before the start of the project, the teachers were given instruction by the researchers about how to implement the WebQuest.

During the 5 week period, the first and second authors were in the school frequently, to guide the teachers, and answer any questions.

In the first phase (orientation) the topic was introduced. The children watched a movie situated in the Middle-Ages in two groups. After this they discussed the movie in their own groups, and made a concept map in small groups on the Middle Ages and aspects of the Middle Ages they saw in the movie. These concept maps were later on discussed in the whole class. They also thought about questions they wanted to find out about the Middle-Ages, and read several texts together with the teacher on the topic. In this phase, background knowledge is activated, and through the movie a common knowledge base is created.

In the next phase, the WebQuest entered the stage. In class, the children further worked on the topic via small writing or reading assignments which were discussed in small group discussions guided by the teacher. At the same time, the children work in dyads on the WebQuest. Children worked in dyads composed by their teachers. Each group worked for about 5 hours in the computer room of the school. When the children worked on the computer the teacher walked around and helped when necessary. The activities in the class and the activities on the computer were complementary. In this phase the role of the teacher was. In this process, the teacher takes on the role of a coach, and makes sure that attention is paid to the web-literacy skills of the children, but also to give instruction and guidance in the process of reading and writing for information purposes.

When the work was finished, a revising phase was entered, in which the work was evaluated. Children also needed to return to the phase of searching and working with information because additional information could be needed in order to revise the work. Then, in the final phase, the children presented their work to others, in the form of posters, talks, and a medieval market for parents and other visitors.

5. Results

The first question was which child characteristics were related to the learning gain.

Correlation analyses showed that the learning gain was positively correlated to ‘Text reading task 2’, r=.249, p=.017, n=94, and a negative correlation with Prior Knowledge was found, r=-.364, p<.001, n=98. Text Reading Task 2 was found to correlate significantly with all other measures of linguistic ability. This task thus reflects a general linguistic ability, and will further be referred to as Reading Comprehension.

The second question aimed at finding out what the learning gains were as measured by the Knowledge test and how these were related to grade and teacher. Leaning gains from pretest (M=10.32, SD=4.347) to posttest (M=17.01, SD=5.465) were significant, t(97)=12.610, p<.001, and with a large effect size, d=1.364.

A univariate analysis of variance with score at posttest as the dependent variable, Teacher and Grade as between-subjects factors, and the significant child characteristics (Reading Comprehension, and Prior Knowledge) as co-variables, revealed main effects of Reading Comprehension, F(1, 80) = 9.279, p =.003, O²p =.104, Prior Knowledge, F(1, 80) = 9.866, p =.002, O²p =.110, and Teacher, F(3,80) = 13.625, p <.001, O²p =.338. There was no interaction between Grade and Teacher (p =.105). The adjusted R squared of the model was.528. Descriptives can be found in Table 1.

Pairwise comparisons of the different teachers showed that the group from teacher D scored higher at posttest than children from the other three teachers (after controlling for Text Reading Task 2, Raven IQ, and Prior Knowledge). The group from teacher A also scored above the group from teacher A.

Table 1. Mean scores (and standard deviations) at pretest (Prior Knowledge) and posttest, and Reading Comprehension (RC) in the different teacher groups.

teacher	pretest	posttest	RC
A (n = 25)	11.56	(3.926)	17.75	(5.296)	29.92	(5.083)
B (n = 22)	9.82	(3.887)	12.42	(4.010)	27.75	(6.215)
C (n = 21)	8.58	(4.221)	15.96	(5.287)	29.60	(3.979)
D (n = 25)	10.50	(4.918)	20.93	(4.415)	29.36	(4.801)

6. Discussion

The aim of the present study was to investigate an optimal way of implementing a WebQuest in the day-to-day school activities. A literature review led to an approach in which principles of Anchored Instruction were combined with the use of a WebQuest. In an effect study, we found learning gains when implementing a WebQuest this way. Children in different grades equally benefited from the WebQuests: all grades showed a substantional learning gain, although there were differences at both pretest and posttest between grades.

More remarkable was the difference in learning gain between the four teacher groups. In one group, learning gain was significantly below that of the other groups, even though the WebQuest was the same and the teachers all received the same guidance from the researchers.

As for differences between the children, it seemed that, Reading Comprehension in the sense of understanding meaningful relationships between sentences was most important for learning gain.

The fact that integrating a WebQuest in the day-to-day school activities turns out to be successful is not surprising. The WebQuest concept has been shown to be effective before, and so have the principles of anchored instruction. There is no reason why combining these should not be beneficial. How large the effect is, as compared to doing a WebQuest in isolation is unclear. In a study using another WebQuest in isolation and another type of knowledge test (Segers and Verhoeven, 2009), the effect size was smaller than in this study (.66 vs. 1.36). Many different circumstances could have influenced the difference in effect though. One, for example being that children spend more time on the subject when it is integrated into the curriculum, so this comparison should be regarded with caution.

The main point we want to stress in this article is that a WebQuest is a tool that can be used very effectively, but needs the teacher to orchestrate the learning experience (cf. Kinzer and Leu, 1997) and the WebQuest needs embedding in the day-to-day activities in the classroom (cf. Bransford et al, 1990). The importance of the teacher is again stressed in the results of the present study, as differences in learning gain between the group cannot be explained by the WebQuest, but by the activities of the teacher. Follow-up research is necessary to unravel what caused the difference. As for child characteristics, reading comprehension was the most important factor related to learning gain. This measure correlated, however, with other child characteristics measures, and thus seems to tap a more general school performance; reading comprehension is a basic ability for learning (Verhoeven and Perfetti, 2008).

Unfortunately, we did not observe the whole process in all four groups, so cannot explain difference between classes by differences in style between the teachers. We also did not assess other capacities, besides language skills and nonverbal IQ, children may need to be able to deal with the flow of information and the processing of multimedia texts. We also realize that in this study, we have not been able to fully grasp the process of knowledge acquisition, and have not been able to disentangle teacher effects and WebQuest effects. Future research should address this issue further.

A second limitation was that we did not have a control group. We studied the effectiveness of implementing a WebQuest, and studied what child characteristics influenced the effect, as well as differences between teachers. It would take another study to compare the effectiveness of this trajectory with a more traditional approach, e.g. without a WebQuest. However, since the Internet seems to have taken its place in schools and is not going to be abandoned - because children need to be able to deal with this medium anyhow - it is perhaps more fruitful to study how it can be implemented optimally (cf. Segers and Verhoeven, 2009).

A final limitation of the study was that this was the first time both teachers and children in this school worked with a WebQuest and thus had to become acquainted with the concept.

Furthermore, the results are based on the use of one WebQuest in one school, and one thus needs to be careful to interpret these to other populations using other WebQuests. The WebQuest contained questions trying to elicit higher order thinking, but many children did not seem to be engaged in higher order thinking or thinking of own questions when working on the computer, but tended to quickly fill in the example questions, sometimes without reading the information sources. With help of the teacher, deeper thinking and reading of the sources was supported.

To conclude, a WebQuest needs to be embedded in the day-to-day activities in class (cf. Ikpeze and Boyd, 2007), using principles of Anchored Instruction. Implementing computer technology in classroom is not an easy task for a teacher and many factors can influence the degree of success. By just using any ICT-learning environment as an isolated activity one discards many possibilities. The practical implication of the present study is that strong support is given for a more integrated approach in which learning to search for meaningful information and processing this information is part of the more general educational curriculum (Verhoeven et al., 2006), and the integration of classroom activities with the use of the Internet thus makes traveling on the Web a more meaningful experience.