University of Michigan

1101 Beal Ave.

Ann Arbor, MI 48109, USA

http://hi-ce.eecs.umich.edu/

E-mail: [email protected], [email protected], [email protected]

The learner-centered design of software suggests the need to design scaffolding—fadeable supports—in educational tools. We describe an approach, Guided Learner-Adaptable Scaffolding (GLAS), in which the learner controls the fading of scaffolding, with guidance and support provided by the system. Using GLAS, we have developed a tool, TheoryBuilder, that supports learners in building and testing dynamic models of complex systems. We have conducted extensive classroom testing with students who used the tool several times throughout a year. An analysis of the data demonstrates the success of the GLAS approach in developing an adaptable tool to support the diverse and changing needs of learners.

Learner-Centered Design, Scaffolding, Fading, Adaptable Interfaces, Education Applications.

To address the needs of a population of users who are also learners, in our recent work we have proposed a new methodology for the learner-centered design (LCD) of software [9, 10]. Learners often require extra support and motivation to engage in unfamiliar tasks. Learners are a diverse population, varying in knowledge, skills, interests, and learning style. And finally, learners will learn—as they grow and develop understanding and expertise over time and through interaction with the software, their needs for software support and functionality will change as well.

The needs of learners suggest the use of scaffolding in educational software. Scaffolding refers to support provided so that the learner can engage in activities that would otherwise be beyond their abilities. Scaffolding, as provided by human tutors, has been well-established as an effective means of supporting learning [2, 8]. Building scaffolding into software offers the opportunity to provide for diversity through individualized support that accommodates learners of different skills, backgrounds, and learning styles, and to support growth by making more powerful functionality available as the learner develops expertise.

A critical component of scaffolding is that it be capable of fading; as the learner’s understanding and abilities improve, the computer, much like a human tutor, needs to back off and give the learner more autonomy, fewer hints, etc. In the field of educational software research, scaffolding is a new concept that is still being defined [2, 4, 9, 10]. Many techniques have been explored that provide various supportive structures for learners, but typically that support does not fade within the software itself.

The question of how to design scaffolding—fadeable supports—is informed by research in the field of adaptive and adaptable interfaces. Adaptive interfaces can be implemented as scaffolding that changes automatically using a model of the learner’s understanding. The main problem is that an extensive model of the learner’s knowledge may be hard to specify or evaluate in more open-ended domains. The alternative, adaptable interfaces, suggests scaffolding faded by the user. One problem is that it may be hard for the learner to make fading decisions. To address this, the software can be designed to provide information and advice, and to encourage self-evaluation, helping the student to measure his or her progress and understanding [6]. Another issue is that the learner needs to understand what the options are [3]. The question of how to design scaffolding is still an open one, although some combination of adaptive and adaptable fading appears to be the appropriate direction for future research.

We have therefore developed a design approach that we call Guided Learner-Adaptable Scaffolding (GLAS). GLAS is designed as discrete, fine-grained scaffolding of various types, faded under control of the learner, with guidance from the software to aid the learner in making informed decisions. We have developed scaffolding design and fading mechanisms, based on the GLAS approach, for a wide range of scaffolding types.

GLAS has been applied to the design of TheoryBuilder, which is the successor to Model-It, a tool that we developed to support the complex and educationally valuable task of scientific modeling [5, 10]. Research with Model-It provided us with a great deal of information about the kinds of supports and functionality that learners need to engage in modeling activities. TheoryBuilder incorporates a broad range of scaffolding to specifically target those needs. And whereas Model-It’s supports generally did not fade, TheoryBuilder implements the GLAS approach, providing fading of each of its scaffolds under the learner’s control.

TheoryBuilder has been thoroughly tested in actual classroom use with 100 students over several projects throughout a school year. Data collected include the models that the students constructed and videotapes of student conversations and software use. In-depth analysis of this data is used to evaluate the success of the GLAS approach.

TheoryBuilder, also called Model-It 3.0, represents the next phase of our research with Model-It—a tool to make modeling accessible to high school science students [5]. The basic components of each tool are the same. Students construct, verify, and analyze qualitative models by linking together the factors that comprise high-level objects.

To build a model, a student begins by creating objects, and choosing photos or pictures to represent them. Next the student defines factors, measurable quantities or characteristics associated with each object. Then the student defines the relationships between factors, either qualitatively, (Figure 1) or quantitatively, by clicking on the points of the graph or inputting a table of values.

Figure 1: Qualitative relationship definition

Once objects, factors, and relationships have been defined, students can run their models, using meters and graphs to view factor values. While a model is running, students can change the value of factors in real-time and see the results (Figure 2). Student can view their model either with a high-level "world" view of the objects in the model, or an interactive "map" view in which icons representing each object’s factors are linked together with arrows that represent the relationships.

Versions of Model-It have been used by hundreds of 14 to 15-year-old students at a local school over the past four years. Four classroom studies have been conducted, in each of which students were asked to design and create their own models. These models typically exhibited reasonable scientific validity and significant complexity. Students appropriately focused on high level concepts—defining the factors in the system and the relationships between them, running simulations with different factor values, and Figure 2: Map view, running a simulation with the model

determining the validity of the results. Students were comfortable expressing themselves qualitatively, and were able to conceive of a relationship, then quickly add it to their model. We found significant evidence that Model-It can make modeling accessible to pre-college science students. [5, 10, 11].

Our research also suggested areas where students would have benefited from additional support to learn the task and to engage in modeling activities. For example, we found that some students required more support in order to understand and engage in the cognitive activities associated with modeling—e.g., planning, making predictions, testing and evaluation [11]. Interviews and video of students identified areas of confusion for novice learners, as well as ideas for other advanced functionality that might be useful.

TheoryBuilder, the next phase of our project, had two goals. First, to broaden the range of supports provided by the program, in order to address the issues highlighted by our research. And second, to specifically address the idea of what it means to implement scaffolding in interactive learning environments, by designing supports that can be faded as the learner’s understanding and abilities improve.

We propose a particular approach to the design of scaffolding in software: Guided Learner-Adaptable Scaffolding (GLAS). The design is learner-adaptable because, in keeping with the goal of designing learning environments as tools with which the learner designs and creates artifacts (as opposed to, e.g., tutoring programs in which the software directs the interaction), the learner will also be in control of deciding to change and fade scaffolding. The design is guided, because of the potential need for guidance from the system (to supplement, not replace, the classroom teacher) in making those decisions. The goal is for the student, with the software's help, to be able to take on some of the responsibility for his or her learning.

For purposes of our research, and based on the literature [2, 4, 8], we have defined scaffolding as covering the following three categories: supportive scaffolding, reflective scaffolding, and intrinsic scaffolding. For each, we describe its purpose, its implementation in TheoryBuilder and its fading, summarized in Table 1.

Table 1: Fadeable scaffolding strategies and their implementation in TheoryBuilder

Supportive scaffolding is support for doing the task. The task itself is unchanged; scaffolding is provided alongside the task, to offer advice and support. As supportive scaffolding is faded, the task is the same as before, but the expectation is that the learner has internalized the concepts which had been scaffolded. This kind of scaffolding, which includes guiding, coaching, and modeling, is the most often referred to as scaffolding in the literature [e.g., 2, 8].

In TheoryBuilder, guiding scaffolding is provided through messages which appear when appropriate, and which can be faded through a "Stop reminding me" button (Figure 3). If the learner continues to neglect a task after fading a reminder, a message will eventually appear: "I know you asked me not to remind you about this, but I noticed that you haven’t been..." In keeping with our learner-centered approach in which the learner is in control of the tool and not vice versa, these reminders are suggestions; they do not force the learner into an action, and can be ignored.

Figure 3: Example of a guiding scaffold

Coaching and modeling scaffolds provide help and examples explaining the various modeling concepts. They are invoked through contextualized help buttons that appear throughout the software, usually as "?" (e.g., Figure 4) or "Show me an example". They display help using both text and pictures. Being passive scaffolds, these buttons do not fade per se, but rather are faded simply through not being invoked.

Reflective scaffolding is support for thinking about the task (e.g., planning, making predictions, evaluating). It also doesn’t change the task itself, but instead makes the activity of reflection explicit by eliciting articulation from the learner. As Norman asserts, reflective cognition, the mode of "comparison and contrast, of thought, of decision-making," is essential for learning, although more rarely supported by technology [7].

Reflective scaffolding in TheoryBuilder is provided by a Notepad that appears alongside the main window, and by description fields that are part of each component’s editing window. The learner reflects by typing plans, descriptions, predictions and evaluations into appropriate fields of the Notepad, as relevant for each subtask. Many of the guiding scaffolds refer to these text fields, prompting the learner to fill them in when appropriate, (e.g., Figure 3). Another passive scaffold, the text fields themselves do not fade, but once the reminders are faded they are more easily ignored.

Intrinsic scaffolding is our name for supports that change the task itself, by reducing the complexity of the task and focusing the learner’s attention (e.g., training wheels on a bicycle, outlines in coloring books), or by providing mechanisms for visualizing or thinking about a concept (e.g., maps and models for visualization). Intrinsic scaffolding can be designed simply by providing the option of a beginner mode, as in the Training Wheels word processing program [1] or KidPix’s (Broderbund) option of "Small Kids Mode." Ideally, however, scaffolding should support more gradual fading [8]. As the scaffold fades, the task is changed, but associations should remain so that the learner can progress from simpler, more structured, or more concrete tasks to variations in which more of the underlying complexity or abstractness is introduced.

Intrinsic scaffolding is implemented in TheoryBuilder as defaults which hide all but the simplest tools from the novice learner, but make advanced features available as the learner grows and develops expertise. Intrinsic scaffolding is manifested as different views and representations of model components, or different sets of enabled controls and tools. Figure 4 shows the mechanism by which the learner fades intrinsic scaffolding for the relationship editor by turning on advanced options that were previously hidden. The dialog also provides guidance for selecting appropriate options by showing what each option will let the user do. Other advanced features are listed in Table 1.

Figure 4: Example of the fading mechanism for intrinsic scaffolding for the Relationship Editor.

TheoryBuilder also includes other supports that make the task more accessible, but which cannot be called "scaffolds" by our definition because they do not fade within the program. Some of these features are: Task-switching controls (Plan, Build, Test, Evaluate) that identify and the task and provide different sets of tools for each subtask; labels on relationship arrows that display the graph or type of relationship; and the ability to test parts of a model by highlighting specific relationships, as a debugging aid.

For the first half of the year, the class studied stream ecology. For the first modeling project, in November, students built models of the physical and biological factors of a stream, and for the second, in January, students chose their own topic relating to some aspect of the whole stream ecosystem, including physical, biological, and chemical factors. In the spring, the class studied weather and global climate issues such as acid rain, global warming, and the greenhouse effect. For the third modeling project, students chose a research question based on these issues, used the internet to search for information, and built models to demonstrate their theory or understanding about the topic they had researched, using data from their research where appropriate. They presented these models to the class.

Data collected from all students included the models created for each project, and software-generated log files that record student interactions with the program. This data has been used to compile statistical information about the use of scaffolding and fading mechanisms over time.

In order to achieve an in-depth understanding of students’ usage of and reactions to the software, a focus group of 11 students was chosen as a representative subset. For this focus group, process video was collected continuously during each project, consisting of audio recordings of their conversations coupled with video recordings of their computer screen interactions. Post-interviews were conducted to further clarify their motivations and solicit reactions to specific scaffolding and fading mechanisms.

To compile user profiles, video of each focus student was analyzed and episodes involving use of or reactions to scaffolding and fading mechanisms were recorded and categorized. By observing changes and fading of scaffolding over time, we can identify reactions to specific scaffolding components, and overall, the success of the GLAS approach in supporting students adapting and fading the software to meet their changing needs.

The data from these studies is currently being evaluated. Preliminary analysis of the full set of models indicates that the software was quite successful. For the first project students were able to use the basic program to build useful models with only brief instruction, aided by the supportive scaffolding, and shielded by the intrinsic scaffolding from the potentially confusing array of options. In successive projects, a short introduction to advanced features was sufficient for students to choose to fade selective scaffolds and build larger, more complex, and more accurate models.

Supportive scaffolding: Use of coaching and modeling scaffolding faded over time, as students learned how to use the software, although help associated with advanced features continued to be used as these features were turned on. For the first project we tried having some help appear automatically the first time the student encountered a concept, but it appeared that students mostly ignored the help unless they had invoked it themselves.

More students faded guiding scaffolds over time as they learned the task; by the third project 63% of the students had turned off at least one reminder. Others simply learned to ignore the reminders. (Regrettably, there was no mechanism for remembering student preferences between projects, so all reminders were initially turned on at the start of each project). In general, in later projects students tended to ignore all supportive scaffolding unless they had invoked it themselves (e.g., by clicking on a help button).

Table 2 shows how often each reminder was faded, for each project. The reminder to test the model periodically was the most commonly faded, by 45% of the students in the second and third projects. This reminder first appeared after three relationships had been created or modified, and every five relationships thereafter, so it was the most frequently encountered, and perhaps thereby most frequently became annoying. Despite turning off the reminder, however, students tested their models more frequently in later projects, suggesting that they had retained the strategy. Other reminders, such as to open meters or to fill in descriptions and explanations, were faded less frequently, probably because students usually did these tasks without prompting and so encountered the reminders less often.

Table 2: Number of models (out of 45) in which each reminder was faded, over the three projects.

Reflective scaffolding: Students were required by the teacher to fill in the text fields that comprise reflective scaffolding (plans, descriptions, explanations, evaluations) as part of their grade. As a result, rather than fading, there tended to be increased and more thoughtful usage of these fields in successive projects. The exceptions were the prediction and assessment questions that accompanied the testing task, which were not printed out and therefore not graded. The result was that students tended to skip these questions in later projects, and in fact, as Table 2 shows, the guiding reminders to fill in these fields were the second and third most commonly faded supportive scaffolds.

As further evidence that students had developed the habit of filling out reflective scaffolding, we saw that even though there was no reminder to fill in the explanation field for the advanced weighting and combining editor, students who used it still tended to write an explanation.

Intrinsic scaffolding: More students faded the intrinsic scaffolding, i.e., used more of the advanced features, in later projects as they developed expertise, as shown in Table 3. For the first project the only options used by any students were the most basic: 20% of the student pairs used the option of specifying the slope of a relationship, and 9% used the table option to draw their own relationship graph. By the third project 85% of the models used one or both of those options, and of the most advanced options, 41% used delay and 26% used rate relationships, 28% used the weighting tool, and 11% used the combining function to calculate the product or difference of two factors’ effects.

Table 3: Percentage of models using each advanced option, over the three projects.

The table option was used most often in the second project, because the textbook used for that topic included a number of applicable graphs, while for the third project, on climate, the resources used by students usually did not include that level of detail. The third project did see the most use of the other advanced options, even the rate relationships which our previous research showed was rarely ever used. Here, though, owing perhaps to the growth of expertise due to the increased time students had spent with the program, many students were able to use rate relationships to model such temporal phenomena as the release of ozone-depleting gasses or the melting of the polar ice caps.

In this section, we present a qualitative analysis of two students from our focus group. These two students represent the diversity of our learner population, having very different backgrounds, approaches, and learning styles.

Greg is mathematically and logically inclined, even enrolled in a higher level math course than most of his classmates. He was quick to learn the program, and from the first day, branched out to explore and try new ideas. He was thoughtful about model-building, reasoning about chains of relationships.

Figure 5 shows the model Greg built for project 3. Greg had read that an increase in global temperature would increase storm severity, and he built a model to express his theory about why this would occur: that flooding would cause greater humidity in coastal areas as compared to inland areas, and this difference would create a pressure differential that would increase storm severity.

Figure 5: Greg’s model for Project 3

Supportive scaffolding: Greg used the coaching help buttons frequently in the first project as he explored the program, and thereafter only rarely, to learn about new advanced features. In a post-interview he said that the buttons were most helpful early on, but they were "nice to have around." The reminders were also used most at the beginning. During the first project, Greg read and followed reminders five times, but in the second and third projects, Greg was more impatient with the reminders and used the "Stop reminding me" button to fade most of them.

Greg’s reaction to the reminder to test was particularly interesting. The first day learning the program, he never tested the model, ignoring all reminders, saying that he wanted to finish building relationships first. The second day, building a new model, he decided to try following the testing reminder, and it led him to discover a mistake in his model. Having learned that testing can help catch errors early, Greg continued to test periodically in later models, internalizing the habit even after fading the reminder.

Reflective scaffolding: Greg was impatient with the reflective scaffolds, describing himself as a "verbal thinker" and feeling that he already had the whole model planned out in his mind, and just wanted to build it. He thought the testing questions were "pointless, because I can think of what I expect to happen and compare it to what happens without writing it down." Indeed, he did state very good predictions out loud and test his model against them. On the other hand, he wrote very good explanations, and said that he particularly enjoyed filling in the evaluation because he was proud of his model.

Intrinsic scaffolding: Greg explored advanced options from the first project, and used the guiding information in the dialogs to learn what the options would do. Still, he didn’t choose to turn on all the advanced options (fading the intrinsic scaffolds) at first. For example, he read the Factor option for numerical values and decided "we don’t need that." Later in the project, he did find a need to use numbers, and went back to the dialog to turn it on, using the associated help to figure out how to use it. In post-interviews, Greg said that while he understood that options were hidden so as not to confuse students who didn’t want to use them, he didn’t think it would’ve confused him, and if he were the only user he would like them all on from the beginning.

Greg eventually tried out all of the advanced options, pushing the computational limits of the software. For his third model (Figure 5, above), Greg used all the advanced features, including rate relationships to show rate of flooding, and the combining tool to calculate the difference in humidity between coastal and inland areas. He also used feedback loops, something that was never explicitly taught, in order to show that the more water there is on land the more is evaporated, and in turn the less water is left on land.

Amanda describes herself as "not comfortable with computers." Nevertheless, she applied herself to the modeling task and tried to learn the software, focusing on what was necessary to get a good grade. She had a very good understanding of the scientific content, wrote detailed scientific explanations, but didn’t always understand how to translate her knowledge into a model.

Figure 6 shows Amanda’s third project, which shows the human impact on ozone depletion, and suggests that alternatives to CFC-producing products could help alleviate the problem. When asked how her use of the program had changed over time, Amanda said that even though she tried to keep things simple, she felt "we understood it better now and I think we did a more thorough, better job on this one."

Figure 6: Amanda’s model for Project 3

Supportive scaffolding: Amanda used the coaching help buttons most in the first project, and later only to learn about advanced options such as weighting. She seemed to prefer to get help from the teacher or paper handouts. She thought that the guiding reminders were definitely helpful at the beginning, and even by the third project when she mostly ignored them, she never faded them, saying that they "didn’t bug me ‘cause I knew I could turn them off whenever I wanted."

Reflective scaffolding: Amanda spent a lot of time writing detailed descriptions and explanations of the components of her model and how they related. She thought that writing explanations for each relationship "helped so that you felt like you could explain the model more... and it made the relationships clear in your mind, ‘cause you knew exactly how everything affected everything." Amanda even felt that planning ought to be required, because if not she might skip it, and then not be making the best use of her time. She also liked the evaluation questions because it helped her plan what to say in her presentation.

Intrinsic scaffolding: Amanda used some advanced features, but tended to stick with the simpler layers of functionality that were easier for her to understand. During the building of her second model, a fellow classmate "helped" her by turning on all the advanced relationship options, and the teacher, looking at their model, suggested and showed them how to use the Table view. She and her partner tried to follow others’ suggestions, but on their own they stayed with the most basic options. When asked what she thought the program would be like if it started with all the advanced options visible, she said "I think it would have been confusing.... But if it would’ve been explained really well, probably we could’ve done it, I don’t know."

She said about her third model that they had chosen a simple project and just used what they needed to make it work. Her goal was to get the basic model working, and add advanced options later. In fact, although they did not use the slope, table, or delay features that had been suggested to them in the previous project, they did choose to add weighting and combining at the end to make their model more accurate.

Weighting had a relatively simple interface (Figure 7) and was conceptually intuitive. As Amanda said, "We want to weight it so nitrous oxide has less of an effect on ozone depletion than CFC’s do." The combining option was more awkward and required mathematical insight in order to recognize the need for it. The problem was that she wanted her model to show that CFC’s are high only if there are both a large human population and low usage of non-CFC producing alternatives. She had discovered during testing that the default which averaged the effects was not working, although it was hard to understand why. With help, Amanda was able to change the combining function to "product" and achieve her goal, but when asked later if she understood it she said "I understand the concept of what we did, but I might not have been able to do that on the model."

Figure 7: Weighting in Amanda’s 3rd model

In analyzing the effectiveness of GLAS, we recall first our fundamental motivation in designing learner-centered software—to make the task accessible. This goal was certainly achieved; almost all students were able to build reasonably accurate models to represent their understanding of the topic being studied. Second, the motivation for providing scaffolding was to allow students to gradually adapt the software to meet their changing needs. Here, too, we feel that the design was generally successful. Students turned off or ignored help features that they no longer needed, and accessed more advanced features as they felt ready. Both Greg and Amanda learned and changed their usage of the program over time, and each was able to adapt the program in different ways to meet their individual needs.

Greg and Amanda were most different in their fading of intrinsic scaffolding and use of advanced features. We had learned from our research with Model-It that the tool was successful where students could think in terms of the domain concepts, not "programming" a model, but representing their ideas in a way that was natural to them. Similarly, we find that the advanced features designed into TheoryBuilder were successful in proportion to how qualitatively and conceptually they were presented. For example, the advanced option of delay allowed students to specify whether a relationship ought to happen "immediately, quickly, or slowly." Underneath, this was a complex mathematical function, but it is presented in terms the students understand, so it became one of the most frequently used features.

Another option, weighting, was a little more advanced but it still fit well with students’ thinking, and was used correctly by many students. Students found the interface understandable, although a qualitative option might also have been useful since students might not have quantitative data (e.g., in Figure 7, Amanda writes that "CFC’s have a much larger effect on the depletion of the ozone than nitrous oxide" but the specific weighting values they chose were "an educated guess").

The combining option, however, was purely mathematical, and few students could follow there on their own. In presenting the combining option to the class, we had given the example that if the combined effect depends on two factors both being active, than a product might be most appropriate. Scaffolding could be designed along these lines, to bridge from conceptual to mathematical representations. Students might specify the desired effect (e.g., by filling in sentences about when x is high and y is low, then z is {low, medium, or high}). Once students had mastered this level, fading from conceptual to mathematical could help them see how their ideas are translated into equations.

One goal of this research is to take what we have learned about scaffolding and fading design and propose some general design guidelines. For example, we suggest that an interface that presents lots of options will initially confuse many users who are new to the task, even if there are useful defaults. Instead, allowing users to choose the options they want to use will help them tailor the interface in ways that make more sense to them. Through our analysis, we hope to identify a set of guidelines and context for their use.

A future direction for research is to focus more on the computer’s role in helping the user make fading decisions. For example, the system may be able to provide feedback about the appropriateness of the current level of scaffolding, by identifying and informing the learner of problems that might have been caused by fading too much too rapidly, and suggesting alternative scaffolding options. Even open-ended tools that have no content knowledge can, like TheoryBuilder, still provide procedural scaffolding using knowledge of general and modeling-specific problem-solving strategies. The learner’s self-evaluation can also be taken into account—if the learner can articulate, through predictions, how he expected his model to behave, the system may then be able to help the learner in modifying the model to meet his expectations [6].

In summary, the advantage of tools with scaffolding—fadeable layers of support—is to provide learners with an adaptable interface to fit their learning approach and changing needs. Learners take on more of the responsibility for doing the task as they are ready to do so, fading supports that are no longer necessary, and selecting advanced levels of functionality as they want access to more complex tasks.

This research has been supported by the National Science Foundation (RED 9353481 and IRI 9117084), the National Physical Science Consortium, and the U. of Michigan.