Technology-Rich Ethnography for Examining the Transition to Authentic Problem-Solving in a High School Computer Programming Class

This study utilized elements of technology-rich ethnography to create a rich description of a multi-user virtual environment in a high school computer programming class. Of particular interest was the transition that took place in classroom culture from one characterized by a well-defined problem solving approach to one more indicative of open learning environment. Using technology, high school students created learning activities and resources for use by younger students in the virtual environment. Evidence supported that high school students initially benefitted from the new open environment; however, some immutable elements of the classroom environment presented barriers to peer collaboration and motivation for high-level, creative work. Our findings lend support to the argument that teachers in high school computer programming classes should incorporate the following features in their curricula: open-ended problem solving, real-world clients, group work, student autonomy and ample opportunities for student creative expression.

One major concern in education is that many current instructional strategies inadvertently encourage the acquisition of "inert" knowledge (e.g., Bransford, Brown, & Cocking, 2000;Cognitive and Technology Group at Vanderbilt (CTGV), 1997). The problem with inert knowledge is that although it can be recalled when people are specifically prompted to remember it, it is not spontaneously used to solve problems, even when it is relevant (Bransford, Brown, & Cocking, 2000;Gick, 1986;Gick & Holyoak, 1980). In response to this concern, instructional strategies based on experiential learning, meaningfulness, and reflection have been developed and studied in the hope that they can overcome static knowledge that is often disassociated from context and application (Bransford, Brown, & Cocking, 2000;CTGV, 1992CTGV, , 1993CTGV, , 1997. The purpose of the present study was to examine one example of such an instructional strategy for problem solving in a high-school computer programming class as the classroom culture evolved from a traditional learning environment to a more authentic and openended learning environment.

RESEARCH ON OPEN-ENDED LEARNING ENVIRONMENTS
To address the problem of inert knowledge, some educational researchers (e.g., Hannafin & Land, 1997;Hannafin, Land, & Oliver, 1999;Jonassen, 1999) have attempted to design open-ended learning environments (OLEs) imbued with a variety of approaches and technologies that enable learners to engage in cognitively complex tasks that are question and goal driven and require critical thinking, self-direction, problem-solving, and meaningful integration of knowledge (Hannafin & Land, 1997). Some examples of OLEs include anchored instruction (CTGV, 1993(CTGV, , 1997 and project-based learning (Blumenfeld, Soloway, Marx, Krajcik, Guzdial, & Palinscar, 1991; Anchored instruction is an instructional approach first used by Bransford and his colleagues (CTGV, 1993) that provides stories as "anchors" through the use of interactive videodisc technology to create interesting and realistic contexts that encourage active construction of knowledge by learners. It is now broadly used to refer to any technology-based instructional approach, in which learning and teaching activities are designed around an "anchor" or situation such as stories, cases, or problems for learners to explore and problem-solve. Similarly, project-based learning is an instructional approach that engages students in acquiring knowledge and skills through an extended inquiry process structured around complex, authentic problems and carefully designed products and tasks that have reallife consequences and outcomes (Blumenfeld et al., 1991;CTGV, 1997). Both approaches encourage meaningful learning through student-directed investigation. Little research, however, has examined the transition from decontextualized environments to OLEs.
A fundamental difference between traditional and open-ended learning environments is that the problem drives the learning in OLEs, rather than acting as an example of the concepts and principles previously taught (Jonassen, 1999). As Jonassen (1999) argued, "Students learn domain content in order to solve the problem, rather than solving the problem as application of learning" (p. 218). The real-world problems presented in OLEs frame learning needs and set up enabling contexts which guide students in recognizing or generating problems (Hannafin et al., 1999).

PROJECT-BASED LEARNING AND MOTIVATION
Enabling contexts can be externally imposed, externally induced, or individually generated (Hannafin et al., 1999). Project-based learning presents an individually-generated, enabling context, in which students pursue solutions to open-ended problems by formulating questions for investigation, designing plans or proposals, collecting and analyzing information, and creating "artifacts" or products of their understanding (Blumenfeld et al., 1991). Project-based learning requires learners to generate a question or problem that serves to organize and drive learning needs so that the learners can produce a final product or series of products to address the driving question or problem generated by learners (Blumenfeld et al., 1991). In project-based learning environments, students must engage in active, constructive, and authentic learning activities, such as debating ideas, planning and conducting experiments, and communicating their findings.
As Blumenfeld et al. (1991) argued, there are several reasons why project-based learning could support student motivation. One of the most consistent findings in the study of motivation to learn is that students are more likely to continue learning when they find the tasks meaningful, interesting, and moderately challenging (e.g., Ames, 1992;Greene, Miller, Crowson, Duke, & Akey, 2004). In project-based learning students are encouraged to tackle projects of their choosing with a high degree of real world significance. That real world aspect tends to bring with it challenge and relevancy that fosters motivation. Another common finding from motivation research is that students are more likely to continue engagement in learning activities when they believe the learning environment supports their autonomy (e.g., Deci & Ryan, 1987;Greene et al., 2004). Support for autonomy includes allowing students to make choices and encouraging them to be responsible for their learning.
Autonomy is a key aspect of Self-Determination Theory (SDT, . This is an overarching theory that describes basic human needs for competence, autonomy and relatedness, as well as the notions of intrinsic and extrinsic motivation. Intrinsic motivation occurs when a person derives personal satisfaction for engaging in a task and is stimulated by tasks that people find challenging, interesting, and meaningful. SDT can be used to explain motivation across different contexts. Extrinsic motivation occurs when a person engages in a task for the purpose of achieving something outside of the task itself (i.e., a reward). Extrinsic motivation is less likely to support on-going task engagement than is intrinsic motivation (Deci, Koestner, & Ryan, 1999;. Project-based learning provides an ideal context for fostering autonomy, relatedness, and intrinsic motivation because students are able to: a) control many aspects of the task they choose to work on; b) work in groups; and c) select projects that are interesting enough to sustain intrinsic motivation.

EXAMPLES OF PROJECT-BASED LEARNING
One project-based approach provides students with a technology-rich, enabling context and the real experience of working as instructional designers. In these contexts, students participate and collaborate in producing real-world multimedia products for real audiences (e.g., Erickson & Lehrer, 2000;Lehrer, 1993;Liu & Rutledge, 1997). Lehrer (1993) studied 8th grade students' design processes for creating hypermedia projects about the Civil War that were used by their classmates. Liu and Rutledge (1997) investigated the effects of a "learner-asdesigner" environment for at-risk high school students who developed multimedia projects for a local children's museum. Similarly, Erickson and Lehrer (2000) explored how 7th grade students' conceptions evolved throughout the year with regard to a hypermedia environment for peer use in a social studies class. The most striking finding across these studies was the degree of student involvement and engagement in the projects. In addition, students reported a sense of ownership over their projects and learning interests. Through technology-rich, project-based learning, the students deepened their understanding of the topic under study and developed their own perspectives about the learning materials (Lehrer, 1993). Their technical skills and design knowledge improved (Liu & Rutledge, 1997) and their conception of multimedia functionality evolved from mere decoration to the actual conveyance of meaning (Erickson & Lehrer, 2000).
The common characteristic of "student-as-designer" research is the focus on student use of authoring software to develop or create real-world multimedia products for real clients. Instead of learning from computers, students learned with computers (Solomon, Perkins, & Globerson, 1991). In such cases, computers functioned as mediating tools for learning and problem solving. For example, in Lehrer's (1993) study, the authoring software used by students had two components: (1) tools for construction and (2) tools for reflection. These tools helped to guide students' design processes and critical thinking. However, the authoring tools studied were static rather than dynamic. A decade later, 3-D virtual environments emerged as an advanced technology. Compared to traditional hypermedia tools, 3-D interactive environments are more powerful, interactive, dynamic, and interconnected. At present, research into the use of 3-D interaction for creating technology-rich learning environments is in its infancy (Barab, Thomas, Dodge, Carteaux, & Tuzun, 2005;Cobb, Neale, Crosier, & Wilson, 2002;Youngblut, 1998). On the other hand, past research is indicative of the cognitive and motivational benefits of a "student-as-designer" approach. To date, there have been insufficient studies that have observed students transitioning from decontextualized learning environments to more authentic (OLE) learning environments, especially those involving 3-D interaction.
An integral component of this transition is the notion of evolving cultural change. Observing this change in situ can help us understand how to better support the integration of technology and the development of highly contextualized learning environments while leveraging new technologies. In the present study, design-based research and the flexible, dynamic approach of technology-rich ethnography were employed in an attempt to observe this potential change in process and to leverage lessons learned from this observation to inform future deign work as well as the development of theory. Design-based research is an approach to both the design of instructional contexts and data-driven theory construction, with researchers functioning in the dual role of designer and researcher (Brown, 1992;Brown & Campione, 1994;Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). Contexts of learning are not fixed or immutable, but rather open to re-design by the researchers. A major goal is for research findings to be cycled back into the design of instruction and support the development of theory.
The purpose of the present study was to explore student motivation and engagement in problem solving in a high-school computer programming class as the classroom culture evolved from a traditional learning environment to a more open-ended learning environment. The new classroom culture was mediated by Quest Atlantis (QA), an interactive 3-D meta-game environment designed for learning (Barab et al., 2005). This program was used since it was an existing program ideal for our study of open-ended problem solving. High school students who served as participants for this study were asked to be designers and use QA to create fun yet educational experiences for younger students. Of particular interest was any cultural shift that might take place when the classroom environment transitioned to an open-ended learning environment. By "cultural shift," we refer to the aspects of the classroom that were structurally changed by the introduction of the QA activity and/or those aspects that naturally modified in conjunction with the unfolding of the QA activity. A major concern was also the development of appropriate methodological tools for capturing the subtleties of the evolving classroom culture. The following research questions guided our work: Question 1: How did students approach the classroom activities before and after the introduction of the QA project? What was the cultural shift?
Question 2: What was the nature of student motivation for classroom activities before and after the introduction of the QA project?
Question 3: How was the QA environment used to support problem solving? Question 4: What aspects of the classroom environment supported peer collaboration before and after the introduction of the QA project?

METHOD
Researchers have struggled to develop appropriate methodologies for researching contextualized learning in technology-rich settings (Barab, Thomas, Dodge, Newell, & Squire, 2004;Cobb et al., 2002;Youngblut, 1998). In order to study the use of QA in a high school classroom we developed a collection of data generating tools, that we call technology-rich ethnography (TRE). TRE is a collection of data gathering methods that are appropriate for the study of the "goings on" in technology-rich learning environments such as QA. The affordances of the QA environment allowed for the following components of TRE to be employed in the present study: • observation protocols from three researchers; • interviews and focus groups with students and the teacher; • students' logs of daily work; • audio-recorded group interactions; • chats within QA among students; • the projects produced by the students; and • students' evaluation of the student projects The observation protocols were completed by three researchers who were in the classroom for each of the 10 days of the project. The observation protocols prompted the researchers to record their observations on the following aspects of the class: the ecological environment, the principal participants, the most salient points in the environment, and the overall interpretations and reflections of the individual researcher.
The three researchers also conducted interviews with individual students, the teacher and focus groups with small groups of students that conformed to their work group. The researchers usually approached the participants while they were working on their projects and started by asking some informal questions, such as "How is your project going?" "Have you had problems while working on the project?" "What do you like about this activity?" and "What do you think the younger students will learn from this project?" The researchers then moved on to some formal questions concerning their motivation for taking the computer programming class (e.g., "What do you like about computer programming?" "What goals to you have for taking these types of classes?"), the approach they used when they encountered a difficult problem (e.g., "Do you ask a lot of questions of the teacher?" "Do you ask other students questions?" "Do you get answers or explanations from the other students? Are they helpful?"), the general approach they used for this type of class (e.g., "Do you like the challenge of learning something new, or like the feeling of getting better with practice?), and how similar or different their study approach is in taking this class as compared to other classes (e.g., "Do you approach learning in the same way or a different way?").
Students completed logs of their work each day. We recorded their group interactions with audio-tapes using eight different recorders so as to capture their interactions within work-groups. Students used the text-based chat function of QA. All of the chat was logged by the QA server and assembled into text documents. We also had the projects the students developed and their evaluations of one another's projects.
However, it should be noted that observations and interviews constituted the primary TRE data sources that provided most rich and useful information. Other data sources served to complement the primary data as they only provided minimal information. For example, the students either skipped daily group logs or provided minimal information. This was likely because they did not perceive the daily log tasks as meaningful. The group did not often use the chat function in their collaboration. They only used the chat to explore the QA environment when QA was first introduced or when they had some free time to chat during the project.
The researchers did not use much of the audio data recordings of the group interactions due to the poor quality (e.g., noisy background, unidentified speakers, and long periods of silence). The researchers learned from this experience that audio recording alone was not an effective way to capture group interactions. Video-taping along with audio recording was suggested as a viable means for observing and recording group interactions in future studies.

Participants and Context
An advanced computer programming class located in a suburban high school in the southwest volunteered to participate in this study. At the beginning of the school year of the research period, the high school where this study took place had 2,128 enrolled students (83% white, 21% free or reduced lunch eligibility, and a 2.9% dropout rate). There were 24 students in the computer programming class, three of whom were girls. Most of the students were juniors or seniors, but there were also freshmen and sophomores. The teacher was an experienced teacher who was pursuing a master's degree in the area of the design of computer-mediated instruction.
The primary resource used was Quest Atlantis (QA). QA is an educational program that is designed to allow K-8 students to explore virtual worlds and learn how to solve problems in those worlds (Barab et al., 2002). QA is grounded on research-based educational theory and uses a multi-user virtual environment common in many contemporary computer games. QA was designed to challenge students to help others solve problems related to the environment, society, and science. These activities students engage in are designed to be "pro-social" in that they encourage empathy as well as problem-solving. The myth of Atlantis is used as a legend that underpins the game. The QA environment is composed of virtual microworlds such as Culture World and Ocean World. These worlds are populated with villages that contain educational activities called "Quests." For the present study, our participants were given the task of developing or modifying quests for younger students that would fit into the existing QA environment (Barab et al., 2005).
The classroom had a total of 25 student work stations with desktop computers, a small desk for the teacher, a cart for the teacher's computer to project on one wall, and a white board filling the opposite wall. Of the 25 student work stations, 21 were arranged in rows of 4 or 5 down the center of the room. Two stations faced the wall opposite the white board and another two were situated in one corner of the room next to the white board.
Before the QA project in this class, the researchers were simply observers. After the QA activity began, the researchers also served as facilitators in the classroom and, in this way, influenced the environment under investigation. An inherent challenge in the process of participatory ethnography is the inevitable tension between both emic and etic (experience-near and experience-distant) perspectives (Geertz, 1976(Geertz, , 1983. Researchers must balance the actions they take in the environments or cultures they engage with the meaning they glean from their observations. The researchers here acknowledge their influence on the environment under investigation. The multiple role of the researcher and this emic/etic tension is well-grounded in several traditions of qualitative research (Lincoln & Guba, 2000;Kincheloe, 1991).

Procedures
All three researchers visited the classroom three times before the start of the Quest Atlantis project. Observation protocols were filled out on those days. Following each observation, the researchers discussed the observations extensively among each other and with the teacher in order to gain a solid understanding of the original classroom culture. Also prior to the initiation of the project, the teacher assembled the students into groups of three or four students, making sure that two students in each group were proficient in basic computer programming. One of the groups was an "all girl group," which was composed of the only three girls in the class, who had never worked together before the QA project.
On the first day of the project, one of the researchers demonstrated QA and then explained the task to be performed by the students. By the second class period, the students had explored the QA environment and had generated ideas for their projects. The students were given question prompts designed to facilitate their group collaboration on the project, and they were asked to answer or work through the questions once each day so that they would be guided through the project process. The question prompts (see Appendix A) were organized logically by different project stages and based on a synthesis of problem-solving literature (Bransford & Stein, 1993;Jonassen, 1997;Ge & Land, 2004): (a) brainstorm ideas for the project, (b) define the project, (c) plan for the project, (d) design and develop the project, (e) monitor progress on the project, and (f) evaluate the project. The project lasted 10 class periods. On the last two days, students demonstrated their projects to the entire class and completed evaluation forms of at least two of their peers' projects.

Analysis
Data analysis began as the three researchers discussed the daily observations. First, global observations about the classroom were noted, and then specific groups of students were discussed. The observation protocols were completed each day by each researcher, but the three researchers met frequently to discuss observations and interpretations. If there were different interpretations, the researchers negotiated the meanings until they reached a common understanding. By the end of data collection, global summaries had been completed through on-going discussion. Those summaries were then applied and discussed in relation to each research question and compared with the observation summaries collected before the QA project.
The transcribed interviews also yielded much of the data for the analysis. Transcripts of audio recordings were imported into the qualitative research software package NVivo. The researchers proceeded through the process of open coding (Glaser, 1978, Glaser & Strauss, 1967 to identify categories. These were later collapsed into broader conceptual categories. These broader categories were further refined into themes informed by other data, such as students' projects, logs, and evaluations. Using Stake's (1995) notion of "categorical aggregation," the researchers assembled instances from the data that were then identified for each category. The researchers then drew on the emergent themes to address the research questions.

Before the Project-Summary
The teacher had a programming objective for each day that built on the concept that they were learning. The goal was for each student to figure out how to get the new Java code to run. As the teacher projected the code for the day's lesson on the wall at the front of the classroom, the students tried to get the code to run on their own computers. As the students worked on their programming code, the teacher circulated among the students, meeting with students who sought help. Students often conferred with one another, especially the group of boys who were situated in the middle of the room. All chairs rolled and this group of boys often rolled around the class to look at the work of others. Although there were no formal groups, many of the boys often collaborated on how to make the program run and then change the program to make it do something different.
Once students finished the day's work, many students experimented with changing parameters of the code. Other students simply seemed pleased that the program finally would run and did not explore further. Students were allowed to play games, check e-mails and do homework once their work was finished for the day. Students generally appeared comfortable and calm during the class with some getting excited when they figured out the program or found an interesting take on modifying the program. There was one student who often called attention to himself. He was very exuberant whenever he did something well, whether it was working on the day's code or playing a game with peers in class. By the end of the class period, it seemed that nearly all the boys were playing an online action game with one another. There was another student who was either chatting with one of the girls or asking the teacher for permission to leave for the bathroom. The teacher admitted that he gave this student a lot of latitude because of the boy's personal life. This student was rarely seen working on anything relevant to the class.

During the Project-Summary
The students were attentive during the introduction to QA and to the project, but it was obvious that they were ready to immediately explore within the program. One researcher led the introduction and spoke briefly to the class each day to report on progress, show examples, and answer questions. Students were attentive to the daily introduction, though, most were also "multitasking" on their computers.
The teacher's formation of groups and the institution of a group task was a major change in how students approached the learning. Most of the groups seemed to be brainstorming and collaborating easily with one another by the second day. The students in the "all girl" group did not talk very much to each other and, in fact, seemed mostly to speak with each other only when one of the researchers was attempting to facilitate their working together. One of them did nearly all of the typing. The girl next to the typist did a lot of conceptualizing, and the third girl was pretty much left out. This was both observed by the researchers and noted by this participant in an interview.
In most groups, one student took responsibility for typing while the others helped him/her with idea generation. After the idea for the project was developed, the group members tended to divide up the tasks and some would type and others would look for resources related to their project. One member of the group that created a chess tutorial talked about how their group divided their design work: Student #2: What I did is the basic outline of what the rules of chess (are) and give out points for that and then J. did something about how the individual pieces move and T. is doing something about . . . strategy so they (the users) don't have to learn but if they want to, they can.
None of the groups used the question prompts that were both handed out to them on paper and e-mailed to them. One group said they used it the first day, but most students ignored the prompts and simply explored and brainstormed in their own ways. The students also did not change roles each day as they were asked to by the researchers. Instead, they followed whatever pattern seemed more natural to their dynamics. While this natural approach worked for many of the groups, there were three that had trouble working collaboratively on a project. It seemed that the adults were effective in facilitating the students' work only when they were either asked a question by a group, or when they posed questions or gave feedback. This is consistent with results reported by Greene and Land (2000). Greene and Land (2000) found that despite having multiple supports available in the learning environment, when students were struggling with an aspect of their project they were aided most by interacting with the instructors regarding the specific problem. As in the present study, the groups that seemed to work well together in Greene and Land's study (2000), did not need instructor intervention even when they encountered problems.

The Projects
The students were tasked with creating quests that would fit into the Quest Atlantis online 3-D space. They created text for the quests as well as accompanying resources for users to aid them in completing the quests. The texts of the quests were entered in the QA database so that they would have the look and feel of the hundreds of other quests in the QA online space. These quests were then linked to rotating 3-D disks that represent quests in the online space. When a user encounters and then mouse clicks on a spinning QA disk in the 3-D space, a quest is launched. A 2-D window then pops up with the text of the quest. Each of the quests created by the high school students also contained at least one electronic resource that was created by the high school students to be used by users who attempt to complete the quest. The stylized QA letters on the disks are color coded according to the number of points a user can earn for completion of the quest. More difficult quests award more points for completion (see Figure 1).
The high school students created a total of eight different quests. The first quest was called "Checkmate" and was about the game of chess. This group decided on a chess theme after seeing chess pieces in one of the QA microworlds. For this quest, the students created animated images that showed how chess pieces can move in a game. They also created other images with text that explained other rules of chess (see Figure 2). The second quest was named "Basic Nutrition" and was on the food pyramid. The third quest was entitled "Track and Field" and provided some information on this popular sport. The fourth quest was called "Screen-Shot Bug Report." For this quest, users were asked to report any mistakes in the online 3-D world of QA and then report these "bugs" to the 3-D space administrators by submitting screenshots of whatever problem they wish to report. Accompanying this quest was a screen shot these students took of one such "bug" to serve as an example. This group planned on developing a Java-based program that could be used to report "bugs" but realized that they needed to learn more programming before they could actually do it.
The fifth quest was called "Texas Holdem." This quest taught this popular card game by way of a PowerPoint presentation that included animated slides and accompanying text. The sixth quest was called "Blacksmithing" and included some text and online resources on the craft of blacksmithing. This group thought blacksmithing was an appropriate quest after exploring a QA microworld that featured a knight avatar and medieval themes.
The seventh quest was called "The Iron Fish" and was on submarines. For this quest, the high school students in this group created a Website on submarines that featured cutaway illustrations of submarines (see Figure 3). Finally, the all-girl group noticed that there were no animals in the online 3-D space of QA. They considered this to be a shortcoming, so they chose to introduce a quest on horses. None of the projects involved any programming. One group expended considerable effort developing a Website on submarines that was part of their quest, but most groups just developed their quests, added links to external sites with information, and then created PowerPoint presentations that would serve as resources for users to engage when accomplishing their quests.

FINDINGS BY RESEARCH QUESTIONS
The following findings are framed by way of a contrast between the culture of the computer programming class before and during the implementation of the Quest Atlantis project. A discussion of these themes is presented for each question followed by summary tables.
Question 1: How Did Students Approach the Classroom Activities Before and After the Introduction of the QA Project? What was the Cultural Shift?
The themes' task specificity, ownership and autonomy, project outcomes, individual accountability, and classroom dynamics emerged related to this question. Table 1 illustrates the differences before and during the QA project.
Before the QA project, the students were assigned a specific daily task with a specific solution. For example, according to the researchers' observation notes, in a class before the QA project the instructor first demonstrated the day's work-creating a bouncing ball with the Java programming language. Then the instructor showed them scripts to change the parameters in the program to make the ball large or small, faster or slower, and let the students experiment modifying or changing the scripts he had provided. The students were attentive to teachers' notes in order to make sure that they followed the correct directions and procedures to get their program to run. During the QA project, tasks were more open-ended and unique to each group with no specific problem assigned to the whole class. Groups had the general task of doing design work for QA and they were able to choose what they would do. After students understood what was required of the project, they did not pay much attention to the instructional materials provided to them (e.g., the question prompts) probably because they had sufficient help resources, such as the teacher, the researchers, and peers. They made their own decisions on what was important and what were the useful resources. Although the usual work of the class involved Java programming, all of the groups chose projects that involved the use of PowerPoint or HTML instead of programming Java applets. In this regard, students had more autonomy during the QA project than before. Before QA, the students individually owned the assignment and the solutions, but during the QA project, the groups of students collaboratively owned the projects and the solutions.
Since the QA project also involved group work, students' satisfaction was indicated by progress on their project and good collaboration. The group dynamics, naturally, were greatly affected by absenteeism. One group was led by a student on the high school track team and the group decided to create some instruction on the sport Track. This student was then absent for two successive days subsequent to the first day and this left the other two students in the group confused as what to do next. These other two students were less interested in the topic and they seemed to use the absence of their team member as an excuse to delay all work on the project. Another group that created a Website on submarines was delayed by the absence of one team member who was particularly skilled in Web development.
If a group member was absent, other members were not always prepared to "take up the slack" and the work suffered in quality and in its completeness. However, if a group collaborated very well, the whole group reported a sense of ownership. Therefore, before the QA project, students were more independent, and their performance only affected their own learning outcomes, but during the QA project students were given more autonomy and accountability for their team's project since individual performance could affect group performance and the project outcome.
Both before and after the implementation of the QA project, students worked on tasks until they either completed what they saw as the daily assigned task (after which they would begin playing video games) or until they got frustrated and gave up on the perceived daily task (and started playing video games anyway). However, this persistent pattern with regard to the dynamics of the class was impacted by the implementation of the QA project. The work for the QA project was ill-defined, thus making the daily task somewhat ambiguous with relatively fuzzy parameters largely dependent on what the students themselves chose to do.
As part of the classroom dynamics, most of the students seemed to have adapted to the group work well during the QA project. The conditions seemed to have existed for group collaboration before the QA project, as some students naturally rolled their chairs around to help one another to solve some programming problems. Thus, when students were assigned to groups during the QA project, most of the students adapted to the group activities and the new learning environment. However, a few students who were independent before the QA project remained quiet and inactive during the QA project. The students appeared resistant to the new learning environment. They had to be prompted or facilitated by the teachers or the researchers to work with the other group members.
Another part of the classroom dynamics was the video game playing culture, which seemed to have carried over from the old learning environment to the new learning environment. Before and during the QA project, when students finished their assignments, they were allowed to play games and do other non-class related work. This aspect of the classroom culture will be discussed further in the next section on motivation.
The data from the chat, although limited, added to our knowledge of the dynamics of the class during the QA project. For example, two incidents occurred in the online 3-D space of QA when some students used inappropriate language using the QA chat function. Elementary teachers from other schools that use QA complained about this to the administrators of QA prompting the researchers and the teacher of the high school students to warn them again about the real, immediate, and authentic nature of the live online environment of QA. The high school students were not fully aware that their inappropriate behavior might affect the younger children who use this online environment. These incidents indicated that although the classroom environment had changed, the high school students might not have perceived the degree of the authenticity of the environment, which in a way reflected some of the classroom culture during the QA project.
Question 2: What was the Nature of Student Motivation for Classroom Activities Before and After the Introduction of the QA Project? Table 2 demonstrates the themes related to the question on student motivation, which are explained below. Both before and during the project, collaborations with peers seemed from observations to be a particularly motivating aspect of the class for many boys. These boys were most animated when conferring with others and sharing their work. None of the girls were rolling their chairs to confer with the other students. This was substantiated in the interviews when most of the boys reported that they enjoyed working with peers on different tasks in this computer class. Interestingly, many also said that when faced with a challenge, they prefer to first figure the problem out on their own. If they cannot figure it out, they then go to peers, and finally they go to the teacher if peers are not able to help. Only one student said that going to an adult (her mother) was her first line of seeking help. The finding, which was clear in the formal interview data, of students saying they prefer to figure problems out on their own was not consistent with our observational data that showed many students clearly waiting for the teacher's help early on in their daily programming task.
Before the QA project, students would receive obvious and immediate visual feedback from their computer screens showing them whether their programming efforts succeeded or failed; the outcome of which was either rewarding if their programming was successful, or frustrating if the program failed to run. But during the QA project there was no immediate visual feedback. The outcome was not immediately obvious; in fact the outcome was not visible until the end of the two-week project when group members had assembled all the project pieces.
Before the start of the project, motivation during class for many of the students seemed to be mainly about getting the task of the day done so that they would have time to play video games, surf the net, or do homework. Quite a few of the male students turned to playing games whenever there was a free moment. This is a standard indicator that they experienced intrinsic pleasure from these games (e.g., Stipek). Unfortunately, for these students, class work was simply in the way of game playing, so the task of getting the code of the daily Java assignments to run was mainly a means of allowing them to experience what they enjoyed, which was playing the games. This situation is a classic example of a behaviorist idea known as the Premack Principle (Premack, 1959). Students are allowed to engage in something pleasurable when they finish the daily work, so something intrinsically motivating is used as an extrinsic reward for doing school work. It sets up a very powerful way of extrinsically rewarding work done in class. The problem with this powerful motivator is that there is a wealth of research that shows that extrinsically rewarding school work (and other behaviors) undermines any development of intrinsic enjoyment of that school work (c.f., Deci et al., 1999). A specific example of this was found in one of the logs, where a student wrote that he decided to play a game after doing minimal work. The opportunity to play games was a feature of the classroom culture for most of the boys, and our project was not able to compete with the games for their motivation.
Our observations showed at least two other types of motivation during class time. There were two students, one boy, one girl, who did not seem motivated by any aspect of the class. Instead they chatted with one another most of the time. There also were, of course, students who seemed, from observational data, intrinsically motivated by the programming tasks themselves. These were most obviously the boys who rolled their chairs to confer with one another and demonstrated joy at learning how to make the code do something more than what it was originally intended to do (e.g., a ball bouncing was made to bounce very fast by some of these students and bounce very slowly by others in a "Pong" videogame Java programming project). These students typically did not wait for the teacher to help them, but found a peer when they could not figure something out or wanted to learn something new. They were unlike other students who waited for the teacher to come help them and would give up as they watched the teacher's predictable progress around the room and saw that it would be a long time before he made it to them.
When asked about the QA project during the interviews, students commented that they really enjoyed being able to make choices with most aspects of the project. In fact, being in charge, or being autonomous in their learning, was frequently noted in interviews as the main aspect of the project they enjoyed. For example, one student compared the QA project to the level of control found in other classes. Student #1 said, "Like when a project has to have double space one page of what you think about the ostriches dying in Australia, explain using a metaphor, simile and examples of some multiplication, that drive me insane." One of the girls noted that unlike what they normally do in the class, "We're like, coming up with our own ideas!" Another student (Student #6) summarized his perspective on learning in a manner that we thought demonstrated both his desire to be an autonomous learner and his sophisticated view of learning. He said, "It's better when you've got an inquisitive level and if you put it together yourself, then you'll remember a lot better. Rather than somebody showing you." Some students also noted that the ambiguity of the task caused some discomfort at first because they usually know exactly what they will be doing each class period. Once the groups decide on a topic for their projects, the feeling of discomfort seemed to give way to the enjoyment of planning the project: Researcher: How do you feel about the fact that you were left to make a lot of the decisions on your own? We call it kind of an open-ended in the environment. Student #4: It was kinda weird at first. We didn't quite know what to do at all . . . Then once things started coming together, (it was) kinda nice.
Once the planning phase was clearly underway, we noticed that engagement in the project, an indicator of motivation, started to drop off. Some of this was likely due to the manner in which students chose tasks to do for the group-with some students not having a task defined for their participation for several days. Students often had an excuse for their passivity related to the project (e.g., the content expert was out that day or the key materials were left at home) so that time could be spent gaming, Web surfing or doing homework for other classes. It seemed that the researchers and the teacher were more troubled by the passivity or off-task behavior than were the students.
Another reason for dwindling motivation during the two weeks was likely that students did not figure out how to make the project something that would be at the right level of challenge for the younger students for whom they were designing the projects. During brainstorming, they often had interesting ideas (e.g., a virtual petting zoo, an interactive chess teaching quest, etc.), but they lacked the skills and time necessary to learn the skills to move forward with the most intriguing ideas they developed. They, instead, often chose something very simple for their skill level and either worked to complete the simple project quickly or strung it out by doing minimal work each day. The role of the younger children dropped completely from their consideration.
In interviews, many students admitted that procrastination was their favored method for doing school work. It is important to note that while procrastination came out of an interview question we asked about students' general approaches to learning, we did not observe procrastination during the QA project. Several also said that they found ways to get homework and even projects done during school time so that no work had to be taken home. Here are a few examples of the procrastination theme: Student #3: "What I can put off till the next day, I just do that." Student #1: "I do homework during lunch. . . . and I'm a horrible, horrible procrastinator. I procrastinate projects that were due for 3 months before." Student #5: "I've been more and more of a procrastinator. It's like, now, I'm just usually like, like, the night before studying and stuff (laughs)." The teacher had said that most of the students in the class were high achievers and it was apparent from the interviews that they tended to demonstrate high levels of metacognition (i.e., self-awareness) about their procrastination. They reported that, while procrastination worked for them in high school without threat to their grades, they assumed they would need to learn new strategies for college.
Most students we interviewed seemed very savvy about what we interpreted as their "playing the game" of high school. The game was to get the grades they wanted with the least amount of effort at school or home. They were also aware that the "game" would change when they go to college: Researcher: Do you think that your study habit is going to serve you well in college? Developing quests for younger children was not the motivator that we expected. We believe that without having feedback from clients, the notion of creating for younger students remained too abstract for encouraging ongoing motivation. Without having something concrete like actual feedback or a grade, the students did not seem to persevere with project ideas that were actually complex or challenging.

Question 3: How was the QA Environment Used to Support Problem Solving?
Overall, the QA environment, including the 3-D microworlds and the nature of the task, served as an anchor to stimulate the students' initial representation of the problems for the QA projects, as described previously in "the Projects Chosen" section. The stimulation of the QA environment further engaged the students in the real-world problem solving activities. On the other hand, we also noticed that the QA environment seemed too open allowing students to work with any programs they chose. This in fact discouraged them from engaging some useful but challenging programming tasks which was what they were learning in this course. Instead, most of the groups chose to use HTML or PowerPoint to create their quests. This finding was consistent with Lubin's (2005) finding that some students avoided using their newly learned computer skills, given the choice they enjoyed in the open-ended learning environment.
Since our goal was to examine the cultural shift before and after the introduction of th QA project, we presented further findings to show how QA environment supported problem solving via a comparison of students' problem solving performance before and during the QA project. Two themes were identified which were summarized in Table 3: problem solving and creativity. The QA project was presented as an ill-defined problem, with unclear situations, fuzzy goals, and insufficient information, which required students to seek additional information, specify situations, and define problem goals in order to develop sound solutions. It was also presented as a "real-world," authentic design problem in that students were asked to design learning activities for younger children who were going to use their original materials for learning. The authenticity of the QA project helped the students to think about the target audience for whom they were developing the quests, and thus supported students' reasoning, decision making and value judgment as in real-life situations. For example, in the peer evaluations, two groups of students questioned the educational value of the projects they were evaluating for younger kids. One group of students made a comment to the "Texas Holdem" project group, "Will it (poker) encourage gambling?" Another group made a similar comment to the "Blacksmithing" project group, "(The) weapons described are dangerous to kids." In addition, most of the groups made thoughtful suggestions regarding the appropriateness of the content and the difficulty level of the materials for the young learners. Examples of these suggestions were "need more explanation," "need more pictures," and "need clearer goals." One group asked the "Screen-Shot Bug Report" group what the educational value of their project offered, commenting: ". . . the kids don't care." By comparison, before the QA project students had a daily task of getting some Java code run correctly. A small part of a program was presented each day. The students had to get that part of the code run on their own computers every day. One problem with this was that students who failed to program correctly one day could fall behind and never catch up as one day's task was built upon previous day's tasks. During the QA project, the students were asked to create instruction that would be integrated into the QA environment. The cumulative material problem was not present during the QA project; however, we found that students who relied on group members who were absent on later days tended to disengage altogether.
Before the implementation of the QA project, specific tasks were assigned with specific solutions and outcomes expected. This was after the teacher demonstrated the specific daily Java-related goal. Students did not need to define problems since all the problems were defined for them. Their task was to come up with correct solutions an be responsible for themselves by getting the code presented that day to work. It was observed that the students varied in their abilities and persistence in this type of problem solving. Some students had high reasoning and problem solving skills; they experimented with various parameters of programming through a trial-and-error method, examining the underlying rules and principles, until they figured out the causes of failures and got the program to run. Then they would offer to help their neighbors and would explain to others how they came up with their solutions. Some students sought help from their peers who had succeeded in solving the Java-related problem of the day when they got frustrated after repeated attempts. We also observed that there were a few students trying to figure out the problem on their own by seeking resources from the Internet. However, there were also students who got frustrated and eventually gave up trying and, instead, waited passively for the teacher to come to their help.
During the project, however, no specific tasks were assigned and students could choose the topics of their projects as well as the forms the topics would take. The groups had to spend time brainstorming the project, defining the tasks, identifying the scope of the project, and all the relevant information and useful resources for the QA environment. They also had to keep in mind the client for whom they were going to develop the project. The teacher along with the researchers demonstrated possibilities for their project and emphasized autonomy. Students were encouraged to negotiate their decisions for their projects, discuss multiple possibilities, and select the most viable project for all the group members. When developing the solutions for the QA project, instead of a trial-and-error approach often seen with the Java-code problems, the students shared different perspectives and justified their rationales for the materials to be selected and created and the ways to present information as solutions in light of the real clients and the context.
With regard to creativity, there were limited opportunities for this in the pre-QA classroom. Although students were able to (and even encouraged to) find alternative solutions to the coding problem of the day and experiment with the code, a single path was usually presented and a single outcome expected. During the QA project, the students were free to create almost anything that could appropriately fit into the QA environment and to generate alternative solutions that seemed appropriate for their clients. It was observed that before QA, more students relied on the teacher for help, but during the QA project more students relied on their peers in collaboratively completing their projects. We even found that the teacher wanted the students to find solutions themselves telling the researchers to "back off" when we began to help them too much.

Question 4: What Aspects of the Classroom Environment Supported Peer Collaboration Before and After the Introduction of the QA Project?
It was noted that (a) the teacher's attitude and the researchers' facilitation, (b) peer evaluation and feedback, and (c) the physical setting of the classroom were the central aspects of the environment that supported peer collaboration. Before the QA project, peer interactions were encouraged by the openness of the teacher's attitude, the natural disposition of students offering and seeking peer help, and the physical setting of the classroom. During the QA project, peer collaboration was supported by the intentionally designed mechanisms of formed groups, structured peer collaborations, evaluation and feedback, and facilitated group processes. These mechanisms became important components of the new learning environment. Table 4 demonstrates the themes related to the question on student collaboration and peer interaction.
The posture of the teacher before the QA project was an important component of the classroom environment. The teacher took a fairly "hands-off" approach to things. He readily assisted students and allowed them to engage in "off-task" behavior. The openness of the environment before the QA project supported collaboration and peer interaction by allowing the students to spontaneously offer help to one another. However it also made for an environment in which distractions from daily tasks (such as the Internet and computer games) were plentiful and sometimes formidable.
Before the QA project, those students who were intrinsically motivated to engage in Java programming traveled around the classroom while working so that they could collaborate with other students, offer assistance, receive assistance, and show off their work. Because the chairs in the classroom easily rotated and were equipped with wheels, they did not have to get up to do this. Clearly, the classroom feature of making use of these moving chairs supported collaboration and peer interaction. This was true during the QA project as well. On the other hand, however, the formality of the grouping during the QA project combined with the physical layout of the room may have hampered some collaboration, particularly among students who were seated across a table from the other two group members. The openness also made for an environment that allowed students who were not motivated to avoid daily tasks. Before the QA project, the process of group work was not facilitated. Group work had to happen naturally because the teacher was engaged with assisting individuals almost exclusively. During the QA project, students were arranged into groups. While group collaboration was unstructured before the QA project and was the result of students' intrinsic motivation for their work, during the QA project collaboration was structured with the teacher and the researchers asking students to play different roles and taking daily logs and using question prompts. However, our observations revealed that many students did not perceive the logs as meaningful or authentic. They initially just did them as a requirement and later ceased doing them altogether. The structured and supported peer collaboration encouraged students to rely less on the teacher and more on the group members for assistance during the QA project.
Feedback from the peers or the researchers helped students to refine their projects or see things from different perspectives. For example, one of the researchers pointed out to the group working on a healthy eating quest that the food pyramid had become a controversial topic. Upon the feedback, the group did more research on this topic, and as a result they added a link about the controversy of the food pyramid. A member in the Poker group did not like the idea of poker and was aware of its risk as a topic for younger learners. His concern made the other group members think about ways to make this topic more educational, positive and meaningful.
As described previously, the peer evaluation, which was an activity the researchers asked the groups to do to evaluate each others' projects also showed that the students generated meaningful feedback to the peers, particularly in thinking about the target audience and the educational value of the projects. For instance, while some students encouraged the "Texas Holdem" project group saying that they did a good job in explaining the jargon and had humor, they also commented that the materials were inappropriate and too hard for kids, and that the instruction needed to be simplified. However, due to a lack of time and a drop-off of the students' motivation at the end of the QA project, all the good feedback did not get implemented.

DISCUSSION
Past research on "student-as-designer" has primarily examined the cognitive and motivational gains as students developed competence and confidence in OLEs, but we have not encountered studies that specifically focused on the cultural shift that took place when student problem solving transitioned to an OLE. Our study focused on the cultural shift that took place in a classroom using Quest Atlantis as the open-learning environment.
The cultural shift was noted in the following areas: (a) the nature of the tasks and the learning outcomes; (b) teachers' role and students' role; (c) students' autonomy and accountability; and (d) classroom dynamics. Before the QA project students were assigned a specific programming task, which was well defined and had a single correct solution. The teacher's role was to provide the instruction, guidance and feedback while the students' responsibility was to copy the code, take notes, work on the task, and experiment on different parameters until they achieve the correct coding to get the program to run. All the students had consistent outputs regarding the assignment, but the outcome was immediately visible, which tended to stimulate students' interest. Group collaboration occurred spontaneously for some students who were highly motivated to learn programming.
During the QA project, students generated ideas and searched information for their group projects, which were unique and ill-defined. There were no correct or wrong answers, but rather better quality solutions relative to less-developed solutions in response to their problem contexts. Because of the authenticity of the task, the students were able to align their projects with their clients' needs and address some of the instructional issues with the client in mind. Additionally, because of the ill-structured nature of the tasks, group collaboration appeared necessary, which led to students' satisfaction and generation of ideas for different project outputs that reflected students' divergent thinking and multiple perspectives. The teacher and the researchers were there to facilitate the group work process, which apparently became the primary and the most influential support compared with others, such as question prompts and group logs.
Although we clearly saw the potential for students to work well in collaborative teams on OLE projects, we were not able to sustain their motivation in the present project for them to develop outstanding projects. We identified two intransigent cultural features that the researchers viewed as problematic for changing the culture to support authentic problem solving. First, games as an extrinsic reward for class work-both before and during the QA project, students who felt they had completed the day's work, or who had become frustrated by their perceived inability to accomplish the work, played action video games or surfed the Internet. This was a motivator for the work to be done quickly if inexpertly, which replicates a finding regarding the negative power of rewards for learning (Deci et al., 1999). On the other hand, experience with 3-D environments and other types of games allowed students to generate lots of ideas on how the QA program could be more engaging. In retrospect, we believe that we could have leveraged the students' interest in gaming to sustain intrinsic motivation for the QA task by getting them to focus on the gaming aspect of QA. However, we needed to analyze our pre-and post-QA observation data before we understood how the game playing was operating in the classroom. This study also reveals that 3-D virtual environments can be more appealing but more difficult for students-asdesigners as compared with the static authoring tools investigated in past studies (e.g., Erickson & Lehrer, 2000;Lehrer, 1993;Liu & Rutledge, 1997). The second intransigent cultural feature was that students were not working to potentialboth before and during the QA project, which was also noted by the instructor.
The issues discussed above need to be investigated more closely in future research so that we may better understand what are the effective instructional strategies to utilize students' motivation in playing games to benefit their learning and to facilitate students to work to their potential. In the present study we made use of a mutable platform in the form of a technology-rich educational innovation, in order to examine its implementation in the naturalistic setting of a high school computer class. Our research was informed by designed-based research as we anticipated the present research being the first study in a broader research agenda on technology-rich learning environment and authenticity in secondary computer education. Our findings inform practice in research, design, and teaching as we discuss in the next section.

Implications for Research and Practice
Technology-rich ethnography is an approach that may be useful to other researchers seeking to construct accounts of learning environments and the evolving cultures mediated by new learning tools. We found that flexibility and multiplicity were overarching concepts. Flexibility in research design and teaching is required because of the naturalistic context. The researchers need to be able to change what and how they are looking at the data, before, during and after the implementation because it is not a static environment controlled by researchers.
Multiplicity in research design and in teaching was also indicated. We needed multiple data sources and methods to capture the depth and breadth of the live, rich experiences of the research participants, who are using an innovative mutable platform. Additionally, multiple perspectives in the analysis were desirable due to the complexity of the data that was generated in this naturalistic setting.
Teaching also required different approaches since the students were developing different projects on different timelines based on their unique perspectives. As suggested by the findings of the study, group collaboration is a beneficial and natural learning approach for difficult tasks, but it must be implemented adaptively and flexibly according to different group compositions, prior knowledge, skills and experience. According to research, question prompts are also a useful scaffolding technique to guide learners' problem solving processes (e.g., Ge & Land, 2003;Ge, Chen, & Davis, 2005), but when there are sufficient help resources (e.g., teacher and researcher facilitation and peer collaboration), other supports may seem less effective or efficient. Thus, scaffolding techniques should also be used adaptively and flexibly depending on different situations. The gaming culture may be flexibly turned into a learning opportunity and directed to benefit student learning and sustain motivation.
Autonomy in learning is an essential attribute of a technology rich, open-ended learning environment that requires more effort on the part of both students and teachers. There are also clear implications for flexibility and adaptability in assessment. Another implication is that teachers must resist the temptation to constrict the flexibility and multiplicity implied by this type of learning environment. The teacher and students are partners in the dynamic design and implementation process. The need to provide students with optimal challenge was demanding for a single teacher. The willingness of students to take part in their own learning is critical for their development in an open learning environment. We hope that openness will provide for multiple ways for students to be motivated, while scaffolding the students' growth in multiple ways. However, the intransigence of the classroom or learning culture may mitigate flexibility and multiplicity, as we found for some of the participants in the present study.
Future research should involve studying different aspects of the learning tasks, including the saliency of the client-designer relationship. We found that without explicit and frequent feedback our high school designers tended to overlook the characteristics and needs of the target learners. Additionally, negotiating the difficulty and appropriateness of challenge in the design task and student autonomy in choosing group versus independent work should be examined. We found that elements of the old classroom culture would reassert themselves, stripping away the more challenging aspects of the new OLE culture. This seemed to encourage maladaptive motivation, an inference that should be examined in future work. A related line of research should focus on aspects of student learning, such as how to harness metacognition toward academic effort rather than avoidance of effort. More types of OLEs, in school and non-school settings, should also be examined. Finally research should involve a more explicit focus on fostering student creativity in solving ill-structured problems.

APPENDIX A Question Prompts for the Quest Atlantis Group Project
At least one section of these prompts should be read each day within your groups. They are intended to guide you through the process.

Brainstorming Ideas for the Project:
• What topics are we interested in?
• What area are we knowledgeable about?
• What is something that we think can teach or show younger people, such as elementary students (5th graders for instance) *Note: It doesn't have to be subject area, it can be something you think you are good at, such as laying computer games, playing chess, driving, etc.)

Define the Project:
• Are we going to create a new Quest for Quest Atlantis or create resources for the existing quests? • What area(s) are we going to focus on? • Who are our students, clients, or users? • What do we want them to learn from our project? • Is this project something that can be accomplished within 8 days?
Planning for the Project: • What is our project going to look like?
• What kinds of resources do we need in order to complete the project? (List them.) Where or how can we find those resources? • What skills do we need in order to create this project (using Word; PowerPoint, creating Web pages; creating Java Applet; etc.)? Do we have the skills to develop the project? • How are we going to plan our time? How are we going to collaborate on this project?
Designing and Developing the Project: • What ideas and activities do we want to include in this project?
• What do we know about the chosen ideas and activities? What information have we found? What else do we need? • How are we going to present the ideas and activities? Why do we want to present the information and activities this way? • What components do we need in order to put the project together (e.g., text, graphics, charts, audio files, video files, etc.)? • How are we going to divide our work in order to complete this project?
Monitoring Progress on the Project (after initial planning, this set of questions should be read aloud and answered each day): • What stage of development is the project in currently? (i.e., still planning, in the middle, or nearly finished). • Describe or summarize the project you have decided to do. All group members should be able to do this. • What problems, if any, are we encountering as we work on this project?

Evaluating the Project:
• Does the project meet our expectation or goals that we have intended earlier? Why? • Do we think it will teach our students/users/clients what we intended to teach? Why? • What is missing? Does the project look good? Why?
• Is there anything that we'd like to improve, for example, the font size, text color, graphics, content, etc.? Why would you like to make those changes? • What could we have done differently if we were going to start the project again?