Chapter 4
Home Up

 

CHAPTER 4

RESULTS

    This research was an investigation of the initial offering of the Advanced Placement (AP) Statistics course and students’ learning within this concept oriented statistics class. Qualitative research methods were employed to describe the genesis of the course and to investigate ideas students’ developed in a concept-oriented classroom. Constructivism was the learning theory for the study (Guba & Lincoln, 1994).
    This chapter reports analysis of the data. The researcher gathered data to investigate the initial offering of the AP Statistics course and the effect of concept-oriented instruction on students’ development of statistical ideas. Data sources for the study included various materials relevant to this initial offering of the AP Statistics course and two AP Statistics classes taught by the same instructor during the first year (1996-1997). Information relevant to the initial offering included College Board publications, interviews with Test Development Committee members, comments by other teachers involved with the AP Statistics course and exam, Educational Testing Service (ETS) AP test results, and interviews with the instructor. Two classes taught by the same instructor contained 31 students. Data came from student artifacts gathered from bimonthly project reports, test papers, audio taped project presentations, student interviews, and field notes. Immediately following the administration of the nationwide AP Statistics exam, 12 students were interviewed and audio taped individually while discussing topics relating to the instructor, their specific class, and the AP exam. Analysis of the data is presented using excerpts from all sources. Questions addressing the initial offering of AP Statistics course precede discussion about the research site for this specific academic year. When addressing questions about students’ learning, I investigated patterns based on race, gender, and mathematical abilities. All student writing quoted in this paper is unedited. Pseudonyms are used for all participants. As the researcher, I was the primary research instrument. The research questions were:

1. What are the salient features of the AP Statistics course and the genesis of these features?

2. What teaching strategies are effective in implementing the recommended pedagogy?

3. What effects do projects that employ cooperative group problem solving and writing have on students’ development of statistical concepts?

4. How does gathering data effect students’ understanding of statistics?

5. What components of activity-based, concept-oriented instruction effect students’ performance on the AP exam?

The Initial Offering of the AP Statistics Course

    The first research question, regarding the salient features of the AP Statistics course and the genesis of these features, emerged from the study. I originally intended to focus only on student learning within a concept-oriented course. However, as the year progressed and I became more involved in the overall AP experience, I realized that a holistic description of the course could contribute to the study. Therefore, in order to describe the course and its genesis, I researched a variety of sources. I obtained pamphlets and booklets on AP Statistics published by the College Board (see Appendix E). I selected the final member of my dissertation committee because he was involved in the planning, development, and implementation of the initial offering of the AP Statistics course. I corresponded with several other people who had been involved with the creation of the course and the administration of the first exam: Richard L. Scheaffer, the Chief Faculty Consultant (CFC), members of the Test Development Committee, and Table Leaders from the Reading. I subscribed to the AP Statistics listserv and printed relevant comments. In addition, I referred to the listserv’s archives when necessary.

The AP Program

    The Advanced Placement Program began in the early 1950s in response to a concern within the educational community to provide appropriate courses for talented high school students (Watkins et al., 1997). In 1954, the College Entrance Examination Board voted to implement the AP Program. The College Board enlisted the Educational Testing Service (ETS) to develop, administer, and grade the examinations. During these first years, there were 11 courses available. By 1987, there were 15 courses offered. Today, many subjects have more than one examination. In 1997, the following examinations were offered: Art: History, Studio--General Portfolio, Studio--Drawing Portfolio; Biology; Chemistry; Computer Science; English: Language and Composition, Literature and Composition; French: Language, Literature; German Language; Government and Politics: American, Comparative; History: American, European; Latin: Vergil, Catullus-Horace; Mathematics: Calculus AB, Calculus BC; Music: Listening and Literature, Theory; Physics: B, C-Mechanics, C-Electricity and Magnetism; Spanish: Language, Literature; and Statistics.

AP Statistics

    In the 1980s, officials at the College Board and ETS were concerned that many students were not enrolling in a mathematics class in 12th grade. The AP Statistics course evolved as a result of this interest in expanding AP offerings in mathematics. Initial suggestions, as early as 1980, received little interest. At the request of the Advanced Placement Calculus Test Development Committee, ETS and the College Board conducted a feasibility study in 1987 to examine what course might be successful. Questionnaires went to mathematics department chairs at 300 colleges and universities that received AP Calculus scores in 1986 (Armstrong & Jones, 1987). Questions asked college officials if they would grant credit and/or placement in multivariable calculus, elementary statistics, discrete mathematics, applied matrix algebra, or linear algebra. Questionnaires were also sent to 841 secondary school mathematics department chairs to determine if they had an interest in offering an additional AP mathematics course (Armstrong & Jones, 1987). ETS received and analyzed responses from 212 colleges and 541 schools. As a result of the first questionnaire, the second study targeted a statistics course. Sixty-six out of 105 colleges surveyed indicated they would offer credit and 72 out of 105 said they would offer placement for an AP Statistics course. Therefore, the AP Statistics Task Force (1992) recommended that the College Board implement an AP Statistics course in the 1995-96 academic year (Advanced Placement Statistics Task Force, 1992). The Test Development Committee in Statistics was formed to devise course materials and write the AP exam. The 1995-96 Test Development Committee in Statistics consisted of the following: Rosemary A. Roberts, Bowdoin College, Brunswick, Maine, Chair; Fred C. Djang, Choate Rosemary Hall, Wallingford, Connecticut; Kinley Larntz, University of Minnesota, St. Paul; Christopher R. Olsen, George Washington High School, Cedar Rapids, Iowa; Diann C. Resnick, Bellaire Senior High School, Texas; Richard L. Scheaffer, University of Florida, Gainesville; Walter O. Walker, Eckerd College, St. Petersburg, Florida; and Ann E. Watkins, California State University, Northridge. ETS Consultants were James S. Armstrong, Jeffrey G. Haberstroh, and Michael A. Ponisciak. According to Jeff Haberstroh, of ETS, most committee members serve 3 or 4 years. In 1996-97 Kinley Larntz was replaced by two members: Daniel Teague, North Carolina School of Science and Mathematics, Raleigh; and Jessica Utts, University of California-Davis, Davis. The current committee (1997-98) lost Rosemary Roberts and added the following: Katherine Halvorsen, Smith College, Northampton, Maine; and Roxy Peck, California Polytechnic University-San Luis Obispo, California. As of June 1998, Ann Watkins is serving as the chairperson. At the conclusion of the 1998 AP Reading, Roxy Peck replaced Richard L. Scheaffer as the Chief Faculty Consultant.
    In the AP Statistics Course Description, the Test Development Committee provided recommendations to determine whether this course is appropriate for a student. "The AP Statistics course is an excellent option for any student who has successfully completed a second-year course in algebra," they suggested, "regardless of the student’s intended college major" (College Entrance Examination Board, 1997, p. 2). With second-year algebra as the prerequisite course, students would most likely enroll in AP Statistics as a junior or senior. Strong mathematical students could take this course simultaneously with AP Calculus or AP Computer Science. Other students could choose to take AP Statistics in lieu of precalculus or AP Calculus. The Test Development Committee, however, recommended that any student interested in taking calculus in college should complete precalculus in high school (College Entrance Examination Board, 1997).
    The AP Statistics Test Development Committee provided an outline of the course content (see Appendix F). This outline was published in the Advanced Placement Course Description: Statistics (College Entrance Examination Board, 1997). The topics were divided into four major themes: exploratory data analysis, planning a study, probability, and statistical inference (College Entrance Examination Board, 1997). Exploratory analysis included the use of graphs and summary statistics to determine patterns and exceptional cases in data. Planning a study entailed posing or clarifying a research question, providing details of data collection, and a method of analysis. Probability was used to examine patterns of data and as a link to inference. Statistical inference involved drawing conclusions from samples and making predictions about populations.

The First AP Statistics Examination

    On May 6, 1997, the first AP Statistics exam was administered worldwide. Seven thousand, six hundred sixty-nine students sat for this exam. It consisted of 35 multiple choice (MC) questions, five free response questions (FR), and one investigative task (IT). The MC section was 90 minutes. The MC questions were equally weighted and counted as half the total AP grade. MC questions were graded with a correction factor to compensate for random guessing (College Entrance Examination Board, 1997). The FR and IT were combined in a 90-minute session. The FR was 3/8 and the IT 1/8 of the total AP grade. Grading of FR and IT questions followed a holistic rubric (see Appendix G). According to Watkins, et al. (1997), "the response will be considered as an entire package. Note that there are two aspects to the scoring: statistical knowledge and communication of that knowledge" (p. 57). The Chief Faculty Consultant combined section scores and then converted the total raw score to the program’s five point scale: (5) Extremely well qualified, (4) Well qualified, (3) Qualified, (2) Possible qualified, and (1) No recommendation. In July 1997 grades were sent to students, their high schools, and colleges they selected.

The First AP Statistics Reading

    The Reading took place the second week of June 1997. Fifty-six statistics educators gathered at the College of New Jersey in Trenton to read the FR and IT sections. Since this was the initial statistics offering, the Reading site was located near Princeton in case additional staff was needed. Richard L. Scheaffer, University of Florida, served as the Chief Faculty Consultant (CFC). Rosemary Roberts, AP Statistics Test Development Committee Chair, acted as Assistant Chief Faculty Consultant. The remaining Committee members served as Table Leaders or Readers. Other Table Leaders were Patti Collings, Chris Franklin, Brad Hartlaub, Duane Hinders, and Ken Koehler.
    The College Board divided the United States into six regional offices: Middle states, Midwest, New England, South, Southwest, and West. ETS sought to obtain Readers representing each region by region, gender, ethnic background, and college or secondary schools affiliation. Overall, ETS targeted 50% college--50% high school participation. In 1997, more students registered for the AP Statistics exam than originally anticipated. Therefore, ETS had to recruit approximately 20 extra Readers on short notice. The result was approximately 60% high school and 40% college.
    The CFC screened all applications for Table Leaders and Readers. Once applicants were selected, the CFC submitted a form to the Performance Scoring Services at ETS. Applicants were notified of dates and conditions of acceptance. The primary purpose of the Table Leader was to ensure consistency in scoring. According to J. Haberstroh of ETS (personal correspondence, February 10, 1998), criteria used to select Table Leaders included experience with subject matter and an ability to work as a diplomatic facilitator. For this Reading, all Table Leaders had also been involved with the development of the AP Statistics course and exam. Subject matter background was equally important for Readers. In addition, ETS sought a certain level of professionalism. J. Haberstroh explained. "We want to have people who contribute to the overall professional aspect of the process," he said (personal correspondence, February 10, 1998). Selected participants can serve as a Reader for up to six years. Table leaders can serve an additional six years.
    The Chief Faculty Consultant, Assistant Chief Faculty Consultant, and Table Leaders arrived three days prior to the Reading. This leadership group prepared the rubric that all Readers would use (see Appendix G). Prior to the Reading, the Chief Faculty Consultant paired each Reader with a partner. During the Reading, all eight tables sat six Readers and one Table Leader. To insure consistency, Table Leaders back read a portion of graded papers from all Readers at his or her table. They also assisted when one of his or her readers had questions regarding grading.

Test Development Committee Members’ Responses

    I contacted all eight members of the Test Development Committee and asked them to respond via E-mail to the same questions (see Appendix H). Unfortunately, only four Committee members responded to this request. The Chief Faculty Consultant (CFC) supplied The CFC Annual Report on AP Statistics.
    The first question asked, "Did students perform as well on the open-ended questions as you had expected? If not, what might be some possible reason for poor performance?" Three said no and the fourth said he expected them to perform ‘miserably’ (HC, personal correspondence, October 1, 1997). One member believed students performed poorly because "some of the students had never seen the material" (ID, personal correspondence, October 5, 1997). All four members suggested students and teachers lack experience with Free Response questions. Students need more practice with writing as part of assessment, particularly stating assumptions and reasons for choosing an analysis technique.
    The second question probed Committee Member’s Reading experience as it related to other teachers at the Reading. I asked, "What did you learn from other teachers who were there as Readers, with regards to the course and their own classroom preparation for the exam?" One member stated teachers "feel there is plenty of time to teach the course, if the course is a year long, and not just one block" (RF, personal correspondence, June 28, 1997). Another member wrote that the Readers left with a better understanding of "what is to be expected. There is no doubt now what assumptions are required for tests and the fact that they have to be explicitly stated in a response" (ID, personal correspondence, October 5, 1997).
    Readers use of a rubric was a significant piece of the grading process. I asked Committee Members "What is your opinion/impression/reaction to the rubric and holistic grading?" Two members suggested some Readers may have been skeptical. But by the end of the Reading, everyone agreed it was fair, consistent, and a good method of grading (HC, ID, RF, NA, personal correspondence). One member had experience grading the AP Calculus exam using an analytical method. He said, "Holistic grading gives us an opportunity to discuss the response made by each student so that we can come to an overall conclusion about the students’ overall understanding of the questions and the appropriateness of the answers" (RF, personal correspondence, June 28, 1997). One member identified a specific advantage to using a rubric. "We saw students come up with innovative ideas about the open-ended questions that enabled them to get quite a few points," she said, "even though they did none of what we thought of as the standard, obvious solution" (NA, personal correspondence, July 2, 1997).
    Technology was an important element of the AP Statistics course. Students were required to have a graphing calculator for the AP exam (College Entrance Examination Board, 1997). I asked Committee Members if they are in favor of allowing graphing calculators use during the examination. All four members responded "yes." "As teachers, we cannot teach in one way and test another," says one member. "You cannot do statistics without technology. We have to test using technology" (ID, personal correspondence, October 5, 1997). Another member, who was in favor of using technology, suggested some portions of the exam could be given without use of graphing calculators. "I am in favor of kids taking portions of the exam without calculators," he wrote, "as this increases the kinds of questions that can be asked" (RF, personal correspondence, June 28, 1997). One member expressed concern about students using calculators to cheat.

Some sort of computational device is absolutely necessary for a statistics exam. Predictably, some students misused and overused their calculators. Some Readers were a bit discouraged by this, but those of use who have been teaching since the days before calculators know students resort to mindless computation when they don’t know what else to do, whether or not they have a calculator. The bigger problem, which I wasn’t fully aware of until the Reading, was that students can download tremendous amounts of textual information into a TI-83. That does bother me. This is supposed to be a closed book test. We will have to figure out some way to deal with that. Clearing memories isn’t as flawless an idea as it might first appear. The students with TI-82's would then lose the programs that make it emulate a TI-83. (NA, personal correspondence, July 2, 1997)

Overall, committee members supported the use of technology during the exam but were willing to explore alternative applications.
    Two other questions were presented to the Committee Members, but only one yielded relevant information. I asked "What is your overall impression of the initial offering of the course and the examination?" All four members stated favorable impressions. One member noted that "the number of exams far exceeded our projection and we had to scramble for Readers at the last minute, but I don’t think that hurt us" (RF, personal correspondence, June 28, 1997). Another felt "relief that everything went so well and that so many students and teachers understand that statistics is an important subject" (NA, personal correspondence, July, 2, 1997). One member, who was involved with the course and the exam from the beginning, said, "I am sure there is room for improvements, but considering how much work we did based on ‘hunches, estimation, and flat guesswork,’ I thought things went great!" (HC, personal correspondence, October 1, 1997).

1997 AP Statistics Exam Results

    Table 1 shows national, southern regional, and Georgia results by number of students and percentage where the grades were reported to a college or university. Additional information regarding national, regional, and state data is available from ETS.

Table 1

1997 AP Statistics Exam Results Reported for the Nation, the Southern
Region and Georgia

                    National          Southern Region          Georgia

AP Score       Percent                 Percent                 Percent

      5               15.7%                   16.6%                   20.0%

      4               22.1%                   19.6%                   17.6%

      3               24.4%                  21.9%                   23.8%

      2               19.8%                   20.3%                  21.7%

      1                18.0%                  21.6%                   16.8%

The AP Statistics Listserve

    J. Swift, who was teaching statistics in Nanaimo, British Columbia in the mid-1980s, initiated efforts to connect statistics teachers using phone/modems through the Bulletin Board Systems (BBS) network (personal correspondence, February 4, 1998). The Woodrow Wilson National Fellowship Foundation month at Princeton in 1984 supported this type of communication and provided hardware for the BBS. In the early 1990s, changes in technology provided a more convenient and efficient method to open communication lines among teachers. At this time, many teachers began to subscribe to the List for Statistics at North Carolina State University, including J.C. Chappelle, Brookestone School, Columbus, Georgia and J. Swift, Coordinator, Information Systems, School District 70, Alberni, British Columbia, Canada. Both planned to teach the AP Statistics course when it was introduced. J.C. Chappelle posted a message to the existing Edstat listserv, sometime in 1993, suggesting that it might be useful to start a list directed at the high school statistics teacher who was planning to teach the AP Statistics course. J. Swift enlisted the help of the Education Technology Centre in British Columbia to host the APStat Listserv and he began managing the list. According to Swift in April 1998, 644 have subscribed to the list and only the AP Computer Science list has more subscribers (personal correspondence, April 11, 1998). Swift also indicated that since the AP Statistics list proved to be so successful, the College Board initiated lists for other AP subjects (personal correspondence, February 5, 1998).
    The purpose of the APStat listserve was to provide academic support to any teacher involved with the AP Statistics curriculum. Various topics were introduced repeatedly as new teachers join the list. Two nonacademic topics reoccurred that created a tremendous amount of debate. These topics were choice of textbook and choice of technology.
    The most frequent question that appeared from new teachers concerns textbooks. There were two major types of statistics textbooks, traditional and progressive. There were a variety of texts available of the traditional flavor. These texts all presented the material in the approximately the same order, placed an emphasis on probability, and provided data with detailed instructions on the type of analysis to conduct. The other type of statistics books were more individual in their design. Some progressive texts eliminated or reduced topics in the traditional texts (probability, binomial distribution) and included topics not mentioned in most traditional texts (sampling techniques, experimental design). Another difference in the two types of textbooks was the organization. Traditional texts placed probability topics early and bivariate data analysis later. Progressive texts emphasized exploratory data analysis, including boxplots and normal probability plots. Overall, progressive textbooks shifted the emphasis from calculations and formulas to conceptual understanding. Even though several texts were recommended by the Test Development Committee, it appeared that this is a common source of frustration and confusion to a novice AP Statistics teacher. This may be because most of the texts recommended by the Committee were progressive texts that require interaction with the data. It is possible that these recommended texts are unfamiliar to a teacher who was taught from traditional texts. As this specific topic was not stressed in my research, I did not emphasis these conversations while viewing the listserve. However, those who participated in this discussion consistently recommended the three texts available by David S. Moore (one text is coauthored with George McCabe) and the other progressive texts, Devore and Peck, Siegel and Morgan, Waldrop, and Iman.
   Another frequent topic of conversation revolved around technology. Since the Test Development Committee recommended that students interact with data, the choice of technology has become a significant issue. The contributors to the listserv fell in one of two camps. The experienced teachers preferred either a graphing calculator (specifically the TI-83) or the use of statistical software on a computer. While there was some discussion about the choice of software, the most heated debate occurred with respect to calculators versus computers.
    Many high school teachers preferred a graphing calculator, specifically the TI-83, because of the convenience and cost for the student. Texas Instruments provided overhead screens that attach to a teaching calculator and enabled the instructor to project the text to a wall screen. With this attachment, the instructor could use his or her calculator while the students see the screen at the same time. This allowed students to follow the sequence of instructions on their own unit simultaneously. Another advantage of the calculator was availability and convenience. Students carried a graphing calculator wherever they needed to. And, the calculator was not an overly expensive item to purchase. Many campus bookstores sold or rented the calculator of the mathematics department’s choice. The computer offered neither of these advantages. A school had to provide a classroom with computers or have a lab where the class could meet to utilize the technology. In addition, unless the school arranged for a site license from the publisher, students had to purchase software bundled with the text (although the software might not cost as much as the graphing calculator). Unless a student had a computer at home, they were forced to use the facilities at the time and convenience of the school. However, the computer offered advantages the calculator did not. This was where the debate always led. Many software packages allowed students to examine graphs and statistics simultaneously. It was advantageous to view graphs side by side. It was easy to adjust the size and scale of graphs. Computers offer printing as an option. In addition, most software packages allow the user to analyze large data sets. The TI-83, for example, was restricted to data sets of 999 observations. Statistical software easily handled much larger data sets. Another technological advantage of the computer was the ability to download data sets from the Internet. As our society becomes more dependent on technology services, such as the World Wide Web, availability of data from the web will increase as an advantage. Procalculator teachers responded with the idea that large data sets are not necessary. If the goal was for the student to interact with the data, then allow them to gather the data. There were effective ways to gather data during class: how many miles do you live from school, age and height of both parents, number of children in the family, estimate the length of a piece of string, and favorite car color. I observed that students enjoyed working with data that had meaning for them. Large data sets obtained from a book or the Internet frequently held no personal meaning for students in the classes I observed.
    While the AP Statistics listserve provided a forum for discussion about texts and technology, the most significant conversations concerned academic questions. This listserve provided a non threatening forum for novice statistics teachers to ask questions from more experienced instructors. Frequently the experts disagreed on academic topics. Discussions involving ambiguous interpretations revealed the aspects of statistical thinking that are difficult to master. Unlike other fields of mathematics, statistics frequently involved subjective interpretations. For novice teacher, this was frustrating and difficult. The attitude of contributing members was to assist anyone seeking help. Several teachers posted other website addresses with related information, including data sources, test questions, and project topics. Since students had access to the listserve, listserve members have noted that test questions posted were not secure.
    In May when the AP Statistics exam was being administered, the listserve conversation was active. Initially, one teacher in the Far East raised the issue of security. Since students in that time zone were physically administered the test earlier, the likelihood of insecure test questions because a realistic problem. No one on the listserve was able to contribute a solution but everyone became aware of the problem. The week following the administration of all AP tests, teachers posted feedback from their students. Overall, of the teachers who offered this feedback, most of their students indicated the multiple choice section was straightforward and not difficult. The free response section, which included one investigative task, received a variety of evaluations from students. Some students thought these problems were not difficult, while other students thought these problems were extremely challenging.
    After teachers received ETS scores for their classes, many posted the results to the listserv. While it was interesting to read these postings, the ETS national scores published later verified a voluntary response bias. Many of the teachers who posted their scores were at or above the national average for the number of 3s, 4s, and 5s.

The Instructor

    Even though the purpose of this study focused on student learning, finding an experienced teacher was the main criterion in choosing the research site. Qualified instructors are an integral component of a successful AP course. (College Entrance Examination Board, 1997; Watkins et al., 1997). The Test Development Committee recommended that AP Statistics instructors complete at least one college-level statistics course, and preferably two or more (Watkins et al., 1997). Lee completed two undergraduate and three graduate courses in statistics. He had 20 years experience teaching statistics. For this initial offering of AP Statistics, he taught all three sections that were offered at the research site. In addition, he taught the Introduction to Statistics course part-time at a local community college.
    The second research question, what teaching strategies are effective in implementing the recommended pedagogy, emerged from the study as a result of trying to describe the AP Statistics course holistically. Originally, I did not intend on focusing on the instructor as I considered student learning issues. However, during data analysis, many issues emerged from the students that could only be addressed by investigation of the teacher. To address these emergent issues, I referred to field notes that recorded dialogues and interactions between the instructor and students, and among the students. In addition, during academic year 1996-1997, Lee and I frequently discussed his techniques and his motivating logic. I recorded notes of these discussions in a journal. After the first year ended, I conducted a structured interview with Lee. But, much of the data that addressed this question were gathered throughout the 1997-1998 academic year as he responded to follow up questions that emerged during data analysis.

Lee

    Lee is calm, patient, and relaxed. These personality traits are apparent in his teaching. Several students in their interviews indicated the classroom atmosphere was relaxed and comfortable. Students indicated they felt comfortable asking questions in this learning environment.
    Lee was the mathematics department chairperson. In this capacity, he arranged teacher course assignments, approved students to take specially recommended classes, interviewed prospective faculty members, obtained substitute teachers, selected and/or approved textbooks and other materials, encouraged teacher involvement professionally through organizations and training, attended official events representing the mathematics department, and solved administrative problems.
    He was a consultant for the College Board in AP Statistics. He conducted workshops that are sponsored by the College Board for AP Statistics teachers. These workshops covered the curriculum while placing an emphasis on activities that instructors can use to facilitate conceptual understanding. College Board sponsored workshops were offered annually in all regions of the country. Many of these one-day workshops were offered during the academic year, but summer workshops were spread over 5 days.   
    Lee has been using concept-oriented instruction for the past 10 years. He began experimenting with constructivist teaching techniques in the late 1980s after attending several progressive workshops and institutes: Mathematics and Technology Conference at Philips Exeter, Woodrow Wilson Modeling at Princeton, and Discrete Modeling at the University of Maine. These workshops focused on interactive model building. He stated that, given his mathematics modeling experiences, teaching statistics as an interactive, concept-oriented course was a natural progression (personal correspondence, March 17, 1998).
    Constructivists advocated that teachers devise scenarios enabling students to investigate, conjecture, and construct their own understandings (Confrey, 1990; Noddings, 1990). Lee’s teaching exemplified the constructivist learning theory by placing students at the forefront of the learning process. For example, one day he gave them a set of instructions, but did not tell them how to do any of the requested things. Students discovered their own method, and then checked with others for verification (Field Notes). Another day one student collected the data from the class and said, "I got the data but what confidence level?" Lee responded, "I don’t know, you’re the statistician." The following dialogue occurred in January. This interaction typified Lee’s pedagogy.

Lee arrived in class with a small bag of chocolate chip cookies and gave one to each student.
Lee: Estimate the number of chips in a cookie.
Student 1: How do we do this?
Lee: I don’t know, you decide.
They count chocolate chips and report their data.
Lee: Okay, the mean is 7.333. So what’s your answer?
Three separate students volunteered their confidence intervals.
Lee: What did you use?
Student 2: 95%
Five others agreed that 95% was appropriate.
Lee: T or z?
They all say t.
Lee: Why?
Three students chimed in: small sample size.
Student 3: We don’t know the population standard deviation.
Lee: Are we comfortable using t? What should we be concerned about?
Student 2: Is it normal?
Lee: How can we determine this?
Student 5: Look at a normal probability plot.
Lee: Okay, so now with that, is it reasonably normal?
Student 6: It’s pretty linear so the data should be normal.
Lee: What else can we do?
Student 4: Use chi-square.
Lee: What else can we do to test normality?
Student 7: Graph a histogram.
Lee: What would be a good scale?
Student 4: How about x-bar plus 3 standard deviations.
Lee: Ideally you ought to be thinking about these assumptions. Now, this cookie company claims the average number of chips is greater than 7. Test this claim.

Student 3: The test statistic is 1.4084 and the p-value is 0.09.
Lee: Which means?
Students 8 and 9: Do not reject the null.
Lee: Which means?
Students 5 and 9: The cookie company is wrong.
At this point, Lee accessed the ‘draw’ feature on the calculator.
Lee: What is this picture? What is the curve?
Student 5: A t-curve.
Lee: What t-curve?
Student 6: Our t-curve.
Lee: Is it the t-curve for the number of chocolate chips?
Several respond no.
Lee: For some standard t-curve?
Student 2 and 7: Yes
Lee: Which one?
Student 9: The one with 14 degrees of freedom.
Lee: What is 1.4084?
Student 6: That’s where the mean would appear on that curve.
Lee: What’s in the center?
Student 5: Zero.
Lee: What does this mean without referring to our problem?
Student 4: I thought it was the probability that it was random.
Lee: What does the shaded area mean?
Student 9: The probability that we would get that sample if the null was true. (Field Notes)

This dialogue portrayed Lee’s belief that teachers should facilitate learning. "I think allowing students to dig things out for themselves is important," Lee said. "I think I force them to do that sometimes, kicking and screaming as they go" (Interview). He frequently allowed the students to pursue misconceptions. He posed a question to the class. When students offered incorrect ideas and solutions, rather than correcting them, he would continue to prompt their inaccurate ideas. I asked about this teaching technique:

Lee: I certainly think that my non answering of questions is a really valuable skill. I also let them go toward the wrong answer. I think that’s important because that’s the way they’re going to go anyway.
KR: How is that important to their learning?
Lee: Well, because I like to think that somewhere along the line they’re going to be going off on their own directions. There’s only so much structure that can be applied to their learning. Some of them may like the structure, but they just have to have ideas of their own. And statistics lends itself to this sort of thing because they’ve got to think about all the situations and the conditions. As you talk and read about these problems that seem sort of standard and four statisticians are discussing it, each one has their own way of going about it. We’re talking about above average high schools students trying to know "the" way when the experts can’t even agree. (Interview)

As the researcher, I was interested in why Lee used this questioning technique to help the students learn.

KR: What about the student who is in the class discussion, conversing back and forth with you, and you let this misconception continue to go until he gets to the point where he has backed himself into the corner where he realized he’s gone down a blind path. You did this repeatedly. So, how does that help them learn?
Lee: I think that’s the only way in a lot of cases where they are ever going to really think about most of the concepts. If I tell them exactly what to do on every problem, everything will be just fine as long as they have exactly those problems that I have taught them how to do that way. There’s just too many twists and if they have never played with twists then they won’t have any idea what to do.
KR: Why do you think allowing them to pursue their misconceptions while you go back and forth with them is better than just turning to them and saying, ‘here’s the correct answer.’ What do you think about the way they learn that your teaching method supports their learning that way?
Lee: I guess my theory is that every student has his own vision of reality because they tell me about it sometimes and it’s certainly not mine.
KR: Where did their view come from?
Lee: People figure out how to learn things however their brains work. I don’t know that I have thought about this much until lately. In lower school, we teach them these algorithms about how to add and subtract. And for a lot of kids, the way we describe how to do it, that’s not how they think about it at all. They’ll do an amazing convoluted system that gets them the right answers so obviously that’s fine. Apparently my method didn’t make any sense. So I think that’s just the way kids do just about everything. So me laying down the rules, it may indeed be something they can memorize and apply to ‘the verbatim problem’, but if it’s anything but that, I don’t know that they have a chance. They’ve just got to have experience with playing around and trying to figure out what the concepts mean to them. I’m sure if I asked a lot of these kids some real standard questions, then they would explain them in a way that would just boggle my mind. But if that’s how they understand it, then that’s okay, as long as it’s not wrong. (Interview)

Consistent with constructivist views of teaching (Cobb et al., 1990; Confrey, 1990; Davis et al., 1990; Hatano, 1996; Noddings, 1990; von Glasersfeld, 1992) and the Test Development Committee’s recommended pedagogy (College Entrance Examination Board, 1997), Lee believed in actively involving students in the learning process. He involved the students using a variety of activities, including gathering data in the classroom, utilizing technology, and group problem solving. When asked about integrating activities with lectures, he said,

Lee: I don't know that the best way to do it is necessarily to have nonstop activities or nonstop lecture. I think there needs to be a nice mix of different kinds of things going on.
KR: Why do you think that's important?
Lee: I think that an activity, whether it’s a simulation or something else, just makes a lot of concepts much more concrete to students. The writing down of facts is just a very different kind of way of learning something than seeing it happen with real data, or actually quote, doing the problem unquote.
KR: So why a mix? Why not do the entire class using only activities?
Lee: Well because at some point if you allow every student to formulate all their own learning through activities, obviously there can be mistakes. Statistics just generates such different kinds of results and a lot of it is due to randomness, but somewhere along the line you have to make it, you have to get them to focus on what the central notion is. And if that doesn't come out, you have to help them along. Or their peers have to help them along. I think that's part of our job too. We can let them go down the wrong path, and play around, and form their own learning. But then we need to get them to say exactly what it means and to make sure that is reasonably close to being correct.
Lee used a variety of techniques to cover material. His goal, however, was conceptual understanding. During student interviews, many referred to their learning as conceptual. I asked Lee how did the students come to know the idea of learning conceptually.
KR: In my interviews with your students, several of them mentioned conceptual learning. Where do you think they got this notion of conceptual understanding?
Lee: I talked about the idea of concept versus application a lot. I would frequently ask them, ‘What does this mean?’ I tried to be clear that meaning was important and forced them to answer this in class and on tests.
Kim: How did you do this?
Lee: In class, I would throw out questions that got to the underlying concepts. If they didn’t reply with that kind of answer, I would continue to prompt them until someone figured out the correct meaning. On tests, I would ask questions where they couldn’t rely on the calculator for the answer. The wording would be such that they would have to understand the meaning of the terms and context of the material to figure out the answer. (personal correspondence, March 10, 1998)

Lee clearly stated his belief in the importance of involving students in the learning experience. His goal was to provide an environment where students actively experienced situations that helped them develop conceptual understanding.
    Another pedagogical technique the instructor implemented involved utilizing technology in the classroom. His knowledge and experience using the TI-83 was extensive. Also, the students had used the TI-82 extensively in both Geometry and Algebra with Transformations and Precalculus. When asked about the controversial technology issue, the following ensued.

KR: What do you think of technology in this class?
Lee: I think having powerful technology available in the classroom allows the students to focus on the important ideas and not the number crunching aspects of statistics.
KR: Your choice of technology is?
Lee: My choice of technology without question is the TI-83.
KR: What are the advantages you see for the TI-83 over a computer?
Lee: It's totally convenient in the fact that the kids have it in their hand at all times. There is no need to run to the computer lab. Despite the comments by many college people on the listserve, that life can't go on without a computer and that is the way it is done [sic]. I still don't get it, I'm trying really hard. But beyond the fact that it handles lots of data, and it prints really pretty pictures, I still don't see the advantages of a computer over the calculator.
KR: The comment that seems to come up over and over on the listserve from those who prefer computers, is that's what real statisticians use.
Lee: That's just an irrelevancy to me. The last comment that I saw on the listserv which I thought was really good is that my students are quite comfortable with computers, whether its through statistics or not is irrelevant. They are comfortable with computers, they are comfortable with using packages. So in ten minutes, if they have never used Minitab, at the end of the course after using the 83, I could say, now here's how Minitab does exactly the same thing and they would not be shocked, surprised or amazed. I'm really trying hard to understand. I think a lot of college people that say that the 83 isn't the same is because they don't know what the 83 can do. I really believe that. In that sense, I feel like I'm in a stronger position because I know what they both can do.
KR: So you don't feel in any way that your students missed anything from not working more on the computer?
Lee: Not at all.

In the classes I observed, the instructor utilized the TI-83 graphing calculator, instead of computer labs, for several reasons. First, the students had extensive experience with the TI-82 and only needed to be taught the statistical features of the new TI-83. Lee suggested convenience for the student, ease of calculations, and depth of data analysis utilities as other reasons to use the TI-83.
    A unique element of Lee’s classroom was the use of round tables and chairs as opposed to rows of desks. Each table would accommodate four students. But, students sat facing the front of the room so there were three students at most tables. I asked Lee about this arrangement in his classroom.

KR: Why did you choose to arrange the room with round tables?
Lee: I believe that students by talking with each other tend to, they tend to clarify to each other the idea that we as a class are attempting to explain. Their reality is not exactly the same as mine. They have to translate my explanation into terms that make sense to them.
KR: How do tables help facilitate this?
Lee: Proximity is obvious. Also, less implied structure. They feel comfortable. Tables help facilitate them feeling comfortable talking to each other, even while I’m talking.
KR: How does this facilitate their learning?
Lee: I think the main thing is that they are put into a setting where they will discuss concepts with each other. When they can explain their own understanding to each other, that helps them understand.

Lee justified his use of round tables by stating the students were more comfortable talking with each other and asking questions during class. He believed this is significant to their learning process because they all learned when they assisted each other.

A Typical Day

    Lee typically began class by introducing an activity, collecting data, or presenting a problem. He allowed students to ask questions, answer questions, or state their own ideas. These discussions were spontaneous and loosely organized. Often questions or comments would lead them to investigate related topics in detail (Field Notes). In a follow up interview, Lee explained why he was unconcerned if they were seemingly off the subject.
    They have their own personal understanding of the statistical ideas they discover or that I present. It’s very unlikely that many of my students understand sampling distributions, the Central Limit Theorem, confidence intervals or hypothesis tests in the same way and certainly not how I understand it. I think when they’re allowed to discuss these ideas among themselves, they share their individual notions of what’s going on. Many of them do not think linearly (I certainly don’t), so other, seemingly unrelated ideas, are part of the discussion. And when numerous versions of reality come together, I think everyone participating in the discussion comes away with a better, or deeper, understanding of the concept. (personal correspondence, April 2, 1998)
    If students asked questions about specific homework problems, they would address those issues. However, he did not solicit questions directly related to homework assignments. "If homework questions get asked and answered, that’s ok," he said. "But it’s up to the kids to make me do that because ultimately we will talk about the homework content during class" (personal correspondence, March 23, 1998). Lee did cover the syllabus material thoroughly (Field Notes).
    Beginning in January, another type of typical day occurred weekly on "project day." Students met with their groups and discussed their projects. Lee was available for questions, but he did not circulate the room. If they needed information, permission to go to the library, or statistical advice, they approached him. He gave advice and made suggestions, but rarely gave them detailed instructions. I asked Lee why he gave them vague answers to their questions about their projects. "I think the project and my lack of guidance helped them to connect all the seemingly disparate pieces of the statistical puzzle," he said. "Before the project, they probably saw the ideas as being unconnected to much of reality" (personal correspondence, March 17, 1998). Findings from students’ interviews support Lee’s idea that the project did help students’ construct overall understandings of statistical concepts.

Lee as a Constructivist

    Confrey (1990) proposed a model that connects classroom instruction to the constructivist learning theory. This framework consisted of six components: the promotion of student autonomy, the development of the reflective process, the construction of case histories, the identification and negotiation of tentative solution paths, the retracing and group discussion of the paths, and the adherence to the intent of the materials (Confrey, 1990). Inherent in this framework was the teacher’s commitment to active learning. Confrey suggested that the responsibility for and control over learning must shift from the teacher to the student.
    The first component necessary to shift responsibility to students required that students make a commitment to their answers. To accomplish this instructionally, teachers can ask students if their answers are correct, engage students at least by requiring that they explain what they attempted, act primarily as a facilitator, and/or involve students in evaluating their own work. Lee accomplished each of these elements, although not with equal weight. The discussion began with a question, posed by Lee or by a student. When another student answered the question, sometimes he responded with a question, sometimes with silence allowing others to explain or refute, and other times he probed that individual students’ thinking and forced him or her to explain their idea (Field Notes). These interactions between Lee and his students were the norm. This mode of instruction guided most classroom sessions.
    Next, Confrey (1990) suggested that students must "modify and adapt their constructions" (p. 116). She suggested that they face situations that are problematic, take action to solve the problem, and examine their action to determine if the problem has been resolved. Lee’s teaching included these strategies. He frequently began class by distributing a handout or assigning a problem on the board. Students worked at the round tables but usually sought to answer the question individually. After an initial solo attempt, they consulted others by asking questions or checking their answers. Their discussions focused on trying to determine whose answers, and therefore procedures and processes, were accurate. Then, as a class, they discussed and answered the question. Lee consistently forced them to explain what the problem was. He then allowed students to describe their strategies and forced them to defend their choice of strategy.
    Confrey’s (1990) third component evolved from student teacher interactions. She referred to the teacher’s familiarity of a student’s knowledge and abilities as a "case history" (p. 118). She suggested that teachers learn their students’ strategies and use this understanding to develop appropriate actions toward their problem resolution. Lee was willing and interested in developing these case histories with every student. Since he had the desire, small class sizes (14 students in fourth period, 17 students in sixth period) made this easy. Some students, however, were more open than others and Lee’s knowledge of their learning processes was more detailed. His accomplishment of this became apparent in our informal conversations. When we discussed a specific project, for example, he frequently conveyed insights into their learning and processes that revealed how well he knew that student. He used this knowledge to direct that student toward appropriate actions that would address his or her concern.
    The fourth component of Confrey’s (1990) framework suggested that teachers use case history knowledge and flexibility to adapt the class objectives depending directly upon students’ questions and understandings. One of Lee’s strengths as a teacher was flexibility. In our interview, he revealed that whenever students digressed, he saw this as an opportunity to investigate their thinking. He was never concerned that "getting off the subject" was that. He believed that students’ questions and issues, which seemingly detracted from the day’s purpose, were important to investigate. Many days, he admitted, he came to class with little or no predetermined structure. He presented a problem and let the students’ actions and responses determine where the investigation led. He was never concerned that they would not eventually cover all the material. His saw his primary responsibility as leading the class to investigate and discuss whatever they believed was relevant to the problem and its solution.
    Confrey’s (1990) fifth component involved reviewing the students’ solutions. She suggested this provides a variety of positive opportunities for students: reflecting on their process, looking at the specific problem holistically, and contributing to a sense of accomplishment. It is unclear if this must be accomplished on an individual basis. Lee provided these opportunities, but most frequently within the group settings. After presenting problems or while reviewing corrected test papers, he encouraged questions and discussion while leading them to an overall sense of the problem and its solution.
    The last component of Confrey’s (1990) framework required a commitment from the teacher to the mathematics content. While the instructor might not have a specific agenda of how to cover the material, it is essential that he or she had a clear picture of the information that needed to be addressed. After observing Lee extensively, it is easy to conclude that Lee was aware of the material required to cover the syllabus and prepare his students for the AP Exam (Field Notes).
    Overall, Confrey’s (1990) framework provided suggestions for teachers willing to shift their pedagogy and emphasize student activity, construction, and reflection. She admitted these techniques may appear to impede progress as related to traditional teaching techniques to cover material. However, she believed the knowledge constructed through these types of activities and processes was more accurate, more meaningful, and more powerful.

Interviewed Participants

    As the researcher, I was interested in the effect of concept-oriented instruction as recommended by the AP Statistics Test Development Committee. Lee and I examined a variety of characteristics to choose the specific students. A total of 12 students were chosen to interview in depth. It was important to obtain a diverse group of students representing all levels of the academic talent spectrum. I was also interested in observing juniors and seniors, males and females. Another important criterion for each student was to be outgoing and receptive to my presence and research. For example, three boys sat at one table in fourth period who rarely spoke to the instructor, the class or to me. These three students were easily eliminated for their shyness. The instructor and I frequently engaged in conversations concerning students’ academic abilities and other factors we thought might influence students’ learning. For example, I was interested in the only sophomore student taking his first AP class. His parents, however, would not sign the consent form and he was eliminated.
    It was important to select informants from both observed classes. The atmosphere of these two classes was quite different. Fourth period students engaged in discussions contained to their tables and students at neighboring tables. A student would frequently converse with the instructor. Other students would join the conversation while some continued to stay within the confines of their table. Sixth period, on the other hand, was frequently loud and sometimes ignored the instructor altogether. They would be engaged in statistical problem solving, yet would carry on conversations with students across the room rather than enlisting the assistance of someone closer. The noise level of this class was consistently higher than fourth period. Frequently, this class was behind schedule with respect to the other two classes because they would slow down instruction of the material. Therefore, two different groups from each class were targeted to study their project progress in detail. I interviewed at least one member from each of the project groups that were chosen. Listed below are the reasons that each participant was chosen.

Adam (AD): As a member of the mathematics team he represented the upper academic spectrum for juniors. He enrolled in Calculus BC his senior year. He is of Indian decent.
Beverly (BH): She was a senior who was not recommended for the AP Statistics class. She approached the mathematics chair, Lee, and requested permission to take this class. She was the only student out of all 40 enrolled who had not taken Precalculus. She was interviewed in detail to represent the group project BDL (Beverly, David, Lisa).
Carter (CF): He attended Governor’s Honor Program (GHP) the previous summer in mathematics. His interest in mathematical theory and rigor were significant. He was an African American junior who was recommended to take Calculus BC his senior year.
David (DW): He is also an African American male. Compared to the other 30 students, his mathematical abilities were average. He enrolled in Calculus AB his senior year. His attitude was excellent.
Ethan (EB): As a member of the debate team, he would frequently converse with the instructor in a more detailed way than other students. He displayed stubbornness about giving up before he believed he understood the concept. He was a junior of average academic talent. He enrolled in Calculus AB for his senior year.
Fran (FE): She was selected because she had Calculus BC the prior year as a junior and made a 5, the highest possible score, on that test. She falls on the upper end of the academic scale and was a senior. She was the student selected to represent the group known as FMN (Fran, Mark, Nancy).
Gretchen (GH): Academically, she represented a below average junior girl. However, her interest in learning was strong. She asked the instructor many questions during class and worked well with the other members at her table. She sparked my interest because she sometimes had an idea of the concept, but would continue to question and pursue until satisfied. She also consistently did the assigned homework. She was placed into Calculus AB for her senior year. She was selected to represent the GRS (Gretchen, Rachel, and Sam) group project.
Hanson (IF): His family heritage at this school was prominent. Academically, he fell in the middle of the junior class and was recommended to take Calculus AB for the next year.
Ingrid (IS): She was chosen because she consistently indicated an interest in learning statistics and expressed curiosity for my research. She was recommended to enroll in Calculus BC this year. She was Hindu Indian. She was selected to describe the ITW (Ingrid, Terry and Wesley) group project.
Josh (JK): He was chosen to represent the lower academic end for juniors. He was recommended for Calculus AB his senior year.
Lisa (LB): She was chosen to represent the below average female junior. She enrolled in Calculus AB her senior year. Like the others chosen for this academic reason, she worked with others well, was not intimidated to ask questions of the instructor, and was receptive to my presence.
Sam (SD): He represented a second generation Hindu Indian. Academically, he fell in the mid range for junior males. He was recommended to take Calculus BC in his senior year.

These were all honors students and each junior was advised to take an AP Calculus class their senior year. Overall, of the 12 students chosen, 5 were recommended to take Calculus AB, 5 were recommended to take Calculus BC and 2 graduated. I interviewed 7 males and 5 females. Nine students interviewed attended sixth period and 4 attended fourth period. Other students are quoted as members of the four selected groups: Mark, Nancy, Rachel, Terry and Wesley.
    The interviews were conducted the first 2 weeks following the administration of the AP Statistics exam. I was concerned about their ability to recall details on a test that was completed and, in many ways, no longer relevant to them. Because seniors at this school completed their course requirements and graduated before other classes are completed, I interviewed seniors first. Others were interviewed as early as possible. Since I observed the classes for the entire academic year and knew the students, I opted to audio tape only. Students were interviewed individually and privately. For the interview, we simply found a quiet, secluded spot near the classroom. Student interviews were conducted during class time. Every student appeared comfortable during the interview. Several students who were not selected indicated interest in participating.
    I asked each participant interview questions designed to probe areas of interest related to the research questions. Each student addressed the same questions (see Appendix A), but not in the same order because "the sequence of questions varies with each respondent, depending on prior answers" (Romberg, 1992). Frequently a student would answer a question not yet asked, while others would stray off the topic altogether. If they digressed, we explored their emerging ideas before returning to the list of questions. Also, it was common for a student to respond, yet not directly answer the question. I allowed them freedom to speak about whatever they felt important. I realized some of this information was not relevant to my research questions, but it was critical that they felt completely at ease to say what they wanted. In any of these events, I would probe until I felt all questions had been answered or until I believed the student was unable to provide the requested information.

The Students’ Perspective of Statistics

    In each interview, I asked the students to tell me about their statistics class. All 12 replied that the course was different from other mathematics classes. Five students suggested the course material was more applied. "Math was really different this year," Ingrid said. "Last year we were just totally manipulating numbers. That’s all it was. If we were sitting there writing the equation for a semicircle, ok, big deal. The thing about stat, I can admit that I saw the point to it" (IS, Interview). Ethan expressed a similar opinion.
    It’s a lot clearer from the very beginning why, what the final goal is when learning all these different steps. Whereas opposed normally [sic], you learn all these steps and maybe eventually when you are done, you learn why you’re doing it. Here you know why you are doing what you are doing from the very beginning because you just have the whole concept of trying to compare numbers that have been [sic], everything seems like it’s just trying to make two numbers that are apples and oranges comparable, whether its different numbers of samples or different characteristics and you are trying to account for that. You know that’s the goal the whole time and you can kind of see how everything you’re doing works toward that goal. (EB, Interview)
    Gretchen agreed with Ethan and Ingrid. "We took problems of life, like trying to figure out averages and things like that, real situations," Gretchen said. "I liked it, because unlike precal I can see how it relates to life. It is usable" (GR, Interview). Sam connected the practicality to a specific field of study.

It was more practical than other math classes. In precal and calculus, I don’t see any practical use to it. What are you going to do with a parabola or a derivative? At least here you can actually tell where people get their numbers that they do and how they make confidence intervals. If you ever study medical papers, that’s all they have. My sister is studying to become a doctor and she had to take a stat class to figure out what all the intervals and stuff were. (SD, Interview)

Five out of 12 students interviewed described differences between statistics and other mathematics classes related to practical applications.
    Consistent with Lee’s teaching emphasis (Lee, Interview), seven students reported the difference between statistics and other mathematics classes as conceptual. Hanson stated this difference clearly. "It was just different from math because it is very conceptual instead of working a set pattern," he said. "It’s like an English type of math. You describe things instead of solving for them" (HF, Interview). Ethan described his idea of conceptual learning.
    In other math classes you are doing math, like number crunching. But here, a lot of the number crunching is a lot less important. You don’t get equation sheets in other classes because what you’re learning is the equations. Here, it’s not learning the equations, but what they mean. And, you have to know, it’s a lot less clear. In other math it’s written in stone which equation you use. But that’s what the goal is here. It’s not if you know these equations, but do you know when to use them and how they work. That’s the biggest difference--not knowing the equations but when to use them and how they work. (EB, Interview)
    Sam explained a similar impression of statistics. "Its not solving for this variable," he said. "It’s more like, it’s not exactly what I think of as math. You do have some numbers but it’s more important to understand the concepts. In math usually its just plug in this number and you solve for x. Statistics is more idea oriented" (SD, Interview). David reiterates this same idea. "He tried to make us shy way from just punching numbers in the calculator," he said, "and talked more about what it means, understanding what we were doing" (DW, Interview). Adam expressed the same idea. "You could spend more of your time doing the concepts instead of the nit picky details with it" (AD, Interview). Overall, half the students interviewed suggested that statistics was more conceptually oriented than other mathematics classes they had previously taken.
    Five students stated a somewhat similar opinion by expressing that statistical manipulations were unlike those in other mathematics classes. Adam stated his opinion. "I guess you had to do a lot of reading as opposed to deriving, less computation," (AD, Interview) he said. Josh had a different idea. "When you are solving something you are not going to get an exact answer," (JK, Interview) he said. Two students suggested that statistics is not mathematical. Fran was surprised.

KR: Tell me about the AP Statistics class
FE: I went into it and I thought it would be pretty math based. I thought it would be more like you have a situation and you have to create a way to model the situation and analyze it from the model. But then after the first couple of days I realized, I obviously had no idea about the different statistical processes, I thought that was pretty math based. But a lot of it, I was really surprised. It seemed at least to me 80% of the entire course was not the same kind of math logic as it is for like calculus or prealgebra. I’m better at really concrete math, like those other classes.

Carter took this opinion one step further.

KR: Was there anything in this class that was similar to other math classes?
CF: I really didn’t think so. This is the first math class that I have been able to go into with a calculator and no paper and function and sometimes not even the calculator. He would just give us a worksheet and all we would have to do, especially on the exam preparatory worksheets we got the last half of the year it was so, it just seemed kind of pointless to me that we could just for almost every answer on the page we could type something in the calculator or draw back on some further knowledge to come up with an obvious answer to the question without ever having to get out a pencil or conceptualize a problem in your head and then write it down and then solve it. It just seemed that so little of it actually had to do with math.

These two students who indicated that statistics was not mathematical had the best mathematics background out of all three classes. Carter attended Governors Honors Program the previous summer and Fran scored 5, the highest score, on the Calculus BC test the previous academic year.
    Attempting to get an idea of their overall attitude about mathematics and statistics, I asked them what was their favorite class and if they liked statistics. Nine students said they liked statistics and referred to the practical applications. The mathematically weakest student explained her overall impression of AP Statistics.

Beverly: I liked it because what I got out of the class I think mostly, was a good grasp of the general concepts. The people who had a stronger background, a more in depth background than I did, probably got more of the nit picky stuff. I got more of a general sense of what normality is, how to plan a statistical study, instead of the step by step formulas. I think for myself, since I don’t plan to become a mathematician, I plan to teach high school English, it really served me well because I have a general grasp of what’s going on in this general area. I think it was a good class for me. (BH, Interview)

Three students indicated they did not like statistics. These three were among the top four mathematical students. They said they missed the rigor of pure mathematics and proofs. These are the same two students who said statistics was not like other mathematics classes. No race or gender patterns emerged regarding opinion of statistics.

The Projects

    The third research question examined the effect that projects, that employ cooperative group problem solving and writing, had on students’ development of statistical concepts. The Test Development Committee recommended student projects as part of the AP Statistics curriculum. The AP Statistics Course Description stated reasons to include projects (College Entrance Examination Board, 1997).

Students working individually or in small groups can plan and perform data collection and analyses where the teacher serves in the role of a consultant, rather than a director. This approach gives students ample opportunity to think through problems, make decisions, and share questions and conclusions with other students as well as with the teacher. (p. 10)

Following this recommendation, Lee included group projects as part of the curriculum. Lee and I discussed at length the process, timing, goals, and grading for the students, but he made most of the decisions. I asked Lee if I influenced his decisions regarding any of these elements of the overall process. "Your only direct influence was writing the specific questions on their progress reports," he said. "I would have made them do something, but I let you take care of that part" (personal correspondence, February 3, 1998). Therefore, questions on the bimonthly reports were designed to probe and investigate their thought processes, decisions they made, and why (see Appendix I).    
    Lee assigned students to groups after asking students to specify whom they would or would not like to work with. Lee’s concern was potential personality conflicts, rather than a balance of mathematical abilities. Most of these students were self-motivated and disciplined. He was not concerned about work load inequity. After students submitted their preferences, he discussed the groupings with me. By this time, several students had been identified for in depth interviewing. Lee and I conferred about which groups would serve the purposive sampling. Two project groups from each researched class were selected to investigate in detail.
    Their projects consisted of formulating a research question, designing the experiment, and conducting the statistical analysis. From January through April, students met once a week with their groups in class and worked on their projects. During these class periods, they had access to the computer lab, the Internet, the library, and other resources on campus. Lee served as a consultant. He provided direction for students when they needed assistance. I asked Lee why he chose to intervene as little as possible. He responded with the following.

I truly want them to find their own way in the project. I envision the
project (whether successfully or not) as a culmination and tying togetherof the AP Statistics experience. It is a very global understanding I am aiming for and for every detailed instruction I give them, the more localized the experience becomes. I wanted them to experience the frustration and/or exhilaration of focusing on an interesting question, designing a method and collecting the data to answer the question, analyzing the data, and finally trying to make sense of the data to see if it truly helped to deal with the original question. It's a monumental task for anyone, let alone 17-18 year olds, but it removes statistics from the classroom and brings the subject to life. (personal correspondence, April 14, 1998)

Data collection and analysis were conducted solely by the students. The guidelines for their projects and accompanying poster followed the rules for the national high school competition sponsored by the American Statistical Association (ASA; see Appendix J). On April 15 and 16, 1997, each group gave presentations of their projects and posters to the class. Lee’s assessment of their projects included their bimonthly project reports, their oral presentation, and their written report. He graded holistically, encompassing all three required assessment assignments.

Gathering Data from Students’ Projects

    In order to address the third research question about how projects that employ cooperative group problem solving and writing influence students’ development of statistical concepts, I gathered bimonthly project reports, audio taped project presentations, and asked the students a variety of interview questions.
    All students submitted bimonthly reports to Lee describing their progress. They were asked open-ended questions and each student answered them individually (see Appendix I). Four groups were chosen to follow in detail. I was able to attend sixth periods’ project days most weeks from January through April, but had limited direct contact with fourth period. Regardless of whether or not I was there all four groups individually audio taped the conversations each week when they met for project day. I attended and audio taped the presentation of all four projects. One student from each group was chosen to represent their project for interviewing. Lee provided copies of their project reports that were submitted for traditional grading and the ASA project competition. Data analysis is presented after each group project is described individually. Acronyms, composed of the first letter of each member’s name in alphabetical order, are used to distinguish one group from another.

Projects’ Construction and Analyses

Project FMN: A Study of Our Nation’s Economy and Its Economic Indicators

    Project Construction. Fran, Mark and Nancy (FMN) began with several project ideas. Initially, each team member gathered preliminary data on different topics. They finally decided to investigate whether published economic indicators accurately reflect changes in the American economy. "We have now selected our problem and it is a comparison of the Consumer Price Index (CPI), Dow Jones Industrial Average (DJIA), rate of inflation, and real Gross Domestic Product (GDP) to determine if they are accurate determinants and reflections of the economy," Mark wrote (MB, Project Report 2). They utilized the Internet and easily obtained historical CPI and GDP data. After a frustrating search, "I found the Dow Jones weekly averages from 1910 until today," Nancy reported (NZ, Project Report 3). However, the following week, "Our group suffered a small setback when I lost the disk with all the information on it," Mark wrote (MB, Project Report 4). Later, Fran reported what they had accomplished.

We have collected GDP and CPI for several years and recorded the data in MINITAB. We have once again found the Dow Jones site we had. That data was on a weekly basis. Since all of our other data is in the yearly form, we cannot really use the data as is. So Mark is getting in touch with a source who is going to find the yearly breakdown on the Dow Jones because we are spending too much time on the Internet. We have used MINITAB and found the basic statistics breakdown for the CPI and the GDP. However, this is only the easy part. Now we have to see how they are related. (FE, Project Report 6).

The next project report indicates they had collected the data and were involved in data analysis. They reported problems with this stage also (Project Report 7, Fran’s interview, project presentation). Fran described the problems and resolution in their project presentation.

One of the problems we had was the units. We thought that you know, if everything was in the same units it would be a lot easier for us, but GDP was in millions per year and CPI is like an index that’s just a percentage. So we thought that might be kind of a problem because we’d have to liked to have millions of dollars versus millions of dollars. That way we could tell, we could base it on the economy, on the money circulating through the economy. But we couldn’t do that. After thinking about it, we decided that it wasn’t more of like, more money, how much money was in circulation, but how did the growth look. (FE, Project Presentation)

The next stages of analysis involved comparisons by year. They ran correlations, regression lines, compared slopes, and examined each indicator individually by year looking for trends and outliers (Project Report 7, Project Presentation). In the interview, I asked Fran how they knew to look at the growth by year.

KR: How did you decide to run them all against the year? How did you know to do that?
FE: I think just, we got all this info and we had it all by year. In looking at it, we didn’t have to do it between the two indicators for the first type of look. It just intuitively made sense to see how it progressed over the years. It was natural to say that first we needed to look at them by year.
KR: Did you plot time on the horizontal axis a lot, say in calculus and/or physics?
FE: Physics, yes, we did a lot. Calculus, not really. It could have been from physics, or even government when we did a little section on economics. I mean a lot of it was money and circulation or inflation versus time and that just seemed to fit here also.

While examining the data graphically, they noticed the curves for all three indicators were similar. Initially this discovery confused them. "I think the difficult part was when we were actually doing GDP to Dow Jones or CPI and they looked similar but were off. We were confused. They looked so similar, just this way [sic] but not the same when comparing the years," Fran said (FE, Interview). Fran decided to compare slopes to test how similar the curves really were. In the interview, I asked her why she investigated the slope.

KR: When you were looking at the stocks, to reanalyze the data, you stated in your project presentation, ‘I went back and looked at the curves based on the slopes between 1960-1970.’ Then you stated that the slope was increasing. How did you know to look at slopes?
FE: That’s probably something I picked up through calculus.

They also compared regression equations. Fran admitted that she expected all three equations to be linear and similar. "I didn’t know why we were linearizing it," she said. "I guess I just had it in my mind that GDP versus Dow Jones would be linear and all three indicators plotted against themselves would be linear because if they are supposed to all indicate the economy, they should have the same kind of conclusion" (FE, Interview). The graphical analysis led them to conclude that CPI and GDP are modeled by similar cubic functions. "But the Dow Jones with the same curve, after we thought about it, kind of had what we termed a time lag," Fran said. "Because the Dow Jones is actually the stock market, and how do people respond to what they see like [sic] increasing and decreasing and what they think will make them money" (FE, Project Presentation).

    While summarizing their project, Mark stated that including certain variables could have improved their results.

I guess there’s a lot of things we could have done to make it better. If we had had time and also resources, we could have seen the effects of tax changes, seen the effects of new government programs. Basically there are a lot of things that affect the amount of real money--interest rates and stuff like that, that can affect the disposable income that people have both to put in the stock market or just to spend which would affect the CPI. So, we could have, if we had more time, we could have factored in all those other things and probably come up with a better model. (MB, Project Presentation)

To conclude, they stated that the best economic indicators were GDP and CPI. "Looking at the Dow Jones, if we shift the Dow Jones over a certain amount of time, and by looking at slopes we found that actually it was almost the same," Fran stated. "But the time lag means it is reacting to the economy, not predicting it" (FE, Project Presentation).
    Project Summary. This group sought to determine if various economic indicators are accurate reflections of the overall American economy. They gathered data from 1959- present from the Internet on the Consumer Price Index (CPI), the Dow Jones Industrial Average (DJIA) and the Gross Domestic Product (GDP). They hypothesized that if all three indicators represent the economy, these indicators should highly correlate with each other. To analyze the data, they used computer software to calculate descriptive statistics, including the mean, median, standard deviation and graphical displays. They discussed variation in the data and made comparisons from one data set to another. Attempting to model each data set and expecting each indicator to have linear behavior led them to a conclusion. They decided that GDP and CPI were immediate indicators of economic growth and decline, but the DJIA had a time lag. Based on this they concluded that GDP and CPI were accurate economic predictors but the DJIA was only a reflection of what had happened.

Project GRS: How Are You Feeling Today
Project Construction.
Gretchen, Rachel, and Sam (GRS) began with lack of a consensus for a project topic. By the second project report, they had agreed on a broad topic but were aware of potential problems. Sam described the project goal.

We want to do a project on Seasonal Effective Disorder (SAD). We have begun research on the topic by searching on the Internet, but some of it is irrelevant. We want to compare how a person with SAD reacts to different weather and how a person who is supposedly normal compares with the SAD person. (SD, Project Report 2)

They planned to write a questionnaire and administer it to people on different days depending on the weather. Each member reported they would obtain a random sample of adults and students (Project Report 2). At this stage they were aware of confounding variables. Gretchen described their dilemma and a possible solution.

There are numerous reasons why people may be in a bad mood, and we may have problems determining if the clouds cause the person’s depression or if other events of the day are the cause. To try to eliminate these confounding variables, we will ask the people if there are any events that have made the day bad. (GR, Project Report 2)

By the third project report, GRS has confronted more obstacles as a result of modifying their topic. Attempting to test psychologists claim that 10% of the population have SAD, they were unsuccessful at "finding and contacting research groups that have dealt with SAD" (GR, Project Report 3). Each member reported frustration again. Finally, all members reported accomplishments. After discovering that SAD is not prevalent in the South, they shifted their project emphasis to determine if weather effects mood of students at their high school. They wrote the survey instrument and obtained a random sample of students to survey. They expressed concern about students returning the surveys. In the interview, I asked Gretchen why this was a concern.

KR: What about this voluntary response bias? How did you know this could be a problem?
GR: Sometimes in homeroom, we get surveys. I see people just throw them away or some of them give them to other people who want to do them. So you might not even get who you think you are getting. Then we offered them a reward and we knew it was a problem because of the definition in the book. But from watching people in the past we knew we had to do something. So we offered them a reward but didn’t give them anything. (GR, Interview)

They sent questionnaires to the same students on days representing different types of weather. At this point, they were unsure of how to analyze the data. Also, the process of converting survey information to data in the computer was tedious (GR, RH, SD, Project Report 5). In the next project report, Sam wrote that they identified some confounding variables and decided how to control for them. "We have decided not to send out questionnaires during exam time because most people will feel depressed," Sam wrote. "Additionally, we have decided to only send out questionnaires on Tuesday, Wednesday and Thursday. People feel either bad or good on Monday and Friday for no particular reason" (SD, Project Report 7). Compiling data from the surveys consumed their time for the next several weeks. Gretchen reacted to the data. "The people’s responses are interesting," she says. "On the same day, some say the day is beautiful while others say it is awful" (GR, Project Report 8). Rachel reported frustration. "Today would be a great day for our surveys," she wrote, "but tomorrow is Saturday" (RH, Project Report 8). After inputting data, they began analysis. Gretchen described this process during the interview.

KR: Tell me about one specific part of the project, either because it was interesting or difficult.
GR: I guess we had all these categorical variables and we wanted to do all these quantitative things with it. It was hard. While we were doing it, we were just thinking about how we can we get this put in the computer so that we’ll be able to use it easily or make sense. Then we got it, we were going, what are we supposed to do with it. So it was really sort of hard to figure out how to translate that data into meaning something in the class. We couldn’t do a best fit like a lot of other people did because we didn’t have it, so we had to get creative on how, the ways that we were comparing things and looking for associations. In order to find a correlation between all the different grades we had to do a chi-squared test for all the different grades. It took a lot longer to figure that’s what we needed to do instead of just pressing a button on Minitab and you get a graph which is what a lot of people were doing because they had numbers and they could do that. And if they thought correlations, but we had to test for differences. It wasn’t necessarily creative but it just took longer to figure it out. We couldn’t do what we wanted to do or what we thought we could do.
KR: So how did you finally figure out how to analyze this categorical data?
GR: We just playing around with the tests and then we asked Lee, he told us to put it all in one big test. We had lots of little chi-squares here, so we just put it all together to get a general overall, not for breaking it down. We just figured it out from looking at it. Sridar took it home and started playing with it and he came back with a problem. When we began talking about it, it just all fell into place. Before we realized it wasn’t quantitative, I had tried to graph it and then realized no wonder this doesn’t mean anything when graphing it. After that we just began talking about to do with it. Then Saturday night after we put everything in the computer, we just started doing calculations. Once we started talking about it, it just naturally broke down to the tests. (GR, Interview).

Their analysis included calculating correlations between variables, graphical explorations, and chi-square tests. To present their results, they used bar charts showing percentages of responses for each weather day--clear, cloudy, and raining. During their class presentation, Gretchen stated one overall conclusion. "On the days where the weather wasn’t extreme, there was no perceived effect," she said. "But when the weather became the extremes, in either direction, that’s when the effects took place. That’s when people really like notice the weather" (Project Presentation). Sam summed up their analysis including weaknesses during the oral presentation.

I guess we can’t conclude anything, because you can see that most people just responded three or four and they might have done that just because they were sick of the surveys. But we can conclude that people think that the weather really changes their mood in a big way on those two days, but the other two questions, I guess you could say that those were the best indicators that we had to show that people really, [sic] weather didn’t really affect them. (SD, Project Presentation)

    Project Summary. GRS investigated the effect of the weather on a person’s mood. To address this question, they wrote their own survey instrument, devised a system to obtain a random sample (from a limited population), and analyzed the collected data. Since the survey questions used the Likert scale, they analyzed the categorical data using chi-square tests. They discussed problems with the survey, voluntary response bias, and inconsistent answers. They concluded that people perceive that weather effects their mood but the data did not support any association.

Project ITW: The Great Pricing Expose (That Failed)

    Project Construction. From the beginning, these students wanted to expose a social injustice. The initial project reports indicate all three students were interested and enthusiastic about the same idea. As early as the second project reports, they had a plan.
    ITW hypothesized that prices are higher at lower income area grocery stores. They believed that insurance rates are higher in lower income areas, driving costs up, and that supply and demand principles bring prices down in suburbs. To test this hypothesis, they formulated a plan to obtain an outside food pricing list and to stratify the city based on income to obtain a random sample of grocery stores. Wesley explained why they sought an externally created list of grocery items to compare. "We’d really like to have our representative list of groceries come from a done study on what people generally buy," he said. "Any list we compiled on our own would be biased" (WB, Project Report 2). They were also aware of sampling issues. Ingrid discussed why their initial plan required a random sample. "We need official information of official income area divisions so that we can objectively compare one grocery store to another based on income area," she said. "Hopefully, if we are able to obtain this information, the data will be representative" (Project Report 2). They foresaw the possibility of confounding variables and explained how to control for them. Scott summarized these in his project report.

There is the problem that one brand of goods may not be present at one store. So, we will record the prices of a number of goods and then analyze the ones that were at every store. So, we will have to test a relatively large number of goods to avoid such confounding variables as sales and shortages that will have an influence on prices. Also, we will only go to one type of store (Kroger, most likely, as it is the most common) to avoid a confounding variable: that our results may have been caused by store type and not by income level. (TP, Project Report 2)


 
   The next project reports state success at obtaining a grocery list after exhausting many resources. However, obtaining all the information to generate a random sample continues to elude them. Ingrid does not express frustration, yet what they have obtained has been difficult.

When I called the Georgia Department of Labor, I was transferred a gazillion times before someone finally told me to contact the local census bureau. The lady there said each county is divided into about 60 tracts, with about 4000 people each, and that it would be no problem to obtain the median household for each tract. However, she said we needed to decide which counties we wanted and obtain the geographical tract maps from Georgia Blueprint Company. (IS, Project Report 3)

They ordered and received these maps only to find out "they don’t have any street on them except for the borders" (IS, Project Report 4). After many phone calls, Terry describes the process to help this latest problem.

We figured that because there are no streets on the tract maps, we were going to get a city map and plot both the tracts and the Krogers on it. But, Ingrid called the Census Bureau and they have a program downtown that will give you the tract when you type in addresses. We are going there Wednesday after fourth period. (TP, Project Report 5)

The following week they reported more problems. Terry describes what happened. "Many of our addresses were not included on the tracts because the tract match program was for residences only and did not count businesses," he wrote. "So any Kroger that is in a shopping mall, didn’t show up. We will locate those on a city map to determine the tract location" (TP, Project Report 5). However, this did not work easily either. "We were finally forced to find several Krogers by calling the stores to verify the exact location, a laborious process (Written Report)." Once they had the locations of all the stores, they numbered each from 1-67 and generated a random sample. Satisfied with the design, they split up to gather prices from the randomly selected stores. After each student had been to a few different stores, they noticed all the prices were identical. They reported frustration and disappointment (IS, TP, WB, Project Report 7, Written Report). In their presentation, they stated the analysis they would have conducted if the data had varied.

    Project Summary. These students undertook an involved process to answer a question of social value. They expressed sincere concern that grocery store prices are higher in low-income areas where people cannot afford higher prices. They based this hypothesis on economic principles of supply and demand, and insurance rates. Determined to conduct a randomized sample, they went to extreme measures to collect random data from the same grocery chain in various parts of the city. They relentlessly explored various approaches to divide the city into income-based partitions and choose stores representing three different income levels: high, medium and low. After this labor intensive process of designing the experiment, they physically went to the chosen stores to collect prices for various items. It was at this point they discovered the chain they had chosen has fixed prices for all the stores in the area regardless of the neighborhood. It was too late to redo the experiment so they presented their sampling and surveying techniques.

Project BDL: Fast Food and High School Students

    Project Construction. This project received Honorable Mention in the American Statistical Association (ASA) Project competition (American Statistical Association, 1997). Initial project reports reveal this group’s intent to investigate fast food restaurants and health issues. David describes their goal. "Our area of interest is fast food restaurants and nutrition, dealing with their menu items," he said. "We want to find out which major fast food chain can be considered the most healthy and least healthy. We also want to survey students regarding their preferences" (DW, Project Report 2). Beverly discussed the sample and survey. "We plan to collect nutritional data from each restaurant (how we’re not sure)," she wrote. "Then we will select 100 boys and 100 girls from both the 11th and 12th grades based on randomly generated numbers" (BH, Project Report 2). During our interview, I asked Beverly why they targeted the upper two grades. "Juniors and seniors can drive so they have more freedom about how often to eat fast food," she responded (BH, Interview).

They spent the next several weeks gathering nutritional information from 10 fast food restaurant chains. All three students reported problems gathering this data (BH, DW, LB, Project Report 3). To obtain information unavailable on the Internet, they telephoned local restaurants. Simultaneously, they are working on the sampling procedure. Beverly reports they will not send surveys to other schools as originally planned. In our interview, I asked her about this decision.

KR: In the fourth project report you refer to reducing the sample by eliminating other schools and just concentrating on your school. What were the ramifications of this decision?
BH: Well, first of all, it would have been really hard to get the data from the other schools because we just didn’t trust them to return stuff on time and do what we really wanted. So in that respect, our data was more thorough from the school we could study but we knew that in doing that, we couldn’t say anything about any population other than our own school.

They visited the registrar’s office to obtain homeroom lists and social security numbers and were told they could not have access to this information. David described their alternate plan. "As a group, we decided that those numbers were not necessary," he wrote. "It would be just as easy to take the homeroom list and just number the students 1-200 or whatever off that list and do the SRS off those numbers instead of their ID numbers" (DW, Project Report 4). Beverly reported how they obtained the random sample.

Once we had obtained the homeroom list, we went to the computer lab and scanned the homeroom lists, so that we could cut and paste until we had separate lists of boys and girls for each grade. I took the junior lists and Karen took the senior lists to divide up and pick the SRS. In my spare time on Thursday, I sorted out the guys from the girls and then numbered them and used the table of random numbers to pick the two SRS’s. (BH, Project Report 4)

After choosing students for the survey, they began work on the survey instrument. David commented on this process. "We had to take into consideration what type of analysis would be possible when we receive our SRS results," he wrote (DW, Project Report 5). Beverly referred to difficulties with the survey. "The survey has been a debate filled process, but we finally have something we think we can use. We decided we will do a pilot to try it out" (BH, Project Report 5). In the interview, I asked Beverly why they piloted the survey instrument. "Because we weren’t sure if our wording was okay," she said. "It was really very difficult to decide, even once we had the ideas for the questions. It was the wording that took us days to do and we still didn’t end up completely correct" (BH, Interview).
    During the next week, the circulation of the quarterly campus newspaper, the Herald, created an unforeseen problem. Beverly explained. "We had the surveys ready to go out this week," she wrote, "but the Herald that came out on Monday had a lot of information pertaining to our survey in it, fat grams and calories, and we agreed it would definitely be confounding if we released the survey now" (BH, Project Report 6). Lisa describes their reaction to this situation.

This week we had planned to get the surveys handed out, but because of the Herald article, we decided to wait. Instead, we finalized our list of people to be sampled, gave Lee the master copy to print out 250 copies, and gave a pilot survey in our class. We played with the numbers and decided what hypothesis tests we will run and what kind of relationships we will look for when we get our surveys. We needed to do all this anyway so the Herald article probably won’t make any difference after all. (LB, Project Report 6)

The next few weeks they circulated surveys and organized the data. Data analysis began. They calculated means, standard deviations, minimums and maximums. They conducted confidence intervals and hypothesis tests at the 95% confidence level. Analysis included how many times per week for females versus males, favorite restaurant, student ratings of how healthy each restaurant is, comparison of frequency versus perception of nutritional level, and perception of nutritional level versus actual nutritional level. In Beverly’s interview I asked her about any problems they experienced analyzing data.

KR: David is talking about rating the restaurants from one to ten to test each student’s perception of how healthy each restaurant is. What statistically is difficult about what you were trying to do?
BH: It’s really subjective. We realized in compiling the information two or three days before the project was due that we were asking to rate the restaurant against just that restaurant. And in the end we kind of compared the restaurants to each other. So that part is really subjective. There’s really no basis for them to base it on other their perception.
KR: So is there anything you can think of that you would do differently?
BH: Yes, I would go back and rate them, just ask them to rank them in order of preference rather than rate each one separately on a scale of one to ten. For example, I would say Subway was the healthiest and Taco Bell was the least healthy. In the end we realized that was really what we wanted to know but we just didn’t know the best about going about it.

To try and answer this specific research question, they tested the actual ranking against the students’ average for significant differences for each restaurant. They stated overall conclusions in the written report.

No broad conclusions that enveloped every hypothesis test we ran could be found from our results. Instead, the numbers proved that some restaurants are seen as too healthy, others perceived as too unhealthy, and only a few are interpreted correctly by students. These misconceptions are probably related to many variables, including advertising and the marketing tactics of each restaurant. (Written Report)

The final question of their survey listed one menu item from four different restaurants. The survey asked respondents to guess the number of fat grams for each item. To address this portion of their analysis, they began their oral presentation with an activity. As they were organizing their presentation materials, they offered a Hardee’s biscuit to everyone who wanted one (including Lee and me). After everyone consumed these biscuits, they asked us "How many fat grams do you think were in that biscuit you just ate?" (Project Presentation). They collected data and organized it using a stem-and-leaf plot on the board. During the interview, I asked Beverly why they did this.

KR: Why did you bring biscuits in for everybody to eat when you did your presentation?
BH: As a surprise. To us it was like the Italian salad. People know that biscuits are fatty. I mean look at them, they’re greasy and goody, yum yum. But when we found out those biscuits had 21 grams of fat in them, we were floored and thought most other people would be too.
KR: Where did you get the idea for the presentation to let everybody say how many fat grams were in them and collect the data? Then later you did the same thing, gathered data from the class, about the five items you put on the survey. Why did you do this?
BH: I’ll take credit for the biscuits. I just realized this right now when talking to you, in English we’re taught that it’s helpful to have a quirky introduction sentence that will get the interest of the reader and make them want to keep reading. So we thought if we started off with something that made them say, whoa, then their interest might be held better for the rest of the material.
KR: Why did you go around the room and let everybody guess and then collect that data?
BH: Part of it is that is what we did on the survey. We also thought they could surprise them some. If we had said, hey guys there’s 21 grams of fat in the biscuit, it would have, it just wouldn’t have been the same. The impact would be stronger if they had guessed first and they were 10 or 15 grams off and then get the answer.
KR: So do you think when Lee used this same technique on your class all year there ever some instances where you experienced what you just described?
BH: I’m sure we did. I can remember him handing out little sticky tabs that we had to put our weight, or how far we lived from school on them.
KR: No, I mean if you had to think about something beforehand and then be given the answer, did you learn something differently?
BH: I can’t remember anything specific, but I feel sure that I did. Especially for me because I know that is the kind of thing I would respond to, a little trick here and there.
KR: Did you realize that you were imitating his teaching technique?
BH: No, I didn’t have a clue. Here I am pulling English metaphors because I had no idea. But now that I think about it, it’s obvious that that’s what we did. (BH, Interview)

Numerous times throughout the year Lee collected data from the class to actively involve students. This group was unaware they modeled his pedagogy.

    Project Summary. BDL studied issues related to fast food consumption and misconceptions among high school students. After devising a method to obtain a simple random sample from their high school, they wrote and piloted survey questions. Data analysis included graphing techniques, regression analysis, confidence intervals and hypothesis testing. During their class presentation, they were unaware that they modeled one of Lee’s teaching techniques.

Projects’ Analyses

    I asked one member from each group what they learned from the project. Each student initially described the project and details related specifically to their study. I probed them individually seeking to uncover how they learned general statistical concepts. All four students referred to advantages of conducting statistical analysis first hand.
    Ingrid explained her experience. "Maybe because we were just sitting there going because of this, because of this, this will work," she said. "I think it helped because we were constantly thinking about statistics and having to apply it" (IS, Interview). Fran reflected on real world applications.

A lot of the stat stuff that we did in class was just a concept, I didn’t see how it mattered. We didn’t have any great revelations from the project but we saw how it mattered in the real world. A lot of the experiments that other people do, they’re just not as easy as they look. Design wise ours was not too hard. Once we decided what to do, that part was pretty obvious. But the analysis was not real easy. We really had to think about it and talk about it before we figured anything out. (FE, Interview)

Gretchen explained how the project was different from studying the concepts in class. "When you actually do it out, instead of simulating it, you find all the problems," she said. "When you simulate a list of random numbers there is no bias, there’s no confounding variables. So it just helped identify what those things really were" (GR, Interview). Beverly expressed an appreciation for statistical applications in the real world and a deeper understanding of statistical concepts.

KR: What did you learn from the project?
BH: Overall, as far as stats and real life mixed together, how much data is surrounding us all the time and all the different methods and techniques that we learned are being used all the time. When we put together our own study, we experienced what others have done who have put together data. And it’s tough. The knowledge we learned in the class, we actually applied it. You can learn anything in class and just forget it a couple of weeks later. But with the project it was positive reinforcement and basically engraved that knowledge and techniques into our heads. And it helped us appreciate what companies and others do that print out stats, it helped us see what they do.
KR: Anything else?
BH: Yes, because there is a difference between the collection of data and the presentation of data [sic]. To do a whole project, you have to first get the data and then present it but in the middle you have to sort it and do stuff. So when you are presenting it, you have to have it organized and make sense. The numerical analysis gave meaning to all the tools, like the null hypothesis and some of those other symbols. When you put it all together in a project, that’s when it becomes real. That's when it has meaning. (BH, Interview)

Overall, no patterns emerged relating the effect of a group project on learning specific statistical concepts. All projects were unique and learning outcomes varied. However, students from all groups expressed that conducting the analysis did help solidify their overall statistical understandings. Unfortunately, none of the four targeted students participated in the writing of their project report. Thus, the effect of writing was not addressed.

Students Gathering Project Data

    There is widespread agreement among statisticians and educators that students should interact more with data, including collecting their own data (College Entrance Examination Board, 1997; Hogg, 1992; Moore, 1990; Watkins et al., 1997). Therefore, the fourth research question addressed how did gathering data effect students’ understanding of statistics. I gathered research data from copies of project reports, transcripts of tape-recorded sessions of project days, and interviews. Students’ responses to interview questions provide most of the information addressing question two. I asked 12 students, "What did you learn from gathering data?" Overall, five students responded about writing survey questions, four students referenced the sampling procedure, and seven addressed the collection of data.
    Of the six students who referred to the survey, five indicated it was more difficult than they had expected. Carter’s reflection reveals that writing survey items are an integral part of a rigorous statistics project.

I learned how difficult it is to, I learned more about the process of gathering data [sic]. I learned that you need a whole lot of time to actually just sit down and visualize what you want to do before you send out a survey, make sure that you don’t create the survey with more biases. So that if there are problems with it, you can say at least that you didn’t cause them and then you can look and examine other things. (CF, Interview)

Others articulated difficulty writing the survey questions. David explained why he was challenged by this process.

When you are writing a question, after reading the chapter about how wording can manipulate somebody’s answer, looking at a question and seeing 50 different ways that can be interpreted [sic]. It’s kind of frustrating because every word has to be concrete, no latitude for somebody’s own insight at what you are trying to get at. Each word has to be perfect, get to the point, no interpretation whatsoever and that was difficult. (DW, Interview)

Beverly was also challenged with writing survey items. Her reflective process revealed that wording can be critical in designing a survey.

KR: What did you learn about surveys?
BH: How hard it is to make questions to get what you want, trying to figure out what to put on the dumb piece of paper. It surprised us how easy it was to do the random sampling and pick the people. I thought that was going to be much more complicated and the survey would be, here’s a question, here’s another question and just copy and go. But it was really very difficult to decide, even once we had the ideas for the questions. It was the wording that took us days to do and we still didn’t end up completely correct. Even the last thing, we realized that fat grams was a bad choice and we should have done calories because that’s something most people are more familiar with and a lot of people interpreted it as that. We think that a lot of people read it as calories instead of fat grams and that may have messed up our results. (BH, Interview)

Hanson revealed another relevant problem with designing survey instruments. His group’s project included subjective questions. Hanson suggested their data was possibly biased and unreliable.

KR: What did you learn from gathering data?
HF: Setting it up wasn’t hard. But making up the surveys, getting them back, typing them into the spreadsheet was a pain. It’s not very easy. If you are doing something subjective like an opinion poll, there are a lot of answers that you aren’t sure if you should use. You can’t tell if people are being serious or if it was a joke. That could be a big cause for error.
KR: So how did it help you learn statistics?
HF: The reason it was hard to get anything concrete is that the answers were so subjective. Categorizing the answers was subjective. So we could kind of get a general feel for it, but since it wasn’t like a blood test that was either positive or negative. Just plain numbers are easier to work with instead of yes, no, sometimes type answers. So sometimes we had to use the same general field when we weren’t sure that’s what they meant, like are beautician and cosmetologist the same.

Josh was the only student interviewed who did not have trouble writing survey questions. "Writing the questions wasn’t hard," he explained, "because we knew exactly what we were looking for" (JK, Interview). In general, five students expressed difficulty writing survey items and one indicated otherwise. Those who found the process challenging provided different elements that were perplexing.
    When asked what they learned from gathering data, four students referred to the sampling procedure. One thought the simple random sampling process was easier than expected while two said the opposite. The other student, Ethan, did not refer to ease or difficulty as he described their sampling procedure concisely and accurately.

KR: How did you choose whom you sent the survey to?
EB: We had a book of all the schools in Georgia and randomly chose pages with the calculator and then we wrote down every school on that page. Usually there was only one school on a page and we did 25 generations of a random number. It turned out being pretty spread out geographically and was random. It would have been difficult to try and stratify by size. It ended up being a good mix of schools. (EB, Interview)

Lisa and David (who worked on the same project) indicated complications conducting a random sample. They compared the sampling activity to the classroom experience. "When we were doing the surveys, I remember the simple random sample and having to do one. It was different having to do it than just reading about it in class," Lisa said. "It took a lot more time than I thought it would" (LB, Interview). David agreed with Lisa.

KR: What did you learn from gathering your own data?
DW: Non response, wording of questions had to be perfect, how having a simple random sample, having it totally randomized [sic].
KR: Was that part easier or harder than you expected?
DW: Probably harder than I expected because when you think about picking a random sample out of your classmates, its kind of hard. You can’t just go up and pick this guy or that person. Everything has to be totally randomized. That was a little bit more difficult than I thought. It was easier reading in the text book and understanding it than actually do it yourself. (David, Interview)

Beverly was the only student who expressed the notion that sampling process was easier than she anticipated. "It surprised me how easy it was to do the random sampling and pick the people," she said (BH, Interview). Overall, only two students expressed the sampling process as challenging. All but one, however, expressed an appreciation for the difficulty and the necessity of writing concise, unbiased survey questions.
    When asked about collecting data, six students referred to difficulties obtaining information from outside sources. Adam responded that he needed to take more initiative and responsibility.

KR: What did you learn from gathering data?
AD: We ran into a lot of problems with the data collection. We talked to department head here, English department, and wanted the same paper, as many as we could from different teachers, would she be willing to do it and she said yes [sic]. And she would contact her teachers. We wanted every grade every level so we could see if there were any trends across grades. But what ended up happening was she got really busy and lost track of time. With like a week and a half to go, we had no data. So we just went to all our English teachers and got what we could from them. So we had three sets of data from one class each but one from each grade. But all the classes were honors or AP so we didn’t get to see if there was a trend from lower classes and not from ninth grade. Data collection requires like sustained effort as opposed to expecting people just to comply. You have to realize that other people, it might not be on their list of high priorities and you have to make sure you get what you need. Next time, we would probably get permission from the department head but contact each teacher on our own. And do more follow up to make sure they are getting stuff done. (AD, Interview)

Carter also recalled problems gathering the information from their participants. "And then the actual collection of data was a whole lot more difficult than I imagined because people are so much less predictable than you would like them to be," he said. "We learned a lot about that" (CF, Interview). Josh explained the obstacles his group had obtaining the information they needed to conduct their project.

Few people wanted to participate. It was a lot of trouble getting people interested. When we called the schools, if they were public we had to go to the county people and get permission, who then had to call and say it was all right, then see if that school wanted to participate. It turned out, they would say they would get back to us and we had to call them again several times. In the end we only got about 5 schools out of the 60 that we had randomly picked. (JK, Interview)

Ethan’s group experienced similar problems when seeking to obtain permission prior to data collection.

If you’re going to try and collect the data, be real straightforward about it. Don’t ask people if you can collect the data, just send the packet. We tried to call to get permission and we got sent all around. We got the best results in terms of receiving the responses to those we just sent in a nice formal package. (EB, Interview)

Another group had the same difficulty obtaining basic information. They spent several project days making phone calls and searching the Internet. Finally, they received permission and drove to the U.S. Census Bureau to access their database. Ingrid recalled their frustration.

KR: What did you learn from gathering data?
IS: Lots of things. We always assumed that the Internet was this magic place and whatever you were looking for would be there. That was, we were all sort of familiar with the net but we kept running into one roadblock after another. We spent the first two weeks just doing that and not going anywhere. Finally, we started thinking about who could give us the information we were looking for. Another problem was we didn’t have a clear idea of what we were looking for. We just knew we wanted to divide the city but weren’t exactly sure what we wanted. Then we started calling people and that was another adventure in itself. It’s time consuming. Ninety-nine point ninety nine percent were not willing to put the effort to help you. And most of them didn’t have the information and didn’t seem to care. We got transferred everywhere, automated push one, push two. It was frustrating. We finally go the census bureau and get the stuff we needed and it went well from there. (IS, Interview)

Only Gretchen indicated that she enjoyed collecting data. She also suggested that the groups who gathered information on the Internet did not share her experience.

I liked collecting the data and it helped define the whole experiment and bias stuff. I don’t think a lot of people that just got numbers off the Internet, that they quite understood it the same way I did because I was dealing with it and trying to decide what to do with information that we were getting. (GR, Interview)

Out of seven students who discussed collecting data, six reported frustrating experiences. Only one student enjoyed this part of the process. Of the six who reported frustration, five were male. The only student who enjoyed this process was female.
    Overall, students who gathered data using survey instruments and sampling techniques reported several types of complications. Five indicated writing survey questions was difficult, two said the sampling procedure was harder then expected, and six indicated problems gaining assistance while gathering their information. When asked what they learned from gathering data, all 12 students mentioned at least one aspect of the overall process. No overall patterns of race, gender, or mathematical abilities emerged.

Concept-Oriented Instruction and the AP Statistics Examination

    Members of the AP Statistics Test Development Committee designed a course that "adheres to the philosophy and methods of modern data analysis"(College Entrance Examination Board, 1997, p. 9). This process of modern data analysis involved interacting with data and a mathematical model. To address this idea pedagogically, the Committee recommended instruction that includes the use of technology, projects and laboratories, cooperative group problem solving, and writing. Therefore, the fifth research question asked which components of activity-based, concept-oriented instruction effect students’ performance on the AP exam? I asked 12 students questions about the exam and their preparation for it. The components of interest are those recommended by the AP Statistics Test Development Committee. Each component is analyzed individually.

The Projects’ Effect on the Exam

    One component of the concept-oriented class that may have impacted students’ performance on the AP Exam was their projects. During the interviews, I asked each student directly if the project helped prepare them for the exam. Two students said "no," two students indicated an indirect effect, and nine students believed the project had a positive effect on preparation of the AP exam.
    Ingrid and Ethan both expressed the idea of an indirect effect to describe the impact of the project on preparation for the AP exam. "I think it helped because we were just sitting there thinking because of this, because of this, this will work," Ingrid said. " I think it helped because we were constantly thinking about statistics and having to apply it. But not directly" (IS, Interview). Ethan also indicated an indirect effect. "I don’t know that it directly helped," he said, "but I think it helped you recall all the stuff we had done earlier in the year" (EB, Interview).
    Four of twelve students conveyed that the project helped prepare them on specific parts of the AP exam. One student, Beverly, replied that the project had no effect, but then explained to the contrary.

KR: Did all of this help prepare you for the exam?
BH: Well, um, not entirely a whole lot. There was one problem that was about fish in a tank and you had to do a random sample and figure out how to place them in the tanks so some of them were toward the warmer end and some toward the cooler end in a random way. That question was a snap because of random sampling. If there’s anything we do understand it was that because did it. We did the numbering, we did the randomization process. For math, it was a very hands-on process. Also, anything that had to do with confidence intervals and t-tests because we did six hundred of them in the project. (BH, Interview)

The other three students replied that the project helped them answer the open-ended questions, specifically the Free Response question on experimental design and the Investigative Task. Hanson’s opinion represents the overall idea that all three of these students stated. I asked if the project prepared him for the exam.

Yes and no. It helped prepare for the designing an experiment question. It helped for the investigative task a little bit because you know a little bit more what you are looking for in a study than just seeing it because you’ve actually done and tried to look for something. (HF, Interview)

Three students believed the project directly helped prepare them for the AP exam. Their responses indicated the project gave them the opportunity to put many statistical ideas in a larger, more comprehensive framework. "We got to deal with stuff, taking the raw data, knowing how it was put together, because we were manipulating it ourselves," Josh said. "Then putting it into a report and interpreting it, it just really helped to understand" (JK, Interview). Gretchen’s response reiterated Josh’s idea. "Yes, the project did help" she said, "because it tied everything we’d been doing all year together" (GR, Interview). Lisa agreed with Josh and Gretchen. "I think it just really required that we use everything that we had done in class," she responded. "It just kind of reviewed everything actually" (LB, Interview).
    Overall, out of 12 students asked if the project helped them prepare for the AP exam, two said no, two stated an indirect effect, three suggested on overall effect, and the other six pinpointed the exact effect the project had. No gender or race patterns emerged. However, the two students who stated the project had no effect, Carter and Fran, were the top mathematical students. The three students who perceived an overall effect were three of the weakest four mathematically. In a study of the impact of calculus reform, Ganter (1997) found similar results. She concluded that "students who do not perform well on traditional tests" (p. 13) were better suited to group work and long-term projects (Ganter, 1997).

The Classroom Setting

    Another component of Lee’s class that affected student learning was the layout of the classroom. The room contained six round tables that each sat three students. The instructor did not assign seats. During the first few weeks, some students moved around until they found a permanent seat. One student in sixth period moved again in the winter because of a discipline problem. When Lee gave problems to work in class, the students frequently asked their neighbors for help. There was no group grading or group accountability. If they chose to, the setup of the classroom enabled them to work together at any time during regular class periods.
    To determine the effect of group interactions on their learning, I asked each of the 12 students if they liked sitting at the round tables. One student expressed a neutral opinion and one student stated a negative attitude. Ten students shared positive reactions, but 3 of these 10 also stated negative opinions.
    Each student that expressed a favorable opinion suggested that the round tables created a relaxed, more comfortable environment. "It wasn’t like a regular class where you have to sit in an uncomfortable desk," Gretchen stated. "I guess it was just more relaxed and different. It wasn’t the same old thing" (GR, Interview). "In a way, it was like a sense of community," Fran said (FE, Interview). "It was more comfortable," Lisa said (LB, Interview). Beverly elaborated the same idea.

It was a little more informal atmosphere. To me there is something in any classroom that is a bit stifling about desks, even in classrooms that have straight tables. The structure of the desks is very rigid, very enclosing. Whereas at tables, it’s more open, you have more room to spread out, to be more comfortable, and to be able to situate yourself to be ready to learn a little more. (BH, Interview)

Since most of the students responded to the physical aspects of the classroom environment, I prompted them further to pursue if the round tables had an effect on their learning and preparation for the AP exam. Hanson was the only student who did not think they engaged in group interaction and that the tables had no effect on his learning. "We didn’t do a whole lot of group work together with the people at your table during class," he said. "Like he didn’t give us sets of problems to do or anything" (HF, Interview). Ethan expressed a negative reaction to the group dynamics without seeing any learning benefits. "In sixth period I think the round tables added to the confusion and lack of math concentration," he said. "It seemed to make the whole class more relaxed, but I don’t think they were a very good thing" (EB, Interview). Three other students, Adam, Carter and Ingrid, saw advantages and disadvantages to the round tables. All three said "distracting" but also referred to the advantages of being able to discuss ideas with classmates.
    The remaining seven students believed the round tables contributed to their learning experiences. In general, they expressed the idea that having a peer to consult was beneficial. "I learn better than when everyone is in their own desk just staring at the teacher at the board," Adam said. David also directly referred to Lee’s effect.

It was a different atmosphere than just sitting at desks facing the teacher and listening to him. When we were at our round tables, we could talk to each other and listen to him at the same time. It felt like we were socializing while we were learning at the same time. It helped a lot. (DW, Interview)

Gretchen expressed a similar idea. "When he was doing something on the board and you got lost," she said, "it was easier to look over at Rachel and see what I was doing wrong" (GR, Interview). Fran’s response referred to Lee’s technique of not directly answering their questions and linked it to the advantages of being at tables.

He just has a different teaching technique. He is a lot more lax and the tables just allow a sense of creativity and at the same time we are working toward the same goal. Also I think he wants us to talk amongst ourselves to figure things out and maybe talk to our peers to try and work things out. A lot of times he just says, ‘solve this problem.’ Doing it by ourselves, sometimes I didn’t know what to do. But when you have other people, you can ask what they are doing. (FE, Interview)

A unique dialogue occurred with Sam. He expressed a deeper understanding of learning dynamics and connected his idea to the round tables’ environment.

KR: Did you like sitting at the round tables?
SD: Yes, because you can talk to the people instead of just sitting there. Most of the time we were talking about statistics but you could talk about something else. I guess that’s not good but when we were talking about statistics then it was useful because somebody could help you or you could help somebody else. I guess that’s more useful than sitting at desks because sometimes, Lee might not explain it the same way.
KR: If you were the one explaining, did you benefit from that?
SR: I guess a little. I guess I understood it a little bit after I explained it. If you don’t explain it to somebody then you just think you already know it and you don’t really think through all of it. But when you explain it to somebody you really have to understand every single point about it before you can really explain it to somebody. (SD, Interview)

When asked if they like sitting at round tables, overall only one student expressed a completely disfavorable opinion. One student failed to notice the group interactions that the other ten students saw as advantageous. While three students mentioned the classes were loud and other students were distracting, these same three students agreed with the other seven that the round tables provided an environment that was beneficial to their learning. Three of the best five academic students commented on the positive and negative elements of the round tables. All five girls responded that sitting at tables provided a more relaxed, comfortable environment that had a positive impact on learning. Ganter’s (1997) study on calculus reform indicated similar results. She concluded that "reform efforts using cooperative learning and group activities yield especially positive results for women and minorities" (p. 14). No patterns for minorities emerged from my study.

Technology

    In recent years, pedagogy associated with teaching statistics has been impacted by the increased availability of technology in the classroom (Dunham & Dick, 1994; Gordon & Gordon, 1992; Moore, 1990; Shaughnessy, 1992; Watkins et al., 1997). According to the AP Statistics Course Description, "The AP Stat course depends heavily on the availability of technology suitable for the interactive investigative aspects of data analysis" (College Entrance Examination Board, 1997, p. 10). The AP Statistics Test Development Committee recommended the use of a computer when it is available. Since they cannot require students to have computers during the exam, the AP exam questions provide computer output and students are expected to interpret it (College Entrance Examination Board, 1997). The committee also stated that students "will be expected to bring a graphing calculator with statistics capabilities to the examination and to be familiar with its use" (College Entrance Examination Board, 1997, p. 10).
    The research site had computer laboratories available during class time. The instructor scheduled several classes in the "mathematics bay" during fall quarter. Students were instructed on the use of one statistical analysis package. They were expected to navigate through descriptive statistics, graphing techniques, and some forms of interactive analysis. Also, on project days, they had access to the computer lab when needed. Most of the groups conducted their project analysis using the computer software. However, the instructor relied primarily on the TI-83 graphing calculator even though the AP Statistics Test Development Committee recommends use of a computer. His reasons for this included availability for all students at any time, his students had prior experience with this calculator, inconvenience of the computer laboratory, and a strong belief that the TI-83 provides complete statistical analysis for data sets up to 999 observations. In both classes I observed, the TI-83 was utilized every day. Since the students had extensive experience with this calculator, the instructor easily led them through complicated transformations, residual analysis using the spreadsheet, and simulations.

The Computer
To determine the effect of technology on students’ learning, I questioned the students about the calculator and the computer lab. When asked if the computer lab helped them learn, three students indicated they did not use it much and the other nine said it did not help. Most of those who said the computer lab was not helpful suggested it did the same things as the calculator. "I don’t think there’s anything the computer can do that the calculator can’t do except for printing out stuff," Ethan said (EB, Interview). "It was just a blown up version of the calculator," Sam said. "It didn’t really help that much because the calculator did everything the computer does" (SD, Interview). David expressed the same idea. "We weren’t doing anything that was different than what we can do on our calculator," he said (DW, Interview). Adam explained his preference of the calculator. "The calculator was much more useful and user friendly," Adam said (AD, Interview). Hanson indicated the computer software was difficult. "Minitab was very confusing to me," he said (HF, Interview). Gretchen explained a unique impression. "All I felt like I was doing when we did Minitab was just like clicking here and clicking there [sic]," she said.
    Even though most students believed the computer was not helpful in class, 8 of the 12 mentioned that the computer did help them on their projects. "It was a great help with our project in coming up with visual aids and graphs and stuff like that," Hanson said (HF, Interview). Josh expressed a similar opinion. "On the project, Minitab was very useful," he said. "We used it to run the statistics and print good graphs" (JK, Interview). Lisa also referred to the graphing advantages of the computer. "I guess it did help for the project," she said, "because we had to have it all displayed" (LB, Interview). Sam referred to the ability to analyze large data sets. "It did help on the project when we had lots of data and we had to analyze it," he said (SD, Interview). Overall, students saw the advantages of using the computer to generate reports and graphical displays for project presentations. Otherwise, these students preferred the TI-83 to the computer. There were no gender, race, or mathematical patterns. Of those who said the computer lab helped on the project, there were three girls and five boys, five Caucasian, two Indian, one African American, and an even spread over mathematical abilities.

The Graphing Calculator
To examine the effect of the TI-83 calculator on students’ learning, I asked questions regarding the calculator. When asked if the calculator helped them learn statistics, three said no and nine said yes. Of the three that said no, one indicated an overall confusion about the use of the calculator’s statistical functions. "But it is just as hard for me to learn how to use the calculator than to learn the real math," Gretchen said (GR, Interview). This student indicated difficulty understanding how to use Minitab also. Carter suggested that they would have learned the concepts with or without the calculator.

I think the concepts were stressed pretty well, but I think all the things we needed to understand about the way curves looked and the theory behind using certain values to determine things about sets of data, I think that was all pretty well taught. But I don’t think that that’s because of or in spite of the calculator. I think that was something, I think when they showed us the charts, when we would see a normal distribution curve and would talk about the 99-95-68 thing and how the distributions had different percentages, I think that is something you could see visually and that would have been pretty well understood by almost everyone even if we had had to do a couple more calculations. And it might have, doing all the calculations might have taken away the from the focus being that broad and general of learning about things like that, but I still think most of the people in our class would have got a firm grasp on both. (CF, Interview)

Fran’s opinion was unique. She initially thought the calculator helped, but then discovered another effective studying technique.

At first I thought it was better to use the calculator. But when I was studying for the exam, I went back and reread the book. It made a lot more sense to me. When I was doing the problems, I found myself doing things out manually more. So if I had to go back and do it, I might do it more by hand first. I probably would not use my calculator as much. (FE, Interview).

Although these three students indicated the calculator did not help them understand concepts, their reasons were different. One student was inhibited by technology in general, one student needed the textbook readings, and the third student believed the concepts were taught well enough that the calculator neither helped nor hurt. The two that were not inhibited by technology, Fran and Carter, were the two strongest mathematical students.
    Students who thought the calculators were beneficial emphasized being able to concentrate on the concepts. Seven students indicated the calculator directly enabled them to focus on the ideas. I asked Lisa if the calculator helped her learn.

Yes, because I would have been really overwhelmed with all the equations and stuff. Just plugging them in and doing them [sic]. There was nothing in them that was really complicated, they were all just adding and subtracting, but to have the calculator do it for you, you understood more of the big picture instead of getting caught up in having to compute the standard deviation. (LB, Interview)

Ethan stated a similar idea. "But usually if you had a sense of the large concepts," he said, "then the calculator cleared up the fog of the details for you." Josh also expressed this thought. "It did because you could focus more on what you were learning rather than how to calculate it because the calculator just did it for you," he said. "And that’s, I didn’t feel [sic] that was what was important, was to know how to calculate it but rather what it meant" (JK, Interview).
    Three students stated that even though the calculator helped them learn the concepts, they knew how to work the formulas also. "I wouldn’t have been able to do the course without the calculator," David stated. "But now that I look back on it, I know how to do everything by hand. But as far as knocking out the real boring tedious work, the calculator helped a lot" (DW, Interview). Hanson responded a similar impression. "But on the basic problems you wouldn’t have to do all the drudge work and spend the whole time doing equations by hand. So, it was good," he said. "And he asked questions that made sure we knew the formulas anyway" (HF, Interview).
    Overall, nine students thought the calculator enabled them to focus on conceptual development. The two lowest mathematically able students referred directly to the benefit of being able to concentrate on the concepts. Of the three who indicated the calculator was not beneficial, one student was challenged with the computer also. The other two were the top mathematical students and they integrated the calculator with other learning techniques. No gender or race patterns emerged.

Impact of the Constructivist Teacher

    Research suggests teaching practices directly effect student learning. Cobb states that "teachers’ actions do influence the problems that students attempt to solve and thus the knowledge they construct" (Cobb, 1988, p. 92). The AP Statistics Test Development Committee made recommendations regarding preparation for the AP exam. In one publication, they encouraged teachers to familiarize their students with the form and notation of these given formulas (see Appendix K) by making them accessible to their students at the appropriate time during the course (College Entrance Examination Board, 1997). In the free response section of the AP stat exam, students were asked to answer open-ended questions and to complete an investigative task. The open-ended questions required students to relate different content areas as they formulated a complete solution to a statistics or probability problem. (College Entrance Examination Board, 1997).
    To probe the effect of Lee’s teaching on their preparation for the AP exam, I asked the students several different questions. From this data, two techniques he utilized emerged. First, throughout the year he challenged students with difficult questions in class and on the tests. In addition, he completed the AP syllabus material approximately a month before the AP exam. They spent this time reviewing in general and emphasizing specific areas where the students needed help. The instructor provided numerous multiple-choice questions he obtained from various resources. Since most of his test questions had been in the Free Response format, Lee recognized the students needed practice with Multiple Choice questions. When asked, "What did the instructor do that helped you prepare for the AP exam?" eight students mentioned the year end review, five referred to his tests, and four gave miscellaneous responses. In addition to these comments, I was able to relate specific pedagogical techniques that had a direct effect on how they answered one free response question and the investigative task.

Year End Review
    Several students indicated the review time before the AP exam was helpful. "I like how he got the course done early," Josh said. "That left a good month in the middle between the exam time and when we actually finished the syllabus to go over everything that we had problems with" (JK, Interview). Sam expressed a similar opinion. "After we covered everything we went through," he said, "we did those multiple choice questions and those practice tests. That probably helped the most" (SD, Interview). Gretchen pointed out that Lee provided them with multiple choice questions like those on the AP exam. "He gave us a lot of tests to do, what he thought the exam would be like," she said, "and they were pretty dead on" (GR, Interview). Two students referred to a reiterative practice process. Beverly explained.

The last weeks ever since the projects were over, he just ran us through a gamut of multiple choice questions. In the end, we all thought the multiple choice section on the AP exam was much easier than we ever expected. I think that has in part to do with that preparation he provided us with and in part to do with the fact that he just, it was practice, practice, practice toward the end. (BH, Interview)

David said the same pedagogy was helpful. "Probably hurrying through the year so we could practice and giving us as many more tests, giving us tests over and over again," he said (DW, Interview). Overall, 8 of the 12 students interviewed referred to the review period as helpful in preparation for the AP exam.

Routine Testing
   Students also indicated Lee’s tests were helpful in preparing for the AP exam. Adam said, "His exams were much harder than the AP exam. That way we knew we needed to study all year." Ingrid reiterated this impression. "The tests were really thorough," she said. "I think that really helped because when we got to the AP exam, it was a breeze. It was so easy compared to his tests" (IS, Interview). Ethan felt Lee’s test questions were similar to those on the AP exam. "I think a lot of his test questions were really helpful," he said. "If you just went over his tests, that helped" (EB, Interview). Sam stated that "the multiple choice wasn’t as difficult as the questions as we had done in class" (SD, Interview).

Miscellaneous
   While many students referred directly to the review process or Lee’s thorough tests, some students mentioned other aspects of Lee’s teaching. Gretchen mentioned the project as helpful. "It really tied everything together and that helped on the exam," she said (GR, Interview). Two students referred to his pedagogy and implied he taught on a more difficult, challenging level the entire academic year. "I think he taught on a level that was harder than the exam," Lisa said. "We went into more detail, especially with probability" (LB, Interview). Carter was able to articulate the pedagogy that directly helped him.

He basically just told us, not what would be on there, but how to do whatever they asked us to do. There was nothing on the test that we hadn’t seen in class. He prepared us by giving us examples to work within almost every area. On the exams he gave us things like the investigative task so we were prepared for that. I think we were prepared for basically anything they could have thrown at us. I think he taught us everything we needed to know. He assigned the reading to reinforce it if no one understood but I think basically just teaching us and giving us worksheets and things like that. And that was mostly during the end of the year. He taught for most of the first half of the year and then the last half was worksheets and spot checking what he had taught earlier. I think that was really helpful. (CF, Interview)

Hanson contributed his success partly to Lee’s ability to follow the recommended AP Course Description. He also recognized the importance of the AP syllabus.

He followed a lot of the outline that the AP people gave him and he gave us a copy of that and made sure that we were very familiar with everything on the outline. We knew all the terminology and used that as a guide to what he was going to do but he also added a lot of his own stuff. We had the basics and a little more. (HF, Interview)

In general, the students stated the instructor covered all the material thoroughly, gave challenging tests, created test problems similar to the free response questions, followed the AP syllabus, and spent a lot of time reviewing. No patterns of race, gender or mathematical abilities emerged from this question.

Free Response Problem 3
    Attempting to discover the direct effect of Lee’s pedagogy on students’ performance on the AP exam, I probed the students regarding one Free Response (FR) question and the Investigative Task (IT). In FR question 3, students were given a scenario and asked to calculate both a probability and a conditional probability (Appendix C). I chose this question to probe in student interviews because the instructor consistently used tree diagrams to illustrate a variety of probability concepts (Field Notes). Each student interviewed was asked how they solved this problem. All 12 students responded that they used a tree diagram. When asked why they used this specific technique, each student answered that is what they did in class.

The Investigative Task
   Students’ responses about the IT question (see Appendix C) did not yield the same consistency. Several students offered a solution to this problem based on a similar problem the instructor had given them on a test. Ethan was the first to reveal this prior knowledge.

KR: What did you do to solve this investigative task?
EB: When I did my own model, I ran a piecewise. So I ran the regression and a linear test.
KR: How did you know to try a piecewise function?
EB: One on the questions on our second quarter exam was a piecewise fit. So I just ran from that. (EB, Interview)

Carter also recalled the similar problem from an in class test. "On the exams he gave us things like the investigative task," he said, "so were prepared for that" (CF, Interview). I asked Gretchen how she solved the investigative task. "If Lee hadn’t given us that exam question where we transformed something into two separate lines, I don’t know what I would have done," she said (GR, Interview).
    Eight other students indicated they used the TI-83 to fit an appropriate model. They all remembered using r and r2 to analyze their models. Two students transformed the data and three students used residual analysis to decide which model best fit the data. Two students input the data into their TI-83 and regenerated all the information that had been supplied as generic computer output. Ingrid explained how she regenerated the information and then used what the exam provided.

IS: I went ahead and put all this in my calculator, put these lists in. It’s nice looking at this stuff (indicating the computer output) but I really like to be able to manipulate it myself. I just redid everything.
KR: So even though you had all this generic computer output, you went and regenerated it and looked at it again?
IS: Yes, and I played with it. Some of the equations were slightly different from the equations on here so I checked to make sure I had the data exactly right to see where the difference was there. I couldn’t find any reason for a difference, so I went with their equations. I didn’t use mine. (IS, Interview)

Hanson said he was more familiar with the information provided by calculator and had "a lot of extra time" (HF, Interview) to regenerate the same output that was provided. Overall, students used different techniques to solve the investigative task based on what they learned from Lee’s teaching. Several recalled a similar question from the second quarter exam and others relied on familiarity with the calculator to find an appropriate model. No patterns of race, gender or mathematical abilities emerged.

Writing

    Student writing ability should be an important component of the assessment process (College Entrance Examination Board, 1997). Research questions included investigation of this pedagogical technique. However, the instructor did not require extensive open-ended writing on tests. Therefore, writing was not probed during the interviews.

Students’ Learning in a Concept-Oriented Course

    The AP Statistics Test Development Committee described concept-oriented instruction as an approach that actively involves students in the learning process (College Entrance Examination Board, 1997). This approach included "the use of technology, projects and laboratories, cooperative group problem solving and writing" (College Entrance Examination Board, 1997, p. 10). They suggested that concept-oriented instruction allows students to construct their own knowledge. They recommended that teachers structure their course so students can focus on developing their understanding of statistical concepts, rather than manipulations and formulas. A learning environment with this structure provided opportunities for students to experience and make connections with other academic subjects and realistic life examples. In addition, this type of learning was more important to statistical thinking and resembles what practicing statisticians experience (Watkins et al., 1997).
    Concept-oriented instruction can be regarded as one pedagogical application of the constructivist learning theory. As previously stated in chapter 2, constructivism refers to a variety of theories that attempted to explain how students construct, reconstruct, connect and apply mathematical knowledge. According to Ernest (1996), constructivist learning theories were currently dominating mathematics education research.
    Borasi (1996) described vignettes that illustrated classroom activities and experiences that addressed the NCTM Standards and the constructivist learning theory. Borasi’s vignettes emphasized components of constructivist-based pedagogy that were also present in this study. These classroom activities and experiences included an inquiry approach to teaching, a requirement that students justify their methods, presentation of problems in realistic contexts, and instructional support provided within flexible guidelines. Her findings revealed that these strategies empower students to pose and solve problems, learn to see and appreciate mathematics in their lives, and make interdisciplinary connections.
    Consistent with Borasi’s (1996) results, students at this research site were actively involved in a variety of ways. Gathering data in class was common. Informally structured as groups, they engaged in group problem solving. Use of graphing calculators, including simulations, was the norm rather than the exception. Discussions focused on overall understanding of the question and application of the calculator’s numerical results. Assigned problems involved interactive, investigative data analysis.
    Several components recommended by the AP Statistics Test Development Committee (College Entrance Examination Board, 1997) and Borasi (1996) emerged from this study that were relevant to students’ learning and affected their preparation for the AP exam. Students at this research site indicated that use of technology, a project, and group problem solving contributed to the learning process. In addition, students indicated that specific elements affected their learning. These techniques included a relaxed, discussion focused class, an emphasis on explaining their answers, and the freedom to explore their ideas with peers and Lee.

Summary

    Student learning in a constructivist, concept-oriented course was the focus for this study. Students in a private, urban high school were the data source for this constructivist study on the initial offering of AP Statistics. Data gathered represented year long observations through field notes, student artifacts, and in depth interviews. Interview data were collected as quickly as possible following the AP Statistics exam in May 1997. Other data represented a variety of resources and components I felt were relevant to a holistic description of the AP Statistics course. Two questions emerged from the study as parts of the holistic description of the course. Data that addressed the two emergent questions came from the College Board, ETS, the Chief Faculty Consultant, the instructor, and the AP Statistics listserv.
    Students participated in a group project. Each group formulated a research question, gathered data, conducted statistical analysis, and presented their results orally and in a written report. Some groups gathered data using survey instruments they created. Other groups accessed information from the Internet, the library, or government agencies. All projects followed the format for and were submitted to the American Statistical Association’s annual national project and poster competition. Two of Lee’s groups won Honorable Mention, but only one of the four groups selected to research won this honor. Four groups were selected to observe in detail. One student from each of these four groups was selected to interview about the project. Other data were gathered to investigate the effects of concept-oriented instruction. Twelve students were interviewed about the components recommended by the AP Statistics Test Development Committee. Interview questions pertained to gathering data, in class group work, the use of technology, and the AP Statistics class in general. Interviews were audio taped and transcribed.
    Qualitative analysis software, QSR NUD*IST (Qualitative Solutions & Research Pty Ltd, 1995), assisted in data management. Data were imported into QSR NUD*IST (1995) and coded. The AP Statistics Test Development Committee’s recommended pedagogy provided the framework for coding data (see Appendix D). Data were analyzed by comparison and reevaluation. I looked for patterns that described learning processes and exceptions to these patterns. When necessary, clarification was obtained from students and Lee in follow up interviews. Field notes, interviews, and student artifacts revealed similar ideas that satisfied triangulation. To satisfy member checking (Creswell, 1994; Guba & Lincoln, 1994), Lee reviewed all writings and analysis for accuracy.
    Question 1 addressed the initial offering of the AP Statistics course and described the course in a holistic manner. Included in the description were factors relevant to the course’s creation and implementation, the AP Statistics listserv, resources for teachers, and overall impressions from the AP Statistics Test Development Committee’s members.
    The second research question led to investigation of the instructor’s role in effectively teaching a concept-oriented AP Statistics course. Several significant characteristics emerged. Lee stressed concepts, allowed students to discover and construct their own understandings, introduced topics using activities, utilized technology extensively, encouraged group problem solving, and assigned a comprehensive project. For professional development, he attended conferences, contributed and read the listserv, conducted workshops, and collaborated with other teachers.
    With respect to Question 3, no patterns emerged relating the effect of a group project on learning specific statistical concepts. All projects were unique and learning outcomes varied. However, students from all groups expressed that conducting the analysis did help solidify their overall statistical understandings. Students expressed a variety of advantages related to their active involvement with data. No patterns of race, gender or mathematical ability emerged.
    Question 4 addressed the effect of data gathering. Students who gathered data using survey instruments and sampling techniques reported several types of complications. Five indicated that writing survey questions were difficult, two said the sampling procedure was harder then expected, and six indicated problems gaining assistance from those surveyed. When asked what they learned from gathering data, all 12 students mentioned at least one aspect of the overall process. No patterns of race, gender, or mathematical abilities emerged.
    Question 5 asked what components of concept-oriented instruction affected students’ preparation for the AP exam. Each component was addressed and analyzed individually.
    When students were asked if the project helped prepare them for the AP exam, two said no, two stated an indirect effect, three suggested on overall effect, and the other six pinpointed the exact effect of the project. The two students, Carter and Fran, who stated the project had no effect were the top mathematical student from each class. The three students who perceived an overall effect were three of the weakest four mathematically.
    When asked if they like sitting at round tables, only one student expressed a completely unfavorable opinion. One student failed to notice the group interactions that the other ten students saw as advantageous. While three students mentioned that class was loud and other students were distracting, these same three students agreed with the other seven that the round tables provided an environment that was beneficial to their learning. All five girls responded that sitting at tables provided a more relaxed, comfortable environment that had a positive impact on learning. Three of the best five academic students commented on the positive and negative elements of the round tables. No race patterns emerged.
    Students saw the advantages of using the computer to generate reports and graphical displays for project presentations. Except for use on the project, these students preferred the TI-83 to the computer. Nine students thought the calculator enabled them to focus on conceptual development. The two lowest mathematically able students referred directly to the benefit of being able to concentrate on the concepts. Of the three who indicated the calculator was not beneficial, the computer also challenged one of these students. The other two were the top mathematical students and they integrated the calculator with other learning techniques. No gender or race patterns emerged.
    Examining the teacher’s impact, students stated the instructor covered all the material thoroughly, gave challenging tests, created test problems similar to the free response questions, followed the AP syllabus, and spent a lot of time reviewing. On two specific free response questions, all 12 indicated utilizing a pedagogical technique the instructor used repeatedly in class. On the other hand, students used different techniques to solve the investigative task based on what they learned from Lee’s teaching. Several recalled a similar question from the second quarter exam while others relied on familiarity with the calculator to find an appropriate model. No patterns of race, gender or mathematical abilities emerged.
    Overall, students suggested the group project provided an opportunity to view statistics holistically. Data gathering was more difficult than any had anticipated. The project helped the weaker students prepare for the AP exam more than the mathematically strong students. Almost all enjoyed sitting at round tables and many indicated the tables provided an environment that made it easy to ask questions. They preferred the graphing calculator to the computer labs. Lee’s teaching techniques encouraged students to develop sound statistical concepts in a relaxed atmosphere. The next chapter presents discussion, conclusions, and recommendations.