CHAPTER 3

METHODOLOGY

    This chapter presents methodology for a qualitative study of students’ learning in the initial offering of an Advanced Placement (AP) Statistics course. Considerations include rationale, the researcher’s role, research site, participants, data sources, data analysis, trustworthiness, limitations, and summary.

Introduction

    The College Board, public schools, and participating colleges have collaborated to provide college level courses to high school mathematics students through the AP Program (College Entrance Examination Board, 1996). In the fall of 1996, the College Board implemented an AP course in Statistics. Consistent with NCTM’s Curriculum and Evaluation Standards for School Mathematics and the current reform movement, the Test Development Committee for AP Statistics recommended active learning, writing, group projects, computer applications and simulations. The purpose of this study was to investigate the statistical understandings that AP high school students constructed in a concept-oriented, activity-based class with respect to the Test Development Committee’s recommended pedagogy. This study explored the use of technology and an extensive group project to examine the processes students employed to construct statistical concepts and how these processes related to the Test Development Committee’s recommended pedagogy. Specific emphasis was placed on the effect of concept-oriented instruction on students’ performance on the AP Statistics exam. Qualitative research methods, based on the constructivist theory of learning, were utilized to probe students’ thinking and conceptual understanding.

Rationale for a Qualitative Study

    The constructivist theory of learning supported this study as the researcher attempted to understand and describe how students constructed statistical ideas. An important element of the constructivist theory was the idea that students connected new information to existing ideas (Piaget, 1964). Ethnographic methods, including longitudinal observation and in-depth interviewing, were utilized to investigate this connection.
    Qualitative methodology provides a design for research that attempts to describe and understand educational phenomena. To better understand and describe students’ learning, researchers observe, investigate, and document the environment in which the learning occurs (Creswell, 1994; Glesne & Peshkin, 1992; Goetz & LeCompte, 1984; Guba & Lincoln, 1989; Lincoln & Guba, 1985; Wilson, 1977). Qualitative methods were utilized to document the educational experiences of students while seeking to discover unique and common patterns (Goetz & LeCompte, 1984). Goetz and LeCompte (1984) claim ethnographic methods provide rich, descriptive data regarding the experiences of the participants. Qualitative assumptions that supported this study included an emphasis on process, the interest in meaning, the researcher as an instrument, descriptive data analysis, and longitudinal fieldwork (Creswell, 1994).

Researcher’s Role

    Bogdan and Biklen (1992) suggested that initial site visits consist of limited and detached involvement. This allowed an adjustment period for the subjects to acclimate to the researcher’s presence. As the researcher, the extent of my participation had not been determined during the initial visits to the school. The extent of participation emerged as the research focus developed (Bogdan & Biklen, 1992). The instructor and I frequently discussed my participation as a tutor, additional facilitator, and assistant in developing assessment materials. As I began to develop an understanding of the classroom dynamics, it became clear that to address these research questions, my participation would be minimal. Glesne and Peshkin (1992) define the researcher’s role of observation with minimal interaction as ‘observer as participant’. Even though the instructor and I frequently discussed the logistics of his teaching and the research, the instructor maintained control of instructional techniques, order of presentation of material, design of tests, and supervision of the major project. The researcher observed his concept-oriented pedagogy and attempted to explain its effect on students’ understandings according the Test Development Committee’s recommended pedagogical techniques.
    Qualitative methodology utilizes the researcher as an instrument. Creswell (1994) suggested that, "data are mediated through this human instrument, rather than through inventories, questionnaires, or machines" (p. 145). While there are advantages to this design, one result is that researchers bring biases to their studies (Glesne & Peshkin, 1992; Goetz & LeCompte, 1984; Marshall & Rossman, 1995; Wolcott, 1995). Therefore, efforts to understand and interpret the research site were affected by my background and ideas. I brought certain biases to this study as an experienced statistics instructor. My experience was in the 2-year college environment and I had ideas about teaching this material that may have influenced the way I collected, viewed, and interpreted the data. I regarded teaching as interactive discussions between the instructor and students. The instructor’s role was to facilitate learning rather than dispense knowledge. I believed data analysis and inference, not probability and combinatorics, should be stressed within the statistics curriculum.

Research Site

    Describing learning processes in a classroom required observation of that specific learning environment (Glesne & Peshkin, 1992; Goetz & LeCompte, 1984; Lincoln & Guba, 1985). Marshall and Rossman (1995) gave the following criteria in choosing the research site: (a) entry is possible; (b) there is a high probability that a rich mix of the processes, people, programs, interactions, and structures of interest are present; (c) the researcher is likely to be able to build trusting relations with the participants in the study; and (d) data quality and credibility of the study are reasonably assured. For this study, a single research site adequately addressed all four criteria.
    The research site for this study was a coeducational private school with approximately 1850 students from pre-kindergarten through the 12th grade. It was several miles south of a large, southern metropolitan area. The 75-acres campus comprised 35 buildings, practice fields, a stadium, and tennis courts.
    The school admitted students whose interests were balanced between academics, athletics, and the arts. There were approximately three applicants for each student accepted. The Dean of Admissions interviewed each applicant and their parents. The school’s policy was to admit students of good character, conduct, and academic achievement regardless of race, ethnic, or economic background. Students came from twenty-two neighboring counties. Approximately 10% of the high school students were awarded scholarships.
    The school had a variety of official policies. One policy required that the students wear uniforms. The students had several patterns and colors to choose from, but the styles were consistent. Only seniors wore khaki skirts and pants. The boys had summer uniforms, golf style shirts, and winter uniforms, coats and ties. The girls could wear pants in the winter. All high school students were required to work on campus 20 hours a year. To satisfy the work requirement, students acted as monitors, assisted coaches, or became peer leaders. They participated in debate tournaments, volunteered for Habitat for Humanity, or became involved in various clubs on campus including the Environmental Awareness Club, the Key Club, and the Student Government Association. The school also required that each high school student participate in programs designed to assist them in dealing with peer pressure, academic stress, and other relevant issues.
    The academic year was divided into three quarters. There were no classes for credit during the summer. If a student does poorly, they could attend 2-week review sessions in the summer. They must repeat the class if they failed. The school offered 17 Advanced Placement classes including Biology, Calculus, Chemistry, Computer Science, Economics, English Language and Composition, European History, French, German, Music Theory, Physics, Spanish, and U.S. History. Students who enrolled in AP classes were required to take the corresponding AP exam. For the 1996-1997 academic year, 128 upper school students enrolled in at least one AP class. While AP classes were common high school choices, all curricula at this research site was college preparatory. Seniors were required to apply to at least three different colleges. Almost all their graduates attend college.

Participants

The Instructor

    The most important criterion to conduct this study was an experienced instructor who would teach the AP Statistics class in the format recommended by the College Board guidelines. These guidelines recommended the use of technology, cooperative group activities, projects and writing. Therefore, the preferred instructor would have experience teaching high school statistics using technology and a variety of classroom activities. Combining these requirements with Marshall and Rossman’s (1995) criteria, I contacted the instructor, Lee, at this private, urban preK-12 school. For 18 years this instructor had taught in private schools, as well as in African public schools through the Peace Corps, and also at a 4-year college. Although his experience has been in a variety of schools and in a variety of mathematics course, he taught at least one quarter of statistics each of the 18 years. He conducted a pilot AP Statistics class in the previous academic year (1995-1996). The College Board, however, was unable to furnish a pilot exam for students completing a pilot course in AP Statistics. This academic year (1996-1997), Lee taught three AP Statistics class.
    Lee’s teaching philosophy was constructivist. He believed that "every student has his own vision of reality" (Interview). According to Confrey (1990),

When one applies constructivism to the issue of teaching, one must reject the assumption that one can simply pass on information to a set of learners and expect that understanding will result. When teaching concepts, as a form of communication, the teacher must form an adequate model of the students’ ways of viewing an idea and s/he then must assist the student in restructuring those views to be more adequate from the students’ and from the teacher’s perspective. (p. 109)

Lee’s pedagogical technique involved presenting students with questions or problems and guiding them toward solutions. He encouraged discussion, questions, and allowed students to pursue misconceptions. The students frequently engaged in activities using computers, calculators, and manipulatives.
    Lee’s classroom had six round tables, each with three chairs. The room was large and had three windows. There was a counter top and sink in the back of the room. He arranged shelves to create a semiprivate corner on the other side of the back wall. He had a desk and additional shelves in this corner. (The students were aware that this was his personal area and they respected his right to privacy.) There was an overhead projector and screen in the front of the room. The overhead was connected to a TI-83 graphing calculator. He also had a laptop on his desk that interfaced with the calculator and with the overhead. The board was a marker board instead of a chalk board. The markers were usually fresh, rarely old and illegible.
    In addition to their regular classroom, the classes occasionally met in the computer lab. The mathematics bay was a small section off a larger computer room. This bay was u-shaped with tables that held nine computers. There were two chairs for each computer. The students had access to the computers during regular school hours. The computers had a variety of mathematics software packages, including one statistical analysis package, a word processor, and a web browser for Internet access.

The Students

    All students enrolled in AP Statistics had completed a series of courses listed in the upper-school Honors curriculum. Ninth graders were recommended for Geometry and Algebra with Transformations and 10th graders enrolled in Precalculus. Students were enrolled in the next honors class in the sequence as recommended by past and current instructors. Though there was some flexibility for recognizing unique abilities and circumstances, AP Statistics was the recommended mathematics course for junior Honors students.
    Forty students in three different classes enrolled in this initial offering of the AP Statistics class at this research site. For logistical reasons, two of the three AP Statistics classes were selected as research groups. Thirty-one students in these two different AP Statistics classes participated in this research study.
    Fourth period consisted of 14 students ages 16-17. Ten students were male and four were female. Ten students were juniors and four were seniors. Six of these students were the first members of their family to attend this institution. Of the other nine students, seven had siblings who have attended and one student was a second generation attendee. Every student in this class had previously taken at least one AP class. Three students had taken only one other AP class, four students had completed two other AP classes, three students had taken three other AP classes, two students had taken six other AP classes and two students had taken seven other AP classes. Both students who had taken six other AP classes were seniors but the two students who had taken seven AP classes were juniors.
    Seventeen students enrolled in sixth period AP Statistics class. These students ranged in age from 15-18 years old. Thirteen students were male and four were female. Eleven students were juniors, five were seniors and one was a sophomore. Six students in this class were first generation students and 11 students had siblings who had attended (or were attending). One student came from a long line of family members who attended this school. His grandfather, father, aunt, both sisters, and three cousins all attended this school. His grandmother taught here and his father had been on the governing board. In this class, Statistics was the first AP class for one student, the second AP class for one student, the third AP class for one student, the fourth AP class for two students, the fifth AP class for nine students, the sixth AP class for two students and the seventh AP class for one student. The student who had no prior AP classes was the only sophomore enrolled in AP Statistics at the research site. Two of the seniors had four other AP classes, one senior had five other AP classes, and the student with seven other AP classes was a senior. The juniors were spread from one other AP class to five other AP classes while seven juniors had taken four other AP classes.

Data Sources

    Many data sources were utilized to address the research questions. I gathered materials from the College Board, the Educational Testing Service (ETS), the AP Statistics listserv, student artifacts, field notes, interviews with Lee, interviews with 12 students, and a reflective journal.

College Board Materials

    The College Board publishes pamphlets, brochures, and books describing the AP Program, each specific AP course, the curriculum necessary to prepare students for the different tests, previous AP Exam questions, and recommendations from the corresponding Test Development Committee. In addition to these materials, I obtained the AP Statistics, 1997, Chief Faculty Consultant (CFC) Report, and the Recommendation from the Advanced Placement Statistics Task Force to the College Board Concerning the Feasibility of Offering an Advanced Placement Course in Statistics, November 1992. After the initial offering of the AP Statistics exam, Lee provided individual test scores for each student. Last, I obtained national, regional, and state results published by the ETS.

The AP Statistics Listserve

    I subscribed to the AP Statistics listserve in May 1996. Messages were posted to the group inconsistently, yet frequently. Topics considered relevant to the research emerged after months of monitoring the list. I printed comments relevant to the research. I also referred to the archive files on numerous occasions.

Student Artifacts

    Lee collected copies of students’ tests and other written assignments. Many of the test questions were open-ended to elicit explanations. Other questions emphasized concepts, rather than calculations. I read students’ papers looking for patterns across the class and exceptions with respect to understandings and misconceptions. In addition, Lee supplied me with copies of all the handouts students received.

Project Artifacts

    A variety of artifacts were collected as students worked on their group projects from January through April. Lee scheduled at least one day a week when students met with their groups to work on their projects. During these class times, students had access to the library, the computer room, telephones, and other resources as needed. Lee monitored their progress and served as a consultant. He gathered written progress reports every other week. Students described their project, what they accomplished, what problems they foresaw, how would they attempt to overcome these obstacles, what unexpected problems had risen, and what had gone well. Four specific groups, two groups from each targeted class, were chosen to be observed in detail. These four groups audio taped their conversations on project days. Tapes were transcribed and used to triangulate data. After class presentations, one member from each group was interviewed in detail about the project. Lee copied their final project reports. I read these reports prior to their interviews looking for ideas or comments to investigate. Interview questions (see Appendix A) probed specific elements of the project as related to the Test Development Committee’s recommended pedagogy.

Field Notes

    I visited the school two to three times weekly from August 1996-May 1997. On each visit, I collected field notes with a laptop computer. Field notes served as the primary source for describing the learning environment longitudinally (Glesne & Peshkin, 1992). Details were logged with respect to the school environment, the classroom environment, Lee’s teaching, and social interactions. I recorded conversations between Lee and the students and among students. Recorded details described their use of technology, content discussions, and nonacademic activities that depicted a typical or atypical day. I logged data by date and specified which class was observed.

Lee’s Interviews

    Since I knew what information was needed, I conducted a structured interview (Lincoln & Guba, 1985) with Lee (see Appendix B). We met as quickly as possible after the academic year ended in June. Although he occasionally strayed from the original question, we discussed all the topics I believed were relevant to the study. As additional themes and issues emerged during data analysis, he responded to subsequent questions. In addition, he read each chapter and provided feedback regarding the accuracy. Analysis consisted of organizing and describing Lee’s philosophy of teaching, how this affects his pedagogy, and the realized effect in the classroom.

Students’ Interviews

    Students were selected for interviewing as salient features emerged. According to Goetz and LeCompte (1984),