 |
||
|
|
National Mathematics Advisory Panel Report Review (Released to the web March 17, 2008) (Click here to jump to selected excerpts from the advisory panel reports below)
Ordinarily, federal advisory panels, whether "blue ribbon" or not, tend to come up with
bland, meaningless, or unrealistic recommendations after a superficial review of
subject material. However, the National Mathematics Advisory Panel,
which just released its findings to the public, is an exception. The
reports recently released to the public indicate a thorough, in-depth analysis of
existing data and authorship of new data under the panel. The panel members
don't even hesitate to bite the hand that feeds them, as shown by their
criticism of current testing quality under No Child Left Behind. In general the
panel found serious deficiencies in US programs in areas of fractions, decimals,
accuracy in textbooks, and the encouragement of algebra mastery as a gateway to higher math topics.
But, the panel's observations and recommendations related directly to math
programs and practices of the type in Readington should be of keen interest to
our school leadership and will give new ammunition to parents who have been
concerned for years.
The reports from the panel are bold, no-nonsense and brutally honest. They would also represent a sea-change in public school math instruction if districts across the country implement the recommendations. After a review of the hundreds of pages which make up the various reports, readingtonparents.org has excerpted quotes which are particularly applicable to Readington schools and the debates about math which have occurred here over the past few years. The panel observations and recommendations dovetail strikingly well with the concerns voiced by readingtonparents.org and like minded stakeholders. The Everyday Math program used in Readington and also widely across the nation is not mentioned by name, but the panel leaves little doubt about their opinion of this and similar programs. They even mention the "mile wide-inch deep" phrase so often used by critics to describe Everyday Math. The idea of introducing a topic relatively briefly and then returning to it later as a way of reinforcing concepts--known by educators as "spiraling"--is thoroughly trashed by the panel. This spiraling technique is a core principle of Everyday Math. Similarly, the panel harshly criticizes the bloated size of textbooks for Everyday Math and other US national math programs. Other criticisms of national programs and their textbooks include too much emphasis on pictures, a lack of coherence, significant errors and confusing statements or problems, circular definitions and omissions, and extra "enrichment" activities which would be better left to the social studies or science curriculum. The panel even compares current textbooks to historical examples in the US, finding better examples in prior decades. These and other criticisms relate directly to the four new nationally marketed math programs being considered by Readington right now. One math program which is mentioned specifically by name by the panel is Singapore Math. The Singapore program (the US version closely matches the official Singapore program) has much shorter and much more tightly focused textbooks and spiraling is shunned in favor of proficiency of a subject before moving on. Students in Singapore have consistently outperformed their international counterparts in math proficiency. Readingtonparents.org has repeatedly called for an examination of Singapore Math as a replacement for Everyday Math in Readington schools, though the current administration has rejected its use before any serious consideration. The panel created a table they call Major Topics of School Algebra, for use as criteria in examining desirable areas of study which should be included in a good 7th-10th grade program, and Singapore Math faired extremely well. Only two areas were not covered, and the panel noted that these might actually have just been taught outside the grades studied. In lower grades the use of number line concepts are recommended, which Singapore Math implements with its heavy use of bar diagrams. The panel noted that US programs often do not ensure that students can actually conceptualize, estimate or describe what they are examining in mathematical terms, areas in which Singapore Math is superior. Other areas touched on by the panel reports which are pertinent to Readington include learning processes, instructional practices and assessment. Readington has implemented a homogeneous grouping scheme in math which tends to pigeonhole students into distinct tracks. The panel found that heterogeneous approaches such as Team Assisted Individualization and peer tutoring are better choices. Heterogeneous approaches have long advocated by readingtonparents.org. Pull-out programs were not found to be helpful to the overall student population, and a specific, negative connection was made between student math anxiety and lack of performance with a failure to take higher level math courses. Students who are not selected for higher level classes think of themselves as not good in math, creating a self-fulfilling prophesy. All of that is a strike against the current grouping approach in Readington. Interestingly, the panel had very negative things to say about the quality of national and state standardized testing, even though the panel was sponsored by the federal agency pushing the hardest for reliance on such assessments. Calling attention to poorly worded questions, contextual and non-math related factors skewing test results, flawed test development processes, a lack of credentials in the people creating these tests, and flawed or marginal test questions, the panel cast doubt on the wisdom of using these tests as a means of sorting and grouping students, never mind assessing their knowledge. In the area of technology, the panel found that popular new approaches like the use of Smartboards and the internet for math instruction did not have enough empirical study to do a meta-analysis. In spite of the high popularity of items like this in school districts like Readington, there just isn't any data to support or reject their use and must therefore be seen as experimental. One area of technology where the panel did find good data to suggest a recommendation is in computer programming. Specifically mentioned as successful by the panel was the LOGO programming language for elementary students. It works best when integrated into the curriculum, the panel found. Readingtonparents.org profiled and recommended this mature programming language for use by young students back in a 2006 article. Readington has no integrated programming curriculum of any sort. The excerpts shown below from the National Advisory Panel's reports were selected both for their representation of the overall tone of the panel's recommendations and their relevance to local Readington concerns. The full report is worth reading too. Selected excerpts from task group/subcommitee reports
Task Group On Conceptual Knowledge And Skills "International studies show that high-achieving nations teach for mastery in a few topics, in comparison with the U.S. mile-wide-inch-deep curriculum. A coherent progression, with an emphasis on mastery of key topics, should become the norm in elementary and middle school mathematics curricula. There should be a de-emphasis on a spiral approach in mathematics that continually revisits topics year after year without closure." "...Singapore’s fourth- and eighth-graders have consistently outperformed all other countries’ students on the mathematics portion of the TIMSS (Gonzales et al., 2004). Singapore’s compulsory secondary curriculum begins in Grade 7 and extends through the 10th year of schooling." "...To show how the 27 Major Topics of School Algebra compare with current practices, they were matched against algebra topics listed in 1) U.S. state standards for Algebra I and Algebra II courses, 2) current algebra textbooks, 3) Singapore’s 2007 algebra standards for Grades 7 through 10, 4) NAEP’s assessment objectives for its 2005 Grade 12 test, and 5) the American Diploma Project’s benchmarks for a high school exit test, its core Algebra II end-of-course test and its optional modules for this test. " ...In Singapore’s secondary mathematics curriculum, only two topics do not appear to be covered, and they are the fundamental theorem of algebra, and combinatorics and finite probability (Table 3), although it is possible that these topics are covered after Grade 10." "...A striking and significant difference lies in the number of topics and page length of all current Algebra I textbooks, each of which has close to 1,000 pages and attempts to address far more topics than the more focused and much slimmer texts of twenty years ago. For example, the now out-of-print Dolciani algebra textbooks, which were among the most widely used textbooks of their day, had far fewer pages and focused on far fewer topics. It is not clear how many of the topics in current Algebra I and II textbooks students can realistically study in the course of one year, and, more importantly, to what depth they study the major algebra topics." "...Textbook publishers, their authors, and editorial staff do not pay sufficient attention to mathematical accuracy. It should be emphasized that the Task Group is not asking for rigor in a formal mathematical sense. The mathematics should be presented in an age-appropriate fashion, yet be clear and accurate. Circular definitions or the omission of a definition of an important notion being introduced must be avoided, and can be avoided without making the material less accessible....Many of the problems uncovered by the textbook examination will not be apparent to most students, or even to their teachers. However, such problems tend to affect students’ learning in both overt and subtle ways." "...There seem to be two major differences between the curricula in top-performing countries and U.S. curricula: in the number of mathematical concepts or topics presented at each grade level and in the expectations for learning. U.S. curricula typically include many topics at each grade level, with each receiving relatively light development, while top-performing countries present fewer topics at each grade level but in greater depth. In addition, U.S. curricula generally review and extend at successive grade levels many (if not most) topics already presented at earlier grade levels, while the top-performing countries are more prone to expect proficiency in what is taught at each grade level. These critical differences distinguish a spiral curriculum (common in many subjects in U.S. curricula) from one built on proficiency—a curriculum that expects proficiency in the topics that are presented before more complex or difficult topics are introduced." Task Group On Learning Processes "...Learning of complex algorithms is highly dependent on working memory resources and requires repeated use of the algorithm extended over time. Mastery of standard algorithms is dependent on committing these problem-solving steps to long-term procedural memory, at which point the algorithm can be executed automatically with little demand on working memory resources. Algorithms that are mastered are less prone to disruption due to anxiety or in contexts such as high-stakes testing." "...Teachers should not assume that children understand the magnitudes represented by fractions even if the children can perform arithmetic operations with them. Examining children’s ability to perform novel estimation tasks, such as estimating the positions of fractions on number lines, can provide a useful tool for assessing children’s knowledge of fractions. Providing feedback on such number line estimates can improve children’s knowledge of the fractions’ magnitudes." "...Children’s goals and beliefs about learning are related to their mathematics performance. Children who adopt mastery-oriented goals show better long-term academic development in mathematics than do their peers whose main goals are to get good grades or outperform other children. They also are more likely to pursue difficult academic tasks. Students who believe that learning mathematics is strongly related to innate ability show less persistence on complex tasks than peers who believe that effort is more important… Young children’s intrinsic motivation to learn (desire to learn for its own sake) is positively correlated with academic outcomes in mathematics and other domains." "...Anxiety about mathematics performance is related to low mathematics grades, failure to enroll in advanced mathematics courses, and poor scores on standardized tests of mathematics achievement." "...U.S. students do not meet the goal of fast and efficient solving of basic arithmetic combinations or execution of standard algorithms and their competence in these areas is well below that of students in many other developed countries. U.S. students have a poor grasp of most core arithmetical concepts; most U.S. students do not understand the distributive property of multiplication, or know identity elements or the inverse relation between division and multiplication, among other deficits… The fast and efficient solving of arithmetic combinations and execution of procedures requires considerable practice that is distributed across time. The consistent failure of U.S. children to achieve mastery of these topics is a strong indication that most current curricula in the United States does not provide these experiences." Task Group On Instructional Practices "...Research has been conducted on a variety of cooperative learning approaches. One such approach, Team Assisted Individualization (TAI) has been shown to significantly improve students’ computation skills. This instructional approach involves heterogeneous groups of students helping each other, individualized problems based on student performance on a diagnostic test, and rewards based on both group and individual performance. ...The Task Group cautions that only one type of formative assessment has been studied with rigorous experimentation. These are assessments that include random sampling of items that address state standards. These assessments tend to take between 2 and 8 minutes to administer and thus are practical for regular use." "...The regular use of formative assessment particularly for students in the elementary grades is recommended. These assessments need to provide information not only on their content validity but also on their reliability and their criterion-related validity (i.e., correlation of these measures with other measures of mathematics proficiency). For struggling students, frequent (e.g., weekly or biweekly) use of these assessments appears optimal, so that instruction can be adapted based on student progress." "...In TAI, students are grouped in heterogeneous teams of four or five persons. Each student receives a set of mathematics problems tailored to individual performance on a diagnostic test. Students help each other when needed and check each others’ work. Rewards are based on group performance on assignments, quizzes, and tests. Tests at the end of the unit are taken individually…. It can be concluded that the implementation of TAI for students in Grades 3 through 6, in comparison to a form of whole class instruction, benefits computation skills. Note that this finding applies only to the very particular cooperative group strategy of TAI and only to computation, not concepts or problem solving." "...One particular version, Peer-Assisted Learning Strategies (PALS) (http://kc.vanderbilt.edu/pals/), “is a version of classwide peer tutoring. Teachers identify which children require help on specific skills and who the most appropriate children are to help other children learn those skills. Using this information, teachers pair students in the class, so that partners work simultaneously and productively on different activities that address the problems they are experiencing. Pairs are changed regularly, and over a period of time as students work on a variety of skills, all students have the opportunity to be ‘coaches’ and ‘players’." (http://kc.vanderbilt.edu/pals/). The strategy also creates opportunities for a teacher to circulate in the class, observe students, and provide individual remedial lessons. … In summary, it appears that peer tutoring strategies may be promising in teaching young children mathematical operations (which may not be exclusively computation oriented). However, this finding must be treated cautiously because the evidence is only suggestive." "...The review does allow us to make some key conclusions. First, Team Assisted Individualization (TAI), a cooperative learning strategy, has been shown to be effective in teaching computation skills…Working in heterogeneous groups of four or five students, students are encouraged to work together to ensure that all students in the group attain mastery. Why does TAI work? Researchers of TAI have argued that several elements of the technique may enhance learning: students receive immediate feedback from peers (as compared to delayed feedback from teachers during whole class instruction); materials present mathematical skills in a logical, hierarchical sequence; students’ deficient areas are assessed, identified, and targeted with individualized materials; a group reward structure motivates students and encourages teamwork; the intervention blends teacher-directed and student-centered instruction. ...A second cooperative learning strategy, generally known as peer tutoring, also showed signs of promise, with a significant pooled effect size favoring the peer-assisted condition." "...Previous reviews (see summary Table 27) indicate that programming improves students’ performance compared to conventional instruction, with the greatest effects on concepts and applications, especially geometric concepts, and weaker effects on computation. They also have indicated that programming positively affects problem solving, as well as attitudes toward mathematics and instruction in mathematics, more so than other software categories. Certain computer languages, especially the Logo computer language, have stronger positive effects than other computer languages. … As with other types of software, Logo programming can be particularly effective when embedded in a curriculum and then in a context that includes professional development for teachers." "...There were an insufficient number of original empirical studies to conduct an original meta-analysis on the use of the Internet in mathematics instruction." "...Such tools as electronic blackboards and quick-response devices have mostly descriptive studies to support them (e.g., Fadel & Lemke, 2006)…There were an insufficient number of original empirical studies to conduct an original meta-analysis on this topic." "...On the basis of the high quality studies identified in this category by the Task Group, it is reasonable to conclude that there is no significant negative impact of calculators on students’ calculation competence (only one of the studies allowed students to use calculators on the assessment). However, there are several important caveats. These findings are limited to the effect of calculators as used in the 11 studies, including studies up to a year in duration. Also, tests of computational skills did not measure the more basic processes, such as retrieval or decomposition, that students use to solve arithmetic problems, nor did they measure automaticity or procedural execution as might be assessed with timed paper-and-pencil tests…given that the basic computational skills of many Americans are poor, as described in the Learning Processes report, a finding of no effect is not a promising one; more powerful instructional approaches are needed. The synthesis of previous reviews suggests that more recent calculator interventions, especially those putting calculators to “pedagogical use” as an essential element in the teaching and learning of mathematics, have a greater positive effect (the studies in the Task Group’s meta-analysis did not report such comparisons). “Pedagogical use” usually implies extending mathematics learning in certain situations (and perhaps using calculators to check the accuracy of mental or other calculations), rather than using calculators when other methods would be appropriate. The overuse and inappropriate use of calculators, decried by many, may be more harmful than these (relatively short-term) studies indicate. On the other hand, an emphasis on mental arithmetic may ameliorate such problems. There is much researchers still need to study." "...In summary, computer programming can be considered an effective tool, especially for elementary school students, for developing specific mathematics concepts and applications and mathematical problem-solving abilities. Effects may be larger the more computer programming is integrated into the curriculum. Although there was insufficient research on such issues, the Task Group notes that instructional use of programming has fewer “bells and whistles” than other categories of software and demands thoughtful curricula and knowledgeable teachers, all of which may have contributed to the lack of frequency in U.S. classrooms." "..In most cases, specific uses of technology will not facilitate learning optimally unless they are implemented with fidelity. Unfortunately, information is lacking on this critical issue because reviewers and researchers generally have not measured fidelity." "...Enrichment, which attempts to add breadth and depth to the regular curriculum, as well as complexity, also has been studied and has exhibited some positive effects under the same circumstances, limitations, or conditions affecting the interpretability of findings from the literature on acceleration. Yet, many seemingly excellent enrichment programs have not been rigorously evaluated, perhaps because this option for meeting the needs of gifted students has faced less negativity and resistance than is the case for acceleration." "...Homogeneous grouping is an educational approach that meets with much controversy as well. Enrichment tends to dominate in homogeneously grouped classes but it often includes some increased pace of learning…. The Task Group concludes, however, that it is important for school policies to support appropriately challenging work in mathematics for gifted and talented students. Acceleration, combined with enrichment, is certainly a promising, possibly moderately supported (if the entire literature is considered), practice." Task Group On Teachers "...Math specialists can be found working at every level of U.S. public school systems. They hold positions that oversee all or groups of districts within a state, a single district, a single school within a district, classrooms within a school, and even particular students within a classroom. Some math specialists even take on several of these duties all at once (W. Haver, personal communication, April 1, 2007). In middle schools, math specialists are employed most often specifically to teach mathematics, while in elementary schools they may teach their students about multiple subjects, not just mathematics (Fennell, 2006). ...The Pull-Out Model This is a variation of the specialized-teacher model. In this model, math specialists directly instruct individuals or small groups of students within a classroom who have been identified as either failing to meet or exceeding the standards attached to their grade level (V. Mills, personal communication, May 1, 2007). Therefore, this type of math specialists does not address the problem of the deficiency of mathematics instruction in the generic elementary classroom. ...To the extent that the pull-out model is not designed to meet the needs of the generic classroom, this model is not pertinent to the present considerations." Task Group On Assessment ."..Achievement tests are widely used to estimate what students know and can do in specific subject areas. Tests make visible to teachers, parents, and policymakers some of the outcomes of student learning. They also can drive instruction. Due to their important role in education today, especially after enactment of the No Child Left Behind Act, the Panel examined the quality of released items from the mathematics portions of the National Assessment of Educational Progress (NAEP) and six state tests, and reviewed the relevant scientific literature on the appropriate distribution of test content, the setting of performance categories, factors affecting measurement accuracy, and appropriate test design. The NVS panel found many examples of flawed items on NAEP and state assessments that could affect performance of all or some students and trend lines. The Task Group undertook its own examination of released items on state and NAEP tests, looking specifically for non-mathematical sources of difficulty (e.g., particular context portrayed within an item) and found many items on the NAEP and state tests affected by these sources of difficulty, resulting in too many flawed items. The Task Group presents seven types of flawed items illustrating non-mathematical sources of influence that could affect scores. Test developers should be sensitive to the presence of these types of flaws in the test development process...Because flawed and marginal items on NAEP and state assessments could affect performance of students and could affect trend lines, the Task Group probed this issue. " "...How prevalent are poorly worded problems on high-stakes assessments? The Task Group wanted to find out if there was evidence on the frequency of language or wording issues from other analyses of test items on state, NAEP, or commercial mathematics assessments. ...In sum, while the Task Group found many studies on other aspects of mathematics assessments, including item performance and item difficulty, they did not locate any studies that examined how suitable the wording of a test item may be for its mathematical objectives or the effects of wording-related issues in test items on student performance. Therefore, the Task Group proceeded to examine an array of test items from NAEP and state tests to see what kinds of language or wording flaws could be found. ...Many flawed items were found on the state tests in sufficient quantity to raise further concerns about item quality. The examples given above illustrate seven types of flaws that were found. Our findings, when combined with NVS findings on the large percentage of flawed and marginal items, point to possible gaps in test development procedures that need to be addressed. Developers of NAEP and state tests use sophisticated psychometric models and methods to select items and yet, according to NCES, these statistics are unable to detect the type of flaws noted in the NVS study…. Several aspects of the item and test development process may contribute to the large numbers of undetected flawed and marginal items." "...there is a gap in the educational background of psychometricians and item writers. Psychometricians are trained to use highly sophisticated statistical models and data analysis methods for measurement, but are not as familiar with issues of item design with respect to measuring mathematical constructs. Typical item writers and item evaluators often do not have a college degree in the appropriate subject, and typically have little or no training in task and item design." Subcommitee On Instructional Materials "...It might be assumed that textbooks for middle school and high school math would be free of errors. When mathematicians have reviewed already published middle and high school textbooks, however, they have identified a non-trivial number of errors, and a large number of ambiguous and confusing statements and problems." "...U.S. mathematics textbooks are extremely long. Not counting study guides and answers at the end of the books, middle and high school textbooks typically range from 600 to more than 900 pages. The study guides and answers sometimes exceed 1,000 pages. Even elementary school textbooks sometimes exceed 700 pages. The length of math textbooks was much shorter in previous decades and continues to be much shorter in many nations with higher mathematics achievement than the United States. Thus, the great length is not needed for effective instruction." "...Another indicator and source of lack of coherence of some textbooks is the table of contents. Tables of contents should provide students, teachers, and textbook adoption teams with a sense of the organization of the mathematical topics in the book. In some textbooks, however, tables of contents emphasize not the mathematics but rather specific applications (e.g., Ferris Wheels, Penny Jars). Tables of contents that emphasize the mathematical content seem more likely to help students appreciate the coherence inherent in mathematics. ...Other potentially useful ways of decreasing length and increasing coherence are 1) reducing the number of photographs that are not essential to the mathematical content; 2) placing content aimed at providing extra review, enrichment, or motivation in supplements rather than in the main textbook; and 3) excluding applications in which the primary challenge is posed by the social studies or science content."
|
|