ELPS Faculty
Mary Lee Smith Regents' Professor Professor Educational Leadership & Policy Studies Email: mlsmith@asu.edu Office: Farmer 134C

Current Position

Past Positions

Other affiliations

Center for Research on Evaluation, Standards and Student Testing, University of California-Los Angeles
Southwest Educational Policy Studies (director)
Education Policy Studies Laboratory (fellow)

Education Background

Bachelors (History, 1965), Masters (Counseling and Personnel Services, 1967), and Doctoral degrees (Counseling and Research Methodology, 1972), University of Colorado

SCHOLARLY CONTRIBUTIONS (BY TOPIC AREA)

META-ANALYSIS OF PSYCHOTHERAPY EFFECTS

This program of research resulted in the first application of the methodology termed meta-analysis (invented and developed by Gene V Glass). The investigation involved the statistical integration of 500 controlled studies of the effects of psychotherapy. This work remains one of the most widely-cited research studies in psychology and was named as one of the most influential in the field ( Forty Studies that Changed Psychology by Roger R. Hock (Prentice Hall Publishers). Results of this program of research were instrumental in federal policy that added mental health services to the benefits of government employees (see Congressional testimony, cited below).

This research was replicated more than four times by independent analysts, including the U.S. Office of Technology Assessment, as recently as 1999. The publications stemming from this research follow:

Smith, M. L., Glass, G. V, & Miller, T. I. (1980). Benefits of psychotherapy. Baltimore: Johns Hopkins University Press (peer reviewed).

Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760. Reprinted in C. Kiesler & N. Cummings (Eds.). Psychology and National Health Insurance. Reprinted in T. Cook (Ed.). (1978). Evaluation Studies Review Annual (Vol. 3). Beverly Hills: SAGE. Reprinted in I. Lehmann & W. Mehrens (Eds.). (1979). Educational Research in Focus. New York: Holt (peer reviewed).

Glass, G. V, & Smith, M. L. (1978). Reply to Eysenck. American Psychologist, 33, 517-519. Reprinted in T. Cook (Ed.). (1978). Evaluation Studies Review Annual (Vol. 3). Beverly Hills: SAGE.

Glass, G. V, & Smith, M. L. (1980). Ask not for whom the bell tolls. American Psychologist, 35, 223.

Smith, M. L. (1980). Psychotherapy outcomes. In R. Rosethal (Ed.), Quantitative Assessment of Research Domains. San Francisco: Jossey-Bass.

Smith, M. L. (1982). What research says about the effectiveness of psychotherapy. Journal of Hospital and Community Psychiatry, 33, 457-461 (peer reviewed).

Smith, M. L., & Glass, G. V. (1983). Citation classic. Current Contents: Institute for Scientific Information, 15, 20-21.

Smith, M. L. (1982). What research says about the effectiveness of psychotherapy. Journal of Hospital and Community Psychiatry, 33, 457-461.

Glass, G. V, Smith, M. L., & Miller, T. I. (1983). Placebo effects in psychotherapy outcome research. Behavioral and Brain Sciences, 6, 292-293 (peer reviewed).

Shadish, W., Matt, G., Navarro, A., Siegle, G., Crits-Christoph, P. Hazelrigg, M., Jorm, A., Lyons, L., Nietzel, M., Prout, T., Robinson, L., Smith, M.L., Svartberg, M. & Weiss, B. (1997). Evidence that therapy works in clinically representative conditions. Journal of Consulting and Clinical Psychology (peer reviewed).

Glass, G. V, & Smith, M. L. (1978). Statement to the United States Senate Finance Committee on the Benefits of Psychotherapy. Boulder: Laboratory of Educational Research, (offset, policy brief).

Smith, M. L. (1978). Selected results from the meta‑analysis of psychotherapy outcome research (Report to the National Advisory Panel, CHAMPUS Project, American Psychological Association). Boulder: Laboratory of Educational Research, (offset, policy brief).

METHODOLOGY OF META-ANALYSIS

In addition to the application listed above, contributed to the development and elaboration of the methods of meta-analysis, which has become a staple in medical research as well as social science. Co-authored the first major textbook in the field, as well as the first investigation of what is called the “file drawer problem,” which is the bias in the published literature arising from tendencies to publish only that confirm researchers’ hypotheses. Relevant publications follow:

Glass, G. V, McGaw, B., & Smith, M. L. (1981). Metaanalysis in social research. Beverly Hills: SAGE (peer reviewed).

Smith, M. L. (1982). Research integration. In H. E. Mitzel (Ed.), Encyclopedia of Educational Research. New York: The Free Press.

Smith, M. L. (1980). Publication bias in meta-analysis. Evaluation in Education,4, 22-25 (peer reviewed).

MISCELLANEOUS CONTRIBUTIONS TO META-ANALYSIS

Led or participated in several other projects using meta-analysis, including research on teacher expectancies, therapist gender bias, and the effects of varying class size. Relevant publications follow:

Smith, M. L. (1980). Sex bias in counseling and psychotherapy. Psychological Bulletin, 87, 392-407 (peer reviewed).

Smith, M. L. (1984). Psychotherapy bias: Where politics and social science mix. Contemporary Psychology, 29, 289-291 (peer reviewed).

Smith, M. L., & Glass, G. V. (1980). Class-size and its relationship to attitudes and instruction. American Educational Research Journal, 17, 419-433 (peer reviewed).

Glass, G. V, Cahen, L., Smith, M. L., & Filby, N. (1982). School class size: Research and policy. Beverly Hills: SAGE (peer reviewed).

Smith, M. L, & Glass, G. V. (1980). The effect of class‑size on what happens in classrooms. The Education Digest, 45, 16-19.

Glass, G. V, Cahen, L. S., Smith, M. L., & Filby, N. (1979). Class size and learning: New interpretations of the research literature. Today's Education, 68, 42-44.

Glass, G. V, & Smith, M. L. (1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis, 1, 2-16 (peer reviewed).

Smith, M. L. (1980). Meta-analysis of research in teacher expectation. Evaluation in Education, 4, 53-56.

Anderson, R. D., Kahl, S. R., Glass, G. V, & Smith, M. L. (1983). Science education: A meta-analysis of major questions. Journal of Research in Science Teaching, 20, 379-385 (peer reviewed).

ALTERNATIVE METHODOLOGIES

Critical work on meta-analysis generally and experimental designs specifically led to scholarly contributions to the development of alternative methodologies for the study of social and educational programs. Was asked to write an article on the intellectual foundations of qualitative research for the American Educational Research Journal that opened up this premier journal in the field to studies based on ethnographic, participant observation, and other qualitative approaches. Has frequently written about the integration of quantitative and qualitative research and evaluation. Relevant publications follow:

Smith, M. L. (1981). Naturalistic research. The Personnel and Guidance Journal, 59, 585-589.

Smith, M. L. (1982). Benefits of naturalistic research methods. Journal of Research in Science Teaching, 19, 627-638.

Smith, M. L. (1986). The whole is greater: Combining qualitative and quantitative approaches in evaluation studies. New Directions in Program Evaluation, 37-54 (peer reviewed).

Smith, M. L. (1987). Publishing qualitative research. American Educational Research Journal, 24 , 173-184 (peer reviewed).

Smith, M.L. (1994). Qualitative plus/versus quantitative: The last word. New Directions in Program Evaluation, 61, 37-44 (peer reviewed).

Behrens, J. and Smith, M.L. (1996). Data and data analysis. In D. C. Berliner & R.C. Calfee (Eds.), Handbook of Educational Psychology (945-989). New York: Macmillan (peer reviewed).

Smith, M.L. (1997). Mixing and matching: Methods and models. New Directions in Program Evaluation, 74. 73-86 (peer reviewed).

Smith, M.L. & Noble, A.J. (1992). Toward a Comprehensive Program of Evalua­tion for Integrated General and Special Education. In Goodlad, J. I. & Lovitt, T.(eds), Integrating General and Special Education. New York: Merrill.

Smith, M. L., Gabriel, R., Schott, J., & Padia, W. L. (1975). Evaluation of the effects of Outward Bound. (Bureau of Educational Field Services Report). Boulder: School of Education. Reprinted in G. V Glass (Ed.), Evaulation Studies Review Annual, Volume I. Beverly Hills: SAGE.

Morrow, Sue & Smith, M.L. (2000). Qualitative research for counseling psychology. In Brown, S.D. & Lent, R.W. (Eds.) Handbook of Counseling Psychology. New York: John Wiley and Sons, 199-230 (peer reviewed).

Smith, M. L., & Glass, G. V. (1978). Factors influencing the evaluation of educational programs: Analysis of judgmental policies. Studies in Educational Evalua­tion, ,4, 9-18.

RESEARCH ON SOCIAL AND EDUCATION POLICIES

This research falls into the following categories: Class size (already listed in item III), special education diagnosis and placement, grade retention and school readiness practices, high stakes testing. Work on the problems of special education identification and classification, with Professor Lorrie Shepard of the University of Colorado, was the basis of policy changes in Colorado governance as well as other states. Their research on school readiness and grade retention effected changes in states such as California and became part of the official policies of the National Association of Education of Young Children. Research on the effects of high stakes testing on school processes and outcomes has become classic in the field of education policy and is frequently cited in criticisms of policies such as Arizona Instrument to Measure Standards, Texas Assessment of Academic Standards, and the like. Relevant publications follow:

Special Education Policy

Smith, M. L. (1982). How educators decide who is learning disabled: Challenge to psychology and public policy in the schools. Springfield, IL: C. C. Thomas.

Shepard, L. A., & Smith, M. L. (1983). An evaluation of the identification of children with learning disabilities in Colorado. Learning Disabilities Quarterly, 6, 115-127 (peer reviewed).

Shepard, L. A., Smith, M. L., & Vojir, C. P. (1983). Characteristics of pupils identified as learning disabled. American Educational Research Journal, 20, 309-332 (peer reviewed).

Davis, W. A., & Smith, M. L. (1984). The Colorado Learning Disabilities Study: History and politics of an evaluation. Educational Evaluation and Policy Analysis, 6, 27-37 (peer reviewed).

Smith, M. L. (1984). Distorted indicators in special education. Evaluation and Program Planning, 7, 13-14.

Grade-Level Promotion/Retention Policies

Shepard, L. A., & Smith, M. L. (1989). Flunking Grades. Philadephia, PA:
Falmer Press.

Smith, M. L., & Shepard, L. A. 1988. Kindergarten readiness and retention: A qualitative study of teachers' beliefs and practices. American Educational Research Journal, 25 307-333 (peer reviewed).

Shepard, L. A., & Smith, M. L. (1986). Synthesis of research on school readiness and kindergarten retention. Educational Leadership, 44, 78-86.

Smith, M. L, & Shepard, L. A. (1987). What doesn't work: Explaining policies of retention in early grades. Phi Delta Kappan, 69, 129-134.

Shepard, L. A., & Smith, M. L. (1987). The effects of retention at the end of first grade. Psychology in the Schools, 24, 346-357 (peer reviewed).

Smith, M.L. & Shepard, L.A. (1989). What Doesn't Work: Explaining Policies of Retention in the Early Grades. Reprinted in Early Childhood Editions, Tenth Edition. Guilford, CN: The Dushkin Publishing Company.

Shepard, L.A. & Smith, M.L. (1989) Escalating academic demand: Counterproductive policies.Elementary School Journal.

Shepard, L.A. and Smith, M.L. (1989) Synthesis of research on school readiness and kindergarten retention. Readings on Research from Educational Leader­ship. Alexandria, VA: ASCD.

Smith, M.L. (1990). Can we look honestly: A response to teachers' criticism of retention research. Excellence in Teaching, 7, 5-6.

Smith, M.L. & Shepard, L.A. (1990). What Doesn't Work: Explaining Policies of Retention in the Early Grades. Reprinted in K.S. Goodman, L.B. Bird, & Y.M. Goodman (Eds.), The Whole Language Catalog. Hightstown, NJ: American School Publishers.

Shepard, L.A. and Smith (1990). Repeating grades in school: Current Practice and Research Evidence. Policy Briefs. Stanford, CA: Center for Policy Research in Education.

Lyons, T. and Smith, M.L. (1990). Effects of extra-year programs prior to first grade. Early Child Development and Care, 61, 11-18 (peer reviewed).

Shepard, L.A., Smith, M.L. & Marion, S. (1997). Failed evidence on grade retention. Psychology in the Schools (peer reviewed).

High-Stakes Assessment Policies

Glass, G. V, & Smith, M. L. (1978). The technology and politics of standards. Educational Technology, 18, 12‑18 (peer reviewed).

Ellwein, M. C., Glass, G. V, & Smith, M. L. (1988). Standards of competence: Propositions on the nature of testing reforms. Educational Researcher, 17, 1-5 (peer reviewed).

Ellwein, M.C., Glass, G.V, & Smith, M.L. (1988). Predilections, Opinions, and prejudices. Educational Researcher, 17, 21-22.

Smith, M.L. (1991). The meanings of test preparation. American Educational Research Journal, 28, 521-542 (peer reviewed).

Smith, M.L. & Rottenberg, C. (1991). Unintended consequences of external testing in elementary schools. Educational Measurement: Issues and Practice, 10, 7-11 (peer reviewed).

Smith, M.L. (1991). Put to the test. The effects of external testing on teachers. Educational Researcher, 20, 8-11 (peer reviewed).

Smith, M.L. (1992). Put to the test. The effects of external testing on teachers. Reprinted in Annual Editions, Education 92/92, (F. Schultz, ed.), 218-221.

Noble, A.J. and Smith, M.L. (1994). Old and new beliefs about measurement-driven reform. Educational Policy, 8, 111-136 (peer reviewed).

Noble, A.J. & Smith, M.L. (1997). Times for reform: The experiences of two states. In Gandara, P. (Ed.), The Dimension of Time and the Challenge of School Reform. SUNY Press.

Smith, M.L. & Fey, Patricia (2000). Validity and accountability in high-stakes testing. Journal of Teacher Education, 51 (5), 334-344 (peer reviewed).

Smith, M.L., Edelsky, C.E., Draper, K., Rottenberg, C. & Cherland, M. (1989). The role of external testing in elementary schools. Los Angeles, CA: UCLA Center for the Study of Evaluation (monograph).

Smith, M.L. (1993). The role of high-stakes testing in school reform. Washing­ton, D.C.: National Education Association (monograph).

Smith, M.L. (1996). Reforming schools by reforming assessment: Consequences of the Arizona Student Assessment Program. Los Angeles, CA: UCLA Center for the Study of Evaluation (monograph).

Noble, A.J. & Smith, M.L. (1994). Measurement-Driven Reform: The More Things Change, the More They Stay the Same. Technical Report 373. Center for Research on Educational Standards and Student Testing. Los Angeles: University of California.

Noble, A.J. & Smith, M.L. (1994). Measurement-Driven Reform: Research on Policy, Practice, Repercussion. Technical Report 381. Center for Research on Educational Standards and Student Testing. Los Angeles: University of California.

Smith, M.L., Noble, A.J., Cabay, M. Heinecke, W., Junker, S., & Saffron, Y. (1994). What Happens When the Test Mandate Changes? Results of a Multiple Case Study. Technical Report 380. Center for Research on Educational Standards and Student Testing. Los Angeles: University of California.

Smith, M.L., Noble, A.J., Haag, S., Oliveres-Seck, M. & Taylor, K. (1994). Consequences of Measurement-Driven Reform: Survey of Arizona School Practitioners. Los Angeles, CA: Center for Research on Educational Standards and Student Testing (monograph).

SCHOLARSHIP ON THE INTERSECTION OF POLITICS AND POLICY

In recent years, her scholarship has concentrated on the deleterious effects on education and social policy of what Anne Schneider termed degenerative politics, what Deborah Stone referred to as the failure of the rationality project, and what Murray Edelman categorized as political spectacle. Relevant publications follow:

Smith, M.L., Heinecke, W., & Noble, A.J. (1997). Assessment Policy and Poltical Spectacle. Los Angeles, CA: UCLA Center for the Study of Evaluation (monograph).

Smith, M.L., Heinecke, W. & Noble, A.J. (1999). Assessment policy and political spectacle. Teachers College Record, 101 (2), 199-230 (peer reviewed).

Smith, M.L., Heinecke, W. & Noble, A.J. (2000) State Policy Becomes Political Spectacle: A Drama in Four Acts. Teachers College Record Online, ( www.tcrecord.org/Content.asp?) (peer reviewed).

Miller-Kahn, L. & Smith, M.L. (2001). School Choice and Political Spectacle. Education Policy Analysis Archives, http: //epaa.asu.edu/epaa/v9n50.html (peer reviewed).

Haag, Susan & Smith, M.L. (2002). The Possibility of Reform: Micropolitics in Higher Education. Education Policy Analysis Archives, http://epaa.asu.edu/epaa/v10n21.html (peer reviewed).

Smith, M.L.; Miller-Kahn, Linda Jarvis, Patricia; Heinecke, Walter; & Noble, Audrey (Forthcoming in 2003) Political Spectacle and the Fate of American Schools New York: Routledge Press (peer reviewed).

OTHER SCHOLARLY PUBLICATIONS

Smith, M. L. (1973). Notes on "Perspectives on counselor bias: Implications for counselor education." The Counseling Psychologist, 4, 93 (peer reviewed).

Smith, M. L. (1974). Influence of sex and ethnic group on counselor judgments. Journal of Counseling Psychology, 21, 516-521 (peer reviewed).

Smith, M. L. (1979). Counselor "discrimination" based on client sex: Reply to Donahue and Costar. Journal of Counseling Psychology, 26, 180-182.

Smith, M. L., & Glass, G. V. (1987). Research and evaluation in education and the social sciences. Englewood Cliffs, NJ: Prentice Hall.

Morrow, S. & Smith, M.L. (1995). Constructions of survival and coping by women who have survived childhood sexual abuse. Journal of Counseling Psychology, 42, (1), 24-33 (peer reviewed).

PENDING RESEARCH

Smith, M.L. & Jarvis, Patricia. The Intersection of Policy Research and Political Spectacle. Under review by Education Policy Analysis Archives.

Jarvis, Patricia & Smith, M.L. Speaking Power to Truth: Case Histories of Education Policy in a Time of Degenerate Politics. In preparation for Education Policy.

Smith, M.L. & Shepard, L.A. The Necessary Validation: Combining qualitative and experimental approaches in education research. In preparation for the Handbook of Alternative Methodologies, Camilli, Greg & Greene, Judith

Smith, M.L. The mistaken equating of science with experimental design: Validity and ideology. In preparation for the Educational Researcher.

Camilli, Greg, Smith, M.L. and others. Reading Meta-analysis: An attempt to replicate the meta-analysis of the National Reading Panel.

PRESENTATIONS AT PROFESSIONAL MEETINGS

Approximately 30 presentations in a 25-year period, at the annual meetings of the American Educational Research Association, the American Evaluation Association, Center for Research in Evaluation, Standards, and Student Assessment, American Psychological Association, and Society of Psychotherapy Research. Approximately 15 instances of acting as critic or discussant at symposia. Invited presentations at Michigan State University, Ohio State University, University of California, Los Angeles (CRESST), Stanford University (Policy Analysis of California Education), National Education Association, Northeast Regional Educational Laboratory, California Education Association.

INSTRUCTION

At the University of Colorado, developed curriculum in alternative methodologies. Taught courses in qualitative, survey research, evaluation theory and methodology, and introductory courses in research methods. At Arizona State University, led joint committee to develop a curriculum in qualitative methodology, provided leadership in the development and implementation of a new doctoral program in policy studies. Taught courses such as the following: professional seminar in policy studies, theoretical issues in policy studies, evaluation theory, introduction to qualitative research, methods and practices of qualitative research, and writing in qualitative research. These courses receive excellent evaluation ratings and involve students in extensive and intellectually challenging coursework.

Acted as mentor in the graduate education of countless students over a twenty-five year period. Chaired the dissertation committees of many students including those who went on to scholarly careers at institutions such as the University of Virginia, University of California-Davis, University of Utah, University of Delaware, Brigham Young University, and the University of Colorado at Denver. Also acted as research mentor for students now on the faculty of University of San Diego, University of California- Los Angeles, and University of New Mexico. Other former students work at policy centers across the country.

Collaborated with many of these students in publications subsequent to their dissertations. Involved many students in research grants and consequent publications (see section on publications: Ellwein, Davis, Morrow, Noble, Heinecke, Haag, Fey, and, Jarvis.

Named Outstanding Mentor by the Graduate Student Association of the Arizona State University.

SERVICE

In addition to the typical service to the University and professional associations, served in a variety of policy research and advocacy roles, including writing policy briefs, publishing op-ed pieces for newspapers and magazines, doing litigation support research for cases of education inequality (including the successful Flores case in Arizona) and cases involving alleged improprieties in state assessment. Spent several years of service writing materials for the National Education Association as it developed its policies on high-stakes assessment. Provided research consultation to the Virginia Commonwealth and the National Association of the Education of Young Children. Prepared briefs for Designs for Change, a progressive Chicago think tank.

Edited the leading journal in education, the American Education Research Journal, served on editorial boards and evaluated manuscripts for all the major journal publications and many publishing companies. Served in several capacities for the American Education Research Association, including the research training committee, the Special Interest Group on Qualitative Research (chair), and the Research Review Award Committee (chair).

For Arizona State University, chaired and served on several faculty and administrator search committees, personnel and promotion and tenure committees, admissions and program committees, Affirmative Action Committees, Senator at Large in the Faculty Senate, Graduate Council, ASU Strategic Planning Committee, Regents Professor screening committee. Served as Interim Director, Division of Educational Leadership and Policy Studies.