Education 351A  Winter 2006
 David Rogosa

Sequoia 224, rag AT stat DOT stanford DOT edu

According to the Registrar
Course: EDUC  351A, Sec 01
Title: Design and Analysis of Longitudinal Research
Min/Max Units:  3- 3
Grading Basis: Satisfactory/No Credit
Days/Times/Classroom:  T 02:15PM-05:05PM EDUC313
For materials from last version of this course (AY '04-'05)

One substantive emphasis will be questions/problems in assessing improvement of students,
schools and subgroups in educational assessment and school accountability

Recommended TEXT
Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence
by Judith D. Singer & John B. Willett New York: Oxford University Press, March, 2003
Text Main Page  (browsing encouraged)
Datasets, computer programs, and output for text
Amazon.com link

Course Contents

I. Time-1, time-2 data. Measurement of change. Confusions, controversies and courses of action.
II. Analysis of Collections of Growth Curves. (mixed models)
III. Analysis of Durations. (survival analysis)
IV. Special Topics and Applications
    a. Reciprocal Causation
    b. Stability: Change and Sameness, consistency over time
    c. Behavioral Observations (on-off processes)
    d. Failures of Causal (structural equation) Models
    e. Topics in Educational Policy and School Accountability
         Change in achievement gaps; Value-added analysis; Student progress in charter schools
Notes on Computing.
Much can be done with standard statistical computing resources but there are a lot of specialized add-ons and standalones.
Most of the legacy materials from this course are in SAS (including Timepath). Transitioning to R is one of the objectives this year (but not a requirement).
For introductory materials on R see the Stat141 site, especially the R-diary and Course Files and Examples page. An additional resource that I recently came across is a presentation An Introduction to R, John Verzani ; Verzani's text is also very good.
Class 1/10
Course overview and introduction.
Time1-time2 regressions example (from Myths addendum)
Class 1/17
Traditional Measurement of Change via Myths About Longitudinal Research (hard copy handout)
Rogosa, D. R. (1995). Myths and methods: "Myths about longitudinal research," plus supplemental questions. In The analysis of change, J. M. Gottman, Ed. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 3-65. 
 Intro via "Myths about longitudinal research"
1. Two observations a longitudinal study make.
2. The difference score is intrinsically unreliable and unfair.
3. You can determine from the correlation matrix for the longitudinal
data whether or not you are measuring the same thing over time.
4. The correlation between change and initial status is
(a) negative
(b) zero
(c) positive
(d) all of the above
5. You can't avoid regression toward the mean.
6. Residual change cures what ails the difference score.
7. Analyses of covariance matrices inform about change.
8. Stability coefficients estimate
(a) the consistency over time of an individual
(b) the consistency over time of an average individual
(c) the consistency over time of individual differences
(d) none of the above
(e) some of the above
9. Casual analyses support causal inferences about reciprocal effects.
Myths data examples and description from Rogosa home page

Additional (on-line) readings:
Rogosa, D. R., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726-748. [not available online?]
Rogosa, D. R., & Willett, J. B. (1983). Demonstrating the reliability of the difference score in the measurement of change.
Journal of Educational Measurement, 20, 335-343.
available from John Willet's pub page
Rogosa, D. R., & Willett, J. B. (1985). Understanding correlates of change by modeling individual differences in growth. Psychometrika, 50, 203-228.
available from John Willet's pub page
A lenghty restatement of the Myths content with an emphasis on time1-time2 issues not present in the text is
Willett, J. B. (1989). Questions And Answers In The Measurement Of Change. In Ernest Z. Rothkopf (Ed.), Review of Research in Education, Volume 15. Washington, D.C.: American Education Research Association, 345-422.
available from John Willet's pub page
Another time1-time2 reading covering old-fashioned ground
Maris, Eric. (1998). Covariance Adjustment Versus Gain Scores--Revisited.
Psychological Methods, 3(3) 309-327. apa link  (from campus IP)

Longitudinal in the news:
Association between exercise and cognitive decline, National Institute on Aging (NIA) Seattle Study.
Exercise Significantly Reduces Risk of Dementia in Senior Citizens     Staying active helps keep the mind sharp      latest research suggests that exercising your brain and your heart may help

exercise: consider a population with true change between time1 and time2 Uniform [99,101] and measurement error Uniform [-1, 1], i.e. the measurement of change is accurate to 1 part in a hundred.
What is the reliability of the difference score? try error Uniform [-2,2], accuracy one part in 50.

Class 1/24
Continue Traditional Measurement of Change via Myths About Longitudinal Research (hard copy handout) and Time-1,Time-2 data analysis
Myths 5,6,7-- correlates of change. Main ref "Understanding correlates of change..."
Data analysis for Time-1, Time-2 data. Myths supplement; Myths data examples and description from Rogosa home page
Class handout: residual change scores
Longitudinal Data structure and description, ALDA Ch.2
Background: reliability coefficients and accuracy of scores (see shoe shopping)
Class 1/31
overview of Myths 7-9; full treatment in special topics
more on Myth 7
David Rogosa. Casual Models Do Not Support Scientific Conclusions: A Comment in Support of Freedman.
Journal of Educational Statistics, Vol. 12, No. 2. (Summer, 1987), pp. 185-195. Jstor link
Data analysis for Time-1, Time-2 data. Myths supplement; Myths data examples and description from Rogosa home page
Begin analysis of collections of growth curves; Ramus examples from Myths supplement;
Growth Curve Data Analysis for multi-wave data
Main reading   ALDA Chap. 3,4,5,7
Rogosa, D. R., and Saner, H. M. (1995). Longitudinal data analysis examples with random coefficient models.
Journal of Educational and Behavioral Statistics, 20, 149-170.   Jstor link
Also: Reply to Discussants   Jstor link
Data sets for Rogosa-Saner, plus North Carolina data

Longitudinal in the news:
1. Is Rob Reiner full of it?     UC study examines preschool benefits: By third grade, no difference shown among students    
See also PACE reports   Preschool for California's Children
2. British kids are getting dumber.   Children are less able than they used to be      The stupid nation
2/6/06  Full preschool report (thanks to Russ Rumberger)
Preschool Participation and the Cognitive and Social Development of Language Minority Students
Class 2/7
Statistical analysis of collections of growth curves
Timepath97 Site (SAS based; documentation site used to use Java navigation so substitute links are a little clumsy but I made them work)
Additional talk materials: An Assortment of Longitudinal Data Analysis Examples and Problems 1/97, biostat
Overview and Implementation for Basic Longitudinal Data Analysis CRESST Sept '97
Additional Resources: ALDA Chap. 3,4,5,7; Rogosa, D. R., and Saner, H. M. (1995); HLM book Ch.6;
NCSU Course (SAS) : ST 732 - Spring 2005 Applied Longitudinal Data Analysis
Judith Singer HLM/PROC Mixed papers: Multilevel Modelling Newsletter ; JEBS1998
Timepath97 output for Ramus (Myths chapter), Rat and North Caroloina (Rogosa-Saner) data examples.
tp2hlm demonstration using ramus data
Class 2/14
Special Topics in Longitudinal Research

1. Stability: Consistency, Change and Sameness (Myth 8)
Rogosa, D. R., & Willett, J. B. (1983). Comparing two indices of tracking. Biometrics, 39, 795-6.      JStor link
Rogosa, D. R., Floden, R. E., & Willett, J. B. (1984). Assessing the stability of teacher behavior. Journal of Educational Psychology, 76, 1000-1027.
above available from John Willet's pub page
Stability section of Individual unit models versus structural equations (link below)

2. Applications of Structural Equation Models (LISREL, path analysis, Myth 7)
Readings: ALDA Chap 8.
David Rogosa. Casual Models Do Not Support Scientific Conclusions: A Comment in Support of Freedman.
Journal of Educational Statistics, Vol. 12, No. 2. (Summer, 1987), pp. 185-195. Jstor link
Rogosa, D. R. (1993). Individual unit models versus structural equations: Growth curve examples.
In Statistical modeling and latent variables, K. Haagen, D. Bartholomew, and M. Diestler, Eds.
Amsterdam: Elsevier North Holland, 259-281.
Rogosa, D. R., & Willett, J. B. (1985). Satisfying a simplex structure is simpler than it should be.
Journal of Educational Statistics, 10, 99-107. Jstor link [or on John Willett's pub page, whichever scan is better]
Follow-up paper:
Two Aspects of the Simplex Model: Goodness of Fit to Linear Growth Curve Structures and the Analysis of Mean Trends. Frantisek Mandys; Conor V. Dolan; Peter C. M. Molenaar. Journal of Educational and Behavioral Statistics, Vol. 19, No. 3. (Autumn, 1994), pp. 201-215.    Jstor link
3. Reciprocal Effects (Myth 9)
Class 2/21
1. Pick up parts 2 and 3 undone from 2/14

2. Begin: Analysis of Durations and Series of events
Behavioral observations
David Rogosa; Ghassan Ghandour. Statistical Models for Behavioral Observations
Journal of Educational Statistics, Vol. 16, No. 3,
Special Issue: Behavioral Observations. (Autumn, 1991), pp. 157-252. Jstor link
Reply to Discussants. Jstor link
From lost and found: SAS program to transform rectangular data to HLM/Proc Mixed row format
Class 2/28
Analysis of Durations and Series of events continued
Applications of Survival Analysis,
ALDA Chap 9-11, plus readings; data analysis examples, Kaplan-Meier
ALDA Chap 13-15, Cox Regression, SAS PHREG
Intro reference: Statistics at Square One: Survival analysis , another intro another good intro (with math)
R-examples: CRAN: Survival by Terry Therneau    UCLA   Cox Regression
Full course sites on survival analysis: Stanford: Stat 262, Spring 2004
University of Sheffield Johns Hopkins Biostatistics 140.641 North Carolina Analysis of Survival Data (ST745), Spring 2005

Class examples:
1. Miller (p.49) leukemia data (Kaplan-Meier);   SAS    Minitab
2. Kalbfleisch and Prentice (1980) rat survival (Cox regression). Also best subsets Cox regression example, myeloma
Class 3/7
Longitudinal in the news: Reciprocal effects?
TV may not cause kids' attention disorders   J Pediatrics, March 2006.
1. Group Growth: Repeated measures analysis of variance
a. Research paper for urea synthesis data: Comparative Analyses of Pretest-Posttest Research Designs, Donna R. Brogan; Michael H. Kutner, The American Statistician, Vol. 34, No. 4. (Nov., 1980), pp. 229-232.   JSTOR link
     urea synthesis, BK data     Stat141 analysis
data,,,,,,,,,,,,,,,,,,, example analyses
b. Bock vocabulary data
data,,,,,,,,,,,,,,,,,,,,example analyses
c. Lord's Paradox, Lord (1967) pre-post group comparisons
Wainer, H. (1991). Adjusting for differential base rates: Lord's Paradox again. Psychological Bulletin, 109, 147-151.
Lord's Paradox and the Assessment of Change During College    Journal of College Student Development, May/Jun 2004 by Pike, Gary R

2. Continue survival analysis examples: Cox regression

3. Begin Longitudinal data analysis and educational policy

   a. Stability of school scores from educational assessments:
   Confusions about Consistency in Improvement   David Rogosa, June 2003 ;    Education Writers Association April 2004

   b.   value-added analysis
Using a Longitudinal Student Tracking System to Improve the Design for Public School Accountability in California Edward H. Haertel, August 2005
Fitting Value-Added Models in R  Harold C. Doran & J.R. Lockwood
Background Reading: Journal of Educational and Behavioral Statistics, Value-added assessment special issue, Spring 2004.

   c. student progress in charter schools
Background paper: Student Progress in California Charter Schools, 1999-2002

  d. Long term student progress following preschool exposure. (prior links)
Class 3/14   Dead Week Meeting
Continue Educational Policy Examples
Course wrap-up