Statistics 209/Education 260   Spring 2006
Understanding Statistical Models and their Social Science Applications
David Rogosa   rag AT stat DOT stanford DOT edu

Lecture: Wed 3:15-5, Sequoia 200
Additional hour, to be scheduled (see below and seminars page)

Office Hours:
David Rogosa, Sequoia 224.  Thurs 3:15 -  , Friday 3:15 -   (if possible e-mail me beforehand to double-check any conflicts)
TA's,  Karen Kapur,    Wednesdays at 1pm in Sequoia Hall, room 204
          Baiyu Zhou,   3:00pm-4:00pm Tuesday in Sequoia Hall, room 235.

R Computer Lab: Thursday May 18 4PM Sequoia room 211    Linear Mixed Models in R, prepared by Karen Kapur (based on Linear Mixed Models tutorial by John Fox). pdf of lab text
R Computer Lab: Thursday June 1 4PM Sequoia room 211     Matched Sets in R, prepared by Karen Kapur  pdf of lab text

5/6  Take Home Assessment 1 available ; graded TH1 to be returned in-class 5/24     solution notes TH1

6/2  Take Home Assessment 2 available    solution notes TH2

6/13. Finals Week Exam, Tuesday 6/13 12:15-3:15 Seq 200.     exam pdf

Course Overview
For students who have had intermediate-level instruction in statistical methods including multiple regression, logistic regression, log-linear models.
At the very least, the content of the course should provide some consolidation of previous instruction in statistical methods. The goal is also to instill some introspection and critical analysis for the uses of statistical methods common in social science and medical applications.
The focus of the course is on understanding what useful information statistical modeling can provide in experimental and especially non-experimental social science settings. Presentation will emphasize the many misconceptions and misunderstandings in social science applications, such as casual (nee causal) modeling.

Statistical computing presentation will be in, and students are encouraged to use, R, Mathematica, and Matlab. For introductory materials on R see the Stat141 site, especially the R-diary and Course Files and Examples page. The CRAN Task View: Statistics for the Social Sciences provides an overview of relevant R packages. An additional resource that is efficient if you are experienced with another statistical package is a presentation An Introduction to R, John Verzani  For categorical data, especially if you've had a course using Agresti, the lengthy guide by Laura Thompson has more than you want to know.

Course schedule will be a 2-hour weekly meeting for lecture/presentation plus one additional hour for special topics and discussion; one use of this hour will be for relevant on-campus lectures, such as some of the spring quarter MAPSS seminars.see seminars page

Homework and exams. Weekly homework assignments following class content will be posted, with solutions posted the next class cycle. Homeworks are not graded.
Assessment. Two take home problem sets will be scheduled:
TH1 posted 5/5/06 and due in class 5/10/06 covering content weeks 1-4.
TH2 posted 6/2/06 and due in class 6/7/06 covering content weeks 5-8.
In class final, scheduled by registrar, exam week June 9-14. (Tuesday June 13, 12:15)

Textbooks.  The core of the course is working through David Freedman's new text, along with auxiliary texts and materials. One intent of this course is for students to read some statistical literature and actual research reports to augment the texts (on that theme Freedman's text actually includes reprints of four published empirical research papers).
Main text.
Statistical Models: Theory and Practice David Freedman (2005). Publishers webpage
Auxiliary texts.
Regression Analysis : A Constructive Critique  Richard A Berk (2003). Table of contents
     Jan de Leeuw, Preface to Berk's "Regression Analysis: A Constructive Critique"
Data analysis and regression: A second course in statistics. Mosteller, F. and Tukey, J. W. (1977) (the green book)
  Class texts on reserve at Math/CS library

Preliminary Readings: Candidate Case Studies in Cause and Effect

       1. Is TV bad or is it bad parenting? Attention Deficit Disorder and TV
2004 version : Pediatrics. 2004;113:708-713. Christakis DA, Zimmerman FJ, DiGiuseppe DL, McCarty CA. Early television exposure and subsequent attentional problems in children.   Publication   summary    press release    news report      audio NPR interview    interview transcript and publication

2006 reversal? Pediatrics. March 2006. Stevens T and Mulsow M. There is no meaningful relationship between television exposure and symptoms of attention-deficit hyperactivity disorder. Pediatrics. 2006; 117(3):665-672.  
News Reports: TV may not cause kids' attention disorders   Researchers say TV is not to blame for ADHD   TV may not cause kids' attention disorders: study.  Good general commentary in Slate Feb '06

Auxiliary notes This research example raises an important theme of this course-- similarities (often indistinguishability) between social science and medical research. Is the TV and ADHD child development or medical research? (point being the division is often unclear or unuseful)
further aside: ADHD medication: Prescribing of hyperactivity drugs is out of control FDA panel
or try the patch
As with most important issues, definitive wisdom is provided by South Park via Cartman: here, episode 404 (4/19/2000), episode summary   script

       2. Rob Reiner wants me to pay for his Proposition 82. Do you agree?
 Russ Rumberger, UCSB. Preschool Participation and the Cognitive and Social Development of Language Minority Students   
News reports:    UC study examines preschool benefits: By third grade, no difference shown among students     Meathead's mistake: assuming we're dumb
Other research reports and background. PACE reports   Preschool for California's Children    Preschool reform measure won't close learning gap for poor
News updates Editorial: Beyond Reiner  Preschool measure would increase state's fiscal peril  
And definitive insight on Reiner from South Park, episode 713 (12/3/03) summary  script

       3. What can be learned from the Women's Health Initiative?
From the New York Times (reg required)    Women's Health Studies Leave Questions in Place of Certainty       Big Study Finds No Clear Benefit of Calcium Pills

       4. Drug Use and Depression
Ecstasy causes depression in pigs      Depressed pigs shed light on ecstasy
Child anxiety link to ecstasy use      British Medical Journal publication Anja C Huizink, Robert F Ferdinand, Jan van der Ende, and Frank C Verhulst Symptoms of anxiety and depression in childhood and use of MDMA: prospective, population based study BMJ, Feb 2006;

       5. Carbonated soft drinks and Cancer
Yale press release      News report: Soda-Cancer Link Revealed as Myth      
Journal of the National Cancer Institute   pubmed access   abstract  Carbonated Soft Drink Consumption and Risk of Esophageal Adenocarcinoma Mayne et al. J Natl Cancer Inst.2006; 98: 72-75

Class Meeting Outline

No course plan survives the first class meeting

Week 1. 4/5
Course Introduction and Some properties of multiple regression

Trio of news items.
1. Cell phones now bad? latest reversal on brain tumors.
"heavy users of mobile phones had a 240 percent increased risk of a malignant tumor on the side of the head the phone is used".
Long Mobile Phone Use Raises Brain Tumor Risk     Boston Herald      Slashdot
actual paper  Pooled analysis of two case–control studies on use of cellular and cordless telephones and the risk for malignant brain tumours diagnosed in 1997–2003
  aside: demonstrated cell phone cause and effect: impact trauma   more
2. Alcohol no longer good?
UCSF points out flaw in studies tying alcohol to heart health    Health Benefits of Moderate Drinking May Be Statistical Haze     Alcohol: No Heart Benefit After All?
Kaye Fillmore et al. "Moderate alcohol use and reduced mortality risk: systematic error in prospective studies." Addiction Research and Theory. Advanced online publication March 30, 2006.
3. Prayer, more harm than good?
From Harvard  Study calls prayer for sick people ineffective Rate of problems after surgery unchanged by act   Perhaps Neither Prayer Nor Alcohol Can Save Us
Study of the Therapeutic Effects of Intercessory Prayer (STEP) in cardiac bypass patients: A multicenter randomized trial of uncertainty and certainty of receiving intercessory prayer American Heart Journal, Volume 151, Issue 4, Pages 934-942 (April 2006). [compliance note]
Prior Duke   flip   flop

Lecture topics
Quick Tour of course and course materials
Main Topic: Meaning of regression coefficients: simple and multiple regression (including logistic)
Technical facts and foibles:
a. adjusted variables and regression coefficients--values of coefficients depend crucially on what else is used in the regression fit
     conditioning vs controlling
b. effects of errors in measurement on regression coefficients

Lecture materials
1. handout # 1. Coleman data: adjusted-variables multiple regression   data file, 20 schools
2. handout # 2. Hard-copy of: Table 1, Sweedish cell-phone study; Tables 4,5 of UNC Sexual Media Diet study

Text readings/resources
Freedman, Chap 4 (multiple regression); Chap 2 (straight-line regression); Chap 6 (logistic/probit regression)
Mosteller-Tukey, Chap 13 (Woes of regression coefficients)
Berk, Chap. 6,7 (Using and Interpreting Multiple Regression) Berk online: Chap 6  Chap7
Errors of Measurement in Statistics, W. G. Cochran , Technometrics, Vol. 10, No. 4. (Nov., 1968), pp. 637-666. JStor URL
esp sections 8,9,11
Some Effects of Errors of Measurement on Multiple Correlation, W. G. Cochran Journal of the American Statistical Association Vol. 65, No. 329 (Mar., 1970), pp. 22-34 JStor URL
esp sec 8 discussion.

Exercises, week 1.
Solutions, week 1.
Week 2. 4/12
Experiments vs observational studies and Neyman-Rubin-Holland formulation

In the news (overflow)
1. Sex in Media Drives Earlier Teen Passion    Mass Media May Prompt Kids to Try Sex: Study
actual study   Sexy Media Matter: Exposure to Sexual Content in Music, Movies, Television, and Magazines Predicts Black and White Adolescents’ Sexual Behavior  UNC Teen Media Center (NICHD funded)
2. Smell of fear helps in cognition   Scent of fear impacts cognitive performance
3. Sweedish cellphone study (4/5 #1) continued: The FDA Strikes Back.  Use of Wireless Communication Devices and the Risk of Brain Cancer
4. Secondhand smoke linked to diabetes   actual Alabama study  Active and passive smoking and development of glucose intolerance among young adults in a prospective cohort: CARDIA study, British Medical Journal

Lecture topics
A. First pass: experiments vs observational studies
       Surveys of results from experimental and observational studies;  handout
B. Spurious Correlation: some historical notes
C. Introduction to Neyman-Rubin-Holland formulation for causal effects.
       presentation of NRH formulation for comparative studies based on Appendix of Holland (1988)
       Illustration using encouragement design representation in Holland (1988).   by request  copies of selected overheads.

Text readings/resources
Freedman, Chap 1, 6.4
Berk, Chap 5, 10.5  pdf of chap5
David A. Freedman pubs:  From Association to Causation: Some Remarks on the History of Statistics;  
                                  Statistical Models and Shoe Leather, Sociological Methodology, Vol. 21. (1991), pp. 291-313. JStor link

Spurious correlation?
Correlation and Causation: A Comment, Stephen Stigler Perspectives in Biology and Medicine, volume 48, number 1 supplement (winter 2005)
Correlations Genuine and Spurious in Pearson and Yule, John Aldrich Statistical Science, Vol. 10, No. 4. (Nov., 1995), pp. 364-376.  Jstor link
Spurious Correlation: A Causal Interpretation. Herbert A. Simon Journal of the American Statistical Association, Vol. 49, No. 267. (Sep., 1954), pp. 467-479. Jstor link

Causal Inference, Path Analysis, and Recursive Structural Equations Models Paul W. Holland Sociological Methodology, Vol. 18. (1988), pp. 449-484.
Abstract Rubin's model for causal inference in experiments and observational studies is enlarged to analyze the problem of "causes causing causes" and is compared to
path analysis and recursive structural equations models. A special quasi-experimental design, the encouragement design, is used to give concreteness to the discussion by
focusing on the simplest problem that involves both direct and indirect causation. It is shown that Rubin's model extends easily to this situation and specifies conditions
under which the parameters of path analysis and recursive structural equations models have causal interpretations.

Additional relevant items:
Experiments vs Observational studies:
Bringing Evidence-Driven Progress To Education:
main report November 2002           US DOE press release       December 2003 confab, "what works"
 Classic paper on Medical experimentation. Statistics and Ethics in Surgery and Anesthesia. John P. Gilbert; Bucknam McPeek; Frederick Mosteller Science, New Series, Vol. 198, No. 4318. (Nov. 18, 1977), pp. 684-689.     JTSOR link

Holland-Rubin models for comparative experiments (causal inference), Related technical reading
Statistics and Causal Inference, Paul W. Holland pp. 945-960 JASA 1986, another JSTOR link
Commentaries Donald Rubin, David Cox
Rubin, D. B., 1974, Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, Journal of Educational Psychology, 66, 688-701.
     Education and social science:
Winship's repository Counterfactual Causal Analysis in Sociology
Useful intro lecture notes from Jonathan Wand, Political Science

More Freedman
On Specifying Graphical Models for Causation, and the Identification Problem     
Freedman, D. A.Statistical Models for Causation Freedman, D. A. Political Science seminar Feb 2004
Freedman is Right as Far as He Goes, but There is More, and It's Worse. Statisticians Could Help, William M. Mason, Sociological Methodology, Vol. 21. (1991), pp. 337-351. JStor link

Exercises, week 2.
Solutions, week 2.
Week 3. 4/19
Path analysis and causal modeling  multiple regression with pictures

In the news
  trio of relatively simple treatment-control comparative studies (with controlled assignment)
1. Carbonated soft drinks actually are bad: obesity not cancer. Livescience      Pediatrics publication  Effects of Decreasing Sugar-Sweetened Beverage Consumption on Body Weight in Adolescents: A Randomized, Controlled Pilot Study Pediatrics 2006; 117: 673-680.      Documentary proof, Super Size Me
2. Greed is good? Scientists Prove It: Nobody Likes a Freeloader The competitive advantage of sanctioning institutions. Science. 2006 Apr 7;312(5770):108-11.
3. Chocolate Milk May Improve Recovery After Exercise. medscape      International journal of sport nutrition and exercise metabolism.

Also more from the Women's Health Initiative (local focus).: No Link Between Estrogen-only Therapy, Breast Cancer In Postmenopausal Women   Estrogen Again

Lecture topics
1. Path analysis introduction and examples (incl Blau-Duncan from Freedman chap 5).   class handout
2. Does path analysis identify causal effects? Demonstrations of failure for Holland's encouragement design, Rogosa longitudinal example.   class handout
3. Structural equation models: introduction and examples.   class handout

Text readings/resources
1. Freedman, Chap 5. (and background from Chaps 2,3,4)
  more on response schedules (text sec 5.4) in Statistical Models for Causation: A critical review
2. Does this stuff ever work?
Paul Holland: Encouragement design results (1988 soc meth)
David Rogosa. Casual Models Do Not Support Scientific Conclusions: A Comment in Support of Freedman.
Journal of Educational Statistics, Vol. 12, No. 2. (Summer, 1987), pp. 185-195. Jstor link
Theme Song   Ballad of the casual modeler    Lyrics   http://www.stanford.edu/class/ed260/ballad.mp3
Technical details on Rogosa longitudinal examples:
     Rogosa, D. R. (1993). Individual unit models versus structural equations: Growth curve examples.
     In Statistical modeling and latent variables, K. Haagen, D. Bartholomew, and M. Diestler, Eds. Amsterdam: Elsevier North Holland, 259-281.
     Rogosa, D. R., & Willett, J. B. (1985). Satisfying a simplex structure is simpler than it should be.
     Journal of Educational Statistics, 10, 99-107. Jstor link
     original publication on the longitudinal path analysis:   Some Models for Analysing Longitudinal Data on Educational Attainment. Harvey Goldstein
      Journal of the Royal Statistical Society. Series A (General), Vol. 142, No. 4. (1979), pp. 407-442.  Jstor link

Path analysis intros
Path Analysis: Sociological Examples. Otis Dudley Duncan The American Journal of Sociology, Vol. 72, No. 1. (Jul., 1966), pp. 1-16. Jstor link
D.A. Freedman, Comments on Standardizing Path Diagrams: What Are the Parameters?
A recent reconsideration by a wise psychologist: The Path Analysis Controversy: A new statistical approach to strong appraisal of verisimilitude Meehl, Paul E; Waller, Niels G Psychological Methods. Vol 7(3), Sep 2002, pp. 283-300.  available from SU PsychInfo database

Structural equation modeling is a major industry in social and behavioral science with many texts (such as Principles and Practice of Structural Equation Modeling 2nd Edition Rex B. Kline; here's a long list), specialized courses (U. C Irvine MGMT 290   NC state PA 765), dedicated journals (Structural Equation Modeling: A Multidisciplinary Journal), and specialized computer programs (e.g., LISREL, EQS, AMOS).
Maximum likelihood factor analysis: A General Method for Analysis of Covariance Structures, K. G. Joreskog, Biometrika, Vol. 57, No. 2. (Aug., 1970), pp. 239-251.
Structural equation modeling from Scientific Software International
home of * Structural Equation Modeling (LISREL) Student editions, documentation, examples, etc
sem Structural Equation Models package in R,   sem manual
Two good structural equation model reviews:
Structural Equation Models William T. Bielby; Robert M. Hauser Annual Review of Sociology, Vol. 3. (1977), pp. 137-161. JStor link
Breckler, S. J. (1990). Applications of Covariance Structure Modeling in Psychology: Cause for Concern? Psychological Bulletin, 107, 260-273. (available through psychinfo database SU libraries which I never find easy, here's a link that seems to be permanent

Exercises, week 3.
Solutions, week 3.
Week 4. 4/26
Multilevel data. Contextual effects, aggregation bias.

In the news
Criminal justice needs experiments (randomized controlled field studies) too:       Study Fuels Debate Over Police Lineups
     strangely related news: protocols matter (Duke lacrosse): Details Of Controversial Photo Lineup Revealed
Another reversal on calcium: is compliance the key?  Calcium helps prevent bone breaks, study shows. Results Contradict Previous Report; Consistent Use A Key Factor    Archives of Internal Medicine Effects of Calcium Supplementation on Clinical Fracture and Bone Structure

Lecture topics
1. Background: nested data, ecological fallacy, aggregation bias, levels of analysis
2. Traditional approaches to multilevel analysis: contextual effects, school effects.   class handout
3. Advanced analyses: random effects models, linear and non-linear.   class handout


Text readings/resources
Berk 10.3

Aggregation bias, Ecological fallacy.
D.A. Freedman. "Ecological inference and the ecological fallacy." International Encyclopedia for the Social and Behavioral Sciences. Elsevier (2001) vol. 6 pp. 4027–30. N. J. Smelser and Paul B. Baltes, eds.
D.A. Freedman. "The ecological fallacy." In the Encyclopedia of Social Science Research Methods. Sage Publications (2004) Vol. 1 p. 293. M. Lewis-Beck, A. Bryman, and T. F. Liao, eds
A Rule for Inferring Individual-Level Relationships from Aggregate Data, Glenn Firebaugh American Sociological Review Vol. 43, No. 4 (Aug., 1978), pp. 557-572   JStor URL
A good sociological/medical overview. Ecological effects in multi-level studies. Blakely TA, Woodward AJ. J Epidemiol Community Health. 2000 May;54(5):367-74.  pubmed   full text
American Journal of Epidemiology Vol. 139, No. 8: 747-760 Invited Commentary: Ecologic Studies—Biases, Misconceptions, and Counterexamples S Greenland, J Robins
The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology J. Michael Oakes Social Science & Medicine 58 (2004) 1929–1952

Educational multilevel data.
The Analysis of Multilevel Data in Educational Research and Evaluation Leigh Burstein Review of Research in Education, Vol. 8. (1980), pp. 158-233. Jstor link
Methodological Advances in Analyzing the Effects of Schools and Classrooms on Student Learning, Stephen W. Raudenbush; Anthony S. Bryk Review of Research in Education, Vol. 15. (1988 - 1989), pp. 423-475. Jstor link
Analyzing Multilevel Data in the Presence of Heterogeneous within-Class Regressions Leigh Burstein; Robert L. Linn; Frank J. Capell
Journal of Educational Statistics, Vol. 3, No. 4. (Winter, 1978), pp. 347-383. Jstor link

examples from analyses of voting data.
Bias in ecological regression   Stephen Ansolabehere and Douglas Rivers
David A. Freedman et al., "Ecological Regression and Voting Rights," Evaluation Review 1991, pp. 673-711, Berkeley Law postimg
Klein, S. P. and Freedman, D. A. (1993), “Ecological regression in voting rights cases,” Chance, 6, 38–43.
D.A. Freedman, S.P. Klein, M. Ostland, and M.R. Roberts. "Review of 'A Solution to the Ecological Inference Problem.' " Journal of the American Statistical Association, vol. 93 (1998) pp. 1518–22; with discussion, vol. 94 (1999) pp. 352–57.

Current statistical analyses in social science: multilevel models.
Using SAS PROC mixed:    Judith Singer HLM/PROC Mixed papers: Multilevel Modelling Newsletter ; JEBS1998 Using SAS PROC MIXED to Fit Multilevel Models, Jstor
HLM - Hierarchical Linear and Nonlinear Modeling (HLM): descriptions and student edition HLM6
Freedman, D. A. (census adjustments). Hierarchical Linear Regression
Using R: lme4 (lmer and nlme) and mlmRev.    John Fox lme tutorial   Doug Bates SASmixed package    U. Washington, Hierarchical Modeling for the Social Sciences
Fitting linear mixed models in R Using the lme4 package Douglas Bates (pp.27-30)
London exam data example in Examples from Multilevel Software Comparative Reviews Douglas Bates
mlmRev data examples. Also, Tennessee’s Student Teacher Achievement Ratio (STAR) from Creating an R data set from STAR Douglas Bates

Exercises, week 4.
Solutions, week 4.
Week 5. 5/3
The many uses and forms of analysis of covariance (including regression discontinuity designs)

In the news
Studies on dementia often confuse causes with consequences. Wall Street Journal 4/28/06, S. Begly. (hardcopy).
The Nintendo factor: Report: Brain Training May Not Help Cognitive Decline

Lecture topics
1. Review: formulation and purposes analysis of covariance (including role in multilevel analysis)
      handout from Fox lme tutorial: HS&B multilvel model as ancova;
      failures of regression adjustment in observational studies
2. Analyzing treatment effects as a function of covariate(s) (incl. Johnson-Neyman technique)
      handout: CNRL ancova, ancova alternatives and Johnson_Neyman in R    ascii for comparing nonparallel regression lines ex   
3. Regression Discontinuity Designs (asssignment on basis of covariate) and corresponding regression adjustments
       handout regression discontinuity data exs

Text readings/resources
Freedman 5.6
Berk 8.3  pdf Berk chap 8
Rogosa, D. R. (1980). Comparing nonparallel regression lines.   Psychological Bulletin, 88, 307-321.
Rubin, D. B., (1977), "Assignment to a Treatment Group on the Basis of a Covariate", Journal of Educational Statistics, 2, 1-26.   Jstor link

Regression Discontinuity Designs
Useful primers by Wm Trochin:  The regression-discontinuity design   regression-discontinuity analysis
Short bibliography
Trochim W.M. & Cappelleri J.C. (1992). “Cutoff assignment strategies for enhancing randomized clinical trials.” Controlled Clinical Trials, 13, 190-212.  pubmed link
Capitalizing on Nonrandom Assignment to Treatments: A Regression-Discontinuity Evaluation of a Crime-Control Program Richard A. Berk; David Rauma Journal of the American Statistical Association, Vol. 78, No. 381. (Mar., 1983), pp. 21-27. Jstor
Berk, R.A. & de Leeuw, J. (1999). “An evaluation of California’s inmate classification system using a generalized regression discontinuity design.” Journal of the American Statistical Association, 94(448), 1045-1052.  Jstor
U.C. Merced lecture notes (public health)  Wisconsin econometrics  London School of Economics U Arizona
Econometric treatments using Neyman-Rubin causal formulation.  
Another look at the Regression Discontinuity Design
Eligible Non-Participant And Ineligible Individuals As A Double Control Group In Regression Discontinuity Designs, Erich Battistin, Enrico Rettore, Proceedings of Statistics Canada Symposium 2002  more econometrics
the original paper: Thistlewaite, D., and D. Campbell (1960): “Regression-Discontinuity Analysis: An Alternative to the Ex Post Facto Experiment,” Journal of Educational Psychology, 51, 309–317.
educational application: regression discontinuity design to examine the causal effect of summer school and grade retention on student achievement.

Johnson-Neyman technique and aptitude-treatment interaction (ATI)
Regions of Significant Criterion Differences in Aptitude-Treatment-Interaction Research Leonard S. Cahen; Robert L. Linn American Educational Research Journal, Vol. 8, No. 3. (May, 1971), pp. 521-530. Jstor
Identifying Regions of Significance in Aptitude-by-Treatment-Interaction Research Ronald C. Serlin; Joel R. Levin American Educational Research Journal, Vol. 17, No. 3. (Autumn, 1980), pp. 389-399. Jstor
Defining Johnson-Neyman Regions of Significance in the Three-Covariate ANCOVA Using Mathematica Steve Hunka; Jacqueline Leighton Journal of Educational and Behavioral Statistics, Vol. 22, No. 4. (Winter, 1997), pp. 361-387.  Jstor
discussion of substantive issues: Trait-Treatment Interaction and Learning David C. Berliner; Leonard S. Cahen Review of Research in Education, Vol. 1. (1973), pp. 58-94. Jstor

Background/historical papers: analysis of covariance
Weisberg, H. I. Statistical adjustments and uncontrolled studies. Psychological Bulletin, 1979, 86, 1149-1164.
Covariance Adjustment in Randomized Experiments and Observational Studies Paul R. Rosenbaum Statistical Science, Vol. 17, No. 3. (Aug., 2002), pp. 286-304.   Jstor
Some Aspects of Analysis of Covariance, A Biometrics Invited Paper with Discussion. D. R. Cox; P. McCullagh Biometrics, Vol. 38, No. 3, (Sep., 1982), pp. 541-561.   Jstor
Analysis of Covariance: Its Nature and Uses William G. Cochran Biometrics, Vol. 13, No. 3, Special Issue on the Analysis of Covariance. (Sep., 1957), pp. 261-281. Jstor
The Use of Covariance in Observational Studies W. G. Cochran Applied Statistics, Vol. 18, No. 3. (1969), pp. 270-275. Jstor
Estimation of the Slope and Analysis of Covariance when the Concomitant Variable is Measured with Error James S. Degracie; Wayne A. Fuller Journal of the American Statistical Association, Vol. 67, No. 340. (Dec., 1972), pp. 930-937. Jstor
Deep background Neter-Wasserman text (Applied linear statistical models. Neter, Kutner, Nachtsheim & Wasserman 1996. Fifth edition. Homewood IL: Irwin, Inc.) chapters 22 and 8.

Exercises, week 5.
Solutions, week 5.
Week 6. 5/10
Instrumental variable methods, simultaneous equations
Empirical research on reciprocal effects (e.g. TV and ADHD), including cross-lagged correlation.

In the news
Relevant to Th1 problem 1:  Deskless class helps fight child obesity
  Puppy fat will dog a child into adulthood: study    Development of adiposity in adolescence: five year longitudinal study of an ethnically and socioeconomically diverse sample of young people in Britain, BMJ, May 2006

Lecture topics
1. Intro IV (Disattenuation, ancova adjustments, "selection effects") and other IV applications
       class handout:     ascii for measurement error ex
2. Simultaneous equations (2SLS, IV in butter, ed and fertility, Freedman), nonrecursive models
       class handouts:    ascii for supply-demand ex     ascii for Ducan et al aspirations ex     ascii for Rindfus Education-fertility ex   
3. Reciprocal effects and non-recursive models in longitudinal data

Text readings/resources
Freedman, Chap 8
Berk 9.5

Two-stage Least Squares in R (tsls in sem package) by John Fox (also "systemfit")
Fox, J. (1979) Simultaneous equation models and two-stage least-squares. In Schuessler, K. F. (ed.) Sociological Methodology 1979, Jossey-Bass. Jstor

Instrumental variables, Epidemiology exposition:   An introduction to instrumental variables for epidemiologists, Sander Greenland, International Journal of Epidemiology 2000;29:722-729 (or link through pubmed.org), note: compliance discussion for week 7
Technical reference. Joshua D. Angrist; Guido W. Imbens; Donald B. Rubin "Identification of Causal Effects Using Instrumental Variables"
Journal of the American Statistical Association, Vol. 91, No. 434. (Jun., 1996), pp. 444-455. JStor note: compliance discussion for week 7

Application of instrumental variables:   The Effect of File Sharing on Record Sales An Empirical Analysis
Effect of job training programs

Reciprocal effects: Rogosa, D. R. (1980). A critique of cross-lagged correlation. Psychological Bulletin, 88, 245-258.
Granger Causality. Nobel 2003. Complete Granger
Relationships--and the Lack Thereof--Between Economic Time Series, with Special Reference to Money and Interest Rates. David A. Pierce Journal of the American Statistical Association, Vol. 72, No. 357. (Mar., 1977), pp. 11-26. Jstor

Exercises, week 6.
Solutions, week 6.
Week 7. 5/17
Compliance and experimental protocols; encouragement designs; intent to treat

In the news
CA Proposition 82: decided by social science research? Preschool campaign heats up with first TV ads
Competing websites: http://www.yeson82.com/site/     http://www.noprop82.com/
Even the Mercury News is against

Lecture topics
1. Compliance background: Intent-to-treat analyses, Dose-response data analysis (Efron-Feldman), IV estimators (Greenland, Angrist et al, week6).   Class handout
2. Booil Jo presentation: Potential Outcomes Approach: A Brief Introduction

Text readings/resources
Berk, 11.4.1
Causal Inference, Path Analysis, and Recursive Structural Equations Models Paul W. Holland Sociological Methodology, Vol. 18. (1988), pp. 449-484.
Compliance as an Explanatory Variable in Clinical Trials. B. Efron; D. Feldman Journal of the American Statistical Association, Vol. 86, No. 413. (Mar., 1991), pp. 9-17. Jstor
Booil Jo, Dept of Psychiatry   Estimation of Intervention Effects with Noncompliance Journal of Educational and Behavioral Statistics

Compliance Background: Intent-to-Treat (ITT), the FDA mandate
simple definitions:   research practitioner ;    Intent-To-Treat Analysis Versus As-Treated Analysis Jonas H. Ellenberg, Phd Drug Information Journal, Vol. 30, pp. 535–544, 1996
Intent-to-treat Analysis of Randomized Clinical Trials Michael P. LaValleyBoston University ACR/ARHP Annual Scientific Meeting Orlando 10/27/2003
What is meant by intention to treat analysis? Survey of published randomised controlled trials Sally Hollis and Fiona Campbell British Medical Journal 1999;319;670-674

Compliance Publications based on Neyman-Rubin causal models:
Direct and Indirect Causal Effects via Potential Outcomes Donald B. Rubin Scandinavian Journal of Statistics Volume 31, Issue 2, Page 161-170, Jun 2004 .
Principal Stratification in Causal Inference  Constantine E. Frangakis and Donald B. Rubin, Biometrics, 2002, 58, 21–29.
Addressing Complications of Intention-to-Treat Analysis in the Combined Presence of All-or-None Treatment-Noncompliance and Subsequent Missing Outcomes. Constantine E. Frangakis; Donald B. Rubin Biometrika, Vol. 86, No. 2. (Jun., 1999), pp. 365-379. Jstor link
Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City Barnard, Frangakis, Hill, and Rubin Journal of the American Statistical Association June 2003, Vol. 98, No. 462, Applications and Case Studies
Battistin, E. and Rettore, E. (2002). “Testing for Programme Effects in a Regression Discontinuity Design with Imperfect Compliance.” Journal of the Royal Statistical Society A, 165(1), 39-57.

Exercises, week 7.
Solutions, week 7.
Week 8. 5/24
Matching and propensity score methods

In the news
more TV news, from Vanderbilt: Video Wasted On Toddlers, Unless It's Interactive
Research Publication: Child Development, Vol. 77, Issue 3, Young Children's Use of Video as a Source of Socially Relevant Information by Troseth GL, Saylor MM, and Archer AH (Vanderbilt University) . Stanford accessible link

Lecture topics
0. Compliance recap (including NYC vouchers experiment overview).
1. Traditional matching methods: pair matching, Mahalanobis distance. Matching for increased precision or bias-reduction.
2. The advent/onslaught of propensity score matching methodology for treatment-control comparisons
pdf file, lecture 8

Text readings/resources
Berk, 11.4.2
useful bibliography
Non-technical overview      Donald Rubin Nonrandomized Comparative Clinical Studies   another version, Annals of Internal Medicine
Strategies for Using Propensity Scores Well.  A Workshop given by Thomas E. Love, Ph. D., Case Western Reserve University 6th International Conference for Health Policy Research October 28, 2005     another version of Love workshop ASA
Joffe, Marshall M. and Paul R. Rosenbaum. 1999. "Invited Commentary: Propensity Scores." American Journal of Epidemiology 150(4):327-33.

R packages and examples: Optmatch add-on package for R   optmatch manual  vignettes
Optmatch application paper: Full matching in an observational study of coaching for the SAT.(Scholastic Assessment Test) Journal of the American Statistical Association; 9/1/2004; Hansen, Ben B.
Another application (including matchit): Attributing Effects to a Get-Out-The-Vote Campaign Using Full Matching and Randomization Inference Jake Bowers and Ben Hansen
Also:
MatchIt: Nonparametric Preprocessing for Parametric Casual Inference Daniel Ho, Kosuke Imai, Gary King, Elizabeth Stuart
Multivariate and Propensity Score Matching Software for Causal Inference Jasjeet S. Sekhon

    Propensity etc Original Technical Publications [jstor links]
Rosenbaum and Rubin, “Reducing Bias in Observational Studies Using Subclassification on the Propensity Score, JASA 79[387], September 1984, 516-524. JStor
Rosenbaum, P. R. And D. B. Rubin, 1983, The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika 70[1], April 1983, 41-55. JStor
P. Rosenbaum, Chapters 2 and 3 (on exact inference for treatment effects) in Observational Studies, New York: Springer, 1995.
Dropping out of High School in the United States: An Observational Study Paul R. Rosenbaum Journal of Educational Statistics, Vol. 11, No. 3. (Autumn, 1986), pp. 207-224.  Jstor
Paul R. Rosenbaum; Donald B. Rubin. "Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score" The American Statistician, Vol. 39, No. 1. (Feb., 1985), pp. 33-38   JStor
D. Rubin, Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies, Statistical Science 5[4], November 1990, 472-480. JStor
Rubin, D. B., 1974, Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, Journal of Educational Psychology, 66, 688-701.
Rubin, D. B., 1978, Bayesian Inference for Causal Effects: The Role of Randomization,” Annals of Statistics 6[1], January 1978, 34-58. JStor

Exercises, week 8.
Solutions, week 8.
Week 9. 5/31
Time-1, Time-2 data in experimental and non-experimental designs.
Lord's paradox, Measurement of change, Repeated Measures Analysis of Variance, and even value-added analysis

In the news
Sleep your way to a slimmer body      Sleep Deprivation Linked to Weight Gain

Lecture topics
0. Additional propensity score and matching illustrations
1. Lord's paradox and revisiting regression adjustments for pre-post designs
2. Comparing groups on multiple measurements: repeated measures anova etc
3. Extension to current implementations of value-added analysis
pdf file, lecture 9

               Background readings

1. Lord's Paradox, pre-post group comparisons. Lord, F. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68, 304-305.L
Wainer, H. (1991). Adjusting for differential base rates: Lord's Paradox again. Psychological Bulletin, 109, 147-151. psychinfo link (or obtain through psychinfo database)
or Wainer and Brown Three Statistical Paradoxes in the Interpretation of Group Differences: Illustrated with Medical School Admission and Licensing Data
a quick low-level read: Lord's Paradox and the Assessment of Change During College    Journal of College Student Development, May/Jun 2004 by Pike, Gary R
Another time1-time2 reading covering old-fashioned ground including Lord's paradox. Maris, Eric. (1998). Covariance Adjustment Versus Gain Scores--Revisited. Psychological Methods, 3(3) 309-327. apa link  (from campus IP)

2. Repeated measures analysis of variance
Comparative Analyses of Pretest-Posttest Research Designs, Donna R. Brogan; Michael H. Kutner, The American Statistician, Vol. 34, No. 4. (Nov., 1980), pp. 229-232.   JSTOR link
     urea synthesis, BK data     Stat141 analysis     data,,,,,,,,,,,,,,,,,,, example analyses
Models for Pretest-Posttest Data: Repeated Measures ANOVA Revisited Earl Jennings Journal of Educational Statistics, Vol. 13, No. 3. (Autumn, 1988), pp. 273-280.  Jstor
A good R-primer on repeated measures (a lots else). Notes on the use of R for psychology experiments and questionnaires Jonathan Baron, Yuelin Li.   Another version

3. Value-added analysis.
National School Boards Association: The Value of Value-Added Analysis
J.R. Lockwood, Harold Doran, and Daniel F. McCaffrey. Using R for estimating longitudinal student achievement models. R News, 3(3):17-23, December 2003.
Fitting Value-Added Models in R  Harold C. Doran & J.R. Lockwood
Using a Longitudinal Student Tracking System to Improve the Design for Public School Accountability in California Edward H. Haertel, August 2005
Don Rubin on value-added and Lord's paradox: A Potential Outcomes View of Value-Added Assessment in Education Donald B. Rubin, Elizabeth A. Stuart, and Elaine L. Zanutto, Journal of Educational and Behavioral Statistics
Dead Week 6/7
overflow and course summary. discussion of case studies and on-campus lectures
From week 9: Comparing groups on multiple measurements: repeated measures anova examples and computing (including crossover designs)
pdf file, dead week lecture

In the news
1. Permissive parents and fat kids. Disciplinarian parents have fat kids-US study   actual study: Parenting Styles and Overweight Status in First Grade PEDIATRICS Vol. 117 No. 6 June 2006, pp. 2047-2054
2. Maybe it's not television after all. Premature babies face higher risk of hyperactivity    Medline ADHD repository

Recap David Freedman (5/30) materials, two related papers, plus a handout:
http://www.stat.berkeley.edu/~census/oxcause.pdf
http://www.stat.berkeley.edu/~census/neyreg.pdf
http://www.stat.berkeley.edu/~census/modelobs.pdf