Alessandro (Ale) Rinaldo - Fall, 2025
SDS 387 is an intermediate graduate course in theoretical statistics for PhD students, covering two separate but interrelated topics: (i) stochastic convergence, (ii) selected topics in learning theory and (iii) linear regression modeling. The material and style of the course will skew towards the mathematical and theoretical aspects of common models and methods, in order to provide a foundation for those who wish to pursue research in statistical methods and theory. This is not an applied regression analysis course.
Syllabus: Syllabus
Lectures: Tuesday and Thursday, 9:00am - 10:30am, FAC 101B
TA: Hien Dang, hiendang@utexas.edu - Office hours: Tuesday, 2:00–3:00 pm in WEL 5.228H
Ale's Office hours: by appointment
Homework submission and solutions: use Canvas
| Due date | |
| Homework 1 | |
Class canceled (Ale was sick)
Lecture 1: Introduction and course logistics. Deterministic convergence and convergence with probability one. Limsup and liminf of sequences of events.
Nicola's Baritellt candidacy talk, part of which covered the content of his latest paper with Stephen Walker and Bernardo Flores.
Lecture 2: Limsup and liminf of events. Borel-Cantelli's Second Lemma. Convergence in probability and comparison with convergence with probability one.
References:
See Ferguson's book, chapters 1, 2 and 4.
A nice webpage summarizing the different modes of stochastic convergence and providing some good examples to illustrate their differences.
Lecture 3: WLLN and SLLN. Glivenko Cantelli Theorem and DKW inequality. For proof of the Glivenko-Cantelli Theorem, see Theorem 19.1 in van der Vaart's book.
Lecture 4: Proof of Glivenko Cantelli Theorem and DKW inequality. Lp spaces and convergence.
Lecture 5: Lp convergence, Minkowski, Holder and Jensen inequalities. Relations between Lp convergence and convergence in probability and with probability one. Convergence in distribution for univariate random variables. C.d.f.'s in multivariate settings.
Lecture 6: Convergence in distribution. Relation with other forms of convergence. Marginal vs joint convergence in distribution. For the proof of the claim that convergence in probability implies convergence in distribution, see page 330 of Billingsley's book /Probability and Measure.
Lecture 7: Uniqueness of stochastic limits. Portmantreau Theorem (see, e.g, chapter 2 in van der Vaart's Asymptotic Statistics book).
Lecture 8: Characteristics functions and Continuity Theorem, Cramer-Wald device. I suggest reading Chapter 3 of Ferguson's book (in particuar, Theorem 3(e) has a neat proof). For a reference to multivariate Taylor series expansions, see, e.g., Advanced Calculus by G. Folland, available here.
Lecture 9: Slutsky's theorem, more on convergence in distribution. Big-oh and little-oh notation. Prohorov's theorem about tightness of stochastic sequences.
Lecture 10: CLT for i.i.d. variables using characteristic functions. Triangular arrays, Lindeberg Feller and Lyapunov conditions.
Lecture 11: Lindeberg Feller, examples and multivariate extension. Berry-Esseen bounds. A good reference for this lecture and the last is the book Sums of Independent Random Variables, by V.V. Petrov, Springer, 1975. Another classic and good reference is Approximation Theorems of Mathematical Statistics by Serfling, Wiley, 1980.
Lecture 12: High dimensional Berry-Esseen Central Limit Theorems. Some references:
Bentkus V. (2003). On the dependence of the Berry-Esseen bound on dimension. — J. Statist. Planning and Inference, 113(12):385-402.
Bentkus, V. (2005). A Lyapunov-type bound in Rd. Theory of Probability & Its Applications, 49(2):311–323
Raic, M. (2019). A multivariate Berry–Esseen theorem with explicit constants. Bernoulli, 25(4A):2824–2853.
Specific references for the class of hyper-rectangles are:
Chernozhukov, V., Chetverikov, D. and Koike Y. (2023). Nearly optimal central limit theorem and bootstrap approximations in high dimensions, Annals of Applied Probability, 33(3): 2374-2425
Bong, H., Kuchibhotla A. K. and Rinaldo, A. (2023). Dual Induction CLT for High-dimensional m-dependent Data, arXiv:2306.14299
In class I alluded to Theorem 1 of the following paper as a very general and useful Berry-Esseen bound for smooth (thrice continuously differentiable) functions of independent random variables:
Chatterjee, S. (2006). A generalization of the Lindeberg principle, Ann. Probab. 34 (6), 2061-2076.
Lecture 13: Review of linear algebra. See references in the class notes.
Lecture 14: Review of linear algebra. See references in the class notes.
Lecture 15: Review of linear algebra. See references in the class notes. Projection of a random variable onto a vector space of random variables - see chapter 11 of Asymptotic Statistics by A. van der Vaar.
Lecture 16: Projection of a random variable onto a vector space of random variables (cont'd). Introduction to regression modeling.
Lecture 17: Linear regression modeling: inference vs prediction. Projection parameters and predictive risk. HIghly recommended reading: Assumption lean regression. Highly recommended reading:
Buja, A., Brown, L., Berk, R., George, E., Pitkin, E., Traskin, M., Zhang, K., Zhao, L. (2019). Models as Approximations I: Consequences Illustrated with Linear Regression, Statistical Science, 34(4), 523-544.