eduzhai > Applied Sciences > Computer Science >

Orthogonal Statistical Learning

  • KanKan
  • (0) Download
  • 20210425
  • Save

... pages left unread,continue reading

Document pages: 86 pages

Abstract: We provide non-asymptotic excess risk guarantees for statistical learning ina setting where the population risk with respect to which we evaluate thetarget parameter depends on an unknown nuisance parameter that must beestimated from data. We analyze a two-stage sample splitting meta-algorithmthat takes as input two arbitrary estimation algorithms: one for the targetparameter and one for the nuisance parameter. We show that if the populationrisk satisfies a condition called Neyman orthogonality, the impact of thenuisance estimation error on the excess risk bound achieved by themeta-algorithm is of second order. Our theorem is agnostic to the particularalgorithms used for the target and nuisance and only makes an assumption ontheir individual performance. This enables the use of a plethora of existingresults from statistical learning and machine learning to give new guaranteesfor learning with a nuisance component. Moreover, by focusing on excess riskrather than parameter estimation, we can give guarantees under weakerassumptions than in previous works and accommodate settings in which the targetparameter belongs to a complex nonparametric class. We provide conditions onthe metric entropy of the nuisance and target classes such that oraclerates---rates of the same order as if we knew the nuisance parameter---areachieved. We also derive new rates for specific estimation algorithms such asvariance-penalized empirical risk minimization, neural network estimation andsparse high-dimensional linear model estimation. We highlight the applicabilityof our results in four settings of central importance: 1) heterogeneoustreatment effect estimation, 2) offline policy optimization, 3) domainadaptation, and 4) learning with missing data.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...