Methods for imperfect, adaptive and structured feedback
Modern machine learning increasingly relies on pipelines in which learned models guide decisions, decisions determine which data are observed, and those data are reused for inference and further optimization. The same mechanisms that make these systems effective—adaptive collection, flexible prediction, learned representations, and optimization through estimated components—can invalidate classical guarantees. Component-wise guarantees need not survive composition: deployment changes the sampling law, optimization amplifies estimation error, and plug-in procedures propagate bias downstream.
My research develops statistical theory and algorithms for these interfaces. I study where guarantees fail when learning, optimization, and inference interact; identify conditions and constructions—such as directional stability and orthogonality—that preserve validity and efficiency; and design principled corrections when they do not. The broader goal is to enable expressive learning systems to improve decisions without compromising the conclusions drawn from them. This programme spans inference after adaptive experiments, nuisance-robust distributional causal inference for structured outcomes, nuisance-robust methods for functionals of solutions of inverse and nested problems, policy learning, and earlier work in representation learning.
Statistical Foundations of Inference after Adaptive Experiments
Adaptive experiments use past outcomes to choose future assignments, then reuse the resulting trajectory for inference. For estimators whose leading term is a martingale sum, a Gaussian limit typically requires two ingredients: no individual increment is asymptotically influential, and the normalized quadratic variation—or, under mild conditions, the accumulated conditional variance—stabilizes. Adaptive allocation can violate the second requirement because the realized assignment path may preserve non-negligible information about earlier outcome fluctuations; in fixed-horizon examples, an inverse-propensity estimator with known assignment probabilities then converges to a non-Gaussian mixture of normals. It can also violate the first requirement when some assignment probabilities are of order $1/T$, allowing a few inverse-propensity-weighted observations to retain non-vanishing influence. Consequently, naive Wald intervals based on the standard martingale Gaussian approximation can be miscalibrated.
I develop procedures that restore validity while preserving the information accumulated by the adaptive design. For structured outcomes, I construct a learn-then-test procedure that estimates an informative witness direction, projects a doubly robust RKHS score onto it, and applies predictable variance normalization, yielding calibrated distributional tests under adaptive assignment [AI1]. In my recent work, I introduce directional stability, a target-specific condition requiring only the design information relevant to the parameter of interest—formalized through its Riesz representer—to stabilize, rather than the entire information matrix. To characterize efficiency in this setting, I develop local asymptotic normality and convolution theory for sequences of experiments whose adaptive sampling laws change with the horizon. Within this theory, I show that directional stability is sufficient for classical one-step estimators to remain asymptotically normal and attain the semiparametric efficiency bound, without generic adaptive reweighting [AI2].
Questions:
How can distributional tests remain calibrated when adaptive assignment changes the covariance geometry of the outcome representation? [AI1]
What target-specific stability is sufficient for classical one-step inference to remain valid and efficient under adaptive sampling? [AI2]
How should semiparametric efficiency be defined when the sampling law itself changes with the experiment? [AI2]
Distributional Causal Inference for Structured Outcomes
Most causal inference reduces intervention effects to means, quantiles, or other scalar summaries. This can miss changes in dispersion, modes, tails, or rare configurations, while the field still lacks a general statistical theory and broadly applicable machine-learning methods for causal inference with images, sequences, graphs, and other structured outcomes. I address this gap by developing kernel-based procedures that represent entire interventional outcome distributions while remaining robust to estimation errors in propensity and conditional outcome models.
I first introduced a global, doubly robust representation and test for determining whether two interventions induce the same outcome distribution [SO1]. An omnibus rejection, however, establishes that the distributions differ without revealing how. I therefore developed a second procedure that learns informative outcome prototypes and evaluates the discrepancy at those locations, yielding an interpretable and semiparametrically efficient test that localizes the effect while remaining valid after data-driven prototype selection [SO2].
How can entire interventional outcome distributions be represented, estimated, and compared for structured outcomes? [SO1]
How can a global distributional difference be localized and interpreted without sacrificing statistical efficiency or validity after data-driven selection? [SO2]
Nuisance-Robust Learning and Inference for Nested and Inverse Problems
I develop nuisance-robust methods for finite-dimensional functionals of solutions to nested and inverse problems. In mediation and bilevel optimization, these functionals are built from iterated regressions or a lower-level population optimizer and its derivative; in proxy and instrumental-variable analysis, they are built from bridge or least-squares solutions to conditional moment equations. Plug-in estimation is fragile: errors propagate through nested layers, while inverse operators discard information, making exact solutions unstable, non-unique, or potentially nonexistent. I therefore target the final functional directly using orthogonal scores, Riesz corrections, complementary bridge equations, and cross-fitting.
For mediation, I developed a kernel-localized orthogonal estimator for continuous-treatment mediated response curves [N1]. A companion comparative mediation study benchmarks classical, multiply robust, and double-machine-learning estimators across binary, continuous, and multidimensional mediators, including under model misspecification, weak overlap, and in a UK Biobank application [N2]. For latent confounding, in joint work we developed density-ratio-free doubly robust bridge estimators based on kernel mean embeddings [N3]. In more recent joint work on instrumental variables, we introduced a projected parameter defined through primal and dual population least-squares problems. It remains well defined when the exact moment equations have no solution, is invariant to non-unique least-squares minimizers, and coincides with the usual structural estimand under exact specification [N4]. We subsequently extended the bridge approach to neural representations and continuous or structured treatments [N5]. I also developed the first semiparametric efficiency theory for the unregularized population bilevel gradient, together with an orthogonal estimator controlled uniformly over the outer parameter so that it can serve as a statistical gradient oracle [N6].
Recent questions
What should nonparametric IV analysis estimate when its moment equations are only approximate? Can a projected least-squares parameter recover the usual estimand under exact specification and still support debiased inference otherwise? [N4]
How can the unregularized population hypergradient be estimated without the first-order bias of differentiating fitted lower-level models, and controlled uniformly enough to guide optimization? [N6]
Selected Papers
[N4] Nonparametric Instrumental Variable Analysis Without Structural Equations: Debiased Inference on Functionals of Inverse Problems with No Solutions
Zikai Shen, Nathan Kallus, Dimitri Meunier, Houssam Zenati, Arthur Gretton, Aurélien Bibaut. Preprint, 2026. [Paper]
[N6] Semiparametric Efficient Bilevel Gradient Estimation
Fares El Khoury*, Houssam Zenati*, Nathan Kallus, Michael Arbel, Aurelien Bibaut. Preprint, 2026. [Paper]
Statistical and Algorithmic Foundations of Policy Learning
I develop statistical and algorithmic foundations for policy learning across fully online, batched-online, and offline contextual-bandit settings. From fixed logged data, I first studied direct policy optimization with continuous actions, using smooth importance-weighted objectives and proximal methods to stabilize a nonconvex learning problem [PL1]. In online contextual bandits, I then made nonparametric value-based learning computationally scalable through incremental Nyström approximations [PL2], and studied how hierarchical similarities between actions can reduce the effective complexity of exploration [PL3].
On the policy-based side, I introduced in Sequential CRM a Hölderian error-bound principle linking policy suboptimality to importance-weight variance. Combined with variance-sensitive updates and limited redeployment, this yields fast excess-risk and regret rates for parametric policy classes [PL4]. I also studied how covariance information revealed by semi-bandit observations can improve online exploration [PL5]. Returning to offline continuous-action learning, I completed the pipeline with joint context–action kernel models and principled offline model selection and evaluation [PL6].
Building on these foundations, my recent work extends the fast-rate principle to fully online, agnostic learning over rich nonparametric policy classes: O2PL establishes the first fast best-in-class regret rates in this setting [PL7]. I also introduce a semiparametric natural-gradient construction that debiases the learned policy itself, rather than merely estimating or optimizing a debiased policy value [PL8]. This recent work asks:
What structural conditions allow fully online, agnostic contextual bandits to attain fast regret against the best policy in a rich nonparametric class, without reward-model realizability? [PL7]
Can policy learning itself, not merely the evaluation of a fixed policy, be semiparametrically targeted through a functional natural-gradient flow, so that rich policy classes attain root-N regret while policy approximation and environment-estimation errors interact only through second-order product remainders? [PL8]
Generative Representation Learning and Adversarial Optimization
My earlier work in adversarial learning first focused on generative representation learning for anomaly detection and semi-supervised prediction. I initiated and led a line of work that made GAN-based anomaly detection practical at test time. I introduced an amortized detector that replaced the costly per-example latent optimization used by earlier GAN methods with direct encoder-based inference, achieving state-of-the-art results while reducing test-time computation by several hundred-fold [AR1]. I also contributed to work showing how GANs can encode data-manifold geometry through a Monte Carlo approximation of Laplacian regularization for semi-supervised learning [AR2], [AR3].
I then developed ALAD, strengthening bidirectional adversarial learning with data- and latent-space cycle consistency, spectral normalization, and discriminator-based representations to improve anomaly detection across image and tabular benchmarks [AR4]. In retinal imaging, I applied semi-supervised representation learning to reduce reliance on expert annotations for abnormality classification [AR5]. This line of work became a widely cited reference point in generative anomaly detection, and ALAD was subsequently used beyond the original benchmarks.
A complementary line of work examined optimization in adversarial min–max problems. It characterized the last-iterate dynamics of mirror descent under coherence, showed why standard updates can cycle even in bilinear games, and established convergence of optimistic corrections across the broader class of coherent problems [AR6]. I also studied practical unsupervised anomaly detection for retinal images, where clinically relevant abnormalities may be sparse or localized [AR7].
Questions:
How can adversarial generative models make anomaly detection computationally practical, avoiding per-example latent optimization at inference time? [AR1]
How can generators encode data-manifold geometry and exploit unlabeled observations for semi-supervised learning? [AR2], [AR3]
How can bidirectional adversarial representations improve anomaly detection across image and tabular data? [AR4]
How can semi-supervised representation learning reduce expert-annotation requirements in medical imaging? [AR5]
Under what structural conditions do optimistic first-order methods achieve last-iterate convergence in adversarial min–max problems? [AR6]
How can unsupervised representation learning support practical detection and localization of abnormalities in retinal images? [AR7]
