Ilja Kuzborskij


I am a research scientist at Google DeepMind, London.

I am interested in design and analysis of learning algorithms, with emphasis on theories of generalization, uncertainty estimation, and concentration inequalities.

You can contact me at firstname.lastname@gmail.com

Tech reports

Pointwise confidence estimation in the non-linear ℓ²-regularized least squares
I. Kuzborskij and Y. Abbasi-Yadkori
arXiv. June, 2025.
[PDF]  [Bibtex]  

@misc{kuzborskij2025pointwiseconfidenceestimationnonlinear,
      title={Pointwise confidence estimation in the non-linear $\ell^2$-regularized least squares}, 
      author={Ilja Kuzborskij and Yasin Abbasi Yadkori},
      year={2025},
      eprint={2506.07088},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2506.07088}, 
}

Low-rank bias, weight decay, and model merging in neural networks
I. Kuzborskij and Y. Abbasi-Yadkori
arXiv. February, 2025.
[PDF]  [Bibtex]  

@misc{kuzborskij2025lowrankbiasweightdecay,
      title={Low-rank bias, weight decay, and model merging in neural networks}, 
      author={Ilja Kuzborskij and Yasin Abbasi Yadkori},
      year={2025},
      eprint={2502.17340},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.17340}, 
}

Mitigating LLM Hallucinations via Conformal Abstention
Y. Abbasi-Yadkori, I. Kuzborskij, D. Stutz, A. György, A. Fisch, A. Doucet,
I. Beloshapka, W.-H. Weng, Y.-Y. Yang, Cs. Szepesvári, A. T. Cemgil, N. Tomasev
arXiv. April, 2024.
[PDF]  [Bibtex]  

@misc{yadkori2024mitigating,
      title={Mitigating LLM Hallucinations via Conformal Abstention},
author={Yasin Abbasi Yadkori and Ilja Kuzborskij and David Stutz and András György and Adam Fisch and Arnaud Doucet
and Iuliya Beloshapka and Wei-Hung Weng and Yao-Yuan Yang and Csaba Szepesvári and Ali Taylan Cemgil and Nenad Tomasev},
      year={2024},
      eprint={2405.01563},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks
I. Kuzborskij and Cs. Szepesvári
arXiv. December, 2022.
[PDF]  [Bibtex]  

@misc{kuzborskij2022learning,
title = {Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks},
author = {Kuzborskij, Ilja and Szepesvári, Csaba},
howpublished = {arXiv:2212.13848},
year = {2022},
pdf = {https://arxiv.org/pdf/2212.13848},
url = {https://arxiv.org/abs/2212.13848}
}

Efron-Stein PAC-Bayesian Inequalities
I. Kuzborskij and Cs. Szepesvári
arXiv. September, 2019.
[PDF]  [Bibtex]  

@misc{kuzborskij2019efron,
title =		 {Efron-{S}tein {PAC}-{B}ayesian {I}nequalities},
author =	 {Kuzborskij, Ilja and Szepesvári, Csaba},
howpublished = {arXiv:1909.01931},
year = {2019},
pdf =	 {https://arxiv.org/pdf/1909.01931},
url =	 {https://arxiv.org/abs/1909.01931}
}

Publications

To Believe or Not to Believe Your LLM
Y. Abbasi-Yadkori, I. Kuzborskij, A. György, Cs. Szepesvári
Conference on Neural Information Processing Systems (NeurIPS), 2024.
[PDF]  [Bibtex]  

@inproceedings{yadkori2024tobelieve,
      title="To Believe or Not to Believe Your LLM",
      author={Yasin Abbasi-Yadkori and Ilja Kuzborskij and András György and Csaba Szepesvári},
      booktitle={Neural Information Processing Systems (NeurIPS)},
      year={2024}
}

Better-than-KL PAC-Bayes Bounds
I. Kuzborskij, K.-S. Jun, Y. Wu, K. Jang, and F. Orabona
Conference on Learning Theory (COLT), 2024.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2024better,
title={Better-than-KL {PAC-Bayes} Bounds},
author={Kuzborskij, Ilja and Jun, Kwang-Sung and Wu, Yulian and Jang, Kyoungseok and Orabona, Francesco},
booktitle={Conference on Learning Theory (COLT)},
year={2024},
}

Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation
Y. Deng, I. Kuzborskij, and M. Mahdavi
Conference on Neural Information Processing Systems (NeurIPS), 2023.
[PDF]  [Bibtex]  

@inproceedings{deng2023mixture,
  title={Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation},
  author={Deng, Yuyang and Kuzborskij, Ilja and Mahdavi, Mehrdad},
  booktitle={Neural Information Processing Systems (NeurIPS)},
  year={2023}
}

Tighter PAC-Bayes Bounds Through Coin-Betting
K. Jang, K.-S. Jun, I. Kuzborskij, and F. Orabona
Conference on Learning Theory (COLT), 2023.
[PDF]  [Bibtex]  

@inproceedings{jang2023tighter,
title={Tighter {PAC-Bayes} Bounds Through Coin-Betting},
author={Jang, Kyoungseok and Jun, Kwang-Sung and Kuzborskij, Ilja and Orabona, Francesco},
booktitle={Conference on Learning Theory (COLT)},
year={2023},
}

Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel
D. Richards and I. Kuzborskij
Conference on Neural Information Processing Systems (NeurIPS), 2021.
[PDF]  [Bibtex]  

@inproceedings{richards2021stability,
  title={Stability \& Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel},
  author={Richards, D. and Kuzborskij, I.},
  booktitle={Neural Information Processing Systems (NeurIPS)},
  year={2021}
}

On the Role of Optimization in Double Descent: A Least Squares Study
I. Kuzborskij, Cs. Szepesvári, O. Rivasplata, A. Rannen Triki, and R. Pascanu
Conference on Neural Information Processing Systems (NeurIPS), 2021.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2021role,
      title={On the Role of Optimization in Double Descent: A Least Squares Study},
      author={Ilja Kuzborskij and Csaba Szepesvári and Omar Rivasplata and Amal Rannen-Triki and Razvan Pascanu},
      booktitle={Neural Information Processing Systems (NeurIPS)},
      year={2021}
}

A Distribution-dependent Analysis of Meta Learning
M. Konobeev, I. Kuzborskij, and Cs. Szepesvári
International Conference on Machine Learning (ICML), 2021.
[PDF]  [Bibtex]  [Code]  

@inproceedings{konobeev2020statistical,
title = "{A Distribution-dependent Analysis of Meta Learning}",
author = {Konobeev, Mikhail and Kuzborskij, Ilja and Szepesvári, Csaba},
booktitle =	 {International Conference on Machine Learning},
year = {2021}
}

Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
I. Kuzborskij, C. Vernade, A. György, and Cs. Szepesvári
International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.
[PDF]  [Bibtex]  [Code]  

@InProceedings{pmlr-v130-kuzborskij21a,
  title =	 { Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting },
  author =       {Kuzborskij, Ilja and Vernade, Claire and Gyorgy, Andras and Szepesvari, Csaba},
  booktitle =	 {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
  year =	 {2021}
}

Locally-Adaptive Nonparametric Online Learning
I. Kuzborskij and N. Cesa-Bianchi
Conference on Neural Information Processing Systems (NeurIPS), 2020.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2020locally,
title =		 {Locally-{A}daptive {N}onparametric {O}nline {L}earning},
author =	 {Kuzborskij, Ilja and Cesa-Bianchi, Nicol\`{o}},
booktitle= {Neural Information Processing Systems (NeurIPS)},
year = {2020}
}

PAC-Bayes Analysis Beyond the Usual Bounds
O. Rivasplata I. Kuzborskij, Cs. Szepesvári, and J. Shawe-Taylor
Conference on Neural Information Processing Systems (NeurIPS), 2020.
[PDF]  [Bibtex]  

@inproceedings{rivasplata2020pac,
title =	{{PAC}-{B}ayes {A}nalysis {B}eyond the {U}sual {B}ounds},
author={Rivasplata, O. and Kuzborskij, I. and Szepesv{á}ri, Cs. and Shawe-Taylor, J.}
booktitle= {Neural Information Processing Systems (NeurIPS)},
year = {2020}
}

Distribution-Dependent Analysis of Gibbs-ERM Principle
I. Kuzborskij, N. Cesa-Bianchi, and Cs. Szepesvári
Conference on Learning Theory (COLT), 2019.
[PDF]  [Bibtex]  

@InProceedings{pmlr-v99-kuzborskij19a,
  title =	 {Distribution-Dependent Analysis of Gibbs-ERM Principle},
  author =	 {Kuzborskij, Ilja and Cesa-Bianchi, Nicol\`{o} and Szepesvári, Csaba},
  booktitle =	 {Proceedings of the Thirty-Second Conference on Learning Theory},
  pages =	 {2028--2054},
  year =	 {2019},
  editor =	 {Beygelzimer, Alina and Hsu, Daniel},
  volume =	 {99},
  series =	 {Proceedings of Machine Learning Research},
  address =	 {Phoenix, USA},
  month =	 {25--28 Jun},
  publisher =	 {PMLR},
  pdf =		 {http://proceedings.mlr.press/v99/kuzborskij19a/kuzborskij19a.pdf},
  url =		 {http://proceedings.mlr.press/v99/kuzborskij19a.html}
}

Efficient Linear Bandits through Matrix Sketching
I. Kuzborskij, L. Cella, and N. Cesa-Bianchi
International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2019efficient,
  title =	 {Efficient {L}inear {B}andits through {M}atrix {S}ketching},
  author =	 {Kuzborskij, Ilja and Cella, Leonardo and Cesa-Bianchi, Nicol\`{o}},
  booktitle =	 {Proceedings of Machine Learning Research},
  pages =	 {177--185},
  year =	 {2019},
  editor =	 {Chaudhuri, Kamalika and Sugiyama, Masashi},
  volume =	 {89},
  series =	 {Proceedings of Machine Learning Research},
  address =	 {},
  month =	 {16--18 Apr},
  publisher =	 {PMLR},
  pdf =		 {http://proceedings.mlr.press/v89/kuzborskij19a/kuzborskij19a.pdf},
  url =		 {http://proceedings.mlr.press/v89/kuzborskij19a.html}
  }

Data-Dependent Stability of Stochastic Gradient Descent
I. Kuzborskij and C. H. Lampert
International Conference on Machine Learning (ICML), 2018.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2017data,
  title={{D}ata-{D}ependent {S}tability of {S}tochastic {G}radient {D}escent},
  author={I. Kuzborskij and C. H. Lampert},
  booktitle = {International Conference on Machine Learning (ICML)},
  year={2018}
}

Nonparametric Online Regression while Learning the Metric
I. Kuzborskij and N. Cesa-Bianchi
Advances in Neural Information Processing Systems (NeurIPS), 2017.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2017nonparametric,
  title={{N}onparametric {O}nline {R}egression while {L}earning the {M}etric},
  author={I. Kuzborskij and N. Cesa-Bianchi},
  booktitle= {Neural Information Processing Systems (NIPS)},
  year={2017}
}

Fast Rates by Transferring from Auxiliary Hypotheses
I. Kuzborskij and F. Orabona
Machine Learning, September, 2016.
[PDF]  [Bibtex]  

@article{kuzborskij2016fast,
  author={I. Kuzborskij and F. Orabona},
  title={Fast {R}ates by {T}ransferring from {A}uxiliary {H}ypotheses},
  journal="Machine Learning",
  year=2016,
  pages="1--25",
  issn="1573-0565",
  doi="10.1007/s10994-016-5594-4",
  url="http://dx.doi.org/10.1007/s10994-016-5594-4"
  }

Scalable Greedy Algorithms for Transfer Learning
I. Kuzborskij, F. Orabona, and B. Caputo
Computer Vision and Image Understanding, 2016.
[PDF]  [Bibtex]  
@article{kuzborskij2016scalable,
  author    = {I. Kuzborskij and
	       F. Orabona and
	       B. Caputo},
  title     = {Transfer {L}earning through {G}reedy {S}ubset {S}election},
  journal = "Computer Vision and Image Understanding ",
  volume = "",
  number = "",
  pages = " - ",
  year = "2016",
  issn = "1077-3142",
  doi = "http://dx.doi.org/10.1016/j.cviu.2016.09.003",
  url = "http://www.sciencedirect.com/science/article/pii/S1077314216301370",
  }

When Naïve Bayes Nearest Neighbors Meet Convolutional Neural Networks
I. Kuzborskij, F.M. Carlucci, and B. Caputo
Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[PDF]  [Bibtex]  [Code]  [Supplementary material] 

@inproceedings{kuzborskij2016when,
    title={{W}hen {N}aive {B}ayes {N}earest {N}eighbours
	   {M}eet {C}onvolutional {N}eural {N}etworks},
    author={Kuzborskij, I. and Carlucci, F. M. and Caputo, B.},
    booktitle={Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on},
    year={2016}
  }

Transfer Learning through Greedy Subset Selection (best paper award)
I. Kuzborskij, F. Orabona, and B. Caputo
International Conference on Image Analysis and Processing (ICIAP), 2015.
[PDF]  [Bibtex]  [Code] 

@inproceedings{kuzborskij2015transfer,
  author    = {I. Kuzborskij and
	       F. Orabona and
	       B. Caputo},
  title     = {Transfer Learning Through Greedy Subset Selection},
  booktitle = {Image Analysis and Processing - {ICIAP} 2015 - 18th International
	       Conference, Proceedings, Part
	       {I}},
  pages     = {3--14},
  year      = {2015},
  }

Stability and Hypothesis Transfer Learning
I. Kuzborskij and F. Orabona
International Conference on Machine Learning (ICML), 2013.
[PDF]  [Bibtex]  [Errata] 

@inproceedings{kuzborskij2013stability,
  author    = {I. Kuzborskij and
	       F. Orabona},
  title     = {Stability and {H}ypothesis {T}ransfer {L}earning},
  booktitle = {International Conference on Machine Learning (ICML)},
  pages     = {942--950},
  year      = {2013}
  }

From N to N+1: Multiclass Transfer Incremental Learning
I. Kuzborskij, F. Orabona, and B. Caputo
Conference on Computer Vision and Pattern Recognition (CVPR) 2013.
[PDF]  [Bibtex]  [Code]  [Supplementary material] 

@inproceedings{kuzborskij2013from,
  title =        {From {N} to {N}+1: {M}ulticlass {T}ransfer {I}ncremental {L}earning},
  author =       {Kuzborskij, I. and Orabona, F. and Caputo, B.},
  booktitle={Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on},
  pages={3358--3365},
  year={2013},
  organization={IEEE}
  }

Other topics

Characterization of a Benchmark Database for Myoelectric Movement Classification.
M. Atzori, A. Gijsberts, I. Kuzborskij, S. Elsig, A.-G. Mittaz Hager, O. Deriaz,
C. Castellini, H. Muller, B. Caputo
Transactions on Neural Systems and Rehabilitation Engineering, 2015.
[PDF]  [Bibtex]  

@article{atzori2015characterization,
  title={Characterization of a {B}enchmark {D}atabase for
	 {M}yoelectric {M}ovement {C}lassification},
  author={Atzori, M. and Gijsberts, A. and Kuzborskij, I. and
	  Elsig, S. and Mittaz Hager, A.-G. and Deriaz, O. and
	  Castellini, C. and Muller, H. and Caputo, B.},
  journal={Neural Systems and Rehabilitation Engineering, IEEE Transactions on},
  volume={23},
  number={1},
  pages={73--83},
  year={2015},
  publisher={IEEE}
  }

On the Challenge of Classifying 52 Hand Movements from Surface Electromyography.
I. Kuzborskij, A. Gijsberts, B. Caputo
International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2012.
[PDF]  [Bibtex]  

@inproceedings{kuzborskij2012challenge,
  title={On the {C}hallenge of {C}lassifying 52 {H}and {M}ovements from
	 {S}urface {E}lectromyography},
  author={Kuzborskij, I. and Gijsberts, A. and Caputo, B.},
  booktitle={Engineering in Medicine and Biology Society (EMBC),
	     2012 Annual International Conference of the IEEE},
  pages={4931--4937},
  year={2012},
  organization={IEEE}
}