Depth-2 neural networks under a data-poisoning attack

Research output: Contribution to journalArticlepeer-review

Abstract

In this work, we study the possibility of defending against data-poisoning attacks while training a shallow neural network in a regression setup. We focus on doing supervised learning with realizable labels for a class of depth-2 finite-width neural networks, which includes single-filter convolutional networks. In this class of networks, we attempt to learn the true network weights generating the labels in the presence of a malicious oracle doing stochastic, bounded and additive adversarial distortions on the true labels, during training. For the gradient-free stochastic algorithm that we construct, we prove worst-case near-optimal trade-offs among the magnitude of the adversarial attack, the weight approximation accuracy, and the confidence achieved by the proposed algorithm. As our algorithm uses mini-batching, we analyze how the mini-batch size affects convergence. We also show how to utilize the scaling of the outer layer weights to counter data-poisoning attacks on true labels depending on the probability of attack. Lastly, we give experimental evidence demonstrating how our algorithm outperforms stochastic gradient descent under different input data distributions, including instances of heavy-tailed distributions.
Original languageEnglish
JournalNeurocomputing
Early online date21 Feb 2023
DOIs
Publication statusE-pub ahead of print - 21 Feb 2023

Keywords

  • convolutional neural networks
  • stochastic algorithims
  • data poisoning
  • robust regression

Fingerprint

Dive into the research topics of 'Depth-2 neural networks under a data-poisoning attack'. Together they form a unique fingerprint.
  • MCAIF: Centre for AI Fundamentals

    Kaski, S. (PI), Pan, W. (Researcher), Alvarez, M. (Researcher), Mu, T. (Researcher), Sun, M. (Researcher), Mukherjee, A. (Researcher), Sonee, A. (Researcher), Leroy, A. (Researcher), Lee, J. (Researcher), Wang, J. (Researcher), Parakkal Unni, M. (Researcher), Sloman, S. (Researcher), Menary, S. (Researcher), Quilter, T. (Researcher), Hosseinzadeh, A. (PGR student), Mousa, A. (PGR student), Das, A. (PGR student), Glover, E. (PGR student), DURSUN, F. (PGR student), Zhu, H. (PGR student), Abdi, H. (PGR student), Dandago, K. (PGR student), Piriyajitakonkij, M. (PGR student), Rachman, R. (PGR student), Shi, X. (PGR student), Keany, T. (PGR student), Liu, X. (PGR student), Jiang, Y. (PGR student), Wan, Z. (PGR student), Evans, I. (Support team), Harrison, M. (Support team), Rivasplata, O. (PI), Caprio, M. (PI) & Machado, M. (PI)

    1/10/2130/09/26

    Project: Research

Cite this