Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples

Jianhan Xu, Cenyuan Zhang, Xiaoqing Zheng, Linyang Li, Cho-Jui Hsieh, Kai-Wei Chang, and Xuanjing Huang, in ACL Finding, 2022.

Download the full text

Abstract

Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples. However, the augmented adversarial examples may not be natural, which might distort the training distribution, resulting in inferior performance both in clean accuracy and adversarial robustness. In this study, we explore the feasibility of introducing a reweighting mechanism to calibrate the training distribution to obtain robust models. We propose to train text classifiers by a sample reweighting method in which the example weights are learned to minimize the loss of a validation set mixed with the clean examples and their adversarial ones in an online learning manner. Through extensive experiments, we show that there exists a reweighting mechanism to make the models more robust against adversarial attacks without the need to craft the adversarial examples for the entire training set.

Bib Entry

@inproceedings{xu2022towards,
  title = {Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples},
  author = {Xu, Jianhan and Zhang, Cenyuan and Zheng, Xiaoqing and Li, Linyang and Hsieh, Cho-Jui and Chang, Kai-Wei and Huang, Xuanjing},
  booktitle = {ACL Finding},
  year = {2022}
}