I'm interested in robustness and efficiency of machine learning algorithms. Currently, I am particularly interested in the following topics: Here are my recent publications on these topics. For my full publication please see full publication list .

Robustness of ML Models

Adversarial Attack

Sign-OPT: A Query-Efficient Hard-label Adversarial Attack , Minhao Cheng*, Simranjit Singh*, Patrick H. Chen, Pin-Yu Chen, Sijia Liu, Cho-Jui Hsieh (* Equal Contribution). In International Conference on Learning Representations (ICLR), 2020.

Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach , Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

Code

Evaluating Robustness of Deep Image Super-Resolution Against Adversarial Attacks , Jun-Ho Choi, Huan Zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, Jong-Seok Lee. To appear in ICCV 2019.

The Limitations of Adversarial Training and the Blind-Spot Attack , Huan Zhang*, Hongge Chen*, Zhao Song, Duane Boning, inderjit dhillon, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks , Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, Shin-Ming Cheng. In AAAI Conference on Artificial Intelligence (AAAI), 2019.

Code

Genattack: Practical Black-box Attacks with Gradient-free Optimization , Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, Huan Zhang, Cho-Jui Hsieh, Mani B Srivastava. In Proceedings of the Genetic and Evolutionary Computation Conference (Gecco), 2019.

Code

ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models , Pin-Yu Chen*, Huan Zhang*, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh (* Equal contribution). ACM Conference on Computer and Communications Security (CCS) Workshop on Artificial Intelligence and Security (AISec), 2017.

nominated for best paper award Code

Robustness Verification and Certified Defense

Provable, Scalable and Automatic Perturbation Analysis on General Computational Graphs , Kaidi Xu*, Zhouxing Shi*, Huan Zhang*, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, Cho-Jui Hsieh. (* Equal Contribution) NeurIPS 2020.

Robustness Verification for Transformers , Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh. In International Conference on Learning Representations (ICLR), 2020.

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius , Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Liwei Wang. In International Conference on Learning Representations (ICLR), 2020.

Towards Stable and Efficient Training of Verifiably Robust Neural Networks , Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, Cho-Jui Hsieh. In International Conference on Learning Representations (ICLR), 2020.

Code

A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks , Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, Pengchuan Zhang. To appear in NeurIPS 2019.

RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications , Huan Zhang, Pengchuan Zhang, Cho-Jui Hsieh. In AAAI Conference on Artificial Intelligence (AAAI), 2019.

Code

Efficient Neural Network Robustness Certification with General Activation Functions , Huan Zhang*, Lily Weng*, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel. (* Equal contribution). In Advances in Neural Information Processing Systems (NIPS), 2018

Code

Towards Fast Computation of Certified Robustness for ReLU Networks , Tsui-Wei Weng*, Huan Zhang*, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit Dhillon, Luca Daniel. (* Equal contribution) In International Conference on Machine Learning (ICML), 2018.

Code

Adversarial Defense

Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations , Huan Zhang*, Hongge Chen*, Chaowei Xiao, Bo Li, Mingyan Liu, Duane Boning, Cho-Jui Hsieh (* Equal Contribution). To appear in NeurIPS 2020.

spotlight

Improved Adversarial Training via Learned Optimizer , Yuanhao Xiong, Cho-Jui Hsieh. In European Conference on Computer Vision (ECCV), 2020.

Self-Progressing Robust Training , Minhao Cheng, Pin-Yu Chen, Sijia Liu, Shiyu Chang, Cho-Jui Hsieh, Payel Das To appear in AAAI 2021.

ML-LOO: Detecting Adversarial Examples with Feature Attribution , Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael I. Jordan. In AAAI Conference on Artificial Intelligence (AAAI), 2020.

Convergence of Adversarial Training in Overparameterized Networks , Ruiqi Gao, Tianle Cai, Haochuan Li, Liwei Wang, Cho-Jui Hsieh, Jason D. Lee. To appear in NeurIPS 2019.

spotlight

Rob-GAN: Generator, Discriminator, and Adversarial Attacker , Xuanqing Liu, Cho-Jui Hsieh. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Code

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network , Xuanqing Liu, Yao Li, Chongruo Wu, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

Code

Towards Robust Neural Networks via Random Self-ensemble , Xuanqing Liu, Minhao Cheng, Huan Zhang, Cho-Jui Hsieh. In European Conference on Computer Vision (ECCV), 2018.

Code

Adversarial Attack and Defense for NLP Models

Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples , Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, Cho-Jui Hsieh. In AAAI Conference on Artificial Intelligence (AAAI), 2020.

Code

On the Robustness of Self-Attentive Models , Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, Cho-Jui Hsieh. To appear In Proceedings of Association for Computational Linguistics (ACL), 2019.

long paper

Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent , Minhao Cheng, Wei Wei, Cho-Jui Hsieh. Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019.

long paper

Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data , Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael I. Jordan. Journal of Machine Learning Research (JMLR), 2020.

Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples , Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, Cho-Jui Hsieh. 2018.

Code

Robustness for other ML models (GBDT, KNN, ...)

Provably Robust Metric Learning , Lu Wang, Xuanqing Liu, Jinfeng Yi, Yuan Jiang, Cho-Jui Hsieh. NeurIPS 2020.

An Efficient Adversarial Attack for Tree Ensembles , Chong Zhang, Huan Zhang, Cho-Jui Hsieh. NeurIPS 2020.

On Lp-norm Robustness of Ensemble Decision Stumps and Trees , Yihan Wang, Huan Zhang, Hongge Chen, Duane Boning, Cho-Jui Hsieh. In International Conference on Machine Learning (ICML), 2020.

Evaluating the Robustness of Nearest Neighbor Classifiers: A Primal-Dual Perspective , Lu Wang, Xuanqing Liu, Jinfeng Yi, Zhi-Hua Zhou, Cho-Jui Hsieh. 2019.

Robustness Verification of Tree-based Models , Hongge Chen*, Huan Zhang*, Si Si, Yang Li, Duane Boing, Cho-Jui Hsieh. (* Equal contributio) To appear in NeurIPS, 2019.

Code

A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning. Xuanqing Liu, Si Si, Xiaojin Zhu, Yang Li, and Cho-Jui Hsieh. To appear in NeurIPS 2019.

Robust Decision Trees Against Adversarial Examples , Hongge Chen, Huan Zhang, Duane Boning, Cho-Jui Hsieh. In International Conference on Machine Learning (ICML), 2019.

long talk Code

Fast (Parallel) Training

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes , Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, Cho-Jui Hsieh. In International Conference on Learning Representations (ICLR), 2020.

LAMB in Tensorflow

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh. To appear in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2019.

oral Code

Efficient Contextual Representation Learning Without Softmax Layer , Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang. To appear in TACL, 2019.

Large-batch Training for LSTM and Beyond , Yang You, Jonathan Hseu, Chris Ying, James Demmel, Kurt Keutzer, Cho-Jui Hsieh. The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), 2019.

Fast Deep Neural Network Training on Distributed Systems and Cloud TPUs , Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer. IEEE Transactions on Parallel and Distributed Systems, 2019.

ImageNet Training in Minutes , Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer. In International Conference on Parallel Processing (ICPP), 2018.

Best Paper

Efficient Inference and Model Compression

DRONE: Data-aware Low-rank Compression for Large NLP Models , Patrick CHen, Hsiang-Fu Yu, Inderjit S Dhillon, Cho-Jui Hsieh. In NeurIPS, 2021.

Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search , Li-Cheng Lan, Meng-Yu Tsai, Ti-Rong Wu, I-Chen Wu, Cho-Jui Hsieh To appear in AAAI 2021.

Clustering and Constructing User Coresets to Accelerate Large-scale Top-K Recommender Systems , Jyun-Yu Jiang*, Patrick H. Chen*, Cho-Jui Hsieh, Wei Wang (* Equal Contribution). In Proceedings of the World Wide Web Conference (WWW), 2020.

MulCode: A Multiplicative Multi-way Model for Compressing Neural Language Model , Yukun Ma*, Patrick H. Chen* and Cho-Jui Hsieh (* Equal contributio). To appear in EMNLP 2019.

Long Paper

Fast LSTM Inference by Dynamic Decomposition on Cloud Systems , Y. You, Y. He, S. Rajbhandari, W. Wang, C.-J. Hsieh, K. Keutzer, J. Demmel. To appear in ICDM 2019.

Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks , Patrick H. Chen, Si Si, Sanjiv Kumar, Yang Li, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

A Fast Sampling Algorithm for Maximum Inner Product Search , Qin Ding, Hsiang-Fu Yu, Cho-Jui Hsieh. International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.

GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking , Patrick Chen, Si Si, Yang Li, Ciprian Chelba, Cho-Jui Hsieh. In Advances in Neural Information Processing Systems (NIPS), 2018

A Greedy Approach for Budgeted Maximum Inner Product Search , Hsiang-Fu Yu, Cho-Jui Hsieh, Qi Lei, Inderjit S. Dhillon. In Advances in Neural Information Processing Systems (NIPS), 2017

Code

Gradient Boosted Decision Trees for High Dimensional Sparse Output , Si Si, Huan Zhang, Sathiya Keerthi, Dhruv Mahajan, Inderjit Dhillon, Cho-Jui Hsieh. To appear in International Conference on Machine Learning (ICML) 34, 2017.

AutoML

Improved Adversarial Training via Learned Optimizer , Yuanhao Xiong, Cho-Jui Hsieh. In European Conference on Computer Vision (ECCV), 2020.

Stabilizing Differentiable Architecture Search via Perturbation-based Regularization , Xiangning Chen, Cho-Jui Hsieh. In International Conference on Machine Learning (ICML), 2020.

Code

Efficient Neural Interaction Functions Search for Collaborative Filtering , Quanming Yao*, Xiangning Chen*, James T. Kwok, Yong Li, Cho-Jui Hsieh (* Equal Contribution). In Proceedings of the World Wide Web Conference (WWW), 2020.

Overcoming Catastrophic Forgetting by Generative Regularization , Patrick H. Chen, Wei Wei, Cho-jui Hsieh, Bo Dai

Learning to Learn by Zeroth-Order Oracle , Yangjun Ruan, Yuanhao Xiong, Sashank Reddi, Sanjiv Kumar, Cho-Jui Hsieh. In International Conference on Learning Representations (ICLR), 2020.

Recommender Systems and Ranking

Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers , Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James Sharpnack. To appear in NeurIPS 2019.

Learning from Group Comparisons: Exploiting Higher Order Interactions , Yao Li, Minhao Cheng, Kevin Fujii, Fushing Hsieh, Cho-Jui Hsieh. In Advances in Neural Information Processing Systems (NIPS), 2018

Extreme Learning to Rank via Low Rank Assumption , Minhao Cheng, Cho-Jui Hsieh, Ian Davidson. In International Conference on Machine Learning (ICML), 2018.

SQL-Rank: A Listwise Approach to Collaborative Ranking , Liwei Wu, Cho-Jui Hsieh, James Sharpnack. In International Conference on Machine Learning (ICML), 2018.

Code

Large-scale Collaborative Ranking in Near-Linear Time , Liwei Wu, Cho-Jui Hsieh, James Sharpnack. To appear in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2017.

Oral Code

Goal-Directed Inductive Matrix Completion, Si Si, Kai-Yang Chiang, Cho-Jui Hsieh, Nikhil Rao, Inderjit S. Dhillon. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2016.