I'm interested in robustness and efficiency of machine learning algorithms. Currently, I am particularly interested in the following topics: Here are my recent publications on these topics. For my full publication please see full publication list .

Robustness of ML Models

Adversarial Attack

Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach , Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

Evaluating Robustness of Deep Image Super-Resolution Against Adversarial Attacks , Jun-Ho Choi, Huan Zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, Jong-Seok Lee. To appear in ICCV 2019.

The Limitations of Adversarial Training and the Blind-Spot Attack , Huan Zhang*, Hongge Chen*, Zhao Song, Duane Boning, inderjit dhillon, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks , Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, Shin-Ming Cheng. In AAAI Conference on Artificial Intelligence (AAAI), 2019.

Genattack: Practical Black-box Attacks with Gradient-free Optimization , Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, Huan Zhang, Cho-Jui Hsieh, Mani B Srivastava. In Proceedings of the Genetic and Evolutionary Computation Conference (Gecco), 2019.

ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models , Pin-Yu Chen*, Huan Zhang*, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh (* Equal contribution). ACM Conference on Computer and Communications Security (CCS) Workshop on Artificial Intelligence and Security (AISec), 2017.

Robustness Verification

A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks , Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, Pengchuan Zhang. To appear in NeurIPS 2019.

RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications , Huan Zhang, Pengchuan Zhang, Cho-Jui Hsieh. In AAAI Conference on Artificial Intelligence (AAAI), 2019.

Efficient Neural Network Robustness Certification with General Activation Functions , Huan Zhang*, Lily Weng*, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel. (* Equal contribution). In Advances in Neural Information Processing Systems (NIPS), 2018

Towards Fast Computation of Certified Robustness for ReLU Networks , Tsui-Wei Weng*, Huan Zhang*, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit Dhillon, Luca Daniel. (* Equal contribution) In International Conference on Machine Learning (ICML), 2018.


Adversarial Defense

ML-LOO: Detecting Adversarial Examples with Feature Attribution , Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael I. Jordan. In AAAI Conference on Artificial Intelligence (AAAI), 2020.

Convergence of Adversarial Training in Overparameterized Networks , Ruiqi Gao, Tianle Cai, Haochuan Li, Liwei Wang, Cho-Jui Hsieh, Jason D. Lee. To appear in NeurIPS 2019.

Towards Stable and Efficient Training of Verifiably Robust Neural Networks , Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Duane Boning, Cho-Jui Hsieh. 2019.

Rob-GAN: Generator, Discriminator, and Adversarial Attacker , Xuanqing Liu, Cho-Jui Hsieh. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network , Xuanqing Liu, Yao Li, Chongruo Wu, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

Towards Robust Neural Networks via Random Self-ensemble , Xuanqing Liu, Minhao Cheng, Huan Zhang, Cho-Jui Hsieh. In European Conference on Computer Vision (ECCV), 2018.

Adversarial Attack and Defense for NLP Models

Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples , Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, Cho-Jui Hsieh. In AAAI Conference on Artificial Intelligence (AAAI), 2020.

On the Robustness of Self-Attentive Models , Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, Cho-Jui Hsieh. To appear In Proceedings of Association for Computational Linguistics (ACL), 2019.

Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent , Minhao Cheng, Wei Wei, Cho-Jui Hsieh. Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019.

Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data , Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael I. Jordan. 2018.

Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples , Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, Cho-Jui Hsieh. 2018.

Robustness for other ML models (GBDT, KNN, ...)

Evaluating the Robustness of Nearest Neighbor Classifiers: A Primal-Dual Perspective , Lu Wang, Xuanqing Liu, Jinfeng Yi, Zhi-Hua Zhou, Cho-Jui Hsieh. 2019.

Robustness Verification of Tree-based Models , Hongge Chen*, Huan Zhang*, Si Si, Yang Li, Duane Boing, Cho-Jui Hsieh. (* Equal contributio) To appear in NeurIPS, 2019.

A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning. Xuanqing Liu, Si Si, Xiaojin Zhu, Yang Li, and Cho-Jui Hsieh. To appear in NeurIPS 2019.

Robust Decision Trees Against Adversarial Examples , Hongge Chen, Huan Zhang, Duane Boning, Cho-Jui Hsieh. In International Conference on Machine Learning (ICML), 2019.


Fast (Parallel) Training

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes , Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Cho-Jui Hsieh. 2019.

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh. To appear in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2019.

Efficient Contextual Representation Learning Without Softmax Layer , Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang. To appear in TACL, 2019.

Large-batch Training for LSTM and Beyond , Yang You, Jonathan Hseu, Chris Ying, James Demmel, Kurt Keutzer, Cho-Jui Hsieh. The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), 2019.

Fast Deep Neural Network Training on Distributed Systems and Cloud TPUs , Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer. IEEE Transactions on Parallel and Distributed Systems, 2019.

ImageNet Training in Minutes , Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer. In International Conference on Parallel Processing (ICPP), 2018.

Efficient Inference and Model Compression

MulCode: A Multiplicative Multi-way Model for Compressing Neural Language Model , Yukun Ma*, Patrick H. Chen* and Cho-Jui Hsieh (* Equal contributio). To appear in EMNLP 2019.

Fast LSTM Inference by Dynamic Decomposition on Cloud Systems , Y. You, Y. He, S. Rajbhandari, W. Wang, C.-J. Hsieh, K. Keutzer, J. Demmel. To appear in ICDM 2019.

Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks , Patrick H. Chen, Si Si, Sanjiv Kumar, Yang Li, Cho-Jui Hsieh. International Conference on Learning Representations (ICLR), 2019.

A Fast Sampling Algorithm for Maximum Inner Product Search , Qin Ding, Hsiang-Fu Yu, Cho-Jui Hsieh. International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.

GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking , Patrick Chen, Si Si, Yang Li, Ciprian Chelba, Cho-Jui Hsieh. In Advances in Neural Information Processing Systems (NIPS), 2018

A Greedy Approach for Budgeted Maximum Inner Product Search , Hsiang-Fu Yu, Cho-Jui Hsieh, Qi Lei, Inderjit S. Dhillon. In Advances in Neural Information Processing Systems (NIPS), 2017

Gradient Boosted Decision Trees for High Dimensional Sparse Output , Si Si, Huan Zhang, Sathiya Keerthi, Dhruv Mahajan, Inderjit Dhillon, Cho-Jui Hsieh. To appear in International Conference on Machine Learning (ICML) 34, 2017.

Recommender Systems and Ranking

Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers , Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James Sharpnack. To appear in NeurIPS 2019.

Learning from Group Comparisons: Exploiting Higher Order Interactions , Yao Li, Minhao Cheng, Kevin Fujii, Fushing Hsieh, Cho-Jui Hsieh. In Advances in Neural Information Processing Systems (NIPS), 2018

Extreme Learning to Rank via Low Rank Assumption , Minhao Cheng, Cho-Jui Hsieh, Ian Davidson. In International Conference on Machine Learning (ICML), 2018.

SQL-Rank: A Listwise Approach to Collaborative Ranking , Liwei Wu, Cho-Jui Hsieh, James Sharpnack. In International Conference on Machine Learning (ICML), 2018.

Large-scale Collaborative Ranking in Near-Linear Time , Liwei Wu, Cho-Jui Hsieh, James Sharpnack. To appear in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2017.

Goal-Directed Inductive Matrix Completion, Si Si, Kai-Yang Chiang, Cho-Jui Hsieh, Nikhil Rao, Inderjit S. Dhillon. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2016.