Zhou Ren (任洲) Senior Research Scientist (CV, Google Scholar, Linkedin) Snap Inc., Venice, CA Email: zhou.ren [at] snapchat.com Contact me if you are interested in internship@Snap.

About me

• Zhou is a technical innovator, actively seeking solutions to build and enhance real-world products. At Snap, Zhou has been leading efforts of applying visual understanding to support 1). creative content creation, 2). content monetization, and 3). content security. He is Research Tech Lead of two major production innovations, which handle multi-modal signals such as visual, textual, and audio inputs; involve many research scientists and engineers; cooperate with multiple engineering teams; and are building applications on both frontend and backend for Snap.

• Zhou is an active researcher. His research interests include Computer Vision, Multimedia, Natural Language Processing and Machine Learning. He have worked on problems of multi-modal joint understanding, video understanding, image captioning, hand gesture understanding, object detection, reinforcement learning, and adversarial machine learning, etc. He received his Ph.D. degree in Computer Science from University of California, Los Angeles (UCLA) in 2016, and a M.Eng degree from Nanyang Technological University (NTU) in 2012. Before that, he received his Bachelor’s degree with highest honor from Huazhong University of Science and Technology (HUST) in 2010.

• Selected honors: (1) Runner-up winner in NIPS 2017 Adversarial Attack and Defense Competition (among 107 teams); (2). the leading author of a work been nominated to the “CVPR 2017 Best Student Paper Award”; (3). the leading author of the paper winning the “IEEE Trans. on Multimedia 2016 Best Paper Award”; (4). He has developed the first part-based hand gesture recognition system using Kinect sensor with Nanyang Technological University and Microsoft Research Redmond (Demo1, Demo2).

News

• [07/18] Been invited to attend a panel discussion at ICME 2018, together with Dr. Tao Mei, Dr. Wenjun Zeng, Prof. Xilin Chen, Prof. Mohan Kankanhalli, and Prof. Junsong Yuan.

• [07/18] One paper accepted by ACM Multimedia 2018 as an Oral! Two papers accepted by ECCV 2018. One paper accepted by ICLR 2018. And a book chapter published in Springer on the topic of Adversarial Attacks and Defenses.

• [11/17] We have won the 2nd place in NIPS’17 Adversarial Defense Challenge among 107 teams; and were invited to present in NIPS’17 Competition Track (for details, please check our work).

• [08/17] Our work on hand gesture recognition is ranked the most cited paper in IEEE TMM since 2011. Our TWO works on hand gesture recognition are ranked the most cited paper && the 2nd most cited paper in ACM MM 2011.

• [07/17] Our paper on Reinforcement Learning-based Image Captioning has been nominated to the “Best Student Paper Award” at CVPR 2017!

• [03/17] One paper accepted by CVPR 2017 as an Oral. And one paper accepted by BMVC 2017 as an Oral.

• [06/16] Our paper “Robust Part-based Hand Gesture Recognition Using Kinect Sensor” has been selected to receive the 2016 IEEE Trans. on Multimedia Prize Paper Award (TMM Best Paper Award)!

Research

• My research interests lie in the fields of Computer Vision, Multimedia, Natural Language Processing, and Machine Learning. I have worked on multi-modal joint understanding, image captioning, video captioning, hand gesture understanding, object detection, shape understanding, reinforcement learning, and adversarial machine learning, etc.

• My current research focuses include: 1) video understanding, 2) multi-modal joint understanding, 3) hand pose estimation, and 4) adversarial machine learning.

Publications

(Note: “^” indicates the co-author is the student I mentored during whose internship or during an university collaboration)

1. on Multi-modal Joint Representation Learning

2. on Hand Gesture Recognition and Pose Estimation

3. on Object Detection and Representation Learning

4. on Adversarial Machine Learning

5. on Shape Representation and Shape Coding

6. on Medical Image Processing

Selected Patents

• System and method for robust hand gesture recognition using commodity depth sensor, Singapore provisional patent application, filed in 10/2011
  Co-invented with Junsong Yuan, Jingjing Meng

• Modeling semantic concepts in an embedding space as distributions, US patent application, filed in 01/2016
  Co-invented with Hailin Jin, Zhe Lin, and Chen Fang

• Embedding-driven image captioning using deep reinforcement learning and lookahead beam search, US patent application, filed in 11/2016
  Co-invented with Xiaoyu Wang, Ning Zhang, Xutao Lv, and Jia Li

• Generating Data in a Messaging System for a Machine Learning Model, US patent application, filed in 12/2017
  Co-invented with Zehao Xue

• Query Matching to Media Collections in a Messaging System, US patent application, filed in 01/2018
  Co-invented with Roger Luo, Sushobhan Nayak, Xinran He, and Christophe Van Gysel

• Device Location based on Machine Learning Classifications, US patent, granted in 05/2018
  Co-invented with Ebony Charlton, Sumant Hanumante, Dhritiman Sagar

• Embedding space for images with multiple text labels, US patent, granted in 07/2018
  Co-invented with Hailin Jin, Zhe Lin, and Chen Fang

Mentored Students

• Lluis Castrejon (2017 Summer), PhD student at MILA, University of Montreal
  

• Zhe Li (2017 Summer), PhD student at University of Iowa
  

• Hongyu Xu (2017 Summer), PhD student at University of Maryland, College Park
  

• Cihang Xie (2017 Fall - 2018 Spring), PhD student at Johns Hopkins University
  

• Sheng Liu (2017 Fall - present), PhD student at The State University of New York at Buffalo
  

• Liuhao Ge (2018 Spring - Summer), PhD student at Nanyang Technological University
  

• Tan Yu (2018 Summer), PhD student at Nanyang Technological University
  

• Shibi He (2018 Summer), PhD student at University of Illinois Urbana-Champaign
  

• Jonghwan Mun (2018 Summer), PhD student at Pohang University of Science and Technology.

Teaching

• Teaching Associate of CS32 Introduction to Computer Science II (data structures and algorithms) with Prof. Carey Nachenberg
  [discussion materials]

• Teaching Assistant of CS31 Introduction to Computer Science I (C++ programming) with Prof. David Smallberg
  [discussion materials]

Professional Activities

• Program Committee of CVPR 2017, FG 2018, IJCAI 2018, ECAI 2018, ACM Multimedia 2018, AAAI 2019.

• Reviewer of FG 2016 2017 2018, WACV 2017, CVPR 2016 2017 2018 2019, ICCV 2017, ECCV 2018, IJCAI 2018, etc.

• Reviewer of IEEE Trans on Pattern Analysis and Machine Intelligence; IEEE Trans on Image Processing; IEEE Trans on Circuits and Systems for Video Technology; IEEE Trans on Multimedia; IEEE Trans on Human-Machine Systems; Computer Vision and Image Understanding; Machine Vision and Application; Journal of Computer Science and Technology; etc.