Our research is generously supported by various of funding agencies, including National Science Foundation (NSF), Army Research Office (ARO), DARPA, IARPA, and Office of Naval Research (ONR), industry partners such as Google and IBM, as well as philoanthropic foundations including Alfred P. Sloan Foundation.

  • image

    Machine learning for life and medical sciences

    My research group will be spearheading new projects and initiatives in the direction of inventing and applying machine learning and other artificial intelligence technology to challenging problems in life and medical sciences.

    If you are interested in joining our efforts, please contact me. We are particularly interested in researchers and graduates who either have interdisciplinary background or are not afraid to learn and to explore in this exciting area.

  • image

    Vision + Language

    My research group will be undertaking new projects in the intersections of Computer Vision and Natural Language Processing.

    Visual perception and languages (either spoken or written) are the major means for human to communicate to each other. Scientifically fascinating questions arise by looking at them together: given an image, what kind of knowledge can we extract from it? How can we express this knowledge precisely to another person who has not seen the image? Given a document, can we automatically generate a picture book such that another person who has not read the texts can understand what is intended to communicate? Such problems (and many similar ones) require us to examine the intricate and fundamental link between these modalities. The advances in machine learning and artificial intelligence, as well as the ever-increasing amount of data, have enabled us to study those problems more thoroughly and fruitfully.

    If you are interested in joining our efforts, please contact me. We are particularly interested in resaerchers and graduates who either have interdisciplinary background or are not afraid to learn and to explore in this exciting area.

  • image

    Zero-shot learning

    Is it possible for computer vision systems to recognize visual object categories that they have never seen before? More precisely, in the paradigm of zero-shot learning, a learner has access to only a subset of the labels in the labeling space (and its associated exemplar images). Nonetheless, our goal for the learner is to recognize future occurrences of images from all possible categories. This is an important research problem with great application potential for automatic object recognition in the wild where the number of possible visual categories continuously rises and there is little hope to collect adequate labeling samples for those categories fast enough.

    In this project, we develop new learning methods on tackling this challenge. We have demonstrated that it is possible to train vision systems on the ImageNet images from 1,000 visual categories yet attaining meaningful results on recognizing a disjoint set of 20,000 visual categories.

    Selected References

    • image

      Synthesized classifiers for zero-shot learning.

      Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, and Fei Sha. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, June 2016. (Oral)

    • image

      An empirical study and analysis of generalized zero-shot learning for object recognition in the wild

      Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha, . Proceedings of the European Conference on Computer Vision (ECCV), Oct 2016 (Spotlight)

    • image

      Predicting visual exemplars of unseen classes for zero-shot learning

      Soravit Changpinyo, Wei-Lun Chao, and Fei Sha. arXiv 2016

  • image

    Video summarization

    Video has rapidly become one of the most common sources of visual information. The amount of video data is daunting --- it takes over 82 years to watch all videos uploaded to YouTube per day! Automatic tools for analyzing and understanding video contents are thus essential. In particular, automatic video summarization is a key piece in helping human users by compactly representing videos without losing important information.

    In this project, we have developed several new supervised learning methods for video summarization. While video summarization has conventionally been addressed as unsupervised learning, we have demonstrated that supervised learning techniques are significantly advantageous --- by framing the summarization process as an instance of structured prediction, supervised learning models can mimick more accurately how human annotators decide on what to include in summaries, thus attaining far better summarization results.

    Selected References

    • image

      Diverse sequential subset selection for supervised video summarization

      Boqing Gong, Wei-Lun Chao, Kristen Grauman, and Fei Sha. Proceedings of Nueral Information Processing Systems (NIPS), Lake Tahoe, Dec 2014.

    • image

      Summary transfer: exemplar-based subset selection for video summarization,

      Ke Zhang, Wei-Lun Chao, Fei Sha, and Kristen Grauman. Proceedings of the IEEE Conference on Computer Vision and Pattern Recongition (CVPR), Las Vegas, June 2016

    • image

      Video summarization with long short-term memory

      Ke Zhang, Wei-Lun Cha*, Fei Sha, and Kristen Grauman. Proceedings of the European Conference on Computer Vision (ECCV), Amseterdam, Oct. 2016

  • image

    Domain adaptation for visual object recognition

    Statistical learning algorithms often assume the training data and the test data are drawn from the same (unknown) distribution. While this assumption facilitates rigorous theoretical analysis and empirical comparison, its validity is often challenged in real-world problems, where the testing environment of a learning agent frequently deviates from the well-controlled setting in which the agent is trained.

    Domain adaptation techniques aim to correct the mismatch between these two conditions. The goal is to adapt the classifiers trained on the source domain to work well in the test (target) domain.

    In this project, we have developed new learning methodologies for domain adaptation. Our work examines and exploits intrinsic structures of the data: manifold, (discriminative) clusters and landmarks (instances connecting the source and target domains). The proposed algorithms attain state-of-the-art performances on standard benchmark tasks.

    Selected References

    • image

      Geodesic Flow Kernel for Unsupervised Domain Adaptation.

      B. Gong, Y. Shi, F. Sha, and K. Grauman. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, June 2012. (Oral)

    • image

      Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation.

      Y. Shi and F. Sha. Proceedings of the International Conference on Machine Learning (ICML), 2012

    • image

      Connecting the Dots with Landmarks: Discriminatively Learning Domain-Invariant Features for Unsupervised Domain Adaptation.

      B. Gong, K. Grauman, and F. Sha. Proceedings of the International Conference on Machine Learning(ICML), Atlanta, GA, June 2013

    • image

      Reshaping Visual Datasets for Domain Adaptation.

      B. Gong, K. Grauman, and F. Sha. Proceedings of the Neural Information Processing Systems (NIPS), Lake Tahoe, NV, Dec. 2013

  • image

    Learning similarity and metrics

    Measuring similarity (or dissimilarity, distance) is of fundamental importance to learning methods.

    In this project, we have developed several metric/similarity learning methods, focusing on how to better characterize the proximity relationship among data points. While conventional methods learn to measure distances in linearly transformed spaces, we study how to do so in nonlinear transformed spaces. We have also studied how to measure similarity when the conventional notion of Euclidean distance does not apply, either due to the violation of triangular inequality or the specific characteristics of data.

    Key References

    • image

      Ordinal metric learning

      Yuan Shi, Wenzhe Li, and Fei Sha. Proceedings of AAAI, Pheonix, March 2016.

    • image

      Similarity learning for high-dimensional sparse data

      Kuan Liu, Aurelien Bellet, and Fei Sha. Proceedings of AISTATS, San Diego, Oct 2015

    • image

      Sparse compositional metric learning

      Yuan Shi, Aurelien Bellet, and Fei Sha. Proceedings of AAAI, Montreal, 2014.

    • image

      Two-stage metric learning

      Jun Wang, Ke Sun, Fei Sha, Stephane Marchand-Maillet, and Alexandros Kalousis. Proceedings of ICML, 2014.

    • image

      Similarity component analysis

      Soravit Changpinyo, Kuan Liu and Fei Sha. Proceedings of NIPS, 2012.

  • image

    Kernel methods and deep learning

    (to be constructed)

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

  • image

    Probabilistic models and computational statistics

    (to be constructed)

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

  • image

    Multi-task learning

    (to be constructed)

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

  • image

    Optimization

    (to be constructed)

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.