CS 239 Fall 2018, Paper List

Distributed Storage Systems

• The Hadoop Distributed File System, K. Shavachko, et al. (MSST 2010)
• The Google File System, S. Ghemawat, et al. (SOSP 2003)
• Bigtable: a distributed storage system for structured data, F. Chang, et al. (OSDI 2006)
• Spanner: Google’s Globally-Distributed Database, Corbett, et al. (OSDI 2012)
• Windows Azure Storage: a highly available cloud storage service with strong consistency, B. Calder, et al.  (SOSP 2011)

 

General-Purpose Data-Parallel Engines

• MapReduce: Simplified Data Processing on Large Clusters, J. Dean and S. Ghemawat (OSDI 2004)
• Dryad: Distributed Data-parallel Programs from Sequential Building Blocks, M. Isard, et al. (EuroSys 2007)
• Hyracks: A flexible and extensible foundation for data-intensive computing, V. Borkar, et al. (ICDE 2011)
• Spark: Cluster Computing with Working Sets, M. Zaharia, et al. (HotCloud 2010)
• Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing, M. Zaharia, et al. (NSDI 2012)
• Distributed aggregation for data-parallel computing: interfaces and implementations, Y. Yu, et al. (SOSP 2009)
• Map-reduce-merge: Simplified relational data processing on large clusters,  H.-C. Yang, et al. (SIGMOD 2007)
• MapReduce online, T. Condie, et al.  (NSDI 2010)
• AsterixDB: A Scalable, Open Source BDMS, S. Alsubaiee, et al. (VLDB 2014)

 

Batch-Processing Languages and Systems

• Hive: a warehousing solution over a map-reduce framework. A. Thusoo, et al. (VLDB 2009)
• Spark SQL: Relational Data Processing in Spark. M. Armbrust, et al., (SIGMOD 2015)
• SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets, R. Chaiken, et al. (VLDB 2008)
• The HaLoop approach to large-scale iterative data analysis. Y. Bu, et al. (VLDB 2012)
• FlumeJava: Easy, Efficient Data-Parallel Pipelines, C. Chambers, et al. (PLDI 2010)
• DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language. Y. Yu, et al. (OSDI 2008)

 

Scheduling, Resource Management, etc.

 

• Improving MapReduce Performance in Heterogeneous Environments, M. Zaharia, et al. (OSDI 2008)
• Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center, B. Hindman, et al. (NSDI 2011)
• REEF: Retainable Evaluator Execution Framework, M. Weimer, et al. (SIGMOD 2015)
• Apache Hadoop YARN: Yet Another Resource Negotiator, V. K. Vavilapallih, et al. (SoCC 2013)
• Sparrow: Distributed, Low Latency Scheduling, K. Ousterhout, et al. (SOSP 2013)
• Retro: Targeted resource management in multi-tenant distributed systems, J. Mace, et al. (NSDI 2015)
• Large-scale cluster management at Google with Borg, A. Verma, et al. (EuroSys 2015)
• Latency-tolerant software distributed shared memory, J. Nelson, et al. (USENIX ATC 2015)
• Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks, H. Li, et al. (SoCC 2014)

 

Streaming Data Processing Systems

 

• Storm@twitter, A. Toshniwal, et al. (SIGMOD 2014)
• Apache Flink™: Stream and Batch Processing in a Single Engine, P. Carbone, et al.
• Kafka: a Distributed Messaging System for Log Processing, J. Kreps, et al. (NetDB 2011)
• Building a Replicated Logging System with Apache Kafka, G. Wang, et al. (VLDB 2015)
• Discretized Streams: Fault-Tolerant Streaming Computation at Scale, M. Zaharia, et al. (SOSP 2013)
• Naiad: a timely dataflow system, D. G. Murray, et al. (SOSP 2013)
• SVE: Distributed Video Processing at Facebook Scale, Q. Huang, et al. (SOSP 2017)
• StreamBox: modern stream processing on a multicore machine, H. Miao, et al. (USENIX ATC 2017)
• Trill: A High-Performance Incremental Query Processor for Diverse Analytics, B. Chandramouli, et al. (VLDB 2014)
• Drizzle: Fast and Adaptable Stream Processing at Scale, S. Venkataraman, et al. (SOSP 2017)
• Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark, M. Armbrust, et al. (SIGMOD 2018)

 

Graph Processing

 

• Pregel: a system for large-scale graph processing, G. Malewicz, et al. (SIGMOD 2010)
• Ligra: a lightweight graph processing framework for shared memory, J. Shun, et al. (PPoPP 2013)
• A lightweight infrastructure for graph analytics, D. Nguyen, et al. (SOSP 2013)
• PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs, J. Gonzalez, et al. (OSDI 2012)
• Gemini: A Computation-Centric Distributed Graph Processing System, X. Zhu, et al. (OSDI 2016)

·       One Trillion Edges: Graph Processing at Facebook-Scale, A. Ching, et al. (VLDB 2015)

·       GraphX: Graph Processing in a Distributed Dataflow Framework, J. Gonzalez, et al. (OSDI 2014)

·       Pregelix: Big(ger) Graph Analytics on A Dataflow Engine, Y. Bu, et al. (VLDB 2014)

·       Scalability! But at what COST?, F. McSherry (HotOS 2015)

·       GraphChi: Large-Scale Graph Computation on Just a PC, A. Kyrola, et al. (OSDI 2012)

·       X-Stream: Edge-centric Graph Processing using Streaming Partitions, A. Roy, et al. (SOSP 2013)

·       GridGraph: Large-Scale Graph Processing on a Single Machine Using 2-Level Hierarchical Partitioning, X. Zhu, et al. (USENIX ATC 2015)

·       Arabesque: a system for distributed graph mining, C. H. C. Teixeira, et al. (SOSP 2015)

·       RStream: Marrying Relational Algebra with Streaming for Efficient Graph Mining on A Single Machine, K. Wang, et al. (OSDI 2018)

 

Systems for Large-Scale Machine Learning

 

• Scaling Distributed Machine Learning with the Parameter Server, M. Li, et al. (OSDI 2014)
• Project Adam: Building an Efficient and Scalable Deep Learning Training System, T. Chilimbi, et al. (OSDI 2014)
• TensorFlow: A System for Large-Scale Machine Learning, M. Abadi, et al. (OSDI 2016)
• A Distributed Framework for Emerging AI Applications, R. Nishihara, et al. (OSDI 2018)
• An Automated End-to-End Optimizing Compiler for Deep Learning, T. Chen, et al. (OSDI 2018)
• Gandiva: Introspective Cluster Scheduling for Deep Learning, W. Xiao, et al. (OSDI 2018)
• PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems, Y. Lee, et al. (OSDI 2018)

 

Memory Management for Big Data Processing

 

·       A bloat-aware design for big data applications, Y. Bu, et al. (ISMM 2013)

·       FACADE: A Compiler and Runtime for (Almost) Object-Bounded Big Data Applications, K. Nguyen, et al. (ASPLOS 2015)

·       Broom: Sweeping Out Garbage Collection from Big Data Systems, I. Gog, et al. (HotOS 2015)

·       Trash Day: Coordinating Garbage Collection in Distributed Systems, M. Maas, et al. (HotOS 2015)

·       Interruptible tasks: treating memory pressure as interrupts for highly scalable data-parallel programs, L. Fang, et al. (SOSP 2015)

·       Taurus: A Holistic Language Runtime System for Coordinating Distributed Managed-Language Applications, M. Maas, et al. (ASPLOS 2016)

·       Yak: A High-Performance Big-Data-Friendly Garbage Collector, K. Nguyen, et al. (OSDI 2016)

·       Simple, fast, and safe manual memory management, P. Kedia, et al. (PLDI 2017)

·       Skyway: Connecting Managed Heaps in Distributed Big Data Systems, K. Nguyen, et al. (ASPLOS 2018)