Networking Seminar: Clipper: A Low-Latency, General-Purpose Prediction Serving System

Topic: 
Clipper: A Low-Latency, General-Purpose Prediction Serving System
Thursday, February 1, 2018 - 3:00pm
Venue: 
Gates 104
Speaker: 
Dan Crankshaw (UC Berkeley)
Abstract / Description: 

Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.

Clipper is an open-source, general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by adopting a modular serving architecture and isolating models in their own containers, allowing them to be evaluated using the same runtime environment as that used during training. Clipper's modular architecture provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model. Further, by abstracting models behind a uniform serving interface, Clipper allows developers to compose many machine-learning models within a single application to support increasingly common techniques such as ensemble methods, multi-armed bandit algorithms, and prediction cascades. In this talk I will provide an overview of the Clipper serving system and then discuss some of our work developing Clipper into an active open-source project.


 

Stanford's NetSeminar is a biweekly seminar covering networking-related topics. Speakers come from academia and industry and the talks are open to the public. When possible, talks will be recorded and posted online. We typically start at 12:15pm at Gates Bld — Room 104, unless otherwise mentioned.

NetSeminar is run by graduate students, and it is generously supported by the Stanford Computer Forum.

Bio:

Dan Crankshaw is a PhD student in the UC Berkeley CS department working in the RISELab. After cutting his teeth doing large-scale data analysis on cosmology simulation data and building systems for distributed graph analysis, he turned his attention to machine learning systems. His current research interests include systems and techniques for serving and deploying machine learning, with a particular emphasis on low-latency and interactive applications.