Datacenter servers running latency-critical online applications such as web search and social network are often underutilized as they are (over)provisioned for unpredictable peak service demand. This in turn hurts energy efficiency and thus the total cost of ownership (TCO) due to poor energy proportionality of contemporary servers. To improve energy efficiency, we may consider to deploy aggressive power management policies and run latency-critical applications with latency-agnostic, throughput applications. Such approaches, however, prone to violations of service-level agreement (SLA) whenever servers need to change performance and power states, which prevents servers from performing any tasks for a notable amount of time, and co-running throughput applications generate block I/O requests, which severely interferes with memory requests from latency-critical applications at the memory subsystem.
In this talk, tackling these challenges, I will present two techniques dubbed NCAP and CLAUD. More specifically, NCAP and CLAUD exploit the fact that the rate of network packets encapsulating requests from clients can significantly affect the utilization of datacenter servers. Thus, we propose to enhance network interface cards (NICs) and NIC drivers to measure the rate of latency-critical network packets and send special interrupts to the operating system when the rate exceeds certain values. The special interrupts allow (1) NCAP to preemptively change performance and power states of servers and (2) CLAUD to proactively throttle block I/O requests from co-running throughput applications. NCAP and CLAUD in turn allow datacenter operators to more aggressively deploy power management policies and/or run latency-critical applications with throughput applications without violating SLAs.
Nam Sung Kim is an IEEE Fellow and an Associate Professor at the University of Illinois, Urbana-Champaign and, prior to joining the University of Illinois in the fall of 2015, he was an Associate Professor at the University of Wisconsin, Madison where he was early-tenured in 2013. His interdisciplinary research incorporates device, circuit, architecture, and software for power-efficient computing. Prior to joining the University of Wisconsin, Madison, he was a senior research scientist at Intel from 2004 to 2008, where he conducted research in power-efficient digital circuit and process architecture. He has published nearly 160 refereed articles to highly-selective conferences and journals in the field of digital circuit, processor architecture, and computer-aided design. The top three most frequently cited papers have more than 3500 citations and the total number of citations of all his papers exceeds 6800. He is a recipient of the IEEE Design Automation Conference (DAC) Student Design Contest Award in 2001, Intel Fellowship in 2002, IEEE International Symposium on Microarchitecture (MICRO) Best Paper Award in 2003, NSF CAREER Award in 2010, IBM Faculty Award in 2011 and 2012, and University of Wisconsin Villas Associates Award in 2015. I am a member of IEEE International Symposium on High-Performance Computer Architecture (HPCA) Hall of Fame and IEEE International Symposium on Microarchitecture (MICRO) Hall of Fame.