A Report on NSDI'13:SoNIC: Precise Realtime Software Access and Control of Wired Networks

This is a summary of the presentation by Ki Suh Lee and the question and answer section that followed. If I have misattributed a question, let me know in the comments and I will fix.

Summary

Measuring and controlling interpacket delays can enable better understanding of network characteristics and new applications. For example, measuring these interpacket delays can give better characterization of network traffic, and controlling the interpacket delays can create new covert channels. Unfortunately, current techniques are insufficient to measure these interpacket delays precisely. However, the precision of network measurements could be improved with access to the PHY layer: Counting the number of PHY layer idle characters between packets can give network research applications sub-nanosecond precision.

Manipulating idle characters requires accessing the PHY, which is currently a black box that hides information (including idle character counts). One approach could use something like BiFocals, which uses physics equipment to measure the PHY layer. Unfortunately, this equipment is very expensive ($500k) and only works off-line. Since there is limited access to hardware, the authors propose using software to count idle characters.

The authors implement their approach as a new platform called SoNIC, which ports some of the PHY layer functionality of a 10 Gbps Ethernet link into software.  Specifically, SoNIC ports all of the functionality that manipulates bits into software (everything above the scrambler of the physical coding sublayer), but keeps the functions that translate bits into signals into hardware. This split requires high-performance software, which SoNIC implements using three techniques.  (1) SoNIC software threads are pinned to an individual CPU core. (2) Use polling and an optimized DMA rather than interrupts to interface with the network device. (3) Tightly optimize software (e.g., replace loops with bitwise operators).

These techniques give SoNIC precise measurement and control of interpacket delay and interpacket gaps, which allowed the authors to implement several functions that were previously impossible (including a new timing channel with data rates up to 250 kbps that is undetectable by current techniques).

Question and answer

Dong Zhou, Carnegie Mellon University

DZ: Are there any limitations to applications you can support? SoNIC seems to require a lot of CPU.

KSL: Yes, the applications must be able to work with data faster than 10 Gbps using the remaining resources of the system.

Nikita Borisov, University of Illinois Urbana-Champaign

NB: It's cool that your timing channel attack works even if there are unmodified routers along the path. Did you consider cross traffic while evaluating this attack?

KSL: Yes, we did, but our paper uses only unloaded routers to demonstrate feasibility. We're currently looking at implementing this attack on a real network with cross traffic.

Hongyi Zeng, Stanford University

HZ: This is a cool use of FPGAs. What are the requirements for an FPGA to implement SoNIC?

KSL: The FPGA must have transceivers that can support more than 10.3 Gbps.

HZ: You use a CDF when reporting the interpacket gap. Why is this a CDF, what could cause errors in the hardware.

KSL: SoNIC has no error because it's timestamping at the lowest layer.  Even within the kernel there's a lot of overhead: other tasks, interrupts, etc.

HZ: What about hardware timestamps (say in the NIC)?

KSL: Hardware clocks have lower resolution than our techniques. What's cool about SoNIC is that we get really precise timing from counting the idle characters.

Junda Liu, Google

JL: Every port has 5 dedicated kernel threads, does this require 5 cores?

KSL: Yes, we pin each thread to its own core, but the other cores are shared. More CPU intensive applications (e.g., full packet capture which requires disk access) are impossible right now because of application requirements (must handle > 10 Gbps of data)