Verifying Network-level Properties for Large-scale Networks with Header Transformations in Realtime

Abstract

In current large-scale networks (e.g., datacenter networks), packet forwarding is dynamically customized beyond traditional shortest-path routing to meet various application demands. Such forwarding behavior is tremendously complex to manage and sometimes causes serious network failures. We present Graft, a new realtime data plane verification framework to verify complex forwarding behavior on large-scale networks. For scalable realtime verification, we first propose an optimized algorithm to efficiently compute and manage large packet header spaces and their forwarding paths. Second, we propose a data plane model and algorithms with formal network semantics to precisely model the customized forwarding behavior. We validate its effectiveness using synthetic and production datacenter networks. To the best of our knowledge, we are the first to verify customized forwarding behavior in production large-scale networks. For scalability, we show that Graft is 100x faster than prior works in the synthetic networks and 20000x faster in the production network. For expressiveness, we demonstrate that Graft is enough to model the customized forwarding behavior by verifying the correctness of SRv6-based SFCs in the production network. Finally, we demonstrate that Graft verifies a real failure of a distributed NAT system in the production network.