(NetFlow Filtering at North Carolina State University – Case Study) North Carolina State University (NC State), the largest university in North Carolina, with more than 35,000 students and 8,000 faculty and staff, is a comprehensive university known for its leadership in interdisciplinary innovation. The university is one of the nation’s preeminent research institutions and enrolls accomplished, high-performing students from around the world.
NC State Challenges
By extension of NC State’s success, the university’s IT network has seen significant growth, both in size and complexity. The result is a meaningful increase in telemetry and log data that creates unique cost and scalability challenges related to their network management applications and log collectors.
The challenges facing NC State:
- Flow telemetry data outpacing the license threshold of the management applications
- Increase in compute resources to handle the growth in flow telemetry data
- Duplicate and superfluous flow data
- Network and device performance implications due to complicated flow exporter configurations.
The growth in NetFlow traffic can be especially problematic. The collection and monitoring of network traffic flow data generated by flow-enabled devices is often a nuanced and complicated endeavor. The leading NetFlow analyzers require a Flow Rate License (FRL) for the collection, management, and analysis of flow telemetry. The FRL defines the volume of flows that may be collected and is licensed based on flows per second (fps).
Calculating and managing flow capacity is a difficult challenge for NC State, as it is for many organizations, which often results in exceeding the FRL threshold for the management application(s). The simple solution for NC State would be to increase their license threshold and compute resources to keep up with the growth in NetFlow traffic. Consequently, this has a significant effect on cost and resources.
An alternative solution is to create a more nuanced, yet complicated, flow exporter configuration on the flow-enabled devices. Since a significant amount of NC State’s flow data is considered unnecessary and/or redundant, creating additional flow exporters that define only the specific flow data for each flow analyzer is possible. However, anyone who has spent time managing NetFlow recognizes that this can have a devastating impact on device and network performance.
None of these options were practical for NC State. What was needed was a solution that can filter out unnecessary flow data prior to being sent to NC State’s flow analyzers. While this may appear to be simple, scrubbing unwanted flow telemetry data from NetFlow packets is no easy task.
A NetFlow packet consists of a version-dependent header along with one or more FlowSet records (both data and template). The NetFlow packet header contains the NetFlow version, number of FlowSet records, sequence numbering for the detection of lost packets, source ID to guarantee uniqueness for all flows, and date/time stamp.
An effective solution would need to collect NetFlow packets from any number of flow exporters. Then, remove FlowSet data based on user defined filters. And finally, update and modify the appropriate header fields, reassemble the packet, and forward it to the appropriate NetFlow analyzer. All of this, and at a cost that can provide savings over the aforementioned alternatives.
Tavve’s Solution
Following in-depth research and market assessment, NC State determined that Tavve’s PacketRanger met the solution criteria needed to solve their NetFlow and Flow Based License limitations. Tavve’s PacketRanger is an out-of-band UDP traffic broker that serves as a central collector for NC State’s flow telemetry data generated by flow-capable devices. NC State chose PacketRanger for its ability to filter and broker NetFlow packets (v5, v9, v10/IPFIX).
NC State uses PacketRanger to implement named sets of user-defined conditions to remove FlowSet data from a NetFlow packet before being forwarded to their NetFlow analyzers. This is accomplished by creating one or more filter conditions. The conditions define the FlowSet data to be removed from NC State’s NetFlow packets. Each condition is a combination of one or more filter criteria.
NetFlow filter behaviour is defined as removing flows that match any condition within the filter. The following FlowSet data conditions can be used in the creation of NetFlow filters:
- Source Address
- Source Port
- Destination Address
- Destination Port
- Protocol
- Template ID
A simple filter condition example of a NetFlow filter: Remove All Flows from Source Addresses 10.23.4.[10-100] and Destination Port 53. After creating a NetFlow filter, it can be assigned to NetFlow Forwarding Rules, where the PacketRanger will broker the NetFlow packet to its intended destination(s).
While filtering on specific FlowSet data is a critical requirement, NC State also needed PacketRanger’s capability to broker NetFlow packets from flow-capable devices and forward those packets to any number of management applications. NetFlow packets are processed against user-defined forwarding rules. Each NetFlow packet is tested against the NetFlow forwarding rules to determine how the packet is processed. The PacketRanger will always verify that the NetFlow packet is valid, as indicated in the packet header, and that the packet is from a known device defined in one or more of the forwarding rules source criteria. Forwarding rule processing works to create a unique set of destinations for the forwarded packet, sending only the necessary FlowSet data to the appropriate destination.
The Impact
At the time that NC State contacted Tavve, PacketRanger did not fully comply with NC State’s requirements. But through a rapidly developing partnership, Tavve built the desired functionality for NC State within two months, which was followed by a successful proof of concept phase.
NC State benefits from leveraging PacketRanger:
- More visibility into their network telemetry data
- Savings on Flow Rate License costs
- Optimized compute resources and management application performance
- Reduced device and network performance degradation due to NetFlow overhead
- Simplified NetFlow exporter configurations
- Centralized policy-based routing of NetFlow
- Easier troubleshooting and root cause analysis for NetFlow issues with PacketRanger’s Statistical Analysis
“The entire engagement with Tavve was refreshing. Tavve listened to what we requested instead of prescribing what we must have, while consulting us along the way. Once the requirements were clear, Tavve’s engineering built exactly what we needed.”
Neal McCorkle, Security Architect – North Carolina State University
To learn more, contact us either via www.tavve.com or email at sales@tavve.com