How to properly distribute UDP syslog packets across worker nodes and processes when using F5 LBs?

An F5 load-balancer was setup and should be sending syslog traffic to some Cribl Stream workers using round-robin. Currently there is a single source of events, for testing purposes. The events are almost exclusively going to a single worker, which is unexpected.

The expectation is that the events would be load-balanced and that there would be a fair distribution across the worker nodes. According to F5 team, the connection gets load-balanced, so as more sources added, traffic will start showing on the other workers. That is to be expected but since UDP syslog is being used the events from a single source should already getting load balanced.

With UDP every event is typically a separate payload sent using a different UDP socket. F5 has the following document on how to load balance UDP packets individually.

K3605: Configuring the BIG-IP system to load balance UDP packets individually

1 UpGoat

Note that I’ve run into throughput limitations when using DatagramLB approach outlined in K3605 at ~8keps from a single sender. Not sure if it was environment specific or a more general general limitation, but definitely make sure to stress test.