What are the recommendations for configuring my winlogbeat and filebeat agents to send to multiple cribl stream worker nodes? In my environment, I’m currently seeing 1 or 2 of my worker nodes receiving the majority of the data.
Which output are you using? The ElasticSearch output can handle multiple addresses and will load balance across them for you.
1 UpGoat
Mainly ElasticSearch. I’m sending to an elastic input on the worker nodes.
W/o using a load balancer you should ensure all Stream nodes are listed in output.hosts setting so data is sent to all of them. Additionally, be sure to use the loadBalance and worker directives to maximize concurrency sending to Stream. You may need to play with the value for the worker directive to see what your Beats systems can handle. The default is a low value of 1.
2 UpGoats