Using segment in path to eval host

I am currently bringing in a bunch of Apache logs. The sites are divided up in /var/log/httpd (eg. /var/log/httpd/site1/access.log, /var/log/httpd/site2/access.log). It would be amazing to set the base folder, the max depth, the allow list, and the host segment of the path. Is there a way to read in the path of the file to set the host field? Currently doing this in Splunk with the “host_segment” function in the inputs.conf file.

1 UpGoat

You can use templating in your Directory config like this:

Then, when you run the collector, you’ll see site_name populated as a field of the event.

Thank you for the reply. While in Edge, I replaced the path from a working single folder with ${site_name}. When I check in status, I am not seeing any files listed. I am also not seeing any data in the live data section. Is there something I am missing or a different way I should be accomplishing this?

Works:
/var/log/httpd/sitename.com/access.log

Not working yet:
/var/log/httpd/${site_name}/access.log

Ah, I misunderstood what source you’re working with. My first response was for a Filesystem Collector, but since you’re on Edge I’m guessing that you’re using a File Monitor source instead, correct?

The File Monitor source doesn’t support path matching the same way as the Filesystem Collector does. Another way to accomplish this is to use a Pre-processing pipeline on the File Monitor, with a simple Eval pipeline like:

1 UpGoat