We have updated our Terms of Service, Code of Conduct, and Addendum.

AWS OpenSearch Service and Cribl

Options
Austinr
Austinr Posts: 1 ✭
in General Discussions

Has anyone connected up an AWS Opensearch Service instance to Cribl? I created a local backend user in my domain and gave it what I thought were good permissions but I still get a 401 error when I test the connection. Can anyone share what they did to get this to work?

Tagged:

Answers

  • Kyle McCririe
    Kyle McCririe Posts: 29 ✭✭
    Options

    Hi, there I have set up Stream to send to Opensearch before. Can you please post a screenshot of the error?

    What destination are you attempting to use?

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod
    Options

    Additional note for others reading this thread: Please note that only local users are supported today in Cribl Stream. IAM role authentication has been requested as an enhancement request under ticket CRIBL-5748.

  • Brendan Dalpe
    Brendan Dalpe Posts: 201 mod
    edited July 2023
    Options

    Hi @Austinr, were you able to resolve your problem? I just tested and was able to send data to an AWS hosted OpenSearch deployment.

    How I configured my instance:

    1. Created OpenSearch internal user cribl-workers.

    2. Created new OpenSearch role cribl-stream and mapped the cribl-workers user to the role.

    3. For role permissions, I granted:
      a. indices:data/write/bulk for Cluster permissions
      b. create_index and write under Index permissions mapped to my index pattern my-index-*
      08b40959298db8ee399444af1f8b86c55f7a22b8.png
    4. Added a new Elasticsearch destination in Cribl Stream. I entered my Domain endpoint followed by /_bulk as the API URL.
      6109ea3084bf01295e879ae2b83995c4e70f097d.png
    5. After Commit & Deploy, I ran the test and saw data in my ES instance after adding an Index mapping.
      7e4cf2595599f5e890b4c5a2dc2dadef40e12bd7.jpeg

Categories