We have updated our Terms of Service, Code of Conduct, and Addendum.

Backpressure/Blocking when PQ Drained and Destination Healthy

padilla
padilla Posts: 7 mod
edited March 2023 in Stream

Our destination has PQ enabled. When the destination is accepting data again we are seeing error messages in the logs that the there's blocking when it is set to drop. Why?

{
    "level": "warn",
    "message": "begin backpressure [blocking]",
    "reason": {
        "message": "failed to push to persistent queue due to backpressure: in backpressure mode",
        "stack": "PQBackpressureError: failed to push to persistent queue due to backpressure: in backpressure mode\n at /data/cribl/opt/bin/cribl.js:14:12494168",
        "reason": "output is experiencing increased load",
        "name": "PQBackpressureError",
        "response": {
            "shouldBackpressure": true,
            "reason": "in backpressure mode",
            "forceBlock": true
        }
    }
}

Answers

  • Megan Davis
    Megan Davis Posts: 10 ✭

    This happens when PQ is working on clearing out the data thats been written to disk when the destination has been blocked.

    Then Cribl will read the data off of disk and put it back into Stream to be sent to the destination.

    There is a point where Cribl will stop the data flow completely to read all the data that is currently on disk and then will start processing data from Stream again.

    When Cribl stops the data flow in order to clear the PQ data from disk; this is what is causing the error message that is being seen in the log files about PQ blocking when it is clearing

Categories