We have updated our Terms of Service, Code of Conduct, and Addendum.

How long can a worker node run without a leader node?

Options
Michael Hocke
Michael Hocke Posts: 4
edited September 2023 in Stream

I am in charge of documenting our disaster recovery and high-availability efforts. One of the questions that came up is whether the worker nodes will cease to function (or reduce functionality) when the leader node disappears for a prolonged time. If so, how much time do we have until we must resurrect the leader?

Best Answers

  • Jon Rust
    Jon Rust Posts: 435 mod
    Answer ✓
    Options

    Workers will continue receiving data from push sources, and some pull sources will continue working as well, but may not be optimal. Many pull sources (like REST collectors) will stop working as the Leader coordinates their execution across the Group.

    I dont think there is a set limit for how long Workers will run Leaderless. From experience I can say at least a day.

  • Brandon McCombs
    Brandon McCombs Posts: 150 mod
    Answer ✓
    Options

    The limit is "until the worker node is restarted while the leader is still down." otherwise workers continue functioning as Jon stated.

Answers

  • Jon Rust
    Jon Rust Posts: 435 mod
    Answer ✓
    Options

    Workers will continue receiving data from push sources, and some pull sources will continue working as well, but may not be optimal. Many pull sources (like REST collectors) will stop working as the Leader coordinates their execution across the Group.

    I dont think there is a set limit for how long Workers will run Leaderless. From experience I can say at least a day.

  • Michael Hocke
    Michael Hocke Posts: 4
    Options

    How about the license? Will the workers eventually fall back to a free license when the leader is gone? The key information I am looking for is how much time we can run without a leader before the workers change their behavior.

  • Jon Rust
    Jon Rust Posts: 435 mod
    Options

    No, they will continue operating based on the last license info they saw. Im not sure there is a limit here, but as I said from experience, at least a day.

  • Brandon McCombs
    Brandon McCombs Posts: 150 mod
    Answer ✓
    Options

    The limit is "until the worker node is restarted while the leader is still down." otherwise workers continue functioning as Jon stated.