Cluster in containers across multiple hosts (Docker and Podman)

This guide describes how to configure a cluster where members run in containers on different hosts using Docker or Podman.

Overview

When running cluster in containers across multiple hosts, the default container network (bridge) is not sufficient for cluster communication. Containers cannot reach each other using their internal container IP addresses.

Instead, cluster members must communicate using addresses that are reachable between hosts.

Cluster supports this by:

• Using TCP/IP discovery (not multicast)
• Configuring host-level addresses (IP or DNS)
• Setting a public address that other members can connect to

Multicast-based discovery does not work across multiple hosts and must be disabled.

---

Network requirements

Ensure the following before configuring cluster:

• Each host can reach the other hosts over the network
• Port 5701 (or configured port) is open between hosts
• Container ports are published to the host
• Firewalls allow inbound/outbound traffic on the configured ports

---

Configuration principles

Use host addresses (prefer IP addresses over DNS names)

Container IPs in Docker and Podman bridge networks are not reachable from other hosts.

You must use:

• Host IP addresses (e.g. 192.168.x.x)
• Or DNS names that resolve consistently between hosts

Hazelcast supports both hostnames and IP addresses for TCP/IP member discovery.

However, in static multi-host container deployments, IP addresses are generally preferred because they:

• Are explicit and predictable
• Do not depend on external name resolution
• Avoid issues with DNS caching, multiple records, or misconfiguration

DNS names can be used if:

• They resolve reliably between all hosts
• Each name maps to exactly one node
• There is no load balancing or round-robin behavior

---

Set public-address

By default, Hazelcast advertises its internal container address, which is incorrect in multi-host setups.

You must override this using:

<public-address>HOST_IP:PORT</public-address>

This ensures other members connect to the correct endpoint.

---

Use TCP/IP join

Disable multicast and configure static member discovery:

<join>
    <multicast enabled="false"/>
    <tcp-ip enabled="true">
        <member-list>
            <member>192.168.1.10:5701</member>
            <member>192.168.1.11:5701</member>
        </member-list>
    </tcp-ip>
</join>

It is recommended to include the port explicitly, even if all nodes use the same port.

---

Example configuration

Host A

<hazelcast>
  <network>
    <port auto-increment="false">5701</port>
    <public-address>192.168.1.10:5701</public-address>

    <join>
      <multicast enabled="false"/>
      <tcp-ip enabled="true">
        <member-list>
          <member>192.168.1.10:5701</member>
          <member>192.168.1.11:5701</member>
        </member-list>
      </tcp-ip>
    </join>
  </network>
</hazelcast>

---

Host B

<hazelcast>
  <network>
    <port auto-increment="false">5701</port>
    <public-address>192.168.1.11:5701</public-address>

    <join>
      <multicast enabled="false"/>
      <tcp-ip enabled="true">
        <member-list>
          <member>192.168.1.10:5701</member>
          <member>192.168.1.11:5701</member>
        </member-list>
      </tcp-ip>
    </join>
  </network>
</hazelcast>

---

Verification

When the cluster starts, logs should show:

• Members joining using host IPs
• No reliance on container internal IPs for cluster communication

Example:

Members [2] {
  Member [192.168.1.10]:5701
  Member [192.168.1.11]:5701
}

---

Common issues

Members cannot join cluster

• Incorrect public-address
• Ports not exposed or blocked
• Using container IP instead of host IP

Split-brain cluster

• Mixing container IPs and host IPs
• Incorrect member list configuration

Logs show container IP

• Missing public-address configuration

---

Summary

• Do not use container (Docker/Podman) IPs across hosts
• Use host IP or DNS + port
• Prefer IP addresses in static environments
• Always set public-address
• Use TCP/IP join, not multicast

This ensures that all cluster members can reliably discover and connect to each other across multiple hosts.