Skip to content

CKS: make the source CIDR of the auto-provisioned node SSH firewall rules configurable #13970

Description

@permyakovsv

As an Operator I would like to be able to restrict the source CIDR of the firewall rules that
CKS provisions for node SSH access, instead of having them permanently opened to 0.0.0.0/0.

Current behaviour

When a Kubernetes cluster is created on an isolated network, CKS provisions ingress firewall
rules on the network's source NAT IP for the node SSH ports (2222 .. 2222 + nodes - 1,
plus one extra rule per external node). The source CIDR is hard-coded:

// plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/
//   actionworkers/KubernetesClusterActionWorker.java
protected void provisionFirewallRules(final IpAddress publicIp, final Account account,
                                      int startPort, int endPort) throws ... {
    List<String> sourceCidrList = new ArrayList<String>();
    sourceCidrList.add("0.0.0.0/0");            // <-- hard-coded
    ...
    cidrField.set(rule, sourceCidrList);
    firewallService.createIngressFirewallRule(rule);
    firewallService.applyIngressFwRules(publicIp.getId(), account);
}

It is reached from addFirewallRulesForNodes(). There is no way to override the value:
KubernetesClusterManagerImpl.getConfigKeys() exposes 15 keys (timeouts, network offering,
max cluster size, etcd start port, ...) and none of them relates to network access, and
CreateKubernetesClusterCmd has no corresponding parameter.

The net effect is that on every CKS cluster deployed on an isolated network, SSH of every
control and worker node is reachable from the whole internet, and the operator has no
supported way to change that.

Why narrowing the rule by hand is not a workaround

Editing the rule does not survive normal lifecycle operations.
KubernetesClusterScaleWorker.scaleKubernetesClusterIsolatedNetworkRules() calls
removeSshFirewallRule(), which matches the rule by port number only:

if (Objects.equals(firewallRule.getSourcePortStart(), CLUSTER_NODES_DEFAULT_START_SSH_PORT)
    || (Objects.nonNull(pfRule) && pfRule.getDestinationPortStart() == DEFAULT_SSH_PORT)) {
    rule = firewallRule;
    firewallService.revokeIngressFwRule(firewallRule.getId(), true);
    break;
}

The narrowed rule is therefore revoked and then recreated with 0.0.0.0/0 by
setupKubernetesClusterIsolatedNetworkRules(). The same happens on
addNodesToKubernetesCluster. Because the public ports of the SSH port-forwarding rules are
fixed by CKS itself, there is no way to express a narrowed rule that this matcher would not
pick up. cidrlist of an existing firewall rule is immutable, so it cannot be edited in place
either.

Prior discussion

This has been raised before, inside bug reports about something else:

  • #11779 describes exactly this scenario —
    the reporter removed the default wide-open rules for security reasons, after which cluster
    scaling failed with ManagementServerException: Firewall rule for node SSH access can't be provisioned. Closed as a duplicate of CKS Firewall bug if port start is "ALL" #11758.
  • In #11758, @weizhouapache confirmed
    (2025-10-01) that "several code lines are based on the assumption that the firewall rules
    and port forwarding rules for SSH (to control/worker nodes) start from port 2222"
    , and when
    asked specifically about the security risk of opening 6443 and 2222–22xx to 0.0.0.0/0,
    answered (2025-10-03): "I understand your concerns. I agree we should improve it. it is not
    a simple fix, please keep eye on this issue"
    .

#12806 then closed #11758. That PR fixed the
robustness side of the problem — a missing or NULL-ported rule no longer throws — but
intentionally left the hard-coded CIDR untouched. As a result the security aspect is currently
not tracked by any open issue, which is why I am opening this one.

Proposed feature

Add a configuration key, for example cloud.kubernetes.cluster.ssh.allowed.cidr, scoped to
Account or Domain, defaulting to 0.0.0.0/0 so that existing behaviour is preserved, and use
its value in provisionFirewallRules() in place of the constant. Optionally expose the same
value as a parameter of createKubernetesCluster so it can be set per cluster.

Two related points worth covering by the same setting:

  • The API-port rule (6443) provisioned for the load balancer / port forwarding has the same
    issue.
  • For VPC-based clusters the equivalent path is createVpcTierAclRules()
    provisionVpcTierAllowPortACLRule(), which builds a CreateNetworkACLCmd without setting
    cidrlist, so getSourceCidrList() falls back to 0.0.0.0/0 and ::/0.

A more thorough alternative would be to stop relying on the public IP for management-server →
node SSH altogether (the shared-network path already uses the node's private address via
getKubernetesClusterServerIpSshPortForSharedNetwork()), but a configurable CIDR would already
remove the immediate exposure with a very small change.

Versions

Observed on 4.22.1.0; the code paths above are unchanged in main.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions