Skip to content

Conversation

@PatriiCloud
Copy link

Fixes #790

When HTTP/HTTPS proxy is not configured, empty strings returned by generate_systemd_proxy_config() and generate_apt_proxy_config() were being base64-encoded and causing YAML parsing errors in Cluster API.

The issue occurs because:

  1. utils.generate_systemd_proxy_config() returns "" when no proxy is set
  2. utils.generate_apt_proxy_config() returns "" when no proxy is set
  3. These empty strings get base64-encoded in resources.py
  4. When CAPI decodes them, empty values break cloud-init YAML parsing
  5. Cluster creation fails with validation errors

The fix:

  • Use "or '#'" fallback in resources.py when encoding proxy configs
  • When proxy functions return "", the fallback "#" is used instead
  • "#" is a comment character in both systemd and apt config files
  • Base64-encoding "#" is safe and doesn't break YAML parsing
  • Clusters with actual proxy configs continue to work normally

Changes:

  • magnum_cluster_api/resources.py: Add "or '#'" fallback to proxy config encoding
  • magnum_cluster_api/tests/unit/test_resources.py: Add comprehensive unit tests

Test coverage includes:

  1. Clusters without proxy configuration (fallback to "#")
  2. Clusters with proxy configuration (actual config encoded)
  3. Critical test ensuring empty strings are never encoded

This is a minimal, safe fix that:

  • Prevents cluster creation failures
  • Has no impact on existing proxy-enabled clusters
  • Uses a harmless comment character as fallback
  • Is compatible with all configuration file formats

Related issue: #790

Fixes vexxhost#790

When HTTP/HTTPS proxy is not configured, empty strings returned by
generate_systemd_proxy_config() and generate_apt_proxy_config() were
being base64-encoded and causing YAML parsing errors in Cluster API.

The issue occurs because:
1. utils.generate_systemd_proxy_config() returns "" when no proxy is set
2. utils.generate_apt_proxy_config() returns "" when no proxy is set
3. These empty strings get base64-encoded in resources.py
4. When CAPI decodes them, empty values break cloud-init YAML parsing
5. Cluster creation fails with validation errors

The fix:
- Use "or '#'" fallback in resources.py when encoding proxy configs
- When proxy functions return "", the fallback "#" is used instead
- "#" is a comment character in both systemd and apt config files
- Base64-encoding "#" is safe and doesn't break YAML parsing
- Clusters with actual proxy configs continue to work normally

Changes:
- magnum_cluster_api/resources.py: Add "or '#'" fallback to proxy config encoding
- magnum_cluster_api/tests/unit/test_resources.py: Add comprehensive unit tests

Test coverage includes:
1. Clusters without proxy configuration (fallback to "#")
2. Clusters with proxy configuration (actual config encoded)
3. Critical test ensuring empty strings are never encoded

This is a minimal, safe fix that:
- Prevents cluster creation failures
- Has no impact on existing proxy-enabled clusters
- Uses a harmless comment character as fallback
- Is compatible with all configuration file formats

Related issue: vexxhost#790

Signed-off-by: Guillaume Harvey <[email protected]>
@gabriel-samfira
Copy link

Just hit this one as well. We fixed it locally by changing the enabledIf condition, but this is much cleaner.

@PatriiCloud
Copy link
Author

Thanks! It should also spot if kubernetes change the way around with the unit test to prevent it to come back.

@mnaser
Copy link
Member

mnaser commented Nov 14, 2025

Our CI today runs with no proxy and it works just fine as well as other deployments, so I'm curious what is the root cause that is making this necessary?

Perhaps some more context here would be needed, the PR seems to imply that any deployment of a cluster without a proxy doesn't work today which isn't really valid?

@PatriiCloud
Copy link
Author

What OS is your CI installed with?

Every install I have done on Ubuntu 24.04 is getting this error.

On our side we deploy with kolla-ansible, then we install kubectl and add our kubeconfig from the k8s-magnum-capi.

The only way to make it work was to use this trick in the files. I do not have much more context than that.

@mnaser
Copy link
Member

mnaser commented Nov 15, 2025

Can you share the cluster template and cluster values from magnum?

@jaszil
Copy link

jaszil commented Nov 17, 2025

We are also struggling with empty proxy files.

Openstack version 2025.01 and Magnum_cluster_api v.033.0

When we build a cluster, Magnum reports that it has been successfully created and is healthy. Cluster access via the API also works, but no rolling-updates are possible via Magnum. Clusterapi says that the cluster is not available.

Cluster Infos:

openstack coe cluster show testPub
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                | Value                                                                                                                                                                                            |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| status               | CREATE_COMPLETE                                                                                                                                                                                  |
| health_status        | HEALTHY                                                                                                                                                                                          |
| cluster_template_id  | a9e909ce-ff6c-433f-8b0f-3d1810837424                                                                                                                                                             |
| node_addresses       | []                                                                                                                                                                                               |
| uuid                 | bef790a8-8416-4e80-b1ec-baf95fdd5ab9                                                                                                                                                             |
| stack_id             | kube-ecxu6                                                                                                                                                                                       |
| status_reason        | None                                                                                                                                                                                             |
| created_at           | 2025-11-17T14:10:40+00:00                                                                                                                                                                        |
| updated_at           | 2025-11-17T14:16:02+00:00                                                                                                                                                                        |
| coe_version          | v1.32.6                                                                                                                                                                                          |
| labels               | {'kube_tag': 'v1.32.6', 'keystone_auth_enabled': 'False', 'availability_zone': 'FZJ', 'auto_scaling_enabled': 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True'} |
| labels_overridden    | {}                                                                                                                                                                                               |
| labels_skipped       | {}                                                                                                                                                                                               |
| labels_added         | {'availability_zone': 'FZJ', 'auto_scaling_enabled': 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True'}                                                          |
| fixed_network        | None                                                                                                                                                                                             |
| fixed_subnet         | None                                                                                                                                                                                             |
| floating_ip_enabled  | True                                                                                                                                                                                             |
| faults               |                                                                                                                                                                                                  |
| keypair              | Janos                                                                                                                                                                                            |
| api_address          | https://<public_ip>:6443                                                                                                                                                                      |
| master_addresses     | []                                                                                                                                                                                               |
| master_lb_enabled    | True                                                                                                                                                                                             |
| create_timeout       | 60                                                                                                                                                                                               |
| node_count           | 1                                                                                                                                                                                                |
| discovery_url        |                                                                                                                                                                                                  |
| docker_volume_size   | None                                                                                                                                                                                             |
| master_count         | 1                                                                                                                                                                                                |
| container_version    | None                                                                                                                                                                                             |
| name                 | testPub                                                                                                                                                                                          |
| master_flavor_id     | c4r16                                                                                                                                                                                            |
| flavor_id            | c4r8                                                                                                                                                                                             |
| health_status_reason | {'kube-ecxu6-84qmw-mkmtq.Ready': 'True', 'kube-ecxu6-default-worker-tqxxb-wcgq4-b6qx8.Ready': 'True', 'api': 'ok'}                                                                               |
| project_id           | <project_id>                                                                                                                                                                 |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

clusterctl describe shows that the cluster is unavailable and says that the files may not be empty:

NAME                                                         REPLICAS  AVAILABLE  READY  UP TO DATE  STATUS            REASON        SINCE  MESSAGE
Cluster/kube-ecxu6                                           2/2       2          2      2           Available: False  NotAvailable  64m    * TopologyReconciled: error reconciling the Cluster topology: failed to create patch helper for
│                                                                                                                                             KubeadmControlPlane magnum-system/kube-ecxu6-84qmw: server side apply dry-run failed for modified
│                                                                                                                                             object: KubeadmControlPlane.controlplane.cluster.x-k8s.io "kube-ecxu6-84qmw" is invalid:
│                                                                                                                                             [spec.kubeadmConfigSpec.files[3].content: Invalid value: "":
│                                                                                                                                             spec.kubeadmConfigSpec.files[3].content in body should be at least 1 chars long,
│                                                                                                                                             spec.kubeadmConfigSpec.files[4].content: Invalid value: "":
│                                                                                                                                             spec.kubeadmConfigSpec.files[4].content in body should be at least 1 chars long]

After we have imported the changes from the pull request, everything is fine again.

In both cases, we used the same template and the same values.

Cluster infos:

+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                | Value                                                                                                                                                                                            |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| status               | CREATE_COMPLETE                                                                                                                                                                                  |
| health_status        | HEALTHY                                                                                                                                                                                          |
| cluster_template_id  | a9e909ce-ff6c-433f-8b0f-3d1810837424                                                                                                                                                             |
| node_addresses       | []                                                                                                                                                                                               |
| uuid                 | f0f56567-18ba-4d5b-ad14-6a9c94677b55                                                                                                                                                             |
| stack_id             | kube-09jnq                                                                                                                                                                                       |
| status_reason        | None                                                                                                                                                                                             |
| created_at           | 2025-11-17T12:56:53+00:00                                                                                                                                                                        |
| updated_at           | 2025-11-17T13:01:42+00:00                                                                                                                                                                        |
| coe_version          | v1.32.6                                                                                                                                                                                          |
| labels               | {'kube_tag': 'v1.32.6', 'keystone_auth_enabled': 'False', 'availability_zone': 'FZJ', 'auto_scaling_enabled': 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True'} |
| labels_overridden    | {}                                                                                                                                                                                               |
| labels_skipped       | {}                                                                                                                                                                                               |
| labels_added         | {'availability_zone': 'FZJ', 'auto_scaling_enabled': 'False', 'auto_healing_enabled': 'False', 'master_lb_floating_ip_enabled': 'True'}                                                          |
| fixed_network        | None                                                                                                                                                                                             |
| fixed_subnet         | None                                                                                                                                                                                             |
| floating_ip_enabled  | True                                                                                                                                                                                             |
| faults               |                                                                                                                                                                                                  |
| keypair              | Janos                                                                                                                                                                                            |
| api_address          | https://<public_ip>:6443                                                                                                                                                                      |
| master_addresses     | []                                                                                                                                                                                               |
| master_lb_enabled    | True                                                                                                                                                                                             |
| create_timeout       | 60                                                                                                                                                                                               |
| node_count           | 1                                                                                                                                                                                                |
| discovery_url        |                                                                                                                                                                                                  |
| docker_volume_size   | None                                                                                                                                                                                             |
| master_count         | 1                                                                                                                                                                                                |
| container_version    | None                                                                                                                                                                                             |
| name                 | TestPrPub                                                                                                                                                                                        |
| master_flavor_id     | c4r16                                                                                                                                                                                            |
| flavor_id            | c4r8                                                                                                                                                                                             |
| health_status_reason | {'kube-09jnq-default-worker-l945m-c7hw7-c6hk2.Ready': 'True', 'kube-09jnq-zmrsw-wf24c.Ready': 'True', 'api': 'ok'}                                                                               |
| project_id           | <project_id>                                                                                                                                                                 |
+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Clusterctl describe:

NAME                                                         REPLICAS  AVAILABLE  READY  UP TO DATE  STATUS           REASON     SINCE  MESSAGE
Cluster/kube-09jnq                                           2/2       2          2      2           Available: True  Available  64s
├─ClusterInfrastructure - OpenStackCluster/kube-09jnq-9l96q
├─ControlPlane - KubeadmControlPlane/kube-09jnq-zmrsw        1/1       1          1      1
│ └─Machine/kube-09jnq-zmrsw-wf24c                           1         1          1      1           Ready: True      Ready      8m1s
└─Workers
  └─MachineDeployment/kube-09jnq-default-worker-l945m        1/1       1          1      1           Available: True  Available  136m
    └─Machine/kube-09jnq-default-worker-l945m-c7hw7-c6hk2    1         1          1      1           Ready: True      Ready      136m

Cluster template:

+-----------------------+-----------------------------------------------------------+
| Field                 | Value                                                     |
+-----------------------+-----------------------------------------------------------+
| insecure_registry     | -                                                         |
| labels                | {'kube_tag': 'v1.32.6', 'keystone_auth_enabled': 'False'} |
| updated_at            | -                                                         |
| floating_ip_enabled   | True                                                      |
| fixed_subnet          | -                                                         |
| master_flavor_id      | c4r16                                                     |
| uuid                  | a9e909ce-ff6c-433f-8b0f-3d1810837424                      |
| no_proxy              | -                                                         |
| https_proxy           | -                                                         |
| tls_disabled          | False                                                     |
| keypair_id            | -                                                         |
| public                | True                                                      |
| http_proxy            | -                                                         |
| docker_volume_size    | -                                                         |
| server_type           | vm                                                        |
| external_network_id   | <network_id>                      |
| cluster_distro        | ubuntu                                                    |
| image_id              | 62485eb0-f79a-43cb-8464-6262b50c6899                      |
| volume_driver         | cinder                                                    |
| registry_enabled      | False                                                     |
| docker_storage_driver | overlay2                                                  |
| apiserver_port        | -                                                         |
| name                  | ubuntu-2404-k8s-v1.32.6                                   |
| created_at            | 2025-07-28T07:13:48+00:00                                 |
| network_driver        | calico                                                    |
| fixed_network         | -                                                         |
| coe                   | kubernetes                                                |
| flavor_id             | c8r32                                                     |
| master_lb_enabled     | True                                                      |
| dns_nameserver        | 134.94.32.3                                               |
| project_id            | <project_id>                          |
| hidden                | False                                                     |
| tags                  | -                                                         |
+-----------------------+-----------------------------------------------------------+

@mnaser
Copy link
Member

mnaser commented Nov 17, 2025

@jaszil what version of cluster api are you running if you dont mind me asking?

@jaszil
Copy link

jaszil commented Nov 18, 2025

@mnaser v1.11.2

@mvngne
Copy link

mvngne commented Nov 26, 2025

I'm having the same issues. It would be great if this could be fixed soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

With http_proxy settings configured magnum_cluster_api refuses to connect to kube clusters

5 participants