Skip to content
30 changes: 25 additions & 5 deletions vcluster/_fragments/integrations/istio.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,28 @@ This configuration:
Only `DestinationRules`, `Gateways`, and `VirtualServices` from `networking.istio.io/v1` API Version are synced to the host clusters. Other kinds are not yet supported.
:::

### Network policies

If you are using [network policies](../../configure/vcluster-yaml/policies/network-policy.mdx), consult the [Ambient and Kubernetes NetworkPolicy](https://istio.io/latest/docs/ambient/usage/networkpolicy/) documentation.
Depending on the underlying configuration, the following policies may be required:

```yaml title="vcluster.yaml"
policies:
networkPolicy:
workload:
ingress:
- ports:
# Allow HBONE traffic per https://istio.io/latest/docs/ambient/usage/networkpolicy/
- port: 15008
protocol: TCP
- from:
# Allow kubelet health probe per https://istio.io/latest/docs/ambient/usage/networkpolicy/
- ipBlock:
cidr: 169.254.7.127/32
```

<!--vale off-->
## Route request based on the version label of the app

<Flow id="istio-example">

### Set up cluster contexts
Expand All @@ -69,8 +89,8 @@ Only `DestinationRules`, `Gateways`, and `VirtualServices` from `networking.isti
the `HOST_CONTEXT` is the context of the host cluster, while the `VCLUSTER_CONTEXT` is
the context to access the created vCluster. The `VCLUSTER_HOST_NAMESPACE` is the namespace
where the vCluster is created on the host cluster. The `ISTIO_NAMESPACE` is the namespace to deploy the Istio integration.
The commands in the following steps will be automatically updated to follow your
configuration. For the GatewayAPI version, you should set it according to
The commands in the following steps will be automatically updated to follow your
configuration. For the GatewayAPI version, you should set it according to
your Istio version / GatewayAPI version matrix above.

<PageVariables HOST_CONTEXT="your-host-context" VCLUSTER_CONTEXT="vcluster-ctx" VCLUSTER_HOST_NAMESPACE="vcluster" GATEWAY_API_VERSION="v1.2.1" ISTIO_NAMESPACE="istio"/>
Expand Down Expand Up @@ -132,7 +152,7 @@ Only `DestinationRules`, `Gateways`, and `VirtualServices` from `networking.isti
/>

and label it with `istio.io/dataplane-mode: ambient`:

<InterpolatedCodeBlock
code={`kubectl --context="[[GLOBAL:VCLUSTER_CONTEXT]]" label namespace [[GLOBAL:ISTIO_NAMESPACE]] istio.io/dataplane-mode=ambient`}
language="bash"
Expand Down Expand Up @@ -441,7 +461,7 @@ code={`apiVersion: apps/v1
/>

To apply a `DestinationRule` configuration to the virtual cluster specified by the `VCLUSTER_CONTEXT` context, use the following command:

<InterpolatedCodeBlock
code={` kubectl --context="[[GLOBAL:VCLUSTER_CONTEXT]]" create -f destination_rule.yaml`}
language="bash"
Expand Down
19 changes: 10 additions & 9 deletions vcluster/_fragments/metrics-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,19 +33,20 @@ integrations:
port: 443
```

If you are using network policies, you need to add the metrics server pod to the allowed rules that the control plane can communicate with:
If you are using [network policies](../configure/vcluster-yaml/policies/network-policy.mdx), you need to add the metrics server pod to the allowed rules that the control plane can communicate with:

```yaml
policies:
networkPolicy:
extraControlPlaneRules:
- to:
- podSelector:
matchLabels:
k8s-app: metrics-server
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: "my-metrics-server-namespace"
controlPlane:
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: 'kube-system'
podSelector:
matchLabels:
k8s-app: metrics-server
```

## Config reference
Expand Down
34 changes: 29 additions & 5 deletions vcluster/_fragments/private-nodes.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import PrivateNodeLimitations from './private-nodes-limitations.mdx'

Using private nodes is a tenancy model for vCluster where, instead of sharing the host cluster’s worker nodes, individual worker nodes are joined to a vCluster.
These private nodes act as the vCluster’s worker nodes and are treated as worker nodes for the vCluster.
These private nodes act as the vCluster’s worker nodes and are treated as worker nodes for the vCluster.

Because these nodes are real Kubernetes nodes, vCluster does not sync any resources to the host cluster as no host cluster worker nodes are used. All workloads run directly on the attached nodes as if they were native to the virtual cluster.

Expand All @@ -28,10 +28,10 @@ alt="Overview"

## How private nodes can be provisioned

Private nodes can be provisioned in two different ways:
Private nodes can be provisioned in two different ways:

* **[Manually provisioned](../../../deploy/worker-nodes/private-nodes/join)** - Nodes that were provisioned outside of vCluster. These nodes are joined to vCluster using a vCluster CLI command.
* **[Automatically provisioned](./auto-nodes)** - Nodes that are provisioned on-demand based on the vCluster configuration and resource requirements. vCluster is connected to vCluster Platform and references a node provider defined in
* **[Manually provisioned](../../../deploy/worker-nodes/private-nodes/join)** - Nodes that were provisioned outside of vCluster. These nodes are joined to vCluster using a vCluster CLI command.
* **[Automatically provisioned](./auto-nodes)** - Nodes that are provisioned on-demand based on the vCluster configuration and resource requirements. vCluster is connected to vCluster Platform and references a node provider defined in
vCluster Platform.


Expand Down Expand Up @@ -75,4 +75,28 @@ controlPlane:
enabled: true
```

<PrivateNodeLimitations />
<PrivateNodeLimitations />

### Network policies
If you are using [network policies](../configure/vcluster-yaml/policies/network-policy.mdx), private nodes traffic into the virtual cluster control plane must be allowed.

```yaml title="vcluster.yaml"
privateNodes:
enabled: true

controlPlane:
service:
spec:
type: LoadBalancer

policies:
networkPolicy:
enabled: true
controlPlane:
ingress:
- from:
# Allow incoming traffic from the load balancer internal IP address.
# This example is allowing incoming traffic from any address. Load balancer internal CIDR should be used.
- ipBlock:
cidr: 0.0.0.0/0
```
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: External database
sidebar_label: external
sidebar_position: 2
sidebar_class_name: pro host-nodes private-nodes
sidebar_class_name: pro host-nodes private-nodes
description: Configure an external database as the virtual cluster's backing store for enhanced performance and scalability.
---

Expand Down Expand Up @@ -76,7 +76,7 @@ There are two mutually exclusive options for using an external backing store.
Replace `CONNECTION_STRING` with the connection string for your database. Examples:

- PostgreSQL: `postgres://username:password@hostname:5432/vcluster-db`
- MySQL: `mysql://root:password@tcp(192.168.86.9:30360)/vcluster`
- MySQL: `mysql://root:password@tcp(192.168.86.9:3306)/vcluster`

### Connector configuration

Expand All @@ -95,6 +95,35 @@ controlPlane:
The virtual cluster must be [connected to the platform](/vcluster/configure/vcluster-yaml/external/platform/api-key) to use the connector. This enables centralized management and monitoring of virtual clusters.
:::

## Network policies
If you are using [network policies](../../../../policies/network-policy.mdx), the virtual cluster control plane traffic to the external database must be allowed.

```yaml title="vcluster.yaml"
policies:
networkPolicy:
controlPlane:
egress:
# Allow outgoing traffic to the mysql database server.
- to:
- ipBlock:
cidr: 192.168.86.9/32
ports:
- port: 3306
protocol: TCP

# Allow outgoing traffic to the postgres database server private subnets.
- to:
- ipBlock:
cidr: 10.0.0.0/24
- ipBlock:
cidr: 10.0.1.0/24
- ipBlock:
cidr: 10.0.2.0/24
ports:
- port: 5432
protocol: TCP
```

## Config reference

<ConfigReference />
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,37 @@ In the above example, when you remove the `my-virtual-namespace/my-virtual-servi
`networking.replicateServices.toHost`, the host cluster service `my-host-service` is not automatically deleted from
the host cluster, so you need to delete it manually, if you don't want to keep it in the host cluster.

## Network policies

If you are using [network policies](../policies/network-policy.mdx), traffic to or from the replicated services must be allowed.

```yaml title="vcluster.yaml"
policies:
networkPolicy:
workload:
egress:
- to:
# Example allowing vcluster workload traffic to all pods in the my-host-namespace namespace.
# Depending on your use case, a more restrictive pod selector may be used.
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: my-host-namespace

ingress:
- from:
# Example allowing vcluster workload traffic from all pods in the my-host-namespace namespace.
# Depending on your use case, a more restrictive pod selector may be used.
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: my-host-namespace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this duplicate namespaceSelector intentional? It looks like it might be a copy-paste error.

Suggested change
kubernetes.io/metadata.name: my-host-namespace
ingress:
- from:
# Example allowing vcluster workload traffic from all pods in the my-host-namespace namespace.
# Depending on your use case, a more restrictive pod selector may be used.
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: my-host-namespace

- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: my-host-namespace



```

## Config reference

<ReplicateServices/>
17 changes: 17 additions & 0 deletions vcluster/configure/vcluster-yaml/policies/admission-control.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
sidebar_label: centralAdmission
sidebar_class_name: pro host-nodes
sidebar_position: 5
description: Configuration for ...

Check warning on line 6 in vcluster/configure/vcluster-yaml/policies/admission-control.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Google.Ellipses] In general, don't use an ellipsis. Raw Output: {"message": "[Google.Ellipses] In general, don't use an ellipsis.", "location": {"path": "vcluster/configure/vcluster-yaml/policies/admission-control.mdx", "range": {"start": {"line": 6, "column": 32}}}, "severity": "WARNING"}
---

import AdmissionControl from '../../../_partials/config/policies/centralAdmission.mdx'
Expand All @@ -27,8 +27,8 @@

Central admission webhooks are different from the other policies:

- `LimitRange` and `ResourceQuota` resources are created and enforced on the host cluster. They do not appear as resources in the virtual cluster.

Check warning on line 30 in vcluster/configure/vcluster-yaml/policies/admission-control.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`ResourceQuota`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`). Raw Output: {"message": "[Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`ResourceQuota`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`).", "location": {"path": "vcluster/configure/vcluster-yaml/policies/admission-control.mdx", "range": {"start": {"line": 30, "column": 20}}}, "severity": "WARNING"}

Check warning on line 30 in vcluster/configure/vcluster-yaml/policies/admission-control.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`LimitRange`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`). Raw Output: {"message": "[Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`LimitRange`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`).", "location": {"path": "vcluster/configure/vcluster-yaml/policies/admission-control.mdx", "range": {"start": {"line": 30, "column": 3}}}, "severity": "WARNING"}
- A user could define `LimitRange` and `ResourceQuota` resources inside the virtual cluster, but they have full control over them and can delete them when needed.

Check warning on line 31 in vcluster/configure/vcluster-yaml/policies/admission-control.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`ResourceQuota`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`). Raw Output: {"message": "[Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`ResourceQuota`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`).", "location": {"path": "vcluster/configure/vcluster-yaml/policies/admission-control.mdx", "range": {"start": {"line": 31, "column": 40}}}, "severity": "WARNING"}

Check warning on line 31 in vcluster/configure/vcluster-yaml/policies/admission-control.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`LimitRange`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`). Raw Output: {"message": "[Loft.kubernetes-api-kinds] Kubernetes/Platform API kinds like '`LimitRange`' should not use backticks. Write them as plain text (e.g., StatefulSet not `StatefulSet`).", "location": {"path": "vcluster/configure/vcluster-yaml/policies/admission-control.mdx", "range": {"start": {"line": 31, "column": 23}}}, "severity": "WARNING"}
- Webhooks are different because they need to be configured inside the virtual cluster in order for them to be called by the vCluster's API server.
- vCluster rewrites these definitions to point to a proxy for a host cluster service that handles webhook requests. The host might also have webhook configurations that use this service.
- A user can still install a webhook service or webhook configuration into the virtual cluster outside of this config, but it would run inside the virtual cluster like any other workload.
Expand Down Expand Up @@ -244,6 +244,23 @@
admission webhook "validation.gatekeeper.sh" denied the request: you must provide labels: [gatekeeper]
```

## Network policies
If you are using [network policies](./network-policy.mdx), admission webhooks traffic from the vCluster control plane must be allowed.

```yaml title="vcluster.yaml"
policies:
networkPolicy:
controlPlane:
egress:
- to:
# Allow vcluster control plane traffic to the gatekeeper webhook.
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: gatekeeper-system
podSelector:
matchLabels:
gatekeeper.sh/operation: webhook
```

## Config reference

Expand Down
103 changes: 91 additions & 12 deletions vcluster/configure/vcluster-yaml/policies/network-policy.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Network policy
sidebar_label: networkPolicy

Check failure on line 3 in vcluster/configure/vcluster-yaml/policies/network-policy.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Vale.Terms] Use 'NetworkPolicy' instead of 'networkPolicy'. Raw Output: {"message": "[Vale.Terms] Use 'NetworkPolicy' instead of 'networkPolicy'.", "location": {"path": "vcluster/configure/vcluster-yaml/policies/network-policy.mdx", "range": {"start": {"line": 3, "column": 16}}}, "severity": "ERROR"}
sidebar_position: 1
description: Configure network policies to isolate virtual cluster workloads and implement project-scoped network boundaries.
sidebar_class_name: host-nodes
Expand Down Expand Up @@ -44,24 +44,50 @@
enabled: true
```

#### Custom egress rules {#custom-egress}

Control outbound traffic with specific CIDR blocks:

#### Custom ingress and egress rules {#custom-rules}
Control inbound and outbound traffic with specific ports and IP addresses for vCluster control plane and workloads:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a prerequisite warning about CNI support? In the isolated workloads doc there's a warning that network policies require a CNI that supports NetworkPolicy enforcement - would be helpful to call that out here too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when this is disabled (the default)? It would help users to know that without network policies, vCluster pods can communicate freely with all pods on the host cluster.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a section showing the actual NetworkPolicy resources that get created when this is enabled? Users would benefit from seeing the default YAML (like vc-work-{name} and vc-cp-{name}) and knowing that these are managed by vCluster and manual changes on the host cluster will be overwritten.

```yaml title="vcluster.yaml"
policies:
networkPolicy:
enabled: true
outgoingConnections:
ipBlock:
cidr: 0.0.0.0/0
except:
- 169.254.0.0/16 # AWS metadata service
- 10.0.0.0/8 # Private network ranges
- 172.16.0.0/12
- 192.168.0.0/16

workload:
ingress:
# Allow ingress from anywhere to specific ports
- ports:
- port: 6060
- port: 444

egress:
# Allow egress to a specific address and port
- to:
- ipBlock:
cidr: 172.19.10.23/32
ports:
- port: 7777
protocol: TCP

publicEgress:
# Disable convenience common public egress rule.
enabled: false

controlPlane:
ingress:
# Allow ingress traffic from anywhere to the virtual cluster control plane api
- ports:
- port: 8443

egress:
# Allow egress traffic to a specific address
- to:
- ipBlock:
cidr: 172.19.10.23/32
```

:::note
`ingress` and `egress` config sections accept the same content type as [PodNetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/#podnetworkpolicy-resource)

Check warning on line 88 in vcluster/configure/vcluster-yaml/policies/network-policy.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Google.WordList] Use 'media type' instead of 'content type'. Raw Output: {"message": "[Google.WordList] Use 'media type' instead of 'content type'.", "location": {"path": "vcluster/configure/vcluster-yaml/policies/network-policy.mdx", "range": {"start": {"line": 88, "column": 56}}}, "severity": "WARNING"}
:::

#### Add custom labels {#custom-labels}

Apply labels to generated NetworkPolicies for easier management:
Expand Down Expand Up @@ -114,6 +140,59 @@
- Allows communication within the same project
- Enforces network boundaries for CI/CD pipelines

## Migration from v0.30 config {#migration}
`workload` and `controlPlane` configuration sections are introduced to allow defining additional ingress/egress rules for the specific components.

<div style={{ width: '50%', float: 'left', padding: '5px'}}>
```yaml title="vcluster.yaml (v0.30 and earlier)"
policies:
networkPolicy:
enabled: true

extraControlPlaneRules:
- ports:
- port: 8443


extraWorkloadRules:
- ports:
- port: 6060


outgoingConnections:
ipBlock:
cidr: 172.19.10.23/32
```
</div>

<div style={{ width: '50%', float: 'right', padding: '5px' }}>
```yaml title="vcluster.yaml (v0.31)"
policies:
networkPolicy:
enabled: true

controlPlane:
egress:
- ports:
- port: 8443

workload:
egress:
- ports:
- port: 6060

publicEgress:
cidr: 172.19.10.23/32

```
</div>

## Config reference

| Deprecated Field | New Field |
| ----------------- | ---------------- |
| `extraControlPlaneRules` | `controlPlane.egress` |
| `extraWorkloadRules` | `workload.egress` |
| `outgoingConnections.ipBlock` | `workload.publicEgress` |

<NetworkPolicy/>
Loading