Project Network Isolation
KubeSphere project network isolation lets project administrators enforce which network traffic is allowed using different rules. This tutorial demonstrates how to enable network isolation among projects and set rules to control network traffic.
Prerequisites
- You have already enabled Network Policies.
- You must have an available project and a user of the
admin
role (project-admin
) at the project level. For more information, see Create Workspaces, Projects, Users and Roles.
Note
Enable/Disable Project Network Isolation
Log in to KubeSphere as
project-admin
. Go to your project and select Network Isolation in Project Settings. By default, project network isolation is disabled.To enable project network isolation, click Enable.
Note
When network isolation is turned on, egress traffic will be allowed by default, while ingress traffic will be denied for different projects. But when you add an egress network policy, only traffic that matches your policy will be allowed to go out.You can also disable network isolation by toggling the Enabled button on this page.
Note
When network isolation is turned off, any previously created network policies will be deleted as well.
Set a Network Policy
If the default policy does not meet your needs when network isolation is enabled, you can customize your network policy to meet your needs. Currently, you can add custom network policies in KubeSphere for traffic within the cluster or incoming traffic outside the cluster.
For internal traffic within the cluster
Network policies at the project level within a cluster are used to control whether resources in this project can be accessed by other projects within the same cluster, and which Services you can access.
Assume an NGINX Deployment workload has been created in another project demo-project-2
and is exposed via the Service nginx
on the port 80
with TCP
. Here is an example of how to set ingress and egress traffic rules.
Note
Allow ingress traffic from workloads in a different project
On the Network Isolation page of your current project, click Internal Allowlist.
Click Add Allowlist Entry.
Select Ingress under Traffic Direction.
In Project, select the project
demo-project-2
.Click OK, and then you can see that the project is now in the allowlist.
Note
Allow egress traffic to Services in a different project
On the Network Isolation page of your current project, click Internal Allowlist.
Click Add Allowlist Entry.
Select Egress under Traffic Direction.
Select the tab Service under Type.
Select the project
demo-project-2
from the drop-down list.Select the Service that is allowed to receive egress traffic. In this case, select
nginx
.Click OK, and then you can see that the Service is now in the allowlist.
Note
For incoming traffic outside the cluster
KubeSphere uses CIDR to distinguish between peers. Assume a Tomcat Deployment workload has been created in your current project and is exposed via the NodePort
Service demo-service
on the NodePort 80
with TCP
. For an external client with the IP address 192.168.1.1
to access this Service, you need to add a rule for it.
Allow ingress traffic from a client outside the cluster
On the Network Isolation page of your current project, select External Allowlist and click Add Allowlist Entry.
Select Ingress under Traffic Direction.
Enter
192.168.1.1/32
for Network Segment.Select the protocol
TCP
and enter80
as the port number.Click OK, and then you can see that the rule has been added.
Note
spec.externalTrafficPolicy
in the Service configuration to local
, so that the source address of the packet will not change. Namely, the source address of the packet is the source address of the client.Assume the IP address of an external client is http://10.1.0.1:80
, then you need to set a rule for the egress traffic so that the internal Service can access it.
Allow egress traffic to Services outside the cluster
On the Network Isolation page of your current project, select External Allowlist and click Add Allowlist Entry.
Select Egress under Traffic Direction.
Enter
10.1.0.1/32
for Network Segment.Select the protocol
TCP
and enter80
as the port number.Click OK, and you can see that the rule has been added.
Note
Best practices
To ensure that all Pods in a project are secure, a best practice is to enable network isolation. When network isolation is on, the project cannot be accessed by other projects. If your workloads need to be accessed by others, you can follow these steps:
- Set a gateway in Project Settings.
- Expose workloads that need to be accessed to a gateway via a Service.
- Allow ingress traffic from the namespace where your gateway locates.
If egress traffic is controlled, you should have a clear plan of what projects, Services, and IP addresses can be accessed, and then add them one by one. If you are not sure about what you want, it is recommended that you keep your network policy unchanged.
FAQs
Q: Why cannot the custom monitoring system of KubeSphere get data after I enabled network isolation?
A: After you enable custom monitoring, the KubeSphere monitoring system will access the metrics of the Pod. You need to allow ingress traffic for the KubeSphere monitoring system. Otherwise, it cannot access Pod metrics.
KubeSphere provides a configuration item allowedIngressNamespaces
to simplify similar configurations, which allows all projects listed in the configuration.
root@node1:~# kubectl get -n kubesphere-system clusterconfigurations.installer.kubesphere.io ks-installer -o yaml
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
...
name: ks-installer
namespace: kubesphere-system
...
spec:
...
networkpolicy:
enabled: true
nsnpOptions:
allowedIngressNamespaces:
- kubesphere-system
- kubesphere-monitoring-system
...
Q: Why cannot I access a Service even after setting a network policy through the Service?
A: When you add a network policy and access the Service via the cluster IP address, if the network is not
working, check the kube-proxy configuration to see if masqueradeAll
is false
.
root@node1:~# kubectl get cm -n kube-system kube-proxy -o yaml
apiVersion: v1
data:
config.conf: |-
...
iptables:
masqueradeAll: false
...
...
kind: ConfigMap
metadata:
...
labels:
app: kube-proxy
name: kube-proxy
namespace: kube-system
...
Q: How do I determine the network segment when I set the ingress rule?
A: In Kubernetes, the source IP address of the packet is often handled by NAT, so you need to figure out what the source address of the packet will be before you add the rule. For more information, refer to Source IP.
Feedback
Was this page Helpful?
Receive the latest news, articles and updates from KubeSphere
Thanks for the feedback. If you have a specific question about how to use KubeSphere, ask it on Slack. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.