Table of Contents
Kubernetes – 사설 도메인에 https 적용하기
사설 도메인을 생성 또는 서비스 DNS 에 사설 인증서를 붙여줍니다.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
rootCA 인증서 / 서버 인증서 생성
위 링크를 참조하여 rootCA 인증서를 생성합니다.
아래 명령을 입력하여 *.default.private
, *. repository.private
를 각각 생성합니다.
openssl ecparam -out default.key -name prime256v1 -genkey
openssl req -new -sha256 -key default.key -out default.csr
openssl ecparam -out repository.key -name prime256v1 -genkey
openssl req -new -sha256 -key repository.key -out repository.csr
vi default-extention.ext
---------------------------
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.default.private
---------------------------
vi repository-extention.ext
---------------------------
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.repository.private
---------------------------
openssl x509 -req -sha256 -days 999999 -in default.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out default.crt -extfile default-extention.ext
openssl x509 -req -sha256 -days 999999 -in repository.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out repository.crt -extfile repository-extention.ext
인증서가 정상적으로 생성되었는지 확인할 수 있습니다.
openssl x509 -in default.crt -text -noout
openssl x509 -in repository.crt -text -noout
rootCA 인증서를 Pod 에 배포
rootCA 인증서 확인
인증서는 아래와 같은 내용이어야 합니다.
cat rootCA.crt
-----BEGIN CERTIFICATE-----
MIICBzCCAa0CFFQsOnoVXsnZPXXXXXXXXXXXX
......
6ks6jOgQdifuAiAA9XXXXXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----
인증서 등록
kubectl create configmap default-pemstore --from-file=rootCA.crt -n default
kubectl create configmap repository-pemstore --from-file=rootCA.crt -n repository
Pod 에 배포
vi nginx-deployment-rootCa.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
volumeMounts:
- name: default-pemstore # 여기
mountPath: /etc/ssl/certs/default.pem
subPath: rootCA.crt
readOnly: true
ports:
- containerPort: 80
volumes:
- name: default-pemstore # 여기
configMap:
name: default-pemstore
---------------------------
kubectl apply -f nginx-deployment-rootCa.yaml
확인하기
kubectl get pods -n default
NAME READY STATUS RESTARTS AGE
nginx-deployment-6669bcf7d4-8hdst 1/1 Running 0 42s
nginx 이미지 자체에 pem 파일이 없기 때문에,
추가한 default.pem
파일 한개만 있다.
kubectl exec -it nginx-deployment-6669bcf7d4-8hdst -- /bin/bash
ls -al /etc/ssl/certs/
Pod 에 서버 인증서 배포
인증서 등록
mkdir default
mv default.* default/
mkdir repository
mv repository.* repository/
kubectl create configmap default-server-pemstore --from-file=default/ -n default
# kubectl create configmap repository-server-pemstore --from-file=repository/ -n repository
Pod 에 배포
vi nginx-deployment-rootCa-server.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
volumeMounts:
- name: default-pemstore
mountPath: /etc/ssl/certs/default.pem
subPath: rootCA.crt
readOnly: true
- name: default-server-pemstore # 여기
mountPath: /ssl/
readOnly: true
ports:
- containerPort: 80
volumes:
- name: default-pemstore
configMap:
name: default-pemstore
- name: default-server-pemstore # 여기
configMap:
name: default-server-pemstore
---------------------------
kubectl apply -f nginx-deployment-rootCa-server.yaml
확인하기
kubectl get pods -n default
NAME READY STATUS RESTARTS AGE
nginx-deployment-6669bcf7d4-8hdst 1/1 Running 0 42s
nginx 이미지 자체에 pem 파일이 없기 때문에,
추가한 default.pem
파일 한개만 있다.
kubectl exec -it nginx-deployment-6669bcf7d4-8hdst -- /bin/bash
ls -al /ssl/
nginx https 설정
vi test.default.private-default.conf
---------------------------
server {
listen 80;
server_name test.default.private;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
server {
listen 443 ssl;
server_name test.default.private;
ssl_certificate /ssl/default.crt;
ssl_certificate_key /ssl/default.key;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
---------------------------
kubectl create configmap test.default.private-default --from-file=test.default.private-default.conf -n default
vi nginx-deployment-rootCa-server.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: default
labels:
app: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
volumeMounts:
- name: default-pemstore
mountPath: /etc/ssl/certs/default.pem
subPath: rootCA.crt
readOnly: true
- name: default-server-pemstore
mountPath: /ssl/
readOnly: true
- name: test-default-private-default # 여기
mountPath: /etc/nginx/conf.d/default.conf
subPath: test.default.private-default.conf
readOnly: true
ports:
- containerPort: 80
volumes:
- name: default-pemstore
configMap:
name: default-pemstore
- name: default-server-pemstore
configMap:
name: default-server-pemstore
- name: test-default-private-default # 여기
configMap:
name: test.default.private-default
---------------------------
kubectl apply -f nginx-deployment-rootCa-server.yaml
확인하기
Pod 아이피 확인
kubectl get pods -n default
NAME READY STATUS RESTARTS AGE
nginx-deployment-6669bcf7d4-8hdst 1/1 Running 0 42s
kubectl describe pod nginx-deployment-6669bcf7d4-8hdst
Name: nginx-deployment-6669bcf7d4-8hdst
Namespace: default
Priority: 0
Service Account: default
Node: notebook/192.168.0.4
Start Time: Mon, 05 Dec 2022 16:10:31 +0900
Labels: app=nginx
pod-template-hash=68cff86475
Annotations: <none>
Status: Running
IP: 10.32.0.3
Worker 서버에 도메인 등록
Worker 서버 접속 후 호스트파일을 수정해 준다.
sudo vi /etc/hosts
---------------------------
10.32.0.3 test.default.private
---------------------------
Worker 서버에 인증서 등록
Worker 서버 접속 후 서버에 인증서를 추가해 준다.
# Linux
sudo cp rootCA.crt /etc/ssl/certs/default.pem
sudo update-ca-certificates
# Ubuntu
sudo cp rootCA.crt /usr/share/ca-certificates/mozilla/default.crt
sudo dpkg-reconfigure ca-certificates
sudo update-ca-certificates
https 확인
Worker 서버 접속 후 아래 명령을 실행합니다.
curl http://test.default.private/
curl https://test.default.private/
서비스 DNS 로 접속
서비스 DNS 에 사설 인증서를 붙이려면 default 네임스페이스 이외의 네임스페이스에서 작업해야 합니다.
kubernetes.default
라는 시스템 서비스가 있기때문에 문제가 발생할 수 있습니다.
루트 인증서 확인
kubectl get configmap -n repository
서버 인증서 생성
openssl ecparam -out repository.key -name prime256v1 -genkey
openssl req -new -sha256 -key repository.key -out repository.csr
vi repository-extention.ext
---------------------------
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.repository
DNS.2 = *.repository.svc
DNS.3 = *.repository.svc.cluster
DNS.4 = *.repository.svc.cluster.local
---------------------------
openssl x509 -req -sha256 -days 999999 -in repository.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out repository.crt -extfile repository-extention.ext
rm -rf repository/
mkdir repository
mv repository.* repository/
kubectl create configmap repository-server-pemstore --from-file=repository/ -n repository
Deployment 생성
vi test.repository.conf
---------------------------
server {
listen 80;
server_name test.repository;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
server {
listen 443 ssl;
server_name test.repository;
ssl_certificate /ssl/repository.crt;
ssl_certificate_key /ssl/repository.key;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
---------------------------
kubectl create configmap test.repository.conf --from-file=test.repository.conf -n repository
vi test.repository.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: repository
labels:
app: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
volumeMounts:
- name: repository-pemstore
mountPath: /etc/ssl/certs/repository.pem
subPath: rootCA.crt
readOnly: true
- name: repository-server-pemstore
mountPath: /ssl/
readOnly: true
- name: test-repository-conf # 여기
mountPath: /etc/nginx/conf.d/default.conf
subPath: test.repository.conf
readOnly: true
ports:
- containerPort: 80
volumes:
- name: repository-pemstore
configMap:
name: repository-pemstore
- name: repository-server-pemstore
configMap:
name: repository-server-pemstore
- name: test-repository-conf # 여기
configMap:
name: test.repository.conf
---------------------------
kubectl apply -f test.repository.yaml
kubectl get pods -n repository
Service 생성
vi repository-nginx-svc.yaml
---------------------------
apiVersion: v1
kind: Service
metadata:
name: test
namespace: repository
labels:
app: nginx-service
spec:
type: ClusterIP # 서비스 타입
ports:
- name: http
port: 80 # 서비스 포트
targetPort: 80 # 컨테이너 포트(pod 포트)
protocol: TCP
- name: https
port: 443 # 서비스 포트
targetPort: 443 # 컨테이너 포트(pod 포트)
protocol: TCP
selector:
app: nginx
---------------------------
kubectl apply -f repository-nginx-svc.yaml
kubectl get svc -n repository
kubectl describe svc test -n repository
확인하기
여기 를 참조하여 busybox Pod 을 생성합니다.
kubectl get pod
kubectl exec -it busybox -- /bin/sh
wget http://test.repository/
cat index.html
rm index.html
wget https://test.repository/
cat index.html
rm index.html
exit
wget: note: TLS certificate validation not implemented
busybox 에 있는 wget 은 https 인증을 지원하지 않는다.
kubectl get pod
kubectl exec -it busybox -- /bin/sh
wget https://google.com/
exit
my-svc.my-namespace.svc.cluster-domain.example
서비스 DNS