Table of Contents
Kubernetes – Private Docker Registry with PV 설치
Private Docker Registry 를 설치하는 방법을 설명합니다.
일반 도메인 주소와 https 인증서를 이용해 설정해야 합니다.
Namespace
vi registry-namespace.yaml
---------------------------
apiVersion: v1
kind: Namespace
metadata:
name: docker-registry
---------------------------
kubectl apply -f registry-namespace.yaml
PersistentVolume / PersistentVolumeClaim 생성
sudo mkdir -p /data/docker-registry
sudo chmod 777 -R /data/docker-registry
kubectl get nodes
vi registry-pv.yaml
---------------------------
apiVersion: v1
kind: PersistentVolume
metadata:
name: docker-registry-pv
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/docker-registry
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- your-node-name # 실제 노드 이름으로 변경
---------------------------
kubectl apply -f registry-pv.yaml
vi registry-pvc.yaml
---------------------------
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: docker-registry-pvc
namespace: docker-registry
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: local-storage
---------------------------
kubectl apply -f registry-pvc.yaml
kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE
persistentvolume/docker-registry-pv 100Gi RWO Retain Bound docker-registry/docker-registry-pvc local-storage <unset> 2m27s
TLS
인증서 변경에 대비해서 크론탭등에 등록해서 갱신을 자동화해야 합니다.
# kubectl delete secret docker-registry-tls -n docker-registry
kubectl create secret tls docker-registry-tls --cert=/home/skyer9/cert/fullchain.pem --key=/home/skyer9/cert/privkey.pem -n docker-registry
Deployment
kubectl create secret generic docker-registry-secret \
--from-literal=http-secret=$(openssl rand -hex 32) \
-n docker-registry
vi registry-deployment.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
namespace: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- name: registry
image: registry:2
ports:
- containerPort: 5000
env:
- name: REGISTRY_HTTP_SECRET
valueFrom:
secretKeyRef:
name: docker-registry-secret
key: http-secret
- name: REGISTRY_HTTP_TLS_CERTIFICATE
value: /certs/tls.crt
- name: REGISTRY_HTTP_TLS_KEY
value: /certs/tls.key
volumeMounts:
- name: registry-data
mountPath: /data/registry
- name: tls-certs
mountPath: /certs
readOnly: true
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
volumes:
- name: registry-data
persistentVolumeClaim:
claimName: docker-registry-pvc
- name: tls-certs
secret:
secretName: docker-registry-tls
---------------------------
kubectl apply -f registry-deployment.yaml
Service
vi registry-service.yaml
---------------------------
apiVersion: v1
kind: Service
metadata:
name: docker-registry-service
namespace: docker-registry
spec:
selector:
app: docker-registry
ports:
- name: registry
port: 5000
targetPort: 5000
type: ClusterIP
---------------------------
kubectl apply -f registry-service.yaml
Ingress
Ingress 는 80/443 포트로의 접속만 허용합니다.
즉 서비스를 새로 따려면 도메인도 새로 따야 합니다.
vi registry-ingress.yaml
---------------------------
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: docker-registry-ingress
namespace: docker-registry
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
# GitHub Actions IP 대역만 허용 (예시)
nginx.ingress.kubernetes.io/whitelist-source-range: "192.30.252.0/22,185.199.108.0/22,140.82.112.0/20,143.55.64.0/20,20.201.28.151/32,20.205.243.166/32,20.87.225.212/32,20.248.137.48/32,20.207.73.82/32,20.27.177.113/32,20.29.134.23/32,20.87.245.0/32,20.201.28.152/32,20.205.243.160/32,102.133.202.242/32"
spec:
ingressClassName: nginx
tls:
- hosts:
- k8s.example.co.kr # 도메인을 변경하세요.
secretName: docker-registry-tls
rules:
- host: k8s.example.co.kr # 도메인을 변경하세요.
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: docker-registry-service
port:
number: 5000
---------------------------
kubectl apply -f registry-ingress.yaml
https://k8s.example.co.kr/ 로 접속합니다.
80/443 포트를 사용할 수 없는 경우
vi registry-service.yaml
---------------------------
apiVersion: v1
kind: Service
metadata:
name: docker-registry-service
namespace: docker-registry
spec:
selector:
app: docker-registry
ports:
- name: registry
port: 5000
targetPort: 5000
nodePort: 30010
protocol: TCP
type: NodePort
---------------------------
kubectl apply -f registry-service.yaml
https://노드아이피:30010/ 로 접속합니다.
"주의 요함" 이라는 문구가 남아있을 수 있다.
이럴 때는 다른 브라우저(엣지, 사파리 등등) 를 이용해 접속하면 "주의 요함" 문구가 표시되지 않는 것을 확인할 수 있다.
로그인 제한
기본 설정은 누구나 계정을 생성과 동시에 로그인할 수 있도록 되어 있습니다.
계정 생성을 제한 하려면 htpasswd 등을 이용해 계정정보를 제한해야 합니다.
docker login k8s.example.co.kr:30010
Username: test
Password:
WARNING! Your credentials are stored unencrypted in '/home/skyer9/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
Login Succeeded
# sudo apt install apache2-utils
htpasswd -Bbn username password > htpasswd
# htpasswd -Bb htpasswd another_user another_password
kubectl create secret generic docker-registry-htpasswd \
--from-file=htpasswd=./htpasswd \
-n docker-registry
vi registry-deployment.yaml
---------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
namespace: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- name: registry
image: registry:2
ports:
- containerPort: 5000
env:
- name: REGISTRY_HTTP_SECRET
valueFrom:
secretKeyRef:
name: docker-registry-secret
key: http-secret
- name: REGISTRY_HTTP_TLS_CERTIFICATE
value: /certs/tls.crt
- name: REGISTRY_HTTP_TLS_KEY
value: /certs/tls.key
# 인증 설정 추가
- name: REGISTRY_AUTH
value: htpasswd
- name: REGISTRY_AUTH_HTPASSWD_REALM
value: Registry Realm
- name: REGISTRY_AUTH_HTPASSWD_PATH
value: /auth/htpasswd
volumeMounts:
- name: registry-data
mountPath: /data/registry
- name: tls-certs
mountPath: /certs
readOnly: true
# htpasswd 볼륨 마운트 추가
- name: auth
mountPath: /auth
readOnly: true
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
volumes:
- name: registry-data
persistentVolumeClaim:
claimName: docker-registry-pvc
- name: tls-certs
secret:
secretName: docker-registry-tls
# htpasswd 볼륨 추가
- name: auth
secret:
secretName: docker-registry-htpasswd
---------------------------
kubectl apply -f registry-deployment.yaml
Github Action 등에서의 접속을 허용하려면 아이피 기반 제한이 어려우므로, 비밀번호를 복잡하고 길게 만들어야 한다……