Kubernetes – Private Docker Repository with PV 설치

By | 2023년 1월 18일
Table of Contents

Kubernetes – Private Docker Repository with PV 설치

Private Docker Repository 를 설치하는 방법을 설명합니다.

일반 도메인 주소와 https 인증서를 이용해 설정해야 합니다.

sudo systemctl restart docker
sudo systemctl restart containerd

Namespace

vi repository-namespace.yaml
---------------------------
apiVersion: v1
kind: Namespace
metadata:
  name: repository
---------------------------

PersistentVolume

여기 를 참조하여 PV 를 생성합니다.

vi local-storage-class.yaml
---------------------------
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---------------------------
vi repository-pv.yaml
---------------------------
apiVersion: v1
kind: PersistentVolume
metadata:
  name: repository-pv-0
  namespace: repository
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  claimRef:
    name: claim-docker-repository-0
    namespace: repository
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /DATA/repository          # 호스트 폴더경로
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - es-search02
---------------------------

StatefulSet

일반 도메인 주소와 https 인증서를 이용해 설정해야 합니다.
아니면 관리상 매우 불편해집니다.

vi repository-sts.yaml
---------------------------
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: docker-repository
  namespace: repository
spec:
  serviceName: repository
  replicas: 1
  selector:
    matchLabels:
      app: docker-repository
  template:
    metadata:
      labels:
        app: docker-repository
    spec:
      containers:
      - name: docker-repository
        image: registry:2.7.1
        ports:
          - name: http-port
            containerPort: 5000
        env:
        - name: REGISTRY_HTTP_ADDR
          value: 0.0.0.0:5000
        - name: REGISTRY_HTTP_TLS_CERTIFICATE
          value: /ssl/repository.crt
        - name: REGISTRY_HTTP_TLS_KEY
          value: /ssl/repository.key
        volumeMounts:
        - name: private-rootca-crt
          mountPath: /etc/ssl/certs/repository.pem
          subPath: rootCA.crt
          readOnly: true
        - name: repository-pemstore
          mountPath: /ssl/
          readOnly: true
        - name: claim
          mountPath: /var/lib/registry
      volumes:
      - name: private-rootca-crt
        configMap:
          name: private-rootca.crt
      - name: repository-pemstore
        configMap:
          name: repository-pemstore
  volumeClaimTemplates:
  - metadata:
      name: claim
      namespace: repository
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: local-storage
      resources:
        requests:
          storage: 100Gi
---------------------------

Service

cat repository-svc.yaml
---------------------------
apiVersion: v1
kind: Service
metadata:
  name: docker-repository
  namespace: repository
spec:
  type: ClusterIP
  ports:
    - name: https
      port: 443
      targetPort: 5000
  selector:
    app: docker-repository
---------------------------

클라이언트 설정 (build, push)

샘플은 Jenkins 아이템입니다.
아래 샘플에서는 서비스 도메인을 이용해 빌드하고 있지만,
일반 도메인과 HTTPS 인증서를 이용해 접속해야 별도설정없이 접근이 가능합니다.

pipeline {
    agent {
        kubernetes {
            defaultContainer 'jnlp'
            yaml """
spec:
  # dnsPolicy: Default       # 이게 왜 필요할까?
  containers:
    - name: docker
      image: docker:20.10.22
      command:
        - cat
      tty: true
      # privileged: true
      volumeMounts:
      - name: dockersock
        mountPath: /var/run/docker.sock
  volumes:
  - name: dockersock
    hostPath:
      path: /var/run/docker.sock
"""
        }
    }

    stages {
        stage("Get Source") {
            steps {
                    writeFile file: 'Dockerfile', text: """
FROM docker.elastic.co/elasticsearch/elasticsearch:7.17.8

RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch https://github.com/skyer9/elasticsearch-jaso-analyzer/releases/download/7.17.8/jaso-analyzer-plugin-7.17.8-plugin.zip
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch analysis-icu
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch analysis-nori
                    """
            }
        }

        stage('Docker Build') {
            steps {
                container('docker') {
                    sh "docker build -t docker-repository.repository.svc.cluster.local/search-engine/elasticsearch:7.17.8.${build_number} ."
                    sh "docker push docker-repository.repository.svc.cluster.local/search-engine/elasticsearch:7.17.8.${build_number}"
                }
            }
        }
    }
}

클라이언트 설정 (pull, run)

아래 설정으로 Private Repo 에서 이미지를 받아와 실행시킵니다.

      containers:
        - name: elasticsearch-master
          image: docker-repository.repository.svc.cluster.local/search-engine/elasticsearch:7.17.8.60
          env:
            - name: CLUSTER_NAME
              value: elasticsearch-cluster
            - name: NODE_LIST
              value: "elasticsearch-discovery"
            - name: "ES_JAVA_OPTS"
              value: "-Xms300m -Xmx300m"
            - name: NODE_MASTER
              value: "true"

사설 도메인/인증서를 쓰려면?

  • 모든 Worker Node OS hosts 파일에 서비스 클러스터 아이피를 추가해야 합니다.
  • 모든 Worker Node OS 에 사설 루트인증서를 등록해야 합니다.
  • 모든 Worker Node OS Docker 에 루트인증서를 등록해야 합니다.
  • 모든 Worker Node 에서 아래 명령을 실행해야 합니다.

답글 남기기