nomad 로그 확인하기

노마드 로그는 오류가 발생한 위치에 따라,
Application Level 의 로그를 확인해야 하는 경우도,
또는 nomad 자체의 로그를 확인해야 하는 경우,
그리고 Docker 를 이용했다면 Docker 에서 로그를 확인해야 하는 경우가 발생합니다.

Application Log 확인하기

nomad status

ID          Type     Priority  Status   Submit Date
autoscaler  service  50        running  2021-08-28T12:17:52+09:00
grafana     service  50        running  2021-08-28T12:08:54+09:00
haproxy     service  50        running  2021-08-28T12:50:07+09:00
hello       service  50        running  2021-08-28T14:17:52+09:00
prometheus  service  50        running  2021-08-28T12:54:02+09:00

nomad status hello

ID            = hello
Name          = hello
Submit Date   = 2021-08-28T14:17:52+09:00
Type          = service
Priority      = 50
Datacenters   = dc1
Namespace     = default
Status        = running
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
helloGroup  0       0         1        2       4         0

Latest Deployment
ID          = cd9dbe0a
Status      = successful
Description = Deployment completed successfully

Deployed
Task Group  Desired  Placed  Healthy  Unhealthy  Progress Deadline
helloGroup  1        1       1        0          2021-08-28T14:28:19+09:00

Allocations
ID        Node ID   Task Group  Version  Desired  Status    Created     Modified
d43bab59  a5c3b874  helloGroup  7        run      running   3m18s ago   2m51s ago
5d5a2124  a5c3b874  helloGroup  6        stop     complete  20m17s ago  3m17s ago
00a33b57  a5c3b874  helloGroup  5        stop     complete  30m44s ago  5m6s ago
8195402d  a5c3b874  helloGroup  3        stop     complete  39m54s ago  5m6s ago
f667514c  a5c3b874  helloGroup  3        stop     failed    1h3s ago    5m6s ago
cf08455b  a5c3b874  helloGroup  3        stop     failed    1h48m ago   5m6s ago
ba0fae1f  a5c3b874  helloGroup  1        stop     complete  1h59m ago   5m6s ago

Allocations ID 를 이용해 앱의 로그를 확인할 수 있습니다.

nomad alloc logs d43bab59

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.5.3)

2021-08-28 14:18:07.569  INFO 1 --- [           main] c.e.testpublic.TestPublicApplication     : Starting TestPublicApplication using Java 11.0.11 on notebook with PID 1 (/local/TestPublic-0.0.2-SNAPSHOT.jar started by nobody in /)
2021-08-28 14:18:07.571  INFO 1 --- [           main] c.e.testpublic.TestPublicApplication     : No active profile set, falling back to default profiles: default
2021-08-28 14:18:08.351  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 20256 (http)
2021-08-28 14:18:08.362  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-08-28 14:18:08.362  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.50]
2021-08-28 14:18:08.412  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-08-28 14:18:08.412  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 801 ms
2021-08-28 14:18:08.700  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 20256 (http) with context path ''
2021-08-28 14:18:08.708  INFO 1 --- [           main] c.e.testpublic.TestPublicApplication     : Started TestPublicApplication in 1.526 seconds (JVM running for 1.895)
2021-08-28 14:18:09.254  INFO 1 --- [io-20256-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2021-08-28 14:18:09.254  INFO 1 --- [io-20256-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2021-08-28 14:18:09.255  INFO 1 --- [io-20256-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms

nomad 로그 확인하기

가끔 앱이 실행조차 못하고 종료하는 경우가 있습니다.

이런 경우 노마드 자체의 로그를 확인해 볼 수 있습니다.

sudo mkdir -p /var/log/nomad

로그 폴더를 만들어줍니다.
(노마드가 자동으로 만들어 주지 않습니다.)

sudo vi /etc/nomad.d/nomad.hcl

server {
  enabled          = true
  bootstrap_expect = 1
}

log_file = "/var/log/nomad/"
log_level = "DEBUG"

sudo systemctl restart nomad

ls -al /var/log/nomad/

sudo tail -20 /var/log/nomad/nomad-1630127757917582936.log

Docker 로그 확인하기

노마드 로그상에 도커 생성과 실행 후 실패하는 로그가 있다면 도커의 로그를 확인합니다.

plugin "docker" {
  config {
    # 실행 실패시 이미지 삭제
    # 디버깅시 false 로 할것
    gc {
      container   = true
    }
  }
}

위 nomad client server 의 설정에서 container = true 를 container = false 로 변경하면 도커의 로그를 확인할 수 있습니다.