{"id":5252,"date":"2022-04-16T13:56:50","date_gmt":"2022-04-16T04:56:50","guid":{"rendered":"https:\/\/www.skyer9.pe.kr\/wordpress\/?p=5252"},"modified":"2022-04-16T17:13:46","modified_gmt":"2022-04-16T08:13:46","slug":"intellij-%ec%97%90%ec%84%9c-hadoop-%ec%97%90-spark-submit-%ed%95%98%ea%b8%b0","status":"publish","type":"post","link":"https:\/\/www.skyer9.pe.kr\/wordpress\/?p=5252","title":{"rendered":"IntelliJ \uc5d0\uc11c \uc6d0\uaca9\uc11c\ubc84 Spark Submit \ud558\uae30"},"content":{"rendered":"

IntelliJ \uc5d0\uc11c \uc6d0\uaca9\uc11c\ubc84 Spark Submit \ud558\uae30<\/h1>\n

\ucc38\uc870<\/a><\/p>\n

\ud50c\ub7ec\uadf8\uc778 \uc124\uce58<\/h2>\n

Scala<\/code>, Big Data Tools<\/code> \uc744 \uc124\uce58\ud55c\ub2e4.<\/p>\n

\"\"<\/p>\n

Artifact \uc124\uc815\ud558\uae30<\/h2>\n
    \n
  1. \n

    \uba54\ub274 > \ud30c\uc77c > Project Structure \uc5d0\uc11c Artifacts \uc120\ud0dd<\/p>\n<\/li>\n

  2. \n

    \ucd94\uac00(\ud50c\ub7ec\uc2a4 \ubc84\ud2bc) \ud074\ub9ad \ud6c4, JAR \uc120\ud0dd \ud6c4 From modules with dependencies \uc120\ud0dd<\/p>\n<\/li>\n<\/ol>\n

    \"\"<\/p>\n

      \n
    1. \ubaa8\ub4c8 \uc120\ud0dd \ud6c4, \uba54\uc778 \ud074\ub798\uc2a4 \uc624\ub978\ucabd \ub05d\uc5d0 \ud3f4\ub354 \ubc84\ud2bc \ud074\ub9ad\ud574\uc11c \uba54\uc778 \ud074\ub798\uc2a4\ub97c \uc120\ud0dd<\/li>\n<\/ol>\n

      \"\"<\/p>\n

      Jar \uc0dd\uc131\ud558\uae30<\/h2>\n

      \uba54\ub274 > Build > Build Artifacts \ub97c \uc120\ud0dd\ud558\uba74 jar \uac00 \uc0dd\uc131\ub41c\ub2e4.<\/p>\n

      \"\"<\/p>\n

      Spark Submit<\/h2>\n
        \n
      1. \uc124\uc815 \ucd94\uac00 \ub610\ub294 \uc218\uc815\uc744 \uc120\ud0dd<\/li>\n<\/ol>\n

        \"\"<\/p>\n

          \n
        1. \ucd94\uac00(\ud50c\ub7ec\uc2a4 \ubc84\ud2bc) \ud074\ub9ad \ud6c4, Spark Submit > SSH<\/code> \uc120\ud0dd<\/li>\n<\/ol>\n

          \uc544\ub798 \ub0b4\uc6a9\uc744 \uc801\uc808\ud788 \uc218\uc815\ud574\uc11c \uc544\ub798 \uc774\ubbf8\uc9c0\uc5d0 \ud45c\uc2dc\ud55c \ud3f4\ub354 \ubc84\ud2bc \ud074\ub9ad \ud6c4 \uc785\ub825\ud574 \uc8fc\uba74 \uc790\ub3d9\uc73c\ub85c \ucc44\uc6cc\uc900\ub2e4.<\/p>\n

          \/home\/spark\/spark-3.2.1-bin-hadoop3.2\/bin\/spark-submit --master local --deploy-mode client --class Main --name Remote file:\/\/$HOME\/SparkStreamTest.jar<\/code><\/pre>\n
          \"\"<\/p>\n
          \uc2e4\ud589\ud558\uae30<\/h2>\n
          \uc704 \uc124\uc815\uc744 \ub05d\ub0b4\uace0, \ud504\ub85c\uc81d\ud2b8\ub97c \uc2e4\ud589\ud574 \ubcf4\uba74 \uc790\ub3d9\uc73c\ub85c \uc11c\ubc84\uc5d0 jar \ud30c\uc77c\uc744 \uc804\uc1a1\ud558\uace0,
          \n\uc11c\ubc84\uc5d0\uc11c jar \uac00 \uc2e4\ud589\ub418\ub294 \uac83\uc744 \ud655\uc778\ud560 \uc218 \uc788\ub2e4.<\/p>\n
          \ubc84\uadf8\uc778\uac00???<\/h2>\n
          JAVA_HOME \ubcc0\uc218\ub294 \ubb34\uc2dc\ud558\ubbc0\ub85c \ub514\ud3f4\ud2b8 \uc790\ubc14 \ubc84\uc804\uc774 JDK 8 \uc774\uc5b4\uc57c \ud55c\ub2e4.<\/p>\n
          \/usr\/bin\/java -version<\/code><\/pre>\n
          interactive \ub97c \uccb4\ud06c\ud574\uc57c \uc815\uc0c1\uc791\ub3d9\ud55c\ub2e4.<\/p>\n
          \"\"<\/p>\n","protected":false},"excerpt":{"rendered":"
          IntelliJ \uc5d0\uc11c \uc6d0\uaca9\uc11c\ubc84 Spark Submit \ud558\uae30 \ucc38\uc870 \ud50c\ub7ec\uadf8\uc778 \uc124\uce58 Scala, Big Data Tools \uc744 \uc124\uce58\ud55c\ub2e4. Artifact \uc124\uc815\ud558\uae30 \uba54\ub274 > \ud30c\uc77c > Project Structure \uc5d0\uc11c Artifacts \uc120\ud0dd \ucd94\uac00(\ud50c\ub7ec\uc2a4 \ubc84\ud2bc) \ud074\ub9ad \ud6c4, JAR \uc120\ud0dd \ud6c4 From modules with dependencies \uc120\ud0dd \ubaa8\ub4c8 \uc120\ud0dd \ud6c4, \uba54\uc778 \ud074\ub798\uc2a4 \uc624\ub978\ucabd \ub05d\uc5d0 \ud3f4\ub354 \ubc84\ud2bc \ud074\ub9ad\ud574\uc11c \uba54\uc778 \ud074\ub798\uc2a4\ub97c \uc120\ud0dd Jar \uc0dd\uc131\ud558\uae30 \uba54\ub274 >\u2026 Read More »<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[],"class_list":["post-5252","post","type-post","status-publish","format-standard","hentry","category-scala"],"_links":{"self":[{"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/5252","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5252"}],"version-history":[{"count":7,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/5252\/revisions"}],"predecessor-version":[{"id":5269,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/5252\/revisions\/5269"}],"wp:attachment":[{"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5252"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5252"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyer9.pe.kr\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5252"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}