본문 바로가기
WEB&WAS/Apache Tomcat

WAS) Tomcat Session clustering (2) - 구성

by LILO 2022. 11. 5.
반응형

Apache Tomcat Session clustering 구성 전 준비

1. Apache 설치

 

WEB) Apache 2.4.X Source Installation(소스 설치)

Apache Source 설치를 하는 이유 1) 모듈 및 설정 파일을 자신이 원하는 곳에서 관리하기 위해 2) RPM은 느리게 릴리즈되기 때문에 Source 설치를 함 (보안 취약점 발표시 빠른 대처 가능) 설치 ☞ 설치

lilo.tistory.com

 

2. Tomcat 설치

 

WAS) Tomcat Installation on Linux (Tomcat 설치)

INTRO 앞 글에서 소개한 WAS 제품 중 오픈소스인 Tomcat을 설치하는 가이드를 소개하려고 합니다. WAS) WAS에 대한 이야기 (4) - WAS의 종류 WAS(Web Aplication Server)의 종류 국내에서 많이 사용하는 솔루션을

lilo.tistory.com

 

WAS) Tomcat Multi Instance (2) - 멀티 인스턴스 구성

INTRO Tomcat Multi Instance를 구성하는 과정에 대한 내용을 공유드립니다. Apache Tomcat Multi Instance에 대한 자세한 내용은 아래의 글을 참고하기 바랍니다. WAS) Tomcat Multi Instance (1) - 구조 및 장단점 Apache T

lilo.tistory.com

 

3. Apache와 Tomcat 연동

 

WEB) Apache와 Tomcat 연동하기 (AJP,mod_jk)

INTRO Apache에서 접속하면 Tomcat의 APP이 바로 보일 수 있게 연동을 해주는 작업이 필요합니다. 연동을 하는 이유는 이전의 글을 참고하기 바랍니다. WAS) WAS에 대한 이야기 (5) - WAS와 Web Server를 분리

lilo.tistory.com

 

4. Tomcat Session Cluster 구성전 필요 사항

 

WAS) Tomcat Session clustering (1) - 개념 및 필요 사항

Apache Tomcat(WAS) Session clustering 구성 전 숙지 사항 1. HTTP 프로토콜은 Stateless protocol이다. ☞ stateless protocol: 이전의 요청과 상관없이 각각의 요청은 독립적인 트랜잭션으로 취급하는 protocol ☞ 요청

lilo.tistory.com

 

 

 

 

Apache Tomcat Session clustering 구성

 

테스트는 아래와 같이 구성하여 테스트했습니다.

  SW hostname IP
WEB Apache 2.4.51 web 192.168.0.9/24
WAS1 Tomcat 9 was1 192.168.0.10/24
WAS2 Tomcat 9 was2 192.168.0.11/24

 

테스트에 사용된 Application war 파일은 아래의 파일입니다.

제가 직접 개발한 aplication은 아니고 아래의 JBoss Developer 사이트를 참고하여 필요한 부분만 Archive하였습니다.

 

ReplicatedCounter| JBoss.org Content Archive (Read Only)

Are you sure you want to delete this article?

developer.jboss.org

counter.war
0.35MB

 

Apache 서버에서 workers.properties에서 worker.list를 확인합니다.

(나중에 jvmRoute에 사용될 내용이기 때문에 확인해야 됩니다)

 # vi /app/apache24/conf/extra/workers.properties
 
# LB Settings
worker.list=load_balancer
worker.load_balancer.type=lb
#worker.load_balancer.balance_workers=worker1,worker2,worker3
worker.load_balancer.balance_workers=worker1,worker2
worker.load_balancer.sticky_session=true

# Worker List
worker.worker1.host=192.168.0.10
worker.worker1.port=8109
worker.worker1.type=ajp13
worker.worker1.lbfactor=1

worker.worker2.host=192.168.0.11
worker.worker2.port=8109
worker.worker2.type=ajp13
worker.worker2.lbfactor=1

 

WAS1번에서 Tomcat 설정 파일 중 server.xml에서 Session Cluster 관련 설정합니다.

아래의 설정은 UDP Multicast를 사용한 Session Cluster(Session Replication)입니다.

 <WAS1번 설정>
 # vi /app/tomcat9/instance1/conf/server.xml
 ....  
    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="worker1">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->


    <!-- Session Cluster Setting  -->
        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>

          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
....

 

WAS2번에서 Tomcat 설정 파일 중 server.xml에서 Session Cluster 관련 설정합니다.

아래의 설정은 UDP Multicast를 사용한 Session Cluster(Session Replication)입니다

 <WAS2번 설정>
 # vi /app/tomcat9/instance1/conf/server.xml
 ....  
    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="worker2">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->


    <!-- Session Cluster Setting  -->
        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>

          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
....

 

 

WAS1, WAS2 서버에 배포한 web.xml 파일에서 "<distributable/>" 내용이 있는지 확인합니다.

만약 없다면 내용을 추가하여 배포하거나 테스트 용도의 war라면 배포를 하고 web.xml에서 "<distributable/>" 내용을 추가합니다.

# cat /app/tomcat9/instance1/webapps/ROOT/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

  <display-name>Counter</display-name>
  <description>Counter</description>

  <distributable/>

  <servlet>
     <servlet-name>CounterServlet</servlet-name>
     <servlet-class>org.jboss.example.counter.CounterServlet</servlet-class>
     <init-param>
        <param-name>titleMessage</param-name>
        <param-value>Counter Servlet</param-value>
     </init-param>
  </servlet>

  <servlet-mapping>
     <servlet-name>CounterServlet</servlet-name>
     <url-pattern>/*</url-pattern>
  </servlet-mapping>

  <session-config>
     <session-timeout>30</session-timeout>
  </session-config>

 

WAS1, WAS2 서버에서 Tomcat을 시작합니다.

<WAS 1번 서버>
# /app/tomcat9/instance1/bin/start.sh
Using CATALINA_BASE:   /app/tomcat9/instance1
Using CATALINA_HOME:   /app/tomcat9/engine
Using CATALINA_TMPDIR: /app/tomcat9/instance1/temp
Using JRE_HOME:        /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el8_5.x86_64/jre
Using CLASSPATH:       /app/tomcat9/engine/bin/bootstrap.jar:/app/tomcat9/engine/bin/tomcat-juli.jar
Using CATALINA_OPTS:   -server -Xms256m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -XX:+UseG1GC -verbose:gc -Xloggc:/app/tomcat9/instance1/logs/gc.log_20221105150425 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/tomcat9/instance1/logs/java_pid.hprof -XX:+PrintAdaptiveSizePolicy
Tomcat started.

<WAS 2번 서버>
# /app/tomcat9/instance1/bin/start.sh
Using CATALINA_BASE:   /app/tomcat9/instance1
Using CATALINA_HOME:   /app/tomcat9/engine
Using CATALINA_TMPDIR: /app/tomcat9/instance1/temp
Using JRE_HOME:        /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.332.b09-1.el8_5.x86_64/jre
Using CLASSPATH:       /app/tomcat9/engine/bin/bootstrap.jar:/app/tomcat9/engine/bin/tomcat-juli.jar
Using CATALINA_OPTS:   -server -Xms256m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -XX:+UseG1GC -verbose:gc -Xloggc:/app/tomcat9/instance1/logs/gc.log_20221105150435 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/tomcat9/instance1/logs/java_pid.hprof -XX:+PrintAdaptiveSizePolicy
Tomcat started.

 

 

시작 후 로그를 확인합니다. Member가 잘 join되었는지만 확인하면 됩니다.

<WAS 1번 서버>
# cat /app/tomcat9/instance1/logs/catalina.out

05-Nov-2022 15:08:04.842 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8180"]
05-Nov-2022 15:08:04.851 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-0.0.0.0-8109"]
05-Nov-2022 15:08:04.852 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [294] milliseconds
05-Nov-2022 15:08:04.864 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
05-Nov-2022 15:08:04.864 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.63]
05-Nov-2022 15:08:04.866 INFO [main] org.apache.catalina.ha.tcp.SimpleTcpCluster.startInternal Cluster is about to start
05-Nov-2022 15:08:14.893 INFO [main] org.apache.catalina.tribes.transport.ReceiverBase.bind Receiver Server Socket bound to:[/192.168.0.10:4000]
05-Nov-2022 15:08:14.900 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.setupSocket Setting cluster mcast soTimeout to [500]
05-Nov-2022 15:08:14.907 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for [1000] milliseconds to establish cluster membership, start level:[4]
05-Nov-2022 15:08:15.108 INFO [Membership-MemberAdded] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:[org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0, 11}:4000,{192, 168, 0, 11},4000, alive=6014, securePort=-1, UDP Port=-1, id={112 34 107 -125 74 78 65 -21 -88 36 -20 127 84 -36 -84 12 }, payload={}, command={}, domain={}]]
05-Nov-2022 15:08:15.908 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:[4]
05-Nov-2022 15:08:15.909 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for [1000] milliseconds to establish cluster membership, start level:[8]
05-Nov-2022 15:08:15.915 INFO [Tribes-Task-Receiver[Catalina-Channel]-1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:[104857600] bytes of type: [org.apache.catalina.tribes.io.BufferPool15Impl]
05-Nov-2022 15:08:16.909 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:[8]
05-Nov-2022 15:08:16.919 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/app/tomcat9/instance1/webapps/ROOT.war]
05-Nov-2022 15:08:17.012 INFO [main] org.apache.catalina.ha.session.DeltaManager.startInternal Register manager [localhost#] to cluster element [Engine] with name [Catalina]
05-Nov-2022 15:08:17.012 INFO [main] org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at [localhost#]
05-Nov-2022 15:08:17.016 INFO [main] org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions Manager [localhost#], requesting session state from [org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0, 11}:4000,{192, 168, 0, 11},4000, alive=7516, securePort=-1, UDP Port=-1, id={112 34 107 -125 74 78 65 -21 -88 36 -20 127 84 -36 -84 12 }, payload={}, command={}, domain={}]]. This operation will timeout if no session state has been received within [60] seconds.
05-Nov-2022 15:08:17.117 INFO [main] org.apache.catalina.ha.session.DeltaManager.waitForSendAllSessions Manager [localhost#]; session state sent at [11/5/22 3:08 PM] received in [105] ms.
05-Nov-2022 15:08:17.127 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/app/tomcat9/instance1/webapps/ROOT.war] has finished in [207] ms
05-Nov-2022 15:08:17.127 INFO [main] org.apache.catalina.ha.session.JvmRouteBinderValve.startInternal JvmRouteBinderValve started
05-Nov-2022 15:08:17.128 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8180"]
05-Nov-2022 15:08:17.143 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-0.0.0.0-8109"]
05-Nov-2022 15:08:17.147 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [12294] milliseconds


<WAS 2번 서버>
# cat /app/tomcat9/instance1/logs/catalina.out

05-Nov-2022 15:09:45.432 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8180"]
05-Nov-2022 15:09:45.447 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-0.0.0.0-8109"]
05-Nov-2022 15:09:45.447 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [324] milliseconds
05-Nov-2022 15:09:45.463 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
05-Nov-2022 15:09:45.463 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.63]
05-Nov-2022 15:09:45.464 INFO [main] org.apache.catalina.ha.tcp.SimpleTcpCluster.startInternal Cluster is about to start
05-Nov-2022 15:09:45.477 INFO [main] org.apache.catalina.tribes.transport.ReceiverBase.bind Receiver Server Socket bound to:[/192.168.0.11:4000]
05-Nov-2022 15:09:45.484 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.setupSocket Setting cluster mcast soTimeout to [500]
05-Nov-2022 15:09:45.486 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for [1000] milliseconds to establish cluster membership, start level:[4]
05-Nov-2022 15:09:45.539 INFO [Membership-MemberAdded] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:[org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0, 10}:4000,{192, 168, 0, 10},4000, alive=90625, securePort=-1, UDP Port=-1, id={-72 -63 -29 -79 103 50 71 -64 -122 95 99 -106 -53 -79 -81 120 }, payload={}, command={}, domain={}]]
05-Nov-2022 15:09:46.486 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:[4]
05-Nov-2022 15:09:46.488 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Sleeping for [1000] milliseconds to establish cluster membership, start level:[8]
05-Nov-2022 15:09:46.491 INFO [Tribes-Task-Receiver[Catalina-Channel]-1] org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer pool with max size:[104857600] bytes of type: [org.apache.catalina.tribes.io.BufferPool15Impl]
05-Nov-2022 15:09:47.489 INFO [main] org.apache.catalina.tribes.membership.McastServiceImpl.waitForMembers Done sleeping, membership established, start level:[8]
05-Nov-2022 15:09:47.498 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/app/tomcat9/instance1/webapps/ROOT.war]
05-Nov-2022 15:09:47.604 INFO [main] org.apache.catalina.ha.session.DeltaManager.startInternal Register manager [localhost#] to cluster element [Engine] with name [Catalina]
05-Nov-2022 15:09:47.604 INFO [main] org.apache.catalina.ha.session.DeltaManager.startInternal Starting clustering manager at [localhost#]
05-Nov-2022 15:09:47.608 INFO [main] org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions Manager [localhost#], requesting session state from [org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0, 10}:4000,{192, 168, 0, 10},4000, alive=92629, securePort=-1, UDP Port=-1, id={-72 -63 -29 -79 103 50 71 -64 -122 95 99 -106 -53 -79 -81 120 }, payload={}, command={}, domain={}]]. This operation will timeout if no session state has been received within [60] seconds.
05-Nov-2022 15:09:47.709 INFO [main] org.apache.catalina.ha.session.DeltaManager.waitForSendAllSessions Manager [localhost#]; session state sent at [11/5/22 3:09 PM] received in [105] ms.
05-Nov-2022 15:09:47.717 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/app/tomcat9/instance1/webapps/ROOT.war] has finished in [219] ms
05-Nov-2022 15:09:47.717 INFO [main] org.apache.catalina.ha.session.JvmRouteBinderValve.startInternal JvmRouteBinderValve started
05-Nov-2022 15:09:47.718 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8180"]
05-Nov-2022 15:09:47.727 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-0.0.0.0-8109"]
05-Nov-2022 15:09:47.730 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [2282] milliseconds

 

 

브라우저에 접속하여 Session ID를 확인합니다.

Chrome F12를 눌러 개발자 모드에 진입한 장면

 

해당 사용자는 2번으로 접속된 것을 확인하였고 세션의 가용성이 유지되는지 확인하기 위해 WAS 2번을 종료하고 브라우저를 확인합니다.

Chrome F12를 눌러 개발자 모드에 진입한 장면

반응형

'WEB&WAS > Apache Tomcat' 카테고리의 다른 글

WAS) Tomcat Multi Instance (2) - 멀티 인스턴스 구성  (0) 2022.11.13
WAS) Tomcat과 Scouter APM 연동하기  (0) 2022.11.06
WAS) Tomcat Session clustering (1) - 개념 및 필요 사항  (0) 2022.11.05
WAS) Tomcat Multi Instance 설치 스크립트  (0) 2022.10.30
WAS) Tomcat Multi Instance (1) - 구조 및 장점  (0) 2022.10.24

관련글

티스토리툴바