kafka_zookeper

The purpose of this post is to show how IBM Commerce v9 works with kafka and zookeper. We start from a already configured KAFKA and ZOOKEPER containers installed, we do not cover the installation or setup, maybe I’ll do that in a different post.

Where our adventure begins

We are using docker-compose files to deploy our images. docker ps

CONTAINER ID        NAMES                   IMAGE                                                         CREATED              STATUS
1d2dd42f442a        wcs_store_1             10.176.103.130:5000/commerce/crs-app:9.0.1.2                  4 days ago           Up 4 days (healthy)
483e820d2398        wcs_web_1               10.176.103.130:5000/commerce/ts-web:9.0.1.2                   4 days ago           Up 4 days (healthy)
6c069caa391c        wcs_search_slave_1      10.176.103.130:5000/commerce/search-app:v9012-ltr-search-774   4 days ago           Up 4 days (healthy)
793d37593563        wcs_kafka_1             10.176.103.130:5000/ches/kafka:latest                         4 days ago           Up 4 days (healthy)
16c25f94c9a5        wcs_search_repeater_1   10.176.103.130:5000/commerce/search-app:v9012-ltr-search-774   4 days ago           Up 4 days (healthy)
5d0e7773757f        wcs_utils_1             10.176.103.130:5000/commerce/ts-utils:v9012-ltr-util-774       4 days ago           Up 4 days (healthy)
09277557c010        wcs_zookeeper_1         10.176.103.130:5000/zookeeper:latest                          4 days ago           Up 23 hours (unhealthy)
652def4df73c        wcs_txn_1               10.176.103.130:5000/commerce/ts-app:v9012-ltr-ts-774           4 days ago           Up 4 days (healthy)
d078bb64d7d2        wcs_xc_1                10.176.103.130:5000/commerce/xc-app:9.0.1.2                   4 days ago           Up 4 days

By default docker compose creates its own docker networks components, and below is the listing: docker network ls

NETWORK ID          NAME                DRIVER              SCOPE
6a2eed1fa166        bridge              bridge              local
32d98311bb09        host                host                local
6ee485982f5a        none                null                local
5b0b97016d25        wcs_default         bridge              local

Step 0 - Get ready

Server URLs to create caching entries

With all the configuration identified, we now need:

1. To identify a URL https://myhostname/myurl
2. Ensure that the path for this URL is defined in the cachespecs.xml
3. Use the cachemonitor application to ensure entries are being cached (hint: here is the URL for the cahce monitor application after installed https://1.11.111.2:5443/cachemonitor

Ensure the configuration matches

In order to use kafka inside the container you will need to always unset the JMX_PORT. You will see this unset JMX_PORT; always ahead of the command to run as in step 3 below.

1. login to the server and run: docker exec -it wcs_kafka_1 bash
2. once inside the kafka@kafka enter cd bin
3. unset JMX_PORT;kafka-topics.sh --list --zookeeper zookeeper Which will list

   WCdemo
   WCdemoCacheInvalidation
   __consumer_offsets
   

WCdemo is the topic queue setup in the txn image during the build process. For more information on configuring kafka refer to this other post

Step 1 - Enable caching tracing

1. Enable extended tracing with the following string: com.ibm.commerce.dynacache.*=all

See all the msgs

unset JMX_PORT;kafka-console-consumer.sh --zookeeper zookeeper --topic WCdemoCacheInvalidation --from-beginning

cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+STOREENT_ID+NAME:%:0:%:skipConnectOnceStudent
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+NAME:%:skipConnectOnceStudent
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+STOREENT_ID:%:0
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+STOREENT_ID+NAME:%:0:%:DATABASE_TIMEZONE
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+NAME:%:DATABASE_TIMEZONE
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF+STOREENT_ID:%:0
cache:services/cache/WCStoreDistributedMapCache===WCT+STORECONF