Document toolboxDocument toolbox

.Installation with Docker-compose v13.3.2

Upgrading from 13.3.1

If VRS 13.3.1 is already installed then we will need to create a group in Keycloak that will be assigned to agents upon first time login to VRS.

  1. Run ./deployment.sh in the home directory of the solution.
  2. Run ./install.sh in recording-solution directory.


deployment.sh

Installation Steps

Internet should be available on the machine where the application is being installed and connections on port 9242 should be allowed in the network firewall to carry out the installation steps. 


All the commands start with a # indicating that root user privileges are required to execute these commands. Trailing # is not a part of the command.


Allow ports in the firewall

To start the firewall on CentOS (if it isn't started already), execute the following commands:  

# systemctl enable firewalld
# systemctl start firewalld

To allow the ports on CentOS firewall, you can execute the following commands. (Run on both machines in case of HA). 

# firewall-cmd --add-port=443/tcp --permanent
# firewall-cmd --add-port=8088/tcp --permanent
# firewall-cmd --add-port=5060/tcp --permanent 
# firewall-cmd --add-port=16386-32768/udp --permanent 
# firewall-cmd --add-port=9092/tcp --permanent 
# firewall-cmd --reload


Installation Steps

  1. Please make sure that Solution Prerequisites are met for the desired deployment type. 
  2. Download the deployment script deployment.sh and place it in the user home or any desired directory. This script will:
    1. delete the recording-solution directory if it exists.
    2. clone the required files for deployment
  3. To execute the script, give it the execute permissions and execute it. 

    $ chmod 755 deployment.sh
    $ ./deployment.sh
  4. Change to newly created directory with name recording-solution. This directory contains all the required files.

  5. Run SQL script in SQL Server to create database and tables.  (recording-solution/db_schema.sql).
  6. Create an Application User in CUCM with Standard CTI Enabled permission group. Add all devices that are supposed to be recording enabled to thus user controlled device list. This user will be used to add JTAPI Observer to observe the call END events.
  7. Create a database for Keycloak on same machine as VRS database
  8. Update environment variables  in the following files inside /root/recording-solution/docker/environment_variables folder.

    1. general-environment.env

      Name

      Description

      DB_URL

      VRS database connection URL

      jdbc:jtds:sqlserver://192.168.1.92:1433/vrs;user=sa;password=Expertflow464

      DB_DRIVER

      JDBC driver

      net.sourceforge.jtds.jdbcx.JtdsDataSource


      CC_TYPECisco Contact center type (UCCX or UCCE)
      TZTimezone (Asia/Karachi)
      ENCRYPTION_ENABLED

      To enable/disable recorded file encryption

      true = enabled

      false = disabled

      AMQ_PRIMARYPrimary ActiveMQ URL, (VRS machine IP)
      tcp://192.168.1.242:61616
      AMQ_SECONDARYSecondary ActiveMQ URL, Keep it same as primary if ActiveMQ not available in HA
      AMQ_TIMEOUT3000, keep it same
      AMQ_RANDOMIZEfalse, keep it same
      AMQ_PRIORITY_BACKUPtrue, keep it same
      LOCAL_MACHINE_IPVRS machine IP
      CUCM IPCisco Call Manager IP
      CUCM_APPLICATION_USER_NAMECUCM Application user's username created in step 6.
      CUCM_APPLICATION_USER_PASSWORDCUCM Application user's password created in step 6.
      TIME_CUSHIONThe number of seconds to add to the start and end time of call when calling API from CIM. There are few seconds difference between CIM interaction's start and end time and recording solution start and end time since CIM fetch interactions from Finesse while recording solution gets time from CUCM 
      MAX_RING_TIMEMaximum Call ring time on agent desktop, default is 30 seconds
      JOB_TIMERMaximum interval time for execution call merging submission to the mixer. Default is 100000 milliseconds
      CALL_TIMEOUTSocket timeout for recording rtp packets, set it to 10
      THREAD_TIMEInterval in seconds between two jobs that clears completed calls, set it 10
      FILE_EXTENSIONExtension that archival file will look for file to archive. Set it to "wav"
      DIRECTORY_PATH_TO_MONITORThis and following 9 variables are used for the archival process. This variable will hold the path of the recording
      ARCHIVED_MEDIA_FILES_EXTENSIONThe archival process will archive recordings with this extension, set it to "wav"
      NO_OF_DAYSThe number of days to keep recordings in the primary server. Recordings older than this value days will be archived
      SFTP_HOSTSFTP hostname or IP
      SFTP_PORTSFTP port
      SFTP_USERNAMESFTP username
      SFTP_PASSWORDSFTP password
      ARCHIVAL_JOB_INTERVALArchival process will run every this values seconds and archive any pending archival recordings
      RETRY_LIMITNumber of retries on pending archival recording folders
      ARCHIVE_PATHThe shared path on the archival server where archival process archive recordings
      ARCHIVE_PATH_USERArchive path's machine user
      ARCHIVE_PATH_PASSArchive path's machine password
      ARCHIVAL_PROCESS_NODE

      This variable value should be "active" on once machine and "passive" on second machine in HA.

      "active" machine archival process will sent files to SFTP server and then delete. "passive" machine 

      process will only delete local file. (HA Only)

      UCCE_DB_URL

      UCCE awdb database connection URL, used for UCCE deployment only.

      jdbc:jtds:sqlserver://192.168.1.87:1433/ucce_awdb;user=sa;password=Expertflow464
      Used for UCCX deployment only...
      UCCX_URLUCCX URL, used for fetching agent details, 
      https://192.168.1.101
      UCCX_USERNAMEUCCX user,  should have privileges to fetch agents
      UCCX_PASSWORDUCCX user password
      KEYCLOAK_USERmaster username for Keycloak, 
      KEYCLOAK_PASSWORDpassword for Keycloak user
      DB_VENDORKeycloak database engine, keep it default
      DB_USERKeycloak database username
      DB_PASSWORDKeycloak database password
      DB_ADDRKeycloak database machine IP
      DB_DATABASEDatabase name created in step 7
      Update below environment variables once Keycloak is setup in step 11
      KEYCLOAK_REALM_NAMERealm name created in step 11
      KEYCLOAK_CLIENT_IDKeycloak client id from step 11
      KEYCLOAK_CLIENT_SECRETKeycloak client secret from step 11
      KEYCLOAK_URL

      keep default

      FINESSE_URL
      HTTP address of finesse server to validate credentials given by agent.
      KEYCLOAK_ADMIN_USERNAME
      Keycloak admin username from step 11
      KEYCLOAK_ADMIN_PASSWORD
      Keycloak admin password from step 11
      KEYCLOAK_PERMISSION_GROUP
      Keycloak Group from step 11
      KEYCLOAK_USER_PASSWORD
      Hard Coded private password that remains same for every new agent created in keycloak (example: 12345)
      TRUST_STORE_PATH/app/ssl/truststore.jks (The Path of the trust store, this is the certificate required for SSO)
      TRUST_STORE_PASSWORDTruststore Password
    2. Note:  If the finesse deployed with https, then certificate must be provided to access it, we use 'truststore.jks' file in ssl directory to provide certificate store, to place certificate in truststore, simply run this command in directory where certificate is present and later export the truststore to your deployment ssl directory. 

    3. Command to add certificate is certificate store:

      keytool -import -alias ccx -file ccx.cer -keystore truststore.jks -storepass Expertflow464


  9. (HA OnlyHaving environment configurations done, copy the recording-solution directory on VM2 in/root directory using the following command.

    # scp -r /root/recording-solution root@<vm-ip>:/root/
  10. Execute the following commands inside /root/recording-solution directory. 

    # chmod 755 install.sh
    # ./install.sh
  11. Set up keycloak
  12. Once keycloak is setup up, update these environment variables (KEYCLOAK_REALM_NAME, KEYCLOAK_CLIENT_ID, KEYCLOAK_CLIENT_SECRET, KEYCLOAK_ADMIN_USERNAME, KEYCLOAK_ADMIN_PASSWORD, KEYCLOAK_PERMISSION_GROUP) and run ./install.sh again
  13. Run the following command to ensure that all the components are up and running. 

    # docker ps

    Note: We need to run below query in the VRS DB

    alter table sessions add is_archived int default 0
  14. Go to https://VRS-IP/#/login to access the application
  15. Follow this guide to deploy VRS Finesse gadget
  16.  (HA Only) Now go to VM2, update LOCAL_MACHINE_IP  variable to VM2 IP in root/recording/solution/docker/environment variables/recorder-environment.env file  and run below command inside /root/recording-solution to start recorder and activemq services. The two activemq services on VM1 and VM2 will now act as master/slave to provide HA. The two recorder services on VM1 and VM2 will be configured in Cisco Call Manager (CUCM) to provide HA.  

    # chmod 755 install.sh
    # ./install.sh
  17. (HA Only) The directory "/root/recording-solution/recordings/wav"  should also be mounted on network shared file system on both VMs or they should be synchronized with each other . In this way, all services on two VMs will have a shared directory for recording files reading or writing. Follow next step if network shared and synchronized folder is not provided
  18. (HA Only) Recording folder synchronization, follow below steps;
    1. Install lyncd utility on one machine, run below commands. 

      root@host # yum -y install epel-release
      root@host # yum -y install lsyncd
    2. Generate SSH Keys on same. Run below command to generate a key. Use default by pressing enter every time it prompts 

      root@host # ssh-keygen -t rsa
    3. Transfer the SSH key to the other other machine by running below commands, enter other machine root password when prompted 

      ssh-copy-id root@other-machine-ip
    4. vi ~/.ssh/config

      enter below text in config file, replace the Hostname with other machine IP

      Host dest_host
       Hostname 172.16.144.32
       User root
       IdentityFile ~/.ssh/id_rsa
    5. settings {
              logfile = "/var/log/lsyncd/lsyncd.log",
              statusFile = "/var/log/lsyncd/lsyncd-status.log",
              statusInterval = 1
      }
      
      sync {
              default.rsync,
              source="/root/recording-solution/recordings",
              target="192.168.1.125:/root/recording-solution/recordings",
              delete = false,
              rsync={
                     compress = true,
      acls = true,
      verbose = true,
      owner = true,
      group = true,
      perms = true,
      rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"}
      }
    6. Follow above steps for the other machine

  19. (HA Only) Repeat the following steps on both machines.

    1. Download keepalived.sh script and place it in any directory.
    2. Give execute permission and execute the script. This will create a keep-alived directory.

      # chmod +x keepalived.sh
      # ./keepalived.sh

    3. Configure keep.env file inside keep-alived directory

      Name

      Description

      Name

      Description

      KEEPALIVED_UNICAST_PEERS

      IPs of the machines in the cluster. On each machine, this variable should have a list of IPs of all the other machines in the cluster. The format of the list is as below: 

      192.168.1.80

      KEEPALIVED_VIRTUAL_IPSVirtual IP of the cluster. It should be available in the LAN. For example: 192.168.1.245
      KEEPALIVED_PRIORITYPriority of the node. Instance with lower number will have a higher priority. It can take any value from 1-255. 
      KEEPALIVED_INTERFACEName of the network interface with which your machine is connected to the network. On CentOS, ifconfig or ip addr sh will show all the network interfaces and assigned addresses. 
      CLEARANCE_TIMEOUTCorresponds to the initial startup time of the application in seconds which is being monitored by keepalived. A nominal value of 60-120 is good enough
      KEEPALIVED_ROUTER_IDDo not change this value.
      SCRIPT_VAR

      This script is continuously polled after 2 seconds. Keepalived relinquishes control if this shell script returns a non-zero response. It could be either umm or ECM backend API.

      pidof dockerd && wget -O index.html https://localhost:443/

    4. Give the execute permission and execute the script: 

      # chmod +x keep-command.sh
      # ./keep-command.sh

      19. Place the Keystore.jks file containing Keycloak certificate and finesse certificate inside ssl directory in our deployed project. The path to ssl directroy is as follow: {project directory}/recording-solution/ssl/

Troubleshooting





Configure Log Rotation


Add the following lines in 
/etc/docker/daemon.json file (create the file if not there already) and restart the docker daemon using systemctl restart docker. Perform this step on all the machines in the cluster..

{  
    "log-driver": "json-file"
    "log-opts": {
        "max-size": "50m",
        "max-file": "3"
    
}