Avaya_Analytics_4.1.1.0_104_009
Avaya_Analytics_4.1.1.0_104_009
Avaya_Analytics_4.1.1.0_104_009
0
Avaya_Analytics_4.1.1.0_104_009 ReadMe
Note:
The build number (e.g. 5.3.xxx below) that shows up in version checks on the server is an internal build number.
The patch version is not displayed, it must be deduced from orca chart version and release notes.
e.g. Orca chart = 5.3.xxx
Release = 4.1.1.0
Patch number = 999
Patch Name = Avaya_Analy cs_4.1.1.0_xxx_999
From 4.1.1.0 Patch 5 onwards, this number has been incremented to 5.3.xxx so the nal 3 digits may be
lower than what was included in previous patches.
Patch Overview
This patch contains updated non-expiring license key for MSTR.
ti
ti
ti
ti
ti
fi
Pre-Staging Feature Overview
Pre-staging allows all Avaya Analy cs and Common Services so ware to be downloaded prior to
upgrading the system. This reduces the maintenance window required to upgrade the system as
previously this download had to be performed during the maintenance window.
2. Fill out the Deployment Excel provided with this Patch and transfer to the CCM.
This command will take around 15 minutes to run and will output the size of the download
required.
4. Prior to running the next command, please start a “screen” session to avoid the SSH connec on
ming out during the staging process.
This command will download all so ware required to upgrade the system. Depending on
bandwidth, this can take 1-2 hours to complete.
2
ti
ft
ti
ft
ft
ft
ti
Patch Install Steps
This table describes the steps required for each of the supported deployment paths. Links to the
appropriate sec on describing the steps are included.
1) Pre-Upgrade Steps
a) Verify system health by running ccm smoke-test and checkInfra
b) Follow the steps in the “Pre-installation instructions” section in this ReadMe.
2) Upgrade Products
a) If using the pre-staging feature highlighted in the Patch Overview, do so now.
b) Follow the steps in the “Upgrading Avaya Analy cs Services” chapter in this ReadMe to
apply the patch.
3) Post Upgrade Steps
a) Restart the MSTR srv pod.
Note: Ensure you logged in to CCM using cust and then switch to root user.
i) Wait un l pod is 1/1 a er the upgrade. That ensures all mstr objects are updated. If restart
is done before pod is in 1/1 state one more restart may be required.
ii) Find srv pod name: kubectl get pods -n mstr | grep mstr-srv
General
This patch includes:
Updates to mstr services
3
ti
ti
ti
ti
ti
ft
ti
ti
Current Active Line-up
Analy cs 4.1.1.0 follows a cumula ve patch line-up. Unless otherwise stated this patch rolls up all
previous patches.
The following mings are indica ve and approximate mings to help be er plan the maintenance
window. These mings may vary based network bandwidth, hardware capacity, etc.
Approximate Timing
Pre-installation instructions
Important:
1. Before installing this patch ensure you take a database backup using the Analy cs control script.
• See sec on Avaya Analy cs™ backups in Chapter: Maintaining Avaya Analy cs™ using
the Cluster Control Manager console of the Maintaining and Troubleshoo ng Avaya
Analy cs for Oceana guide.
2. Before installing this patch ensure all custom reports have been archived.
o Method
• Run ccm release orca analy cs
• Select “Historical Repor ng”
• Select “Backup Metadata”
• Select “Backup Metadata” again
• Wait for the backup to nish
• Press b for back
• Select 4. Historical Repor ng
• Select 8. Export Backups
• Wait for export to nish
• Press q to quit
• Verify that the backup is now located in the /home/<customer_account_login>/
historical_md_backups/ directory on CCM.
3. Before installing this patch ensure the following creden als are s ll valid.
o Harbor
o vCenter
4
ti
ti
ti
ti
ti
ti
fi
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
tt
ti
ti
ti
Use the checkInfra script to check vCenter creden als, con gura on and network connec vity.
5
ft
ti
fi
ti
fi
fi
ti
fi
ti
ti
fi
ti
ti
ti
ti
fi
ti
ti
ti
fi
ti
fi
fi
ti
ti
fi
fi
ti
ti
ti
ti
Upgrade Avaya Analytics Services
Before You Begin
Ensure the correct installa on method is chosen. This will most likely be the method used at the original
installa on me. Review the following chapters of the Deploying Avaya Analy cs for Oceana guide if
unsure:
Important:
Manual Con g. Map changes are not supported. Patches will overwrite these.
1. Connect to the CCM server using the customer account log in using cust and then switch to root
user.
3. If output similar to above is observed, then DB pods are in Rainy Day/non-standard state.
Primary de ni on is poin ng at replica pod as end point while replica is poin ng at primary pod.
If in this state follow the steps below otherwise proceed with appropriate upgrade procedure.
6
ti
fi
ti
fi
ti
ti
ti
ti
ti
ffl
ti
ti
[root@ccm112 cust]#
6. Check the status of the crunchy pod. Pod status should be the same as below. This is Sunny Day
state in terms of primary/replica roles. If so, con nue with upgrade.
Online Procedure
1. Connect to the CCM server using the customer account log in.
3. Switch to root user and execute the following command as root to remove crunchy watch before
star ng upgrade
4. From the directory on CCM that contains the excel le, enter the following command with cust
user:
screen
If the Prometheus pod is not star ng or an Async pods in a 0/1 Ready state and either of this
cause the upgrade to not complete, then the valida on checks on these pods can be skipped by
adding the --force op on to the end of the command
7
ft
ti
ti
ti
ti
fi
ti
ti
6. When prompted:
d. Con rm Upgrade
tail -f /var/log/avaya/ccm/ccm-main.log
10. Execute the following command as root to set the replica count for crunchy watcher back to 1.
11. Con rm the opera onal state of the Analy cs cluster. Run the following command:
All pods should report running with n/n repor ng ready before con nuing to the next step. Refer
to Post-installa on veri ca on chapter of Deploying Avaya Analy cs for Avaya Oceana for
further details on con rming the status of Analy cs.
Important:
If the ccm upgrade spec command fails, resolve the issue causing the failure and restart the
upgrade by running: ccm upgrade resume command. For more informa on about troubleshoo ng
installa on failures, see Maintaining and Troubleshoo ng Avaya Analy cs™ for Avaya Oceana.
O ine Procedure
Downloading Avaya Analy cs chart and images
Before you begin you must have:
8
ffl
fi
fi
ti
ti
ti
ti
ti
ti
fi
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
• A func onal ccm-agn-ctl container on your laptop.
11. To view a list of the downloaded images, run the following command:
docker image ls
12. To view a list of the downloaded charts, run the following command:
ls /root/downloads/*.tgz
13. (Op onal) If you see a docker pull error, you can view or retrieve the logs within the ccm-agn-ctl
container at /var/log/avaya/ccm/ccm-main.log.
For more informa on on possible issues and the respec ve troubleshoo ng solu ons, see the
Maintaining and Troubleshoo ng Avaya Analy cs for Oceana guide.
agn-ctl start
14. Connect to your air gap network using your Windows PC or laptop.
15. Start the ccm-agn-ctl container by using the following ccm-agn-ctl.bat le in the Windows
PowerShell console: C:\avaya\ccm-agn-ctl.bat
9
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
fi
ti
ti
16. Using the ccm agn deployed container, run the following command: agn upload <CCM FQDN>,
where <CCM FQDN> is the FQDN of your CCM.
17. To access the CCM docker registry and ChartMuseum, enter the username when prompted.
18. Enter the password.
19. Re-enter the password.
The agn command starts the following in a sequence:
20. Processes the available chart and image data on the Windows PC or laptop.
21. Starts uploading the charts and images to CCM. When the upload is complete, the console
displays an image push report.
22. (Op onal) If you see a docker pull error, you can view or retrieve the logs within the ccm-agn-ctl
container at /var/log/avaya/ccm/ccm-main.log.
For more informa on on possible issues and the respec ve troubleshoo ng solu ons, see the
Maintaining and Troubleshoo ng Avaya Analy cs for Oceana guide.
23. To copy the <solu on spreadsheet name>.xlsm le, run the following command in the ccm-agn-
ctl container:scp /root/downloads/
<solu on spreadsheet name>.xlsm <ccmUser>@<CCM FQDN>:,
where <ccmUser> is the CCM customer login account and <CCM FQDN> is the FQDN of your CCM.
24. In the Are you sure you want to con nue connec ng eld, type yes and press Enter.
25. At the prompt, enter the CCM user password.
The ccm-agn-ctl container uploads the images and charts, which you earlier downloaded
on the Windows PC or client laptop, into the local CCM docker registry and chartmuseum.
10
ti
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
fi
ti
ti
fi
fi
ti
ti
ti
If the Prometheus pod is not star ng or an Async pods in a 0/1 Ready state and either of these
cause the upgrade to not complete, then the valida on checks on these pods can be skipped by
adding the --force op on to the end of the command
32. When prompted:
33. Enter your Avaya SSO creden als.
34. Accept the EULA.
35. Enter the vCenter user ID and password.
36. Re-con rm the password.
37. The installa on starts downloading and installing the following:
• The Avaya Analy cs™ so ware
38. Run the following command to monitor the progress of the install:
tail -f /var/log/avaya/ccm/ccm-main.log
39. The upgrade command will exit when complete
40. Con rm the opera onal state of the Analy cs cluster. Run the following command:
ccm status --pod-details
All pods should report running with n/n repor ng ready before con nuing to the next step. Refer
to the post-installa on veri ca on chapter of Deploying Avaya Analy cs for Avaya Oceana for
further details on veri ca on.
Important:
If the ccm upgrade spec command fails, resolve the issue causing the failure and restart the upgrade by
running: ccm upgrade resume command. For more informa on about troubleshoo ng installa on
failures, see Maintaining and Troubleshoo ng Avaya Analy cs™.
11
fi
fi
ti
ti
ti
ti
ti
fi
ft
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
Post Installation Instructions
i) Wait un l pod is 1/1 a er the upgrade. That ensures all mstr objects are updated. If restart
is done before pod is in 1/1 state one more restart may be required.
ii) Find srv pod name: kubectl get pods -n mstr | grep mstr-srv
1. Follow the instruc ons in the chapter: Post-installa on veri ca on of Deploying Avaya Analy cs
for Oceana guide.
ccm smoke-test
b. Login into Workspace and verify the real- me reports are upda ng.
5. Remove the CCM Virtual Machine snapshot taken at the beginning of the upgrade procedure.
Failure to do so can result in performance degrada on of the CCM as well as the ESXi host.
12
ti
ti
ti
ti
ti
ft
ti
ti
ti
ti
ti
ti
fi
ti
ti
ti
Rollback Procedure
In the event a patch install cannot be completed, and it is required to roll back to the prior release use
the following procedure.
i. Connect to the CCM server using the customer account log in.
ii. Copy excel sheet backed up during Pre-installa on instruc ons to a loca on on the CCM
server.
iii. From the directory on CCM that contains the excel le, enter the following command:
screen
v. When prompted:
• Con rm Upgrade
vii. Run the following command to monitor the progress of the install:
tail -f /var/log/avaya/ccm/ccm-main.log
viii. To check if the installa on is successful, run the following command on the CCM console:
ccm status
13
fi
ti
ti
ft
fi
ti
fi
ti
ti
kubectl cp <full_backup_Date-Time.bkp> --namespace=mstr <mstr-
md-hashed-number>:/opt/app-root/src/md_full_backup/
<full_backup_Date-Time.bkp>
viii. The output on the console can have to Errors pertaining to the postgres user. These can be
ignored.
psql:/opt/app-root/src/md_full_backup/
full_backup_2020_05_27_16_20_23.bkp:23: ERROR: current user cannot be
dropped
psql:/opt/app-root/src/md_full_backup/
full_backup_2020_05_27_16_20_23.bkp:31: ERROR: role "postgres" already exists
x. List available databases with \l command. Con rm the avaya_analy cs_md db.
postgres=> \l
DR Installation
If installing this patch into a solu on with a disaster recovery site, the following is the recommended
order:
Note: Data from the database will be replicated between the primary site and the DR site in the normal
way. See Deploying Avaya Analy cs for Oceana guide for details.
Before star ng installa on on DR site Ensure DB pods are in standard state as explained in sec on Verify
DB Pods in standard state.
14
ti
ti
ti
fi
ti
ti
ti
fi
ti
ti
ti
ti
15
Deployment Excel Deployment Options
The deployment excel now supports two deployment op ons, Produc on and Lab
• The “Lab” op on is for lab only installa ons adding support for Async as part of a Non-
HA deployment.
• The “Produc on” op on must be selected for all produc on-based deployments as lab
based deployment will not be supported in produc on.
16
ti
ti
ti
ti
ti
ti
ti
ti
Known Issues
Issue 1
Upgrade may fail due to lack of space.
This workaround is only required if the space usage of /var/lib/docker/ is above 75%.
17
tt
ti
ti
ti
ft
Here remove the “4.1.1.13” images and not the “4.1.1.14” images.
6. Important: Before proceeding, take note of each Image displayed, including Image Name, Image
Tag and Image ID. You will need this informa on later.
7. Run "docker image rm" for each of the relevant Image IDs
docker image rm <Image ID 1> <Image ID 2> <Image ID 3> ...
Example: docker image rm 663e8a4e29d0 76b929a2c9a1
8. Once nished cleaning up the Images from CCM, they will also need to be purged from the
cluster registry
9. Create a list of images available in the cluster registry
ccm registry repos --tags | grep orca
ccm registry repos --tags | grep mstr
10. For each image removed in step 7, run:
ccm registry purge <image name> <image tag>
Example: ccm registry purge ex/mstr-av-web11.2.1 4.1.12
Note: there is a space between <image name> and <image tag>
Issue 2
The following message may be reported during install.
WARNING: Service release "orca" did not become ready in the allo ed me. Processing will con nue.
This usually can be ignored as the it is just a warning that a pod is slower than others to start-up. If a er
the rest of the install completes all orca pods look healthy, then there is no issue here.
18
fi
ti
fl
ti
ti
fi
tt
ti
ti
ft
Appendix
Solu ons provided in Avaya_Analy cs_4.1.1.0_104_009
JIRA ID Title Impact
WAVE-50404 MSTR Master Licenses Key Update: Patch Update non-expiry licence key in mstr for
Crea on with new non_expiry licence keys analy cs 4.1.1.0 patch 8.
19
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
fi
ti
ti
ft
ti
ti
fi
ti
ti
ti
ti
ti
ti
WAVE-18678 Agent logon dura ons are doubled when Agent Logon dura ons were repor ng
searching by Agent Groups incorrectly on Historical Reports
WAVE-18572 Hold me dura on is not calculated Hold Time Dura on repor ng inaccurately
correctly when consult call is held for consult calls
WAVE-18280 Agent Login\Logout report missing Historical reports missing data for Agent
records Login/Logout
WAVE-18158 Analy cs 4.1.0.1 Patch 3 Excel Deployment Excel did not re ect the
Spreadsheet no longer updates Available Available Hosts entry correctly
Hosts cell for non-HA con gura on
WAVE-18139 Maintaining and Troubleshoo ng M&T Guide missing informa on to
method needed when 'Send now' op on troubleshoot a issue
sends rst page only of PDF
WAVE-18121 Data on Real me Repor ng Dashboard Delay in midnight reset on Real me
not cleared at Midnight but 8 hours later Dashboards
WAVE-18069 Abnormal on not ready Meal Break code Not Ready dura on calculated incorrectly
which exceed login me on certain Reason Codes
WAVE-17816 Contacts Wai ng does not decrement Contacts Wai ng Real me Dashboard was
correctly repor ng inaccurately
WAVE-16381 Rou ng service group are shown not Rou ng Service Group prompts in
assigned to supervisor in historical Analy cs Historical Reports may show all
reports groups which contain rou ng services
assigned to supervisor while supervisor
may be not assigned sone of these groups.
WAVE-14681 Help les for Standard Func ons (used in Historical Repor ng Help Files were not
Reports) are missing func oning
WAVE-6329 MSTR shows all the groups (not Agent Group Filter was showing all Agent
monitored by supervisor) to which agent groups that an Agent was assigned
is assigned incase agent is also part of regardless of what Agent Group the
monitored group of logged in supervisor Supervisor was assigned
20
ti
ti
ti
fi
ti
ti
ti
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
fi
ti
ti
ti
ti
fl
ti
ti
ti
ti
ti
ti