Tested with UCCX 12-5 for ST 13.5
HW specs of the tested VM
VM | CPU | RAM | Disk Size |
|---|---|---|---|
VM (ST deployed) | 2 |
...
4GB |
...
100GB |
...
SQL VM | 4 vcpu | 8GB | 145GB |
Uploading CSV Files (Diff callers volume) in Caller List (Performance Testing):
Caller list | Upload time | Uploaded | Failed | Total in db | File size | Delete through generic clear list (Time) |
|---|---|---|---|---|---|---|
200k | 2 mins | 200k | 0 | 200k | 4-5 mins | |
300k | 8-9 mins | 300k | 0 | 300k | 7-8 mins | |
200k | 3 mins | 200k | 0 | 200k8064kb8064kb | - | |
200k | 1 min | 99999 | 100001 | 99999 | 5913kb | |
178845 | 57 sec | 99999 | 78846 | 5407kb | ||
100k | 53 sec | 98784 | 1216 | 4677kb | ||
100k | 3 mins | 100k | 0100k | 3994kb | ||
25k | 34 sec | 25k | 25k | 64 secs | ||
25k | 34 sec | 25k | ||||
50k | 24 sec | 50k | 050k | |||
194817 | 27 lacs callers db | 182 secs (also a delay of callers visibility | ||||
594817 | 25 lacs callers db | 708 sec ( some delay in callers visibility in caller list) | ||||
300k (latest) | 324 sec (5-6 min) | 300k | ||||
200k (latest) | 138 secs (2-3 min) | 200k | ||||
178k (latest) | 60 sec (1min) | 99999 | 78846 | |||
178k | 178 sec | 99999 | 78846 | |||
178k | 89 sec | 178k | 5636kb | |||
300k (latest) | 239 sec | 300k | 13954kb | |||
178k | 79 sec | 2635877 | 5636 kb | |||
300k | 108 sec | |||||
650k | 249 sec ( 5-7 mins delay in visibility of callers in caller list) |
Note: A delay of approximately 5 to 8 minutes occurs after uploading the CSV on the UI before callers become visible in the caller list.
...
Load Test:
Software for load test: Apache JMeter
API load testing is done for caller list, Easy Announcement, Business Calendar
GET API for Easy Announcement: Retrieves details of easy announcements, including ID, name, and prompts string.
GET API for Caller List: Checks if a specific caller exists in the list and returns associated prompts.
GET API for Service Status for Business Calendar: Determines the status of a calendar, indicating if the agency is Open, Closed, has a Holiday, or an Exceptional opening or closing. For more detail visit
Load test result is run with the following configuration:
Number of threads (users): 1 user will simulate the load.
Ramp-up period: Each user will be started within 1 second of the test start.
Loop count: Each user will execute the test plan 100,000 times.
Thread lifetime: The test will run for 86,400 seconds (24 hours).
Throughput: The target throughput is 70.
Total callers in db 463948
Load test started for 24 hours for 100k (threads) each for All caller list and caller list easy announcement and business calendar concurrently and detailed load test results can be view below.
...
Thread Groups: Four, running concurrently
Load Test Result for Caller List:
Loop count: 100,000
Total callers in db 463948
Tested Scenarios:
Checking if a provided calling number exists in a All caller list containing 463,948 callers.
Verifying if a calling number exists in a caller list of 100,000 callers.
Concurrency: Testing both scenarios concurrently (All caller list and caller list easy announcement and business calendar).
...
.
Thread Group 1:
Sr.no | API | Duration | Constant Throughput Timer | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 10 sec | ||||||||
GET API | 24 hr (86400 sec) | 70 | 1 | 1 | 60409 | 0.05 sec | 187 sec | 0.7 sec | 2.74 % ( 1657 request 10 sec) 1836 request (5 sec) 3.03% | Fail | |
This table summarize summarizes the test result for All caller all callers List total callers 463948 in All callers.
...
Observations:
The test successfully processed 60,409 HTTP requests within the specified duration (24 hr).
The response time ranged from 0.05 seconds to 187 seconds, with an average response time of 0.7 seconds.
A small percentage (2.74%) of requests experienced response times exceeding the threshold of 10 seconds.
...
Sr.no | API | Duration | Constant Throughput Timer | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 10 sec | ||||||||
GET API | 24 hr (86400 sec) | 70 | 1 | 1 | 60393 | 0.055 sec | 187 sec | 0.7 sec | 2.74 % (1659 request 10 sec) 3.03(1835 request 5 sec) | Fail | |
This table summarize summarizes the test result for a caller List having 100k caller in the caller list
Observations:
The test successfully processed 60,393 HTTP requests within the specified duration (24 hr).
The response time ranged from 0.05 seconds to 187 seconds, with an average response time of 0.7 seconds.
A small percentage (2.74%) of requests experienced response times exceeding the threshold of 10 seconds.
...
Investigate and optimize the performance of the API to reduce the occurrences of response times exceeding 10 seconds.
Thread Group 3:
Sr.no
...
API
...
Duration
...
Constant Throughput Timer
Threads (users)
...
Ramp up
(sec)
...
Number of
sample( total http request hits)
...
Response Time(sec)
GET API
...
Benchmark
...
Result
...
Min
...
Max
...
Avg
...
Response time for GET API
should not be more than 10 sec
...
GET
API
...
24 hr
(86400)
...
70
...
1
...
1
...
100k
...
0.027
sec
...
5.9
sec
...
0.05
sec
...
0.00 %
...
Pass
This table summarize the test result for Easy Announcement
...
Observations:
The test successfully processed 100k HTTP requests within the specified duration (24 hr).
The response time ranged from 0.02 seconds to 5.9 seconds, with an average response time of 0.05 seconds.
A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.
Thread Group 4:
...
Sr.No
...
API
...
Duration
...
Constant Throughput Timer
Threads (users)
...
Ramp up
(sec)
...
Number of
sample( total http request hits)
...
Response Time(sec)
GET API
...
Benchmark
...
Result
...
Min
...
Max
...
Avg
...
Response time for GET API
should not be more than 10 sec
...
GET
API
...
24 hr (86400 sec)
...
70
...
1
...
1
...
100k
...
0.033 sec
...
25 sec
...
0.06 sec
...
0.002 %
...
pass
This table summarize the test result for Business calendar
...
Observations:
The test successfully processed 100k HTTP requests within the specified duration (24 hr.)
The response time ranged from 0.03 seconds to 25 seconds, with an average response time of 0.002 seconds.
A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.
| View file | ||
|---|---|---|
|
| View file | ||
|---|---|---|
|
| View file | ||
|---|---|---|
|
| View file | ||
|---|---|---|
|
one thing noticeable is in caller API for 100k list the number in that calling number expired due to TTL and the cron job started at 1 am so user can notice the http request failing after 1:00 as the number expired and removed after the TTL job as given below.
...
Different load test scenarios for caller list:
Scenario 1:
Load The load test result is run with the following configuration:
Number of threads (users): 1 user will simulate the load.
Ramp-up period: Each user will be started within 1 second of the test start.
Loop count: Each user will execute the test plan 4167 times.
Thread lifetime: 3600 (1 hour).
Elapsed time: 18 min 57 sec
Loop count for 3600 duration: 4167
Loop count for 900 duration: 900
*one thing noticeable thread 1 for caller list having 100k callers elapsed time 18 min:57 sec and for thread group is more than the all caller (Elapsed time for 900 duration: 4 min 30 sec diff). So, response time for All caller (874140 callers) where a caller list name is not mentioned to search a number from that list is more better than the caller with caller list name17 sec (single caller list) 2 mins (all caller list)).
Total callers in db 874140
...
API | Duration | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | |||
|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 2 sec | |||||||
1. | GET API | 3600 | 1 | 1 | 4167 | 0.171 sec | 2.98 sec | 0.20 sec | 0.02 % (1 request) | pass |
...
...
...
2 |
...
. |
API
Duration
Threads (users)
Ramp up
(sec)
Number of
sample( total http request hits)
Response Time(sec)
GET |
Benchmark
Result
Min
Max
API |
should not be more than 2 sec
GET
API
900 | 1 | 1 |
900 | 0. |
17 sec |
0. |
66 sec | 0. |
20 sec | 0. |
00 % ( |
0 request) | pass |
This table summarize the test result for a caller List having 100k caller in caller list
Scenario 2:
Load test result is run with the following configuration:
Number of threads (users): 1 user will simulate the load.
Ramp-up period: Each user will be started within 1 second of the test start.
Loop count: Each user will execute the test plan 900 times.
Thread lifetime: 900 sec (15 mins)
total elapsed time: 4 min 17 sec
All caller (874140)
Thread Group 2:
Total callers in db 874140Thread Group 1:
API | Duration | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | |||
|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 2 sec | |||||||
1. | GET API | |||||||||
3600 | 1 | 1 | 4167 | 0. |
22 sec |
2. |
20 sec | 0. |
271 sec | 0. |
02 % ( |
1 request) | pass |
...
...
...
2 |
...
. |
API
Duration
Threads (users)
Ramp up
(sec)
Number of
sample( total http request hits)
Response Time(sec)
GET API
Benchmark
Result
Min
Max
Avg
Response time for GET API
should not be more than 2 secGET API | 900 | 1 | 1 | 900 | 0.22 sec | 4.8 sec | 0.271 sec | 0.22 % (2 request) | pass |
This table summarize the test result for a caller List having 100k caller in caller list
Following are the load test result for different number of thread(users)
Ramp-up Period (secs) | Number of Threads (Users) | Elapsed time of current running test | Response on UI | Loop Count |
|---|---|---|---|---|
1 | 20 | 2 sec | Callers are visible | 1 |
1 | 50 | 3 sec | Callers are visible in caller list | 1 |
1 | 1 | 7 sec | Callers are visible in caller list | 50 |
1 | 100 | 8 sec | Callers are not visible in caller list during load test | 1 |
1 | 1 | 13 sec | Callers are visible in caller list | 100 |
1 | 200 | 16 sec | Callers are not visible in caller list during load test | 1 |
1 | 1 | 26 sec | Callers are visible in caller list | 200 |
1 | 300 | 29 sec | Callers are not visible in caller list during load test | 1 |
1 | 500 | 50 sec | Callers are not visible in caller list during load test | 1 |
1 | 1 | 1 min 9 sec | Callers are visible in caller list | 500 |
5 | 500 | 47 sec | Callers are not visible in caller list during load test | 1 |
60 | 500 | 1 min 12 sec | Callers are visible in caller list during load test with delay of 5 to 6 secs | 1 |
100 | 500 | 1 min 39 sec | callers are not visible during load test | 1 |
1 | 1000 | 1 min 35 sec | Callers are not visible in caller list during load test | 1 |
1 | 2000 | 02 min 39 sec | Callers are not visible in caller list during load test | 1 |
1 | 100k | 1hour 52 mins | Callers and caller list are not visible after the load test. Container of caller list goes into unhealthy state. | 1 |
Note: this result Elapsed time of current running test can varies depending on configuration in jmeter, network latency, server performance/load.
Load Test Result for Easy Announcement and Business Calendar:
Easy announcement:
Thread Group 3:
API | Duration | Constant Throughput Timer | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 10 sec | ||||||||
1. | GET API | 24 hr (86400) | 70 | 1 | 1 | 100k | 0.027 sec | 5.9 sec | 0.05 sec | 0.00 % | Pass |
This table summarizes the test result for Easy Announcement
Observations:
The test successfully processed 100k HTTP requests within the specified duration (24 hr).
The response time ranged from 0.02 seconds to 5.9 seconds, with an average response time of 0.05 seconds.
A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.
Thread Group 4:
Business calendar:
API | Duration | Constant Throughput Timer | Threads (users) | Ramp up (sec) | Number of sample( total http request hits) | Response Time(sec) GET API | Benchmark | Result | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
Min | Max | Avg | Response time for GET API should not be more than 10 sec | ||||||||
1. | GET API | 24 hr (86400 sec) | 70 | 1 | 1 | 100k | 0.033 sec | 25 sec | 0.06 sec | 0.002 % | pass |
This table summarizes the test results for the Business Calendar
Observations:
The test successfully processed 100k HTTP requests within the specified duration (24 hr.)
The response time ranged from 0.03 seconds to 25 seconds, with an average response time of 0.002 seconds.
A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.