Versions Compared

Version Old Version 19 New Version Current
Changes made by

QA Team

Navira Zainab

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Tested with UCCX 12-5 for ST 13.5

HW specs of the tested VM

...

VM

CPU

RAM

Disk Size

VM (ST deployed)

2

4GB

...

100GB

...

SQL VM

4 vcpu

8GB

145GB

Uploading CSV Files (Diff callers volume) in Caller List (Performance Testing):

Caller list

Upload time

Uploaded

Failed

Total in db

File size

Delete through generic clear list (Time)

200k

2 mins

200k

0

200k

4-5 mins

300k

8-9 mins

300k

0

300k

7-8 mins

200k

3 mins

200k

0

200k

8064kb

8064kb

-

200k

1 min

99999

100001

99999

5913kb

178845

57 sec

99999

78846

5407kb

100k

53 sec

98784

1216

4677kb

100k

3 mins

100k

0

100k

3994kb

25k

34 sec

25k

25k

64 secs

25k

34 sec

25k

50k

24 sec

50k

0

50k

194817

27 lacs callers db

182 secs (also a delay of callers visibility

594817

25 lacs callers db

708 sec ( some delay in callers visibility in caller list)

300k (latest)

324

secs

sec

(5-6 min)

300k

200k (latest)

138 secs

(2-3 min)

200k

178k (latest)

60

secs

sec

(1min)

99999

78846

Note: there is some delay after uploading the csv on UI, time it took for callers visibility in caller list is around 5 to 8 mins.

...

Load Test:

Software for load test: Apache JMeter

API for Load test

API load testing is done for caller list, Easy Announcement, Business Calendar

  • GET API for Easy announcement is used to get announcement(s) of type “General”. This gives the details of an easy announcement, including id, name and prompts string within the easy announcement.

  • GET API for caller list to check whether a specific caller exist in the list or not. It also returns the prompts associated to a particular caller list

  • GET API for the Service Status for Business calendar checks the status of a calendar, i.e. if the agency is Opened, Closed, has a Holiday or an Exceptional opening or closing. For more detail visit

Load test result is given with the following configuration:

Throughput in sample per unit: 70

Number of threads (users): 1

Ramp up period : 1 sec

Loop count: 100k

Thread Lifetime: 86400 secs (24 hr)

It means the 100k requests will be hit in duration of 24 hrs.

Total callers in db 463948

Load test started for 24 hours for 100k (threads) each for All caller list and caller list easy announcement and business calendar concurrently and detailed load test results can be view in attached file.

four thread groups were running concurrently.

Two scenarios were tested for all caller list a calling number is provided in API where it will checks in all caller list that the calling number exist or not in all caller list where total callers in caller list are 463948. While another scenario API includes only calling number and caller list and that caller list has 100k caller in it. so it checks either calling number exist in the caller list of 100k caller or not. These cases were tested to see the overall performance in both scenarios.

Given is the resulted reports for all caller list, easy announcement and business calendar.

...

178k

178 sec

99999

78846

178k

89 sec

178k

5636kb

300k (latest)

239 sec

300k

13954kb

178k

79 sec

5636 kb

300k

108 sec

650k

249 sec ( 5-7 mins delay in visibility of callers in caller list)

Note: A delay of approximately 5 to 8 minutes occurs after uploading the CSV on the UI before callers become visible in the caller list.

Load Test Result for Caller List:

  • Loop count: 100,000

  • Total callers in db 463948

  • Tested Scenarios:

    1. Checking if a provided calling number exists in a caller list containing 463,948 callers.

    2. Verifying if a calling number exists in a caller list of 100,000 callers.

Thread Group 1:

Sr.no

API

Duration

Constant Throughput Timer

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than 10 sec

GET

API

24 hr

(86400 sec)

100k

70

1

1

60409

0.05 sec

187 sec

0.7

sec

2.74 % ( 1657 request 10 sec)

1836 request (5 sec) 3.03%

Fail

This

...

table summarizes the test result for

...

all callers List total callers 463948 in callers.

Observations:

  • The test successfully processed 60,409 HTTP requests within the specified duration (24 hr).

  • The response time ranged from 0.05 seconds to 187 seconds, with an average response time of 0.7 seconds.

  • A small percentage (2.74%) of requests experienced response times exceeding the threshold of 10 seconds.

Recommendation:

  • Investigate and optimize the performance of the API to reduce the occurrences of response times exceeding 10 seconds.

Thread Group 2:

Sr.no

API

Duration

Constant Throughput Timer

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than 10 sec

GET

API

24 hr

(86400 sec)

70

100k

1

1

60393

0.055

sec

187 sec

0.7

sec

2.74 % (1659 request 10 sec)

3.03(1835 request 5 sec)

Fail

This

...

table summarizes the test result for a caller List having 100k caller in the caller list

Observations:

  • The test successfully processed 60,393 HTTP requests within the specified duration (24 hr).

  • The response time ranged from 0.05 seconds to 187 seconds, with an average response time of 0.7 seconds.

  • A small percentage (2.74%) of requests experienced response times exceeding the threshold of 10 seconds.

Recommendation:

  • Investigate and optimize the performance of the API to reduce the occurrences of response times exceeding 10 seconds.

Different load test scenarios for caller list:

Scenario 1:

The load test result is run with the following configuration:

  1. Loop count for 3600 duration: 4167

  2. Loop count for 900 duration: 900

*one thing noticeable thread for caller list having 100k callers elapsed time is more than the all caller (Elapsed time for 900 duration: 4 min 17 sec (single caller list) 2 mins (all caller list)).

Total callers in db 874140

Thread Group 1:

http://Sr.no

API

Duration

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than

10

2 sec

1.

GET

API

24 hr

(86400)

100k

1

100k

0.027

sec

5.9

sec

0.05

3600

1

1

4167

0.171

sec

2.98 sec

0.20

sec

0.02 % (1 request)

pass

2.

GET

API

900

1

1

900

0.17

sec

0.66 sec

0.20

sec

0.00 % (0 request)

Pass

pass

This table summarize the test result for

...

All caller (874140)

Thread Group 2:

Total callers in db 874140

http://Sr.no

API

Duration

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than

10

2 sec

1.

GET

API

24 hr (86400 sec)

100k

3600

1

1

100k

4167

0.

033

22

sec

25

2.20 sec

0.

06

271

sec

0.

002

02 %

View file
namesummaryallcaller.csv

View file
namesummarybusinessc.csv
View file
namesummarycaller.csv
View file
namesummaryea.csv

View file
nameInvalid file id - 90c9f724-4e00-439d-98fb-345c4fe27242
View file
nameInvalid file id - 5c6252ec-43f9-48d0-a296-bcc996a3dc6c
View file
nameInvalid file id - c413a925-b582-4950-87da-9047aea1b5c4
View file
nameInvalid file id - 7961a2c6-d064-404c-a698-984b8caa8d4c

one thing noticeable is in caller API for 100k list the number in that calling list expired due to TTL and the cron job started at 1 am so user can notice the http request failing after 1:00 as the number expired and removed after the TTL job.

...

(1 request)

pass

2.

GET

API

900

1

1

900

0.22

sec

4.8 sec

0.271

sec

0.22 % (2 request)

pass

This table summarize the test result for a caller List having 100k caller in caller list

Following are the load test result for different number of thread(users)

Ramp-up Period (secs)

Number of Threads (Users)

Elapsed time of current running test

Response on UI

Loop Count

1

20

2 sec

Callers are visible

1

1

50

6

3 sec

Callers are visible in caller list

1

1

1

7 sec

Callers are visible in caller list

50

1

100

8 sec

Callers are not visible in caller list during load test

1

1

1

13 sec

Callers are visible in caller list

100

1

200

16 sec

Callers are not visible in caller list during load test

1

1

1

26 sec

Callers are visible in caller list

200

1

300

29 sec

Callers are not visible in caller list during load test

1

1

500

50 sec

Callers are not visible in caller list during load test

1

1

1

1 min 9 sec

Callers are visible in caller list

500

5

500

47 sec

Callers are not visible in caller list during load test

1

60

500

1 min 12 sec

Callers are visible in caller list during load test with delay of 5 to 6 secs

1

100

500

1 min 39 sec

callers are not visible during load test

1

1

1000

1 min 35 sec

Callers are not visible in caller list during load test

1

1

2000

02 min 39 sec

Callers are not visible in caller list during load test

1

1

100k

1hour 52 mins

Callers and caller list are not visible after the load test. Container of caller list goes into unhealthy state.

1

Note: this result Elapsed time of current running test can varies depending on configuration in jmeter, network latency, server performance/load.

Load Test Result for Easy Announcement and Business Calendar:

Easy announcement:

Thread Group 3:

http://Sr.no

API

Duration

Constant Throughput Timer

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than 10 sec

1.

GET

API

24 hr

(86400)

70

1

Later tested Again with different calling number Results are given

...

image-20240318-102116.pngImage Removed

image-20240318-102353.pngImage Removed

...

1

100k

0.027

sec

5.9

sec

0.05

sec

0.00 %

Pass

This table summarizes the test result for Easy Announcement

Observations:

  • The test successfully processed 100k HTTP requests within the specified duration (24 hr).

  • The response time ranged from 0.02 seconds to 5.9 seconds, with an average response time of 0.05 seconds.

  • A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.

Thread Group 4:

Business calendar:

http://Sr.No

API

Duration

Constant Throughput Timer

Threads (users)

Ramp up

(sec)

Number of

sample( total http request hits)

Response Time(sec)

GET API

Benchmark

Result

Min

Max

Avg

Response time for GET API

should not be more than 10 sec

1.

GET

API

24 hr (86400 sec)

70

1

1

100k

0.033 sec

25 sec

0.06 sec

0.002 %

pass

This table summarizes the test results for the Business Calendar

Observations:

  • The test successfully processed 100k HTTP requests within the specified duration (24 hr.)

  • The response time ranged from 0.03 seconds to 25 seconds, with an average response time of 0.002 seconds.

  • A percentage (0.00%) of requests experienced response times exceeding the threshold of 10 seconds.