You can execute benchmark tests against your deployment and verify the performance of your Rhoconnect application using the provided
During the test the Rhoconnect benchmark framework will dynamically create a special ephemeral source adapter inside of
your application and perform the 5-step query use-case mimicking the typical device-to-server communication.
Currently, the Rhoconnect Benchmark measures two primary system-wide metrics: Average Time and Throughput.
|Both metrics implicitly include the network latency and therefore represent the performance of the whole system `device-network-server`.|
To execute the Rhoconnect Benchmark test, use the
rhoconnect-benchmark command. This command has the following options:
||Use URL to specify the targeted Rhoconnect server for benchmarking. If this option is not specified,
||This options specifies the amount of data to be simulated in the query request (in number of records). Default: 100, which roughly translates into 20 kilobytes.|
||Specifies how many concurrent clients to emulate during the test. If this option is omitted, then the number of simulated clients will be determined based on the number of available seats in your Rhoconnect application's license.|
||Specifies how many times each emulated client will perform the test. Default: 10. The more iterations you specify, the more time your test will take, but it will lessen the test result's statistical fluctuations.|
||This option saves the test results in the PATH sub-directory and run the post-processing on the saved files (for example, generate the EXCEL spreadsheet). All the results will be saved under the PATH root directory.|
||This option runs several tests in a loop varying the payload between 1 and NUMBER. The specified interval will be split by 10. For example: -m 500 will produce 10 test executions , first for payload of 1 record, then - for 56, 112, 167, 223, 278, 334, 389, 445, and 500|
||This option runs several tests in a loop varying number of concurrent clients between 1 and NUMBER. The specified interval will be split by 5. For example: -m 10 will produce 5 test executions , first for 1 client, then - for 3, 6, 8, and 10|
||This option allows to enforce NUMBER of seconds delay in the source adapter, thus simulating the backend communication round-trip.|
||This convenience option combines -m 10, -x 500, and -S
||This option executes the advanced benchmark test using the remote AWS EC2 clients. Currently, this option is only available for the Linux/MacOS machines. See Distributed Benchmark section below for details.|
By default, all results are displayed in the console during the test. However, it is possible to save all the results into the file and, additionally, generate the EXCEL files and/or PNG images using the
--save PATH option. The results are saved for two benchmark metrics: Average Time and Throughput.
To enable the automatic generation of the EXCEL spreadsheets from the test results, please install the
spreadsheet gem by using:
$ [sudo] gem install spreadsheet
You can read more about the
spreadsheet gem here.
To enable the automatic generation of the PNG images from the test results, install the
Installation of the GRUFF gem requires an additional installation of several components.
Use this guide to install the GRUFF on your machine.
To fully simulate the distributed nature of the device-to-server system, the Rhoconnect Benchmark command allows to you to create distributed clients through the AWS Cloud Formation.
|Distributed Rhoconnect Benchmark test will be executed via the AWS Cloud Formation and EC2 services using your account and therefore you will be billed for its usage by Amazon.|
In order to run the distributed Benchmark test, you need to have an AWS account and specify its settings in a YAML file, which should have the following structure:
:default: :region: desired_aws_region :aws_access_key_id: your_aws_account_key_id :aws_secret_access_key: your_aws_account_secret_access_key :aws_key_pair_name: your_aws_ec2_security_key_pair :aws_ssh_pem_file: location_of_your_aws_ssh_pem_file
Here is the description of each entry in the AWS settings file:
||Specify the desired AWS region for your EC2 clients. Default is 'us-west-1'. Typically, you need to specify the same region as the location of your Rhoconnect server to minimize the network impact (since communications between different regions will take more time).|
||Your AWS access key ID.|
||Your AWS secret access key.|
||In order to create EC2 clients, you will need to create EC2 KeyPair in the desired region. After that KeyPair is created, you will have its name, which should be specified here.|
||For the above KeyPair, you will have the corresponding keypair .PEM file. Download this file to your machine and reference its location here.|
Once the AWS settings file is created, you need to make sure that the following gems are installed on your machine:
You can install those gems using the
[sudo] gem install <gemname> command.
Once all the pre-requisites are met, you can run the distributed Rhoconnect Benchmark test with the following command:
$ rhoconnect-benchmark -u `server_url` -D `path_to_aws_settings_file`
|In order to run the distributed Benchmark test, you must provide public server's URL accessible from the outside clients.|