Installation Guide - TechEmpower Framework Benchmarks (2024)

If you're looking to quickly get started developing using the vagrant development environment, consider the vagrant development environment guide.

Take a look at the Summary of Script Directories section to figure out which directory has scripts relevant to your use case (development or benchmarking).

In order for SSH to work appropriately, the user on each machine will need to be set up to have passwordless sudo access.

Setting up the user

sudo vim /etc/sudoers

You will need to change the line that reads %sudo ALL=(ALL:ALL) ALL to %sudo ALL=(ALL:ALL) NOPASSWD: ALL. You should be able to exit your shell, ssh back in and perform sudo ls without being prompted for a password.

NOTE Again, you will have to do this on every machine: server, database, client.

Setting up SSH

You will need to also be able to SSH onto each of the 3 machines from any of the3 machines and not be prompted for a password. We use an identity file and authorized_keysfile to accomplish this.

ssh-keygen -t rsa

This will prompt you for various inputs; leave them all blank and just hit 'enter'.Next, you will want to allow connections identified via that key signature, so addit to your authorized keys.

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keyschmod 600 ~/.ssh/authorized_keys

Next, you will need these exact same files to exist on the other 2 machines.If you properly set up your user account on all 3 machines, then this will not promptyou for a password (if it does, go back to "Setting up the user" and fix it).

# Export some variables so you can copy/paste the rest.export TFB_SERVER_USER=[your server user]export TFB_SERVER_HOST=[your server ip]export TFB_DATABASE_USER=[your database user]export TFB_DATABASE_HOST=[database ip]export TFB_CLIENT_USER=[your client user]export TFB_CLIENT_HOST=[client ip]# Set up the database machine for SSHcat ~/.ssh/id_rsa.pub | ssh $TFB_DATABASE_USER@$TFB_DATABASE_HOST 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'scp ~/.ssh/id_rsa $TFB_DATABASE_USER@$TFB_DATABASE_HOST:~/.ssh/id_rsascp ~/.ssh/id_rsa.pub $TFB_DATABASE_USER@$TFB_DATABASE_HOST:~/.ssh/id_rsa.pub# Set up the client machine for SSHcat ~/.ssh/id_rsa.pub | ssh $TFB_CLIENT_USER@$TFB_CLIENT_HOST 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'scp ~/.ssh/id_rsa $TFB_CLIENT_USER@$TFB_CLIENT_HOST:~/.ssh/id_rsascp ~/.ssh/id_rsa.pub $TFB_CLIENT_USER@$TFB_CLIENT_HOST:~/.ssh/id_rsa.pub

Now, test it all out, you should be able to execute all of the following withoutbeing prompted for a password. NOTE The first time you SSH to these machines(read: in this step) you will be prompted to accept the signature - do it.

# Test your database SSH setupssh $TFB_DATABASE_HOST# Accept the signature# You are connected to the database machine!sudo ls# This should NOT prompt for a password and list the directory's contents# If this is not true, go back to "Setting up the `user`" and fix itexit# Test your client SSH setupssh $TFB_CLIENT_HOST# Accept the signature# You are connected to the client machine!sudo ls# We also need to test that we can SSH back to the server machinessh [enter your server ip again]# Accept the signature# You are connected to the server machine!sudo ls# If this works, you are golden!exit# Back on clientexit# Back on initial ssh connection to server machine

Last, the benchmarks require that every framework test be run under a differentuser than the one who executes the run-tests.py script. Creating a runner_user named "testrunner" is simple:

# Change this value to whatever username you wantexport RUNNER=testrunner# Get your primary userexport ME=$(id -u -n)# Create the new testrunner usersudo useradd $RUNNER# Give him a home dirsudo mkdir /home/$RUNNER# Make testrunner the owner of his home dirsudo chown $RUNNER:$RUNNER /home/$RUNNER# Add the testrunner user to every group that the travis user is insudo sed -i 's|:'"$ME"'|:'"$ME"','"$RUNNER"'|g' /etc/group# Add the testrunner user to the travis group specificallysudo sed -i 's|'"$ME"':x:\(.*\):|'"$ME"':x:\1:'"$RUNNER"'|g' /etc/group# Add the travis user to the testrunner groupsudo sed -i 's|'"$RUNNER"':x:\(.*\):|'"$RUNNER"':x:\1:'"$ME"'|g' /etc/group# Add testrunner to the sudoers groupecho "$RUNNER ALL=(ALL:ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers# Set the default shell for testrunner to /bin/bashsudo sed -i 's|/home/'"$RUNNER"':.*|/home/'"$RUNNER"':/bin/bash|g' /etc/passwd

Setting up prerequisites

The suite requires that a few libraries and applications are installed in order to run.First, clone our repository.

git clone https://github.com/TechEmpower/FrameworkBenchmarks.gitcd FrameworkBenchmarkssource toolset/setup/linux/prerequisites.shsudo apt-get install -y python-pipsudo pip install -r config/python_requirements.txt

To install TFB components onto the various servers, you must providebasic login details for each server (e.g. usernames, IP addresses, private key files). While these details can all be passed using command line flags, it's easier to use a configuration file and avoid having huge flag lists in your commands.

cp benchmark.cfg.example benchmark.cfgvim benchmark.cfg

You will need to change, at a minimum, the following:

• client_host Set this to the IP address of your client machine
• client_identity_file Set to /home/[username]/.ssh/id_rsa
• client_user Set to your username
• runner_user Set to your runner's username
• database_host Set this to the IP address of your database machine
• database_identity_file Set to /home/[username]/.ssh/id_rsa
• database_user Set to your username
• server_host Set this to the IP address of your server machine

At this point, you should be ready to install the suite.

We use the --install-only flag in our examples to prevent launching tests at this stage. All of these commands should be run from the app server - it will SSH into otherhosts as needed. For more information on how TFB installation works, see here.

Setting up the load server

toolset/run-tests.py --install client --install-only

Setting up the database server

toolset/run-tests.py --install database --install-only

Setting up the app server

At this point, you should be able to run a benchmark of the entiresuite or selectively run individual benchmarks. Additionally, youcan test your setup by running a verification on one of the stableframeworks. Since we wrote it, we tend to test with gemini:

toolset/run-tests.py --mode verify --test gemini

You can find the results for this verification step under the directory:results/ec2/latest/logs/gemini. There should be an err and an outfile.

You can choose to selectively install components by using the --test and --exclude flags.

# Install just the software for beego (as an example)toolset/run-tests.py --install server --test beego --verbose --install-only# Install all php software but php-fuel (as another example)toolset/run-tests.py --install server --test php* --exclude php-fuel --verbose install-only# Install *all* framework software. Expect this to take hours!# If running on a remote server, use `screen` or `tmux` or `nohup` to # prevent the installation from being terminated if you are disconnected$ toolset/run-tests.py --install server --verbose --install-only