I use cutechess-cli from time to time to evaluate changes in hardware or server configuration to keep nextchessmove.com strong.

Here’s my notes on compiling cutechess-cli on a Ubuntu 15.10 VPS: (Ubuntu 14.04 does not package a sufficiently recent version of Qt.)

sudo apt-get update
sudo apt-get install -y build-essential qt5-default libqt5svg5-dev
git clone https://github.com/cutechess/cutechess.git
cd cutechess
qmake
make

You should now have the latest cutechess-cli binary at ./projects/cli/cutechess-cli.

Here’s the basic gist to compare two machines running Stockfish by playing 100 games, 5 seconds per turn, 8 threads on one and 16 on the other:

./projects/cli/cutechess-cli \
  -games 100 \
  -each \
    st=5 \
    timemargin=100 \
    proto=uci \
  -engine \
    name=old \
    option.Threads=8 \
    option.SyzygyPath=/syzygy \
    cmd="ssh chess@192.168.1.201 ./stockfish/7/bin/stockfish" \
  -engine \
    name=new \
    option.Threads=16 \
    option.SyzygyPath=/syzygy \
    cmd="ssh chess@192.168.1.202 ./stockfish/7/bin/stockfish"