DBT2
dbt2-0.37.49.tar.gz
This version of DBT2 is a heavily extended version based on the
dbt2-0.37 found on SourceForge.
It has added a great number of scripts to make it easy to start
and stop MySQL and MySQL Cluster Servers. In later versions
it also integrates the possibility to start and stop Drizzle
servers.
It can be used to run the benchmarks DBT2 (integrated), TPC-W
(integrated) and sysbench (separate tarball).
It contains automation scripts that makes it possible to run
Sysbench benchmarks completely automated. It is also possible
in the same fashion to run DBT2 benchmarks completely automated.
These automated tests can only be used so far on non-clustered
MySQL Servers. MySQL Cluster can be used with the lower level
scripts.
sysbench-0.4.12.2.tar.gz
The above link is the sysbench version we've currently tested the
automation scripts against when using sysbench , this is version
0.4.12 with the addition provided by Mark Callaghan, Tim Cook
and others.
In addition to all the sysbench 0.4.12 features this version can:
1) Use secondary index instead of the primary key on the test table.
2) Use MySQL HANDLER statements instead of SELECT for primary key
selects.
3) Use a KEY partitioned table instead of an ordinary table
4) Use intermediate reporting instead of only reporting at end of
test case.
5) Use a fixed transaction rate with a jitter.
DBT2 is a database test that is used to /simulate/benchmark
heavy load on a database.It simulates multiple warehouses
and the transactions going on in a warehouse database
This new version of DBT2 is based on dbt2-0.37. It has
significantly improved the ability of DBT2 to be used for
benchmarking of clustered databases and MySQL Cluster
in particular. It contains significant fixes to the DBT2 and
a few fixes where DBT2 used very inefficient queries. It has
simplified parameters by using long names instead of the
one-letter parameters which were impossible to remember.
This version of DBT2 also contains a set of scripts that can be
used to start/stop nodes in a MySQL Cluster. It is fairly well
documented in various README files and extensive
documentation in the parameters of the scripts.
Release history of DBT2:
dbt2-0.37.49: This is a major update, it adds the following features:
1) Adapted sysbench tarball with above 5 new sysbench features and also
made all these features available to use in automated benchmark
environment.
2) Made it possible to use sysbench and dbt2 runs on localhost without
using ssh
3) Updated the README-AUTOMATED file with more documentation on parameters
possible to set for benchmark runs.
4) Use intermediate reporting to generate JPEG files that gives a graph of
results the benchmark gave every 3 seconds. Each test run for each test
case will have its own graph where all threads are provided with different
colors (this is for sysbench test runs).
5) Made it possible to specify builds of MySQL with -O3 and another parameter
to specify the same for sysbench builds. Also made tcmalloc being used
also by sysbench program when defined to be used by MySQL.
dbt2-0.37.48: Adapted the full automation scripts to also fully automate
DBT2 test runs.
dbt2-0.37.47: Fixed issue with LD_LIBRARY_PATH for Drizzle which made
it unavailable to find libdrizzle.
dbt2-0.37.46: It is now also possible to use Drizzle for the automated
sysbench testing, need two gzipped tar files for libdrizzle and for
Drizzle. Also added a README-AUTOMATED file describing how to run
automated tests for sysbench.
dbt2-0.37.45: Now tested also with Linux client and Linux host, added
remote build option on Linux hosts and fixed an at-sign in the wrong
place when starting the MySQL Server. The REMOTE_SRC_INSTALL_DIR was
made mandatory on Linux as well as on Windows.
dbt2-0.37.44: First working version using automated benchmark runs.
It has been tested so far with a Linux client and a Windows host,
more testing is expected and a new update is likely to happen shortly.
dbt2-0.37.43: Completed first version (yet untested) of bench_prepare.sh
together with a new examples directory containing an example file
to be used by automatic benchmark runs (autobench.conf).
dbt2-0.37.42: Added support for the ability to also start MySQL
Servers and NDB nodes on a Windows Server using cygwin. Added
a new script run_oltp.sh to automatically run sysbench benchmarks.
Also added a not yet ready script bench_prepare.sh which will
build all needed MySQL, DBT2 and sysbench tarballs and place
them in a proper place. Added more error handling to the scripts.
Fixed a bug in checking the existence of the binaries which
previously took place on the wrong node.
dbt2-0.37.41: A few fixes to parameters for Dolphin Supersockets
and --home-base fully supported
dbt2-0.37.40: Added documentation on how to run DBT2 using the
new scripts in this DBT2 distribution (in new file
README-ICLAUSTRON)
dbt2-0.37.39: Fixed the use of taskset to control CPU placement
of MySQL Servers. Added more support for setting InnoDB flags
and some more MySQL Server flags to enable simple start of
MySQL Server to run sysbench benchmarks using DBT2 scripts.
dbt2-0.37.38: Added test_harness directory which contains a
suite of scripts used to benchmark Falcon, InnoDB and other
non-clustered storage engines.
dbt2-0.37.37: Fixed a silly mistake in dbt2.sh which made the
benchmark bark in perfectly legal situations.
dbt2-0.37.36: This release introduces a number of changes:
dbt2-0.37.34+35 were internal releases which contained parts of
what is described in the changelog for version dbt2-0.37.36.
dbt2-0.37.33: Recomputed the configure script to ensure NDB API
version doesn't have to be compiled
dbt2-0.37.32: Fixed a missing setting of LD_LIBRARY_PATH in
create_dbt2_files.sh. Also included a patch from Joachim
Worringen merging results into one summary result file for
DBT2 runs.
dbt2-0.37.31: New version that fixes an issue with that grep on
Solaris isn't the same grep as on Linux, made sure that we use
the GNU version of grep on Solaris as well.
dbt2-0.37.30: First version to fix some Solaris issues
dbt2-0.37.29: Fixed so that configure script works properly
without having to use the bootstrap.sh script. Added more
documentation to bootstrap.sh script.
Description of changes in dbt2-0.37.36
1) The start_ndb.sh script now accepts the parameter --taskset 0x3
where 0x3 here means that the process started only can run on CPU0
and CPU1. It's primary purpose is to ensure that the MySQL Server
can be limited to execute on a set of CPU's. It cannot be mixed
with the use --mysqld_safe since mysqld_safe doesn't accept
taskset as a parameter. So it's mainly to be applied for --mysqld.
It can be set in the FLAGS part of the dis_config_c1.ini (where 1
is the cluster id) config file.
2) A warmup and a cooldown period was added to the driver binary to
ensure that multiple instances of driver are executed in parallel
such that no driver is stopped in the measurement period of another
driver. Later releases of the DBT2 software will most likely handle
this measurement period in the analysis program.
3) To enable an environment with using mixed storage engines it is
now possible to store the ITEM table in MyISAM. This is achieved
by setting the variable USE_MYISAM_FOR_ITEM="yes" in the
iclaustron.conf configure file. In this case the item table is
created on all MySQL Servers and filled on all MySQL Servers. In
a realistic use case there would also be MySQL replication set-up
between the MySQL Servers for the table. However since the benchmark
doesn't perform any updates of the ITEM table we avoided this for
simplicity.
4) The run_mysql.sh script previously did a lot initialisation and
got a run number in parallel with many other instances of the same
script. This meant it was possible to get the same run number in
multiple drivers. It also meant a very bad synchronisation between
the drivers. Now this script have a prepare phase and a run phase.
This is achieved by adding --pre-run in the prepare phase of the
script execution and --only-run in the run phase. Thus the same
script is used for both phases and the script without those
parameters will use both phases.
5) There were some issues with the SSH_PORT flag which were fixed
and it was documented as part of the run_dbt2_test.sh script
(used to run the entire test from start to end).
6) A new parameter was introduced (it was already existing in driver
and client but had bugs in the implementation). This parameter is
called --spread and can be used on the top level script,
run_dbt2_test.sh. This means that each driver will use a step between
the warehouse ids it uses. Thus with starting warehouse id 10, last
warehouse 66 and spread 8 it will use the warehouse ids
10,18,26,34,42,50,58 and 66. Since each driver only interacts with
one MySQL Server we can use this to show the effect of using a perfect
load balancer in front of the MySQL Servers. When using spread 4 in a
8-node cluster each MySQL Server will only interact with one node group
and by using spread 8 we can even limit it to only using 1 node as
transaction coordinator (it will however affect all nodes in the node
groups for writes.
7) There was an annoying bug in the acinclude.m4 script which meant that
the bootstrap.sh (which does the autoconf stuff) had to call vi and edit
the configure script after generating it. This m4 script used a lib
directory called libmysql_r which was an obvious bug.
8) A pid file was introduced in the driver program and the pid file is
used to ensure that we get rid of the driver binary before the
run_mysql.sh exits.
9) The write of mix.log was not very efficient, each line was written
separately which causes the benchmark itself to become a bottleneck
at very high loads. Now instead 32kBytes is written at a time, which
is about 1000 transactions.
10) A number of example config files:
iclaustron.conf, dbt2.conf, config_c1.ini, dis_config_c1.ini and
dbt2_run_1.conf was added to make it easier for new users to quickly get
up to speed with the DBT2 package.
11) Fixed the configure.ac script to ensure that it only used
src/ndb/Makefile.am if the NDB API part of the DBT2 test is to be used.
NOTE: the NDB API part is probably broken in this release. I will fix
this up later in an upcoming release.
The MySQL version to use for the benchmarks can be either
a 5.0-based version, a 5.1-based version or a Cluster
Carrier Grade Edition version.
If one is compiling using the NDB API version it has to be a
5.1-based version. Also the NDB API version is a tad beta
quality still.
Personally I've done most of my testing on the MySQL Cluster
Carrier Grade Edition version 6.3, the most recent version tested
being version 6.3.19.
The set of performance enhancements mentioned in the
benchmark papers have been integrated in various CGE
releases, most of them are in the CGE 6.3.13 release.
Two bugs was discovered that impacted the
scalability of DBT2. These bugs were fixed in
MySQL Cluster CGE 6.3.24/7.0.4 and the bugs contains description
of how to fix those bugs in a simple manner also in earlier
CGE releases. With those bugs fixed we've been able to
verify 97% scalability going from 4 data nodes to 8 data
nodes which is within the error margin.