SPECweb2009 Release 1.10 User's Guide

Version 1.10, Last modified 2009-09-24


1 Requirements

These instructions assume that you are familiar with the following:

1.1   Software Requirements

1.1.1 The SPECweb2009 kit

Test Harness consists of:

1.1.2 Additional Software Requirements:

1.2   Hardware Requirements

2 Setting Up The Test Environment

2.1   Running the SPECweb2009 installer

Place a copy of the provided setup.jar or setup.exe on all systems that are part of your test setup. You will need to have a JVM installed; a JDK or JRE version 1.6 or later is recommended.

To Run the Linux installer, invoke the Java installer as follows:

To run the Windows installer:

If the installation of the SPECweb2009 components fails with the following message:

Errors occurred during the installation.

-   JVM not found

Please set the JAVA_HOME environment variable to the root path of your JVM engine.

Linux: export JAVA_HOME=/usr/path_to_java

Windows: set JAVA_HOME=c:\path_to_java

2.2   Web Server Setup

General Directions:

  1. Set localization on the server system to US.

  2. Install and configure your favorite Web server with SSL and PHP, ASP.NET, or JSP support configured in. If using JSP, a separate Java application server may be needed. Check your Web server software documentation for assistance.

  3. Run the SPECweb2009 installer (see above) and set your current working directory to the wafgen directory.
    # pwd /web2009-install/wafgen

Depending on the type of scripts you want to use choose one of the following options:

2.3   Besim Setup

  1. Configure the BeSim system with Web server software installed on port 81 or another port (avoid port 80 to avoid confusion with the primary Web server).

  2. Run the SPECweb2009 installer, do a "BeSim" or "Full" installation, and change your current working directory to the <specweb_install_directory>/Besim directory.

  3. Review the BeSim Make_Readme
    NOTE: For Windows/IIS, a precompiled ISAPI is provided (BeSim.dll); copy this to a directory with ISAPI execute permissions, and skip to step 6.

  4. You may choose from FastCGI* for Zeus or Apache or ISAPI for Zeus or IIS or NSAPI for SUN One and edit the appropriate
    MakeIncl.<type>.<OS> file to reflect your compile and linker options and select one or more Defines.  Then you can build the BeSim code for your selected API. (If using FastCGI , please copy the FastCGI executable from the Redistributed Open Source code directory into the BeSim directory.)

  5. Install this code to your BeSim Web server.

  6. Start your BeSim Web server.

  7. Use the test_besim_bank, test_besim_ecom, and test_besim_support scripts (included in the "besim" subdirectory of the installation) to test whether you're getting valid BeSim responses.  Both Perl scripts and Linux bash scripts are provided.  The Perl scripts require a Perl interpreter as well as the following Perl modules: URI, HTML-Tagset, HTML-Parser, and libwww-perl.  The bash scripts require cURL to be in the path.

    Invoke these scripts with the URL to your compiled BeSim API.
      *Note: For more FastCGI instructions (particularly for the Apache HTTP Server), see Appendix F.


If you notice that one BeSim system has a bottleneck (CPU, disk, network, etc.), there is an option to use multiple physical BeSim systems.  Follow the same steps above to install BeSim on another system, and specify multiple BESIM_SERVERs (delimited by spaces) in Test.config.  See the example Test.config files for more information.

2.4   Client Setup

  1. Install a 1.6 JVM (or later) on all client systems.

  2. Run the SPECweb2009 installer per Section 2.1 on each client and set your current directory to the SPECweb2009 directory created by the installation.

  3. Select one of the client systems to serve as the "Prime Client" (can also be a load generating client).

  4. On the Prime Client, decide on an appropriate set of configuration files for your particular system.  For a Red Hat Linux server running Apache and PHP, you should use the *.Unix-PHP.config files.  Copy or rename these files to SPECweb_Banking.config, SPECweb_Ecommerce.config, SPECweb_Support.config, SPECweb_Power.config and Test.config, i.e.

# cp SPECweb_Banking.Unix-PHP.config SPECweb_Banking.config
# cp SPECweb_Ecommerce.Unix-PHP.config SPECweb_Ecommerce.config
# cp SPECweb_Support.Unix-PHP.config SPECweb_Support.config
# cp SPECweb_Power.Unix-PHP.config SPECweb_Power.config
# cp Test.Unix-PHP.config Test.config

For JSP, follow the same steps but use *Unix- JSP.config files instead . Similarly, for ASP.NET, follow the same steps using * ASPX.config files instead .

  1. Review the SPECweb_<workload>.<OS/Web server>-<Script Engine>.config, Test.<OS/Web server>-<Script Engine>.config, and Testbed.config to understand the parameters in these files, with attention to those in the beginning (configurable) section of the workload and Test configuration files which may need to be updated to match your test bed.

  2. Edit configuration files to match your Web server, BeSim configuration, Client, and network configuration.  Change only those parameters listed above the section marked "# Fixed workload properties" or in Test.config above the line marked " # SPECweb2009 fixed test properties":

Some tips:

2.5   Network Setup

Configure all the test bed systems so that they can communicate with one another; make sure to update your /etc/hosts file (or equivalent). The Prime Client must be able to communicate with each Client, Besim and each Controller in the test environment. The Web server and the BeSim system must be able to communicate with each other.

2.6   Measurement Device (Power and Temperature) Setup

Set up the analyzer/analyzers according to the instructions in the Power and Temperature Hardware Setup Guide. Note that these measurement devices need to be set up for both the SUT System as well as the Storage as per the requirements outlined in the SPECweb2009 Run and Reporting Rules.

2.7   Controller Setup

  1. PTDaemon version 1.3.9 is distributed with the kit. The latest version of PTDaemon can be downloaded from the Accepted Measurement Devices page.

  2. PTDaemon should install into <PTD_Home> when installing from the kit. If installing a newer version, copy PTDaemon into <PTD_Home>.

  3. Install an analyzer or a temperature sensor to the controller's interface (such as USB, Serial port etc).
  4. Start the PTDaemon using the PTDaemon script. Use ptd –h to get a brief help.  An example of the command is

<PTD_home>/ptd –p 1920 -l mylogfile 8 /dev/term/a

In the example above, the power daemon is communicating with the Yokogawa WT210, a type 8 analyzer, via /dev/term/a interface and communicates with the primary client via port 1920. All messages of the power analyzer are logged in mylogfile (logging power analyzer messages is not a requirement, so the –l option may be omitted).

For further details on hardware installation, please see the Power and Temperature Hardware Setup Guide.

2.8   Power Analyzer Setup

Please refer to Section 2 of the SPECweb2009 Run and Reporting Rules for the requirement of power measurement and consult the manufacturer's documentation for instructions that are specific to the exact analyzer used.   Once you have correctly associated the device number with the corresponding integer values that apply to those devices, follow the steps listed below:

  1. Connect the communication interface of the power analyzer to the appropriate port of the controller server with a manufacturer-approved cable.

  2. Open runpower.bat for Windows or runpower.sh for Unix with a text editor, and locate the set DEVICE= for the runpower script respectively. Enter the correct device number for the power analyzer being used.

    set DEVICE=8 (for Windows & Unix)

    Now enter the DEVICE_PORT that the PTD uses to communicate with Controller system.

    set DEVICE_PORT=COM1 (for Windows)

    set DEVICE_PORT=/dev/tty2 (for Unix)

    To verify what port the power meter is using (for Windows) you may left click my computer then click 'Manage'. Click 'Device Manager' and then click 'Ports' this should give you a list of all the ports attached to your computer at that time. Verify the port the power analyzer is connected to.

  1. Set power analyzer communication settings as required by the Power and Temperature Hardware Setup Guide.

  2. Connect the SUT's AC power cord to the power receptacle provided by the power analyzer.

2.9   Temperature Sensor Setup

Please refer to Section 2 of the SPECweb2009 Run and Reporting Rules for the requirement of temperature measurement and consult the manufacturer's documentation for instructions that are specific to the exact sensor used.   Then follow the steps listed below:

  1. Place the temperature sensor in accordance with the rules outlined in this section.

  2. Connect the temperature sensor's communication interface to the appropriate port on the controller server with a manufacturer-approved cable.

  3. Open runtemp.bat for Windows or runtemp.sh for Unix with a text editor, and locate the set DEVICE= for the runtemp script respectively. Enter the correct device number for the temperature sensor being used.

    set DEVICE=1001 (for Windows & Unix)

    Also locate the set DEVICE_PORT= for the runtemp script respectively. Enter the correct port number used to connect the temperature sensor.

    set DEVICE_PORT=USB (for Windows)

    set DEVICE_PORT=/dev/tty1 (for Unix)

To verify what port the temperature sensor is using (for Windows) you may left-click my computer, click Manage. Next click Device Manager and then click Ports this should give you a list of all the ports attached to your computer at that time. Find the port that the temperature sensor is connected to.

3 Running SPECweb2009

3.1   Test.config Parameters

Once the Web server, clients, controllers, and BeSim system have been configured and networked, the next step is to try running a test strobe for one of the workloads. 

1.    To start with a short test, edit the Test.config and the workload specific config files.

In the Test.config, verify that the following parameters are appropriately defined:

There are another three parameters that must be set. Please refer to SPECweb2009 run rule document for detailed description. The following table summarizes the minimum values required for a valid run.

Name

Value (regular)

Value (power)

Comment

WARMUP_SECONDS

300

300

minimum value

THREAD_RAMPUP_SECONDS

180

180

minimum value

THREAD_RAMPDOWN_SECONDS

180

180

minimum value

3.2   Workload Specific Config Parameters

Verify the workload specific config file (SPECweb_Banking.config or SPECweb_Ecommerce.config or SPECweb_Support.config or SPECweb_Power.config) to make sure that the following parameters are defined appropriately.  The list below shows the required values for a conforming run of the benchmark. In order to get a conforming run, parameters that are in the "Do not modify" section of the config file should not be modified. However, the user may want to use shorter intervals in order to test the working of the harness and get some initial test runs done before embarking on a mission to get a conforming run.

For each power analyzer device i, the  parameter PTD_AMP_RANGE[i] specifies the ampere range. Appropriate ranges ensure that SPECweb2009 requirements concerning power measurement uncertainty are met. Ranges can be specified with numerical ampere values as in the following examples. These values represent upper bounds of the ranges. Alternatively, the string auto can be specified, which sets the device to automatic range detection mode. This mode is discouraged, however, as the automatic adjustments may occur unsynchronized with the measurement intervals. As shown in the following examples, workloads Banking, Ecommerce and Support require a single range value, while Power requires a concatenation of six values. If the parameter is not specified, the benchmark will run without changing the range status of the device.

PTD_AMP_RANGE[i]=2 - Amp range setting of power analyzer.
PTD_AMP_RANGE[i]=6^5^4^3^2^1 (where each number separated by a "^" represents the range to use for the load level. Since the Power workload uses 6 different levels, a compliant run will use six different values separated by "^".  For all the other workloads PTD_AMP_RANGE only requires a single number as shown.

A few additional parameters have to be set as well. The following table summarizes how to set their values for a valid run. Changing the values of parameters with fixed values will not only invalidate your run, but may also cause errors during the run.

Name

Value (regular)

Value (power)

Comment

ITERATIONS

3

1

fixed value

NUMBER_OF_DISCRETE_WORKLOAD_POINTS

NA

6

fixed value (not defined for reg. runs.)

RAMPUP_SECONDS

300

-

fixed value (not used for power)

RAMPDOWN_SECONDS

300

-

fixed value (not used for power)

RUN_SECONDS

1800

600

fixed value

INTRA_INTERVAL_RAMP_SEC

-

120

min. value (not used for regular runs)

INTRA_INTERVAL_WARMUP_SEC

NA

60

fixed value (not defined for reg. runs.)

3.3   Starting the benchmark

Next, test the setup using the commands below. 

You may want to use 2 windows or redirect the output of the two Java commands to files or create a script to run the harness and to save the output:

  1. Start the specwebclient process on each client machine. Execute the following command in the directory where the client was installed, or run the start_client script provided in the client directory for more details.
    •   java-Xms512m -Xmx512m -jar specwebclient.jar

  2. Start the Power and temperature daemons. Please refer to the runpower and runtemp script files in the PTDaemon directory for more details.

  3. To start the test harness manager process on the prime client begin the test, execute the following command in the prime client directory, or modify the provided start_prime_client scripts.

    Windows:
    •   java-Xms512m -Xmx512m -cp .;lib\jcommon-1.0.15.jar;lib\jfreechart-1.0.12.jar;bin\specweb.jar specweb

    Linux:
    •   java-Xms512m -Xmx512m -cp .:lib/jcommon-1.0.15.jar:lib/jfreechart-1.0.12.jar:bin/specweb.jar specweb

After test has completed check the redirected output and the contents of the results. Please check whether there are power measurements in the output of the primary client as well.

Note: you may wish to set other Java parameters besides the heap parameters to optimize the specwebclient processes as you increase the test load.  Check the documentation for your JVM.

Did that test work?

Yes? Great, so did these instructions "work" or were there missing details or do you have additional questions that you are now writing up to send to mailto:web2009support@spec.org?

No? Well, there should be enough information in the debug output to start debugging. Please send any corrections or requests for clarification to these instructions to the email address above.  Continue reading and reviewing the Web server, BeSim, and Client Checks below, along with the related README files for the various components in the attached appendixes, particularly Appendix D for common errors and Appendix E for the Support FAQ.

Now, that the basic settings are functional, you can try tests that use more Sessions and adding more clients.

3.4   Trouble Shooting your first run

3.4.1 Web Server Checks

  1. Have you configured your Web server to support the HTTP or HTTPS based on the workload you are running?  Banking is all HTTPS (SSL), Ecommerce requires both HTTP and HTTPS, and Support uses only HTTP.  Note: that you can temporarily override the use of HTTPS in Banking and Ecommerce by setting USE_SSL = 0 in the .config file.  Results will be noncompliant but this may be useful when debugging.

  2. Have you configured you Web server to support PHP, JSP or ASP.NET according to the directions provided with your Web server?  If using JSP and a separate Java application server such as Tomcat also review its documentation and configure appropriately.  Try running the PHP, JSP or ASP.NET "Hello World" equivalent to make sure your Web server's script engine is working.

  3. Have you configured the wafgen.rc files for the workload and run Wafgen to create the fixed and scaling file sets? Did you set the number of SIMULTANEOUS_SESSIONS to a value equal or greater than the value you plan to use in your testing?

  4. Can the Web server system ping the BeSim system?  Connect to the BeSim Web server (telnet [BESIM_IP] [BESIM_port])?

3.4.2 BeSim Checks

  1. Have you installed and configured a Web server on you BeSim system to run on port 81 (or other non-standard port number)?

  2. Have you built and installed the BeSim application (FastCGI, NSAPI, ISAPI) in the appropriate directory under the Web server's document root?

  3. Have you run the test_besim_* scripts provided with BeSim to check that the BeSim setup is working?  If not, see  Section 2.3.

3.4.3 Prime Client Checks

  1. Have you checked the SPECweb_<test type>.config?

  2. Have you checked Test.config? Verify the following parameters in section 3.1 match your test environment.

    TEST_TYPE=SPECweb_<test type>
    WEB_SERVER=<SUT>
    BESIM_SERVER=<BeSimsystem>
    BESIM_PORT=81 [or the non-80 value set up on the BeSimweb server]

  3. Verify that BESIM_INIT_SCRIPT matches the relative path of the BeSim API installed on the BeSim Web server.

  4. Verify that the Prime Client can "ping" the other clients, controllers, the SUT and BeSim system.

  5. Made a quick check that the prime and other clients can successfully do a "telnet [WEB_SERVER_IP] [80 and/or 443]" and that you can successfully perform a "telnet [BESIM_IP] [BESIM_port]" from the prime?

  6. Checked that the prime (and other clients) have set per process limits for threads (i.e. max_thread_proc) and file descriptors (i.e. maxfiles) adequately for the number of simultaneous sessions you plan to test?  For example, if you are running a test for 1500 simultaneous sessions and using 2 clients, each client is responsible for 750 sessions.  Each session uses ~3 connections to the server, so each client process (i.e. java -jar specwebclient) will need just over 750 * 3 threads or 2100+ threads and 750 * 3 file descriptors (socket = FD) or 2100 file descriptors.  If your clients' settings aren't adequate, then your test will fail and you'll get a Java error telling you it couldn't start a thread or establish a connection.

3.5   Collecting Statistics (optional)

The prime client has the ability to poll clients for data at specified intervals during a run and display it as CSV or in a GUI.  Data is collected and displayed only during the warm-up and benchmark run phases.  The following configuration parameters in Test.config control the collection and display of this information:

3.6   Optimizing Performance

Once the SPECweb2009 benchmark is running the workload(s) without errors, you will want to optimize your testbed for optimal performance.  Here are some tips:

4 Appendices

4.1   Appendix A - README file from SPECweb2009 client kit

Here are the basics for setting up SPECweb2009 Clients:

• First, you should at minimum have version 1.6.0 of the java jdk and/or jre installed on the client machines running this code to take
advantage of garbage collection optimizations in the more recent jvms.

• Next, there are six different .config files used for SPECweb2009. There is a SPECweb_<workload>.config for each of the four
workloads (Banking, Ecommerce, Power and Support) that contain both configurable and fixed parameters specific to the workload.
Additionally, Test.config contains parameters that are common to all workloads and includes configurable and fixed parameters.

• Lastly, Testbed.config contains the testbed hardware/software configuration properties.

Note that Test.config and the workload-specific configuration files mentioned above do not exist in a new installation. Instead,
example workload and Test configuration files are provided, and you will need to copy the sample file most suited to your environment.

Example:
• cp Test.Unix-PHP.config Test.config
• cp SPECweb_Banking.Unix-PHP.config SPECweb_Banking.config
• cp SPECweb_Ecommerce.Unix-PHP.config SPECweb_Ecommerce.config
• cp SPECweb_Support.Unix-PHP.config SPECweb_Support.config
• cp SPECweb_Power.Unix-PHP.config SPECweb_Power.config

"specwebclient" is the SPECweb2009 client load generator. On each client machine, after editing the .config files to match your test environment, you need to start this process on all client machines. From the directory where specwebclient.jar is installed, and at a command prompt type:
java -jar specwebclient.jar

Starting "specwebclient" with tuned heap sizes and garbage collection parameters is also recommended. Specifically, for v1.6
of the jdk on a multi-processor client, using parallel and CMS garbage collection is strongly recommended. The flag for
ForceTimeHighResolution is highly recommended, particularly while running the client on multi-core platforms.

Example:
java -cp c:/program files/specweb2009 -Xms500m -Xmx500m -XX:+ForceTimeHighResolution -XX:+UseParNewGC -XX:+UseConcMarkSweepGC –jar c:/program files/specweb2009/specwebclient.jar

The usage for specwebclient is:
java -jar specwebclient.jar -h

Usage: java -jar specwebclient.jar [-v] [-p port] [-n hostname] [-s servername] [-le errorlogfilename] [-lo stdoutlogfilename]

Command line flags:
-v Print version string and exit
-p <port> Override default port number (allows multiple instances on same machine)
-n <hostname> Override hostname (use Alias)
-s <servername> Overrides WEB_SERVER
-le <errorlogfilename> redirects error messages
-lo <stdoutlogfilename> redirects other logging messages

"specweb" is the SPECweb2009 test manager and is run on the Prime Client. It is started after all the "specwebclient" processes have been started.
Starting specweb should begin the test. The "specweb" process exits at the end of the test, and the "specwebclient" process can be configured to
terminate as well (see Test.config). To start the "specweb" process, from the directory where specweb.jar is installed, at a command prompt type:

Windows:
java -Xms512m -Xmx512m -cp .;lib\jcommon-1.0.15.jar;lib\jfreechart-1.0.12.jar;bin\specweb.jar specweb

Linux:
java -Xms512m -Xmx512m -cp .:lib/jcommon-1.0.15.jar:lib/jfreechart-1.0.12.jar:bin/specweb.jar specweb

The usage for specweb is:

Windows:
java -Xms512m -Xmx512m -cp .;lib\jcommon-1.0.15.jar;lib\jfreechart-1.0.12.jar;bin\specweb.jar specweb -h

Linux:
java -Xms512m -Xmx512m -cp .:lib/jcommon-1.0.15.jar:lib/jfreechart-1.0.12.jar:bin/specweb.jar specweb –h

Usage:

java -jar specweb.jar [-C] [-v] [-w workloadname] [-le errorlogfilename][-lo stdoutlogfilename]

Command line flags:
-h Print this message and exit
-v Print version string and exit
-C Overrides any non-compliant settings in the .config files with the default compliant values
-w Overides the TEST_TYPE in the config file (SPECweb_Banking, SPECweb_Ecommerce, SPECweb_Support, SPECweb_Power)
-le <errorlogfilename> redirects error messages
-lo <stdoutlogfilename> redirects other logging messages

"reporter" is the SPECweb2009 report generator and is invoked by "specweb" at the end of a test to produce .TXT and .HTML reports. It can also be invoked
to create a merged .raw file from results from running the 4 workloads on a given testbed for a submission to SPEC; as well as generating combined .TXT and .HTML
report files.

Here is the usage for the reporter:
java -jar bin/reporter.jar -h

SPECweb2009 Reporter 1.10
Copyright (c) 2009 Standard Performance Evaluation Corporation

Command line flags:
-h Print this message and exit
-v Print version string and exit
-c Combine 4 separate workload files into one submittable .raw file
-R Regenerate HTML and ASCII reports from combined rawfile

Example Reporter Usage:

Regenerate reports from a single raw file:
java -jar reporter.jar results\SPECweb_Support.20090323-102204.raw

Combine four compliant workload raw files into one submittable raw file:
java -Xms512m -Xmx512m -cp .;lib/jcommon-1.0.15.jar;lib/jfreechart-1.0.12.jar;bin/reporter.jar reporter -c bank.raw ecom.raw support.raw power.raw submit.raw
Note that raw file of SPECweb_Power run (like the power.raw of the example) must be the last raw input file when combining the 4 workload raw files.

Regenerate reports from a previously combined raw file:
java -jar bin/reporter.jar -R submit.raw

 

4.2   Appendix B - Wafgen README file

4.2.1 Invoking Wafgen:

Unix/Linux:
Wafgen.sh <workload_specific>.rc

Windows:
Wafgen.bat <workload_specific>.rc or alternately:
java -Xms384m -Xmx384m -jar Wafgen.jar <workload>.rc or:
java -Xms384m -Xmx384m -classpath Wafgen.jar org.spec.wafgen.Wafgen <workload>.rc

4.2.2 Wafgen Overview

Wafgen is used to create the file sets used by SPECweb2009. Each workload has a "fixed" and a "scaling" file set.
The fixed file set includes a set of static elements that are commonly referenced by the pages that make up a given workload.
The scaling file set includes a set of directories each containing additional static elements for the workload.
The number of directories created(or referenced by a test) is calculated by:

(SIMULTANEOUS_SESSIONS * DIRSCALING) = Total #directories

The total amount of storage required for a benchmark is also be dependent on the total number of directories in the scaling file set and the size of the contents of each directory (see Storage Requirement below). Since the totalnumber of directories needed could exceed system limits for the number of files per directory (usually ~32,000),
an additional level of subdirectories can be added using the SUBDIR* parameters. By default, banking and ecommerce use SUBDIR's. The scaling directories are round-robined across the specified number of subdirectories.

This kit contains the resource files (.rc) for each workload that are used as input to Wafgen. There is a set tailored for both Windows and Unix/Linux in the windows and unix subdirectories. To use these files copy the appropriate set into the upper directory and edit them as needed to match your test environment. For conforming results - ONLY parameters listed under Section A of the .rc file may be modified. Double check all section A parameters, especially:

DOCROOT (if not set will default to current directory)
BASEDIRNAME (see examples in <workload_specific>.rc files)
BASESUBDIRNAME (if applicable)
SIMULTANEOUS_SESSIONS (may be larger than the corresponding value in SPECweb_<workload>.config)

Also verify that the SPECweb_<workload>.config file parameters in the test harness on the Prime Client are in agreement.

The .rc files for fixed file sets include:

bank_image_props.rc
ecommerce_image_props.rc
support_image_props.rc

The .rc files for scaling file sets include:

bank_usercheck_props.rc
ecommerce_productline_props.rc
support_downloads_props.rc

The .rc files are annotated to provide additional guidance. Reminder: Altering .rc file parameters listed in section B will result
in non-conforming tests, so only modify parameters in section A.

4.2.3 Storage Requirements:

Since the benchmark requires that the System Under Test (SUT) has the same set of hardware components
that could contain the max file set among all four workloads, it is important to understand the storage requirement
for each workload. Below are the data and formulas to help estimate storage requirements for SPECweb2009.

4.2.3.1   Banking:

The fixed file set created by bank_image_props.rc includes:

bank/images - 44 files; Average size: 3570 bytes; Total size: 157096 bytes
bank/dynamic_padding - 15 files; Average size: 21800 bytes; Total size: 327000 bytes

The scaling file set created by bank_usercheck_props.rc creates a number of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.

Each bank/images/{subdir00mm}/user0000nnnnnn contains - 20 files; Average size: 10431 bytes; Total size: 208625 bytes

To estimate the storage requirement for the user check directories use the following formula:

(SIMULTANEOUS_SESSIONS * CLIENT_SESSION_USER_IDS) * 208625

For Conforming results, CLIENT_SESSION_USER_IDS (alias for DIRSCALING) must be 50.

Example: ((1000 SIMULTANEOUS_SESSIONS * 50 CLIENT_SESSION_USER_IDS) * 208625) = 10,431,250,000 bytes

4.2.3.2   Ecommerce and Power:

The fixed file set created by ecommerce_image_props.rc includes:

ecommerce/images - 172 files; Average size: 2032 bytes; Total size: 349557 bytes
ecommerce/dynamic_padding - 11 files; Average size: 73000 bytes; Total size: 803000 bytes

The scaling file set created by ecommerce_productline_props.rc creates a number of productline0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.


Each ecommerce/images/{subdir00mm}/productline0000nnnnnn contains - 30 files; Average size: 18816 bytes; Total size: 564495 bytes

To estimate the storage requirement for the product line directories use the following formula:

(SIMULTANEOUS_SESSIONS * DIRSCALING) * 564495

For Conforming results, DIRSCALING must be 5.

Example: ((1000 SIMULTANEOUS_SESSIONS * 5 DIRSCALING) * 564495) = 2,822,475,000 bytes

4.2.3.3   Support:

The fixed file set created by support_image_props.rc includes:

support/images - 31 files; Average size: 728 bytes; Total size: 22580 bytes
Warning: this file set contains several very small files (30 bytes) so the path name must not be longer than 14 characters. So, "support/images" is OK but "supporting/image" is NOT.
support/dynamic_padding - 6 files; Average size: 59616 bytes; Total size: 357700 bytes

The scaling file set created by support_downloads_props.rc creates a number of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.


Each support/downloads/{subdir00mm}/dir0000nnnnnn contains - 16 files; Average size: 4279501 bytes; Total size: 68472021 bytes


To estimate the storage requirement for the download directories use the following formula:

(SIMULTANEOUS_SESSIONS * DIRSCALING) * 68472021

For Conforming results, DIRSCALING must be 0.25.

Example: ((1000 SIMULTANEOUS_SESSIONS * .25 DIRSCALING) * 68472021) = 17,118,005,250 bytes

Note: The heap parameters used for Wafgen are required to support the large files in the scaling file set for Support workload.

4.2.4 Tips for Building Large Filesets:

To increase the size of an existing file set to support a larger number of Simultaneous Sessions than used initially, the steps are:

Edit the <workload>.rc file for the scaling fileset create previously and change these parameters:
SIMULTANEOUS_SESSIONS=<new higher value>
FIRSTDIRECTORY=<1+highest directory number previously create>

Example:

Originally a support_downloads_props.rc file used:
SIMULTANEOUS_SESSIONS=1000
FIRSTDIRECTORY=1

When a file set to support 2000 Simultaneous Sessions is needed, change:
SIMULTANEOUS_SESSIONS=2000
FIRSTDIRECTORY=251

Then run:
Wafgen <updated_workload>.rc

To run several Wafgen's in parallel to create a large file_set, copies of I the <workload>.rc file can be for each parallel wafgen to be run and make the changes to the SIMULTANEOUS_SESSIONS and FIRSTDIRECTORY directories. To calculate the correct values for FIRSTDIRECTORY in each copy use the formula:

Number of Directories = SIMULTANEOUS_SESSIONS * DIRSCALING
(or for banking the DIRSCALING alias CLIENT_SESSION_USER_IDS is used)

Use the directory scaling formula for the workload (shown above) to calculate the correct values for FIRSTDIRECTORY in each copy.

For example, to create a support downloads file set for a total of 4000 Simultaneous Sessions:

support_downloads_props.rc_1:
SIMULTANEOUS_SESSIONS=1000
FIRSTDIRECTORY=1

support_downloads_props.rc_2:
SIMULTANEOUS_SESSIONS=2000
FIRSTDIRECTORY=251

support_downloads_props.rc_3:
SIMULTANEOUS_SESSIONS=3000
FIRSTDIRECTORY=501

support_downloads_props.rc_4:
SIMULTANEOUS_SESSIONS=4000
FIRSTDIRECTORY=751

Then run 4 parallel wafgens:
Wafgen support_downloads_props.rc_1 &
Wafgen support_downloads_props.rc_2 &
Wafgen support_downloads_props.rc_3 &
Wafgen support_downloads_props.rc_4 &
(Note: depending on the number of CPUs and the I/O subsystem being used this sequence may not take significantly less time than just running a single Wafgen for the whole file set.)

To convert from a file set that was recreated without using *SUBDIR* parameters or to add additional subdirectories, the original file set will need to be re-created from scratch. The <workload>.rc file will need to be edited to configure these parameters. The parameters are preceded by the comment string "#subdirs#" if use of SUBDIRS is not the default (i.e Support).

Multiple disks or mount points may be used to hold subsets of a workload's file sets, for example: /<docroot>/bank may have the images directory as as separate mount point or /<docroot>/bank/images may have separate mount points for each of 10 subdirectories (subdir0001 thru subdir0010).

Warning: The path names have been standardized in the wafgen <workload>.rc files and the SPECweb_<workload>.config files in the harness, so that in the DOCROOT there would be these directories:

bank/images
bank/dynamic_padding
ecommerce/images
ecommerce/dynamic_padding
support/images
support/dynamic_padding
support/downloads

While it is possible to change these names in the .rc and .config files with due care (and making sure not to exceed path length limits for some very small files); it is recommended that users retain this naming to avoid errors due to typos and to simplify debugging.

Note: Using parallel wafgens to create the datasets may impact the performance.  

 

4.3   Appendix C - BeSim Make_Readme file

How to Build BESIM:

UNIX/LINUX:

Optionally you can specify other make targets by adding TARGET='list of targets'

For example:
make zisapi TARGET='clean'
make nsapi TARGET='clean all install'

The install target will use the path specified by DEST to install the besim executable or library, so it should be set to the appropriate place on your Besim webserver directory tree (i.e. /www/fast-cgi). If you plan to use Fast-cgi you must build and install the Fast-CGI development kit which provides the fast-cgi library and include files before build BESIM.
If using Apache for the BESIM webserver, that will need to include mod_fastcgi. See: http://www.fastcgi.com/

Note: Please send any new or updated MakeIncl.* files you want included in a future release to web2009(at)spec.org.

Windows:
Project files are included as an example: BeSim.def, BeSim.dsp, BeSim.dsw

 

4.4   Appendix D - Common Errors and Debugging Tips

By default the DEBUG parameter is set to 1 in Test.config, if a validation error occurs this value is sufficient to print out the
ERROR message related to the invalid response.  The invalid response is also printed out and that is followed by a printout
of the Request that produced the invalid response.  Reading the returned HTML response and the associated HTTP Request
is key to debugging problems.

Status 404 (file not found) errors usually mean that there is a path problem either in the .config files on on the associated
Web server.  In some cases it may indicate that not a large enough file set was created by Wafgen.

Errors Returned by BeSim are found in the body of the returned HTML

 

4.5   Appendix E - Support FAQ

NOTE: For the latest version of this document, please visit http://www.spec.org/web2009/docs/supportfaq.html.

 

4.6   Appendix F - Installing BeSim as a FastCGI for the Apache HTTP Server

There are two pieces necessary to get BeSim for FastCGI working properly under Apache:

  1. Compiling the FastCGI source code and BeSim
  2. Compiling/configuring mod_fastcgi, an Apache module to invoke FastCGI for specific directories

4.6.1 FastCGI source compilation

  1. Compile the FastCGI source code.  The source tree is uded on a BeSim (or full) installation of SPECweb2009. Here are steps to compile it:
    cd <path_to_SPECweb2009>/besim/fcgi-2.4.0/
    ./configure --libdir=/lib
    make
    make install

    NOTE: --libdir=/lib was added above due to the default FastCGI Makefile installing libraries to /usr/local/lib, which is not a default library path on Linux and could cause this error upon execution of the FastCGI:
    besim_fcgi.fcgi: error while loading shared libraries: libfcgi.so.0: cannot open shared object file: No such file or directory

    NOTE #2: For x86_64 versions of Fedora Core 3 and 4, it was observed that --libdir=/lib64 should be specified instead of /lib.

  2. Create a directory for the BeSim FastCGI within your web server directory hierarchy (but preferably outside the document root).  Here are some recommended locations for different setups:

    SuSE: /srv/www/fcgi-bin
    Red Hat: /var/www/fcgi-bin
    Apache 2.x compiled from source: /usr/local/apache2/fcgi-bin

  3. Compile BeSim as a FastCGI (replace /var/www/fcgi-bin/ with the directory created in the previous step):
    make fcgi TARGET='clean all install' DEST=/var/www/fcgi-bin/

  4. This should create besim_fcgi.fcgi and install it into DEST.  If not, check the make output for errors.

4.6.2 Apache FastCGI Web server module: mod_fastcgi

These instructions show how to compile the FastCGI web server module for Apache (mod_fastcgi). For other web servers, please refer to their documentation to see whether FastCGI support is available.

NOTE for SuSE: SUSE Linux (8.1 and above) ships with a precompiled mod_fastcgi for Apache 2.x.  To check if it is installed, do a`rpm -q apache2-mod_fastcgi`.If it's not, install this package from the CDs (see your OS documentation for more instructions).  If this package is installed, you can skip the rest of this section.

NOTE for Red Hat: If using the httpd package that ships with the distribution, the httpd-devel RPM must also be installed, as it contains the necessary utilities and headers to compile Apache modules.  To check, do a rpm -q httpd-devel.

  1. Download the latest mod_fastcgi tarball from http://www.fastcgi.com/dist/ and follow the directions in the INSTALL document (INSTALL.AP2 for Apache 2.x).

    NOTE: As INSTALL.AP2 states, unless you are compiling from source, you will need to modify top_dir in your make command.  For example, Red Hat users should use the following 'make install' command:
    make top_dir=/usr/lib/httpd install

    You should now have a mod_fastcgi.so in your Apache modules directory:

    Red Hat:
    -rwxr-xr-x 1 root root 161408 Sep 16 13:47 /usr/lib/httpd/modules/mod_fastcgi.so
    Compiled from source:
    # ls -l /usr/local/apache2/modules/mod_fastcgi.so
    -rwxr-xr-x 1 root root 156434 Sep 21 10:03 /usr/local/apache2/modules/mod_fastcgi.so

  2. Create a directory for FastCGI to store Unix socket files, and change the permissions so the Web server can have full access to it:

    Red Hat:
    # mkdir -p /etc/httpd/fastcgi
    # chmod 777 /etc/httpd/fastcgi

    Apache 2.x from source:
    # mkdir -p /usr/local/apache2/fastcgi
    # chmod 777 /usr/local/apache2/fastcgi

  3. Add support for handling FastCGIs by making the following changes to httpd.conf (for Red Hat, this is in /etc/httpd/conf/):
    1. Add this line in the "Dynamic Shared Object (DSO) Support" section:
      LoadModule fastcgi_module modules/mod_fastcgi.so

    2. Add the following lines just after theScriptAlias /cgi-bin/ line:
      ScriptAlias /fcgi-bin/ "/var/www/fcgi-bin/" #<fcgi-bin directory from Section 1, Step 2>
      FastCgiIpcDir /etc/httpd/fastcgi # <FastCGI directory from previous step>

      NOTE: Replace /var/www and /etc/httpd with /usr/local/apache2 if compiling from source.
    3. Add the following section, preferably after the<Directory "<CGIDIR>/cgi-bin"> section:
      <Directory "<CGIDIR>/fcgi-bin"> # <fcgi-bin directory from Section 1, Step 2>
        AllowOverride None
        Options +ExecCGI -Includes
        SetHandler fastcgi-script
        Order allow,deny
        Allow from all
      </Directory>
      AddHandler fastcgi-script fcgi

4.6.3 Testing the BeSim FastCGI

  1. Start (or restart) your web server.
  2. At the console, monitor your Apache error log to view any errors or warnings during FCGI execution, i.e.
    # tail -f <APACHE_ROOT>/logs/error_log

  3. You should initially see some FastCGI initialization messages such as:
    [Wed Sep 21 10:36:39 2009] [notice] FastCGI: process manager initialized (pid 25700)

  4. Test the BeSim FastCGI, using the scripts in the subdirectory of the SPECweb2009 installation.  See BeSim Setup for examples.

See BeSimsections above for additional information.

 

The SPECweb2009 benchmark utilizes the JFreeChart library, which is licensed under the Lesser General Public License (LGPL). Source code and a copy of the license are included with the benchmark kit.

Copyright © 2009 Standard Performance Evaluation Corporation.  All rights reserved.

Java® is a registered trademark of Sun Microsystems.