LCG Application Area - LCG Infrastructure: SW - testing

NOTE:
Most of the information given here was updated in the specific HowTo (for CppUnit, Oval, PyUnit and QMTest). We maintain this document because it contains some useful information for those developers working in CMT or Cygwin.

HowTo for Test Execution Frameworks:

  1. Introduction and tools descriptions.
  2. Access to the tools:
      2.1 - HowTo directly access the tools under AFS & Linux
      2.2 - HowTo directly access CppUnit under DFS & Windows
  3. Examples:
      3.1 - HowTo download and run easy CppUnit and Oval test examples.
  4. SCRAM and the testing tools:
      4.1 - HowTo integrate CppUnit and SCRAM under Linux
      4.2 - HowTo use Oval and SCRAM under Linux
  5. Local tools installation:
      5.1 - HowTo install locally the tools under Linux
      5.2 - HowTo use CppUnit under Windows.
      5.3 - HowTo use Oval under Windows with Cygwin.
  6. Useful Links
  7. Others:
      8.1 - HowTo integrate CppUnit and CMT configuration management environment (Linux)
      8.2 - HowTo integrate Oval and CMT configuration management environment (Linux)

1.- Introduction and tools descriptions

This guide covers the access, local installation (if needed), setup and getting start of the test execution frameworks in use for the LCG projects. Currently three tools are provided as free software by the SPI External Software Service. These are:

A set of few examples are also provided to start with the tools and to test that the setup and local installation is OK.

Users with permanent access to AFS/DFS CERN cell can skip the local installation procedure and by default they have access to the tools, after proper tunning of the path variables. They can also download to their machines a set of examples to start with these tools.

Users with NO permanent access to AFS/DFS CERN cell have to follow the installation instructions to get a local copy installation.

Users working in a SCRAM project probably will find the tools installed and preconfigured. They only have to follow the integration steps to produce sw-testing inside the SCRAM project. If the tools are not installed the option to install them in the local area project exists also.

The users within a LCG AppArea consolidated project could jump directly in the specific HowTo for these tools, pointed by the following links:

return to top of page

2. - Access to the tools:

2.1 - HowTo directly access the tools under AFS & Linux

SPI provides pre-installed testing tools on AFS, see the External Software Service. Users with access to AFS CERN cell can use directly the preinstalled tools placed at "/afs/cern.ch/sw/lcg/external/packages/". They only need to modify their initial project setup script, or check if the correct path is already established on it. For example: The "rh72_gcc2952" directory-name means that you are selecting the libraries for RedHat 7.2 linux platform with the gcc 2.95.2 compiler. You can also select other posibilities available in the External Software SPI service. After reloading your initial setup script: 
$ source setup.csh
or 
$ source setup.sh
CppUnit and Oval will be available and you should be able to produce the following (or similar) outputs for Cppunit and Oval (user input is in bold): 
$ cppunit-config
Usage: cppunit-config [OPTION] ...

Generic options
  --version     output CppUnit version information.
  --help        display this help and exit.

Compilation support options
  --cflags      print pre-processor and compiler flags
  --libs        print library linking information

Install directories CppUnit was configured to
  --prefix[=DIR]      
  --exec-prefix[=DIR]

$ cppunit-config --version 
1.8.0
$ cppunit-config --cflags
-I/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh72_gcc2952/include
$ cppunit-config --libs
-L/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh72_gcc2952/lib -lcppunit
These inform you about the CppUnit version your are using, where are the libraries and what are the compilation flags. This information was already set in the environment variables you should use: 
$ echo $CPPUNIT_INCLUDES
/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh72_gcc2952/include
$ echo $CPPUNIT_LDFLAGS
-L/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh72_gcc2952/lib -lcppunit -ldl


For Oval you can get help just typing oval: 
$ oval

Usage: oval [-f] [-v] [-r] [-w] [-b] 

Options:
       -r: apply recursively to the directories.
       -l: do not apply recursively to the directories.
       -v: verbose mode.
       -f: fake mode, commands are not really executed.
       -w: send the log file to the watchers when
           there is a problem ; if no watcher is
           defined, use the package administrators
           instead.
       -b: inform oval that it is run on a batch system,
           then for example oval can avoid useless output.

Use Commands:
     prod: apply build/run/diff.
    build: build the available programs with scram.
      run: execute test programs.
     diff: compare last output with the reference.

Other Commands:
     help: this help.
  version: return oval version number or install a new version.
  targets: describe the valid targets for build/run/diff.
     expr: show the output sections compared with diff.
  runtime: return commands to upgrade the environment.

Other help items:
 ovalfile: description of the configuration file.
  logfile: format of the output file.
See also: file:/afs/cern.ch/sw/lcg/external/Oval/2_15_0/current/doc/html/index.html
See also: http://polywww.in2p3.fr/cms/software/oval/
return to top of page

2.2 - HowTo directly access CppUnit under DFS & Windows

2.2.1 - HowTo find the CppUnit stuff.

In order to use CppUnit with the Visual C++ compiler in a Windows platform with access to DFS CERN cell the first thing is to tell VC++ where are the CppUnit includes and libraries in the DFS space:

The disk "G" should be replaced for the logic disc drive in which you have mounted the DFS area in your pc.


2.2.2 - HowTo setup your MSV6.0 CppUnit test project

You can start from an existing Microsoft Visual C++ workspace and project or you can create them. For later option, open Microsoft Visual C++ tool and make a new workspace. After this add a new project into the workspace with type 'Win32 Console Application' and you are ready now to add source files to the project.

Let's link now Cppunit library to the project. In the project settings:

At this point you are ready to build and run your CppUnit test code.


return to top of page

3.- Examples:

3.1 - HowTo download and run easy CppUnit and Oval test examples

A set of CppUnit, Oval, PyUnit basic examples can be download from the SPI repository. The idea of these examples is to show in a very simple way how CppUnit, Oval, and PyUnit work and how they can cooperate. Once you have the tar-file 'Testing_examples.tar.gz' you can see the explanations in the README file. The examples can run in Linux/Solaris/Windows without use CMT or SCRAM.

return to top of page

4.- SCRAM and the testing tools:

4.1 - HowTo integrate CppUnit and SCRAM under linux

4.1.1- If your present SCRAM project was configured with Cppunit as external tool (if not go to section 4.1.2), CppUnit is already available and you can know it doing: 

$ scram tool list
     Tool list for location /home/user/username/LCG/PROJECTNAME_Release
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 gcc3                 3.2        (default=3.2)
 sockets              1.0        (default=1.0)
 uuid                 1.19       (default=1.19)
 mysqlpp              1.7.9      (default=1.7.9)
 mysql                4.0.4beta  (default=4.0.4beta)
 xerces-c             2.1.0      (default=2.1.0)
 root                 3.04.01    (default=3.04.01)
 edg-rls-client       1.1.2      (default=1.1.2)
 cppunit              1.8.0      (default=1.8.0)

 
 $  scram tool info cppunit
Tool info as configured in location /home/user/username/LCG/PROJECTNAME_Release
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Name : cppunit
Version : 1.8.0
Docfile : cvs://lcgapp.cern.ch/cvs/....
++++++++++++++++++++
lib=cppunit
CPPUNIT_BASE=/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh73_gcc32
LIBDIR=/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh73_gcc32/lib
INCLUDE=/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh73_gcc32/include
LD_LIBRARY_PATH=/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh73_gcc32/lib
4.1.1.B - Check the "BuildFile" in the ../LCG/project_release_name/config directory to see if the classpath of the test directory for your package has the option "UnitTests" instead of "binary". Taka a look also and be sure tat your test directory has only one rule for building and this contains "UnitTests" option. 
$ cat LCG/project_release_name/config/BuildFile | grep package_name
INCLUDE+=$(LOCALTOP)/$(INTsrc)/package_name
#package_name is a library:
<ClassPath path=+Project/package_name+Package/src+library>
<ClassPath path=+Project/package_name+Package/test+UnitTests>
this option instructs SCRAM (scram b) to compile the CppUnit test driver "package_name_TDCppUnit.cpp" against the rest of the Cppunit test classes "class_name_CppUnit.cpp" and to build a "package_name_TDCppUnit" executable for testing placed at ../LCG/project_release_name/bin/platform. SCRAM will also build all the other classes described in the specific BuildFile for the test directory and will produce the corresponding binaries.

4.1.1.C- Include the line: 

<external ref=cppunit></external>
in the BuildFile of your specific test directory.

4.1.1.D Use always the name convention:

4.1.2- If CppUnit is not already configured as external tool in your SCRAM project you can install it in your local area project following the instructions below:
4.1.2.A- Go to your local project site directory and create a basic template for CppUnit: 

 $ scram tool template basic
 $ mv basic_template cppunit
4.1.2.B- Edit the cppunit file and fill it like in the next example: 
<doc type=BuildSystem::ToolDoc version=1.0>
<Tool name=cppunit version=1.8.0>
<info url=http://cppunit.sourceforge.net/> Web-based information.
<Lib name=cppunit>
<Client>
<Environment name=CPPUNIT_BASE >
Base of CppUnit release
</Environment>
<Environment name=LIBDIR type=lib>
<Environment name=INCLUDE>
</Client>
<Environment name=LD_LIBRARY_PATH value=$LIBDIR type=Runtime_path>
</Tool>
4.1.2.C Configure the cppunit tool: 
scram setup -i cppunit 1.8.0 file:./cppunit
taking into account that CPPUNIT_BASE is "/afs/cern.ch/sw/lcg/external/CppUnit/1.8.0/rh73_gcc32/".
4.1.2.D Download from SPI the make file "UnitTests_makefile.mk" and place it in the directory "../LCG/project_release_name/config".
4.1.2.E Now you can follow the instructions from section 4.1.1.B

return to top of page

4.2 - HowTo use Oval and SCRAM under linux

4.2.1 Check if oval is already an external tool of your SCRAM project: 
& scram tool list

Tool list for location /home/user/username/LCG/PROJECTNAME_Release
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 gcc3                 3.2        (default=3.2)
 sockets              1.0        (default=1.0)
 uuid                 1.19       (default=1.19)
 mysqlpp              1.7.9      (default=1.7.9)
 mysql                4.0.4beta  (default=4.0.4beta)
 xerces-c             2.1.0      (default=2.1.0)
 root                 3.04.01    (default=3.04.01)
 edg-rls-client       1.1.2      (default=1.1.2)
 cppunit              1.8.0      (default=1.8.0)
 oval                 2.15.1     (default=2.15.1)

$ scram tool info oval

Tool list for location /home/user/username/LCG/PROJECTNAME_Release
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Name : oval
Version : 2.15.1
++++++++++++++++++++
OVAL_BASE=/afs/cern.ch/sw/lcg/external/Oval/2_15_1/rh73_gcc32
PATH=/afs/cern.ch/sw/lcg/external/Oval/2_15_1/rh73_gcc32/bin
If it is not an external tool of your SCRAM project you should ask your SCRAM project manager but meanwhile you can use it if you add the following line to your initial setup project script: 
 # To set the path for LCG stuff (CppUnit, Oval, ...)
 setenv PATH /afs/cern.ch/sw/lcg/external/packages/0.0.1/rh72_gcc2952/bin:$PATH
After this you will have access to the tool and you can ask for a short on-line help about oval (see section 2)
4.2.2- To start your tests copy a basic OvalFile in your test directory. Because Oval was develop in the SCRAM enviroment, the OvalFile you have to use is rather simple: 
=================================================
================================================= 
OvalFile: basic configuration file to run OVAL
              validation and regression tool and/or 
              CppUnit unit-test tool.

SPI project ->Software Development Infrastructure 
                    LCG project 
=================================================
=================================================

=================================================
 Error report concerning these tests will be send
 to the following e-mail if -w option is used
 Write here your e-mail.
=================================================
<watcher mail=your_mail_address@cern.ch>

=================================================
 Standard OVAL environment to run test programs
=================================================
 For the diff command,define single lines to
 compare, through a Perl regular expression. Oval
 will find lines with "OVAL", but can be changed.
<diffline expr="^\[OVAL">

  Here we define a new environment
<environment name=StandardOval-0>
<bin name="your_first_oval_Test.cpp">   
<bin name="your_second_oval_Test.cpp">  
  Here we will run also the CppUnit tests 
<bin name="your_package_name_TDCppUnit.cpp"> 
  Here we run a script for testing
<script name="testScript.py">
  Here we run a script for  valgrind  memory leak cheks
<script name="ovalgrind" args="MyComp_BasicTest">
<script name="ovalgrind" args="MyComp_TDCppUnit">
</environment>

</pre>
4.2.3- Write also the name of your oval test classes in the BuildFile and build all the executables using:
 
   
$ scram b 


After this you can run the executables  using Oval to create a reference file:


$ oval run 
$  mv your_first_oval_Test.log  your_first_oval_Test.ref 


and now you are ready to use the oval diff between the "log: and the "ref" files:

$ oval diff 


return to top of page
  
    
 
 
 5. - Local tools installation: 

  
       
5.1.- HowTo  install locally the tools (Linux)

    
5.1.1 -To install CppUnit in a machine that has not permanent access to AFS CERN cell, is needed 
    to download the sources and proceed with a local installation following the commands shown below
    these lines. For the last step you will need root access unless you want to install the tool in your own 
    directory.  By default, `make install' will install the package's files in `/usr/local/bin', `/usr/local/man',
     etc.  You can specify an installation prefix other than `/usr/local' by giving to `configure' the option
      `--prefix=PATH'. The command `make check' will run a set of CppUnit tests to verify that the installation  was fine.

    
    
     $ mkdir CppUnit
     $ cp /afs/cern.ch/sw/lcg/external/tarFiles/cppunit-1.8.0.tar.gz CppUnit
     $ cd CppUnit 
     $ tar -zxvf cppunit-1.8.0.tar.gz 
     $ cd cppunit-1.8.0
     $ ./configure
     $ make 
     $ make check 
     $ make install 
     
    

    
    
 After the commands above the tool will be install in `/usr/local/' and the compilation support information can be extracted with:

    
    
    $ cppunit-config --cflags              
     -I/usr/local/include
    $ cppunit-config --libs                 
    -L/usr/local/lib -lcppunit   
    

    
    
 5.1.2
    To install Oval tool,  you have to download again the source and follow the next commands, be aware that you need perl5 installed in your machine.

    
    
    $ mkdir Oval
    $ cd Oval
    $ cp /afs/cern.ch/sw/lcg/external/tarFiles/oval_2_15_0.tar.gz
    $ tar -zxvf oval_lcg.0.0
    $ ln -s oval_2_15_0 current
    $ cd current 
    $ perl5 cook.pl  lxgcc.recipe  
    [cook] ./check.txt > ./check
    [cook] ./bin/oval.txt > ./bin/oval
    $ cp ./bin/oval  /usr/local/bin
        

  In this way Oval will be in your /usr/local/bin directory and you can access it by typing:
  
$oval
Usage: oval [-f] [-v] [-r] [-w] [-b] 

Options:
       -r: apply recursively to the directories.
       -l: do not apply recursively to the directories.
       -v: verbose mode.
       -f: fake mode, commands are not really executed.
       -w: send the log file to the watchers when
           there is a problem ; if no watcher is
           defined, use the package administrators
           instead.
       -b: inform oval that it is run on a batch system,
           then for example oval can avoid useless output.

Use Commands:
     prod: apply build/run/diff.
    build: build the available programs with scram.
      run: execute test programs.
     diff: compare last output with the reference.

Other Commands:
     help: this help.
  version: return oval version number or install a new version.
  targets: describe the valid targets for build/run/diff.
     expr: show the output sections compared with diff.
  runtime: return commands to upgrade the environment.

Other help items:
 ovalfile: description of the configuration file.
  logfile: format of the output file.
See also: http://polywww.in2p3.fr/cms/software/oval/ 




The oval version you are running is accesible typing:

$ oval version
oval_2_15_0


and it is possible to try an exmple which comes with the Oval distribution:

$ pwd 
/home/Oval/current
$ oval prod test/subtest3
oval_2_15_0/test/subtest3/: Prog1 (DIFFS)
$ cd test/subtest3
========================== build ===========================
g++ Prog1.cc -o Prog1
========================== run =============================
hello
[OVAL] input ok
[OVAL] result =1
========================== diff ============================
(DIFFS)
=== log #23 != ref #69
log: [OVAL] result = 1
---
ref: [OVAL] result = 1.2
===========================================================




    
    return to top of page

  
  
    
5.2 - HowTo use CppUnit under Windows.

    5.2.1 -  HowTo get CppUnit

    
 You can download the CppUnit sources from http://sourceforge.net/projects/cppunit. 

    

    5.2.2 -  HowTo compile and install CppUnit libraries
    
 In the following document $CPPUNIT is the directory where you unpacked CppUnit: $CPPUNIT/: include/ lib/ src/ cppunit/. 
    

    

       First, you need to compile Cppunit libraries:
       

       Once it is done (it takes a while..), you need to tell VC++ where are the includes and libraries to use them in other projects. Open the 'Tools/Options...' dialog, and in the 'Directories' tab, select 'includes files' in the combo. Add a new entry that points to $CPPUNIT/include/. Change to 'libraries files' in the combo and add a new entry for $CPPUNIT/lib. Repeat the process with 'sources files' and add $CPPUNIT/src/cppunit/. 
    

    

    5.2.3 - HowTo setup your MSV6.0 CppUnit test project

    
You can start from an existing Microsoft Visual C++ workspace and project or you can create them. For later option, open Microsoft Visual C++ tool and make a new workspace. After this add a new project into the workspace with type 'Win32 Console Application' and you are ready now to add source files to the project. 
    

     
 Let's link now Cppunit library to the project. In the project settings:
     

     At this point you are ready to build and run your CppUnit test code. 
     
      
    

    

    
   

    return to top of page
    

    

    
    
5.3 - HowTo use Oval under Windows with Cygwin

    5.3.1 - HowTo get Cygwin 


    Cygwin is a UNIX environment, developed by Red Hat, for Windows. It consists of two parts:

- A DLL (cygwin1.dll) which acts as a UNIX emulation layer providing substantial UNIX
  API functionality. 

- A collection of tools, ported from UNIX, which provide UNIX/Linux look and feel. 

The Cygwin DLL works with all non-beta, non "release candidate", ix86 versions of
Windows since Windows 95, with the exception of Windows CE. 

Installation Procedure:

5.3.1.1 - Download the "setup.exe" program from Cygwin (http://www.cygwin.com/setup.exe)

5.3.1.2 - Run "setup.exe" from your download directory. This will then download the list of available , and offer to install them for you.  

5.3.1.3 - Choose "Install from Internet". When installing, you will have to specify:

- Install Root directory (e.g. C:\cygwin)
- Default text file type ("DOS" or "Unix")
- Choose the mirror site
- Select at least the following packages: all from Archive, all from Base, all from Devel, your 
prefered editor from Editors, perl from Interpreters, default from Libs, mutt from Mail, openssh from Net, all from Shells, all from System.


          
    

    5.3.2 - HowTo install Oval into Cygwin

    

 To install oval under Cygwin you have to follow the steps described in the section 3.2 of this HowTo. Previous to this you have to be sure that: (1) perl is installed, (2) csh is available,  (3) /usr/bin/mail binary is in place, (4) Cpp compiler is installed (optional). To reach this requirements you can follow the next points:


 
 5.3.2.1 - Perl installed ?. Check if Perl is installed in your local copy of Cygwin:
 
 $ perl -v 
 This is perl, v5.6.1 built for cygwin-multi
 

 if it is not installed follow these steps:
       
- On your desktop click twice in the  Cygwin setup and open the Cygwin configuration tool. 
- Press "Next" and select install from Internet
- Press "Next" and select Root Directory as "C:\cygwin"
- Press "Next" and select Local Package Directory as "C:\Documents and Settings\mgallas\Desktop"
- Press "Next" and select "Direct Connection" four your Internet Connection. 
- Press "Next" and select the mirror site. 
- Press "Next" and select the package "Interpreters" and "Perl" 
- Press "Next" and go until the end.      
        

    

    
5.3.2.2 - csh available?
    
 
 $ cp /usr/bin/tcsh.exe  /usr/bin/csh.exe
 

   5.3.2.3 - /usr/bin/mail in place?   . Although for the moment we not plan to use the 
   mail in Cygwin, Oval needs the binary "mail". So we can copy "mutt.exe" (standard Cygwin mail agent) to mail.exe and not configure the mail.
   
   $ cp /usr/bin/mutt.exe /usr/bin/mail.exe
   

  
   5.3.2.4 - Cpp compiler installed (0ptional)? 
   
$ gcc -v   
Reading specs from /usr/lib/gcc-lib/i686-pc-cygwin/3.2/specs
   

   If "gcc" is not installed you can follow the steps described in 9.2.1 and "gcc" is in the "Devel" package.


5.3.4 -  HowTo setup Oval

     
Once Oval was installed a little setup is needed to include Oval binary into the PATH and the directories where you have your executables. This can be done simply running a initial script:

$ source startup_win.sh      

 
that contains the update path:

$ cat startup_win.sh
PATH=/home/myexecutables:/home/OVAL_RELEASES/current/bin:$PATH
export $PATH


Notice here that, depending on your Cygwin installation, you have a "Cygwin" windows folder and inside it there is a "home" folder. This is the "home" referenced in the "startup_win.sh" script. But you can include into the path other directories, for example the line:

PATH=/cygdrive/c/executables:$PATH


includes the folder "executables" in the hard disk "C:" to the path.





This little setup is enough for use "oval run" and "oval diff" commands. An executable produced under Windows (for example with Ms Visual C++ 6.0) can be run with "oval run NameProgram" command and after that you can look for differences respect to  the reference file with "oval diff NameProgram" command.  The needed OvalFile is the same you use under linux/unix, no changes are needed.




The "oval build NameProgram" command will invoke the gcc compiler installed with Cygwin software and this maybe is not suitable for you. 



    

    

    return to top of page
    
    
 
  

    
     
    
    
6.- Useful Links

      
     
    return to top of page    
   
  

    
    
 Others:  

    
Before SCRAM was chossen as LCG configuration management tool,  testing tools were also prepared to work with CMT. Here we keep the work done for this integration.
 
    
    
8.1- HowTo integrate CppUnit and   CMT  configuration management environment (Linux)

      
    8.1.1 -  In your initial project script (ex: setupPOOL.csh ....) add or check if the lines described in section 2 are present. These lines are for make  CppUnit tool available into the path and to set the aproppiate environment variables (LD_LIBRARY_PATH, CPPUNIT_LDFLAGS, CPPUNIT_INCLUDES ).
   

   
  
   
8.1.2 -       In the "requirements" file, under the specific /cmt directory, the 
      following bold-lines in a Pool example are needed:

#==========================================================================
package DataSvc
version v2r0

branches src cmt doc pool test
use GaudiPolicy v5r*
use Common v2r* pool
use PersistencySvc v2r* pool
use FileCatalog v2r* pool


include_dirs $(DATASVCROOT)
# Include dir for CppUnit, you can check with 
# "ccpunit-config --clib"  and  "echo $CPPUNIT_INCLUDES"
include_dirs $(CPPUNIT_INCLUDES)


# Linker library
library DataSvc *.cpp
#   as many lines as test classes you have registered
library RetrieveTest ../test/src/RetrieveTest.cpp


# The driver CppUnit main program 
application DataSvcTest_CppUnit  *.cpp ../test/src/DataSvcTest_CppUnit.cpp

ignore_pattern RuleChecker

apply_pattern package_linkopts
apply_pattern package_stamps
apply_pattern ld_library_path

#==========================================================================
private
apply_pattern package_shlibflags

# cppunit link - you need to link against all the object files of the class
#                registered
macro_append DataSvcTest_CppUnitlinkopts "$(CPPUNIT_LDFLAGS) RetrieveTest.o"

# valid for all test applications in this package
macro_append TestMessageSvclinkopts "$(Common_linkopts)"
include_dirs $(DATASVCROOT)/src
#==========================================================================

      

where the name "DataSvcTest_CppUnit" should be composed by the name of the component you would like to check in addition to "Test_CppUnit". This will be the cppunit driver class which will manage all the CppUnit tests inside the test directory, in this example only RetrieveTest.cpp.





8.1.3 -  For the CppUnit driver class you simply have to copy this example:



// CppUnit driver class
#include <cppunit/extensions/TestFactoryRegistry.h>
#include <cppunit/TextOutputter.h>
#include <cppunit/ui/text/TestRunner.h>

int main(int,char**)
 {

   // Get the top level suite from the registry
   CppUnit::Test *suite =
                 CppUnit::TestFactoryRegistry::getRegistry().makeTest();

   CppUnit::TextUi::TestRunner runner;

   // Adds the test to the list of test to run
   runner.addTest( suite );

   // Run the tests.
   bool wasSucessful = runner.run();
   // If you want to avoid the CppUnit typical output change the line above
   // by the following one:
   //  bool wasSucessful = runner.run("",false,false,false);

   // Return error code 1 if the one of test failed.
   cout<<"[OVAL] Cppunit-result ="<< !wasSucessful<< "\n" ;

   return 0;
 }
 




8.1.4 -  Use CPPUNIT  in the unit test classes (test registration, CppUnit asserts... ). You can follow  next example from POOL that corresponds to the RetriveTest.cpp class 

  

// MyTestCase.cpp -------------------------------------------------

#include <cppunit/extensions/HelperMacros.h>
//#include ...

// Unit test class definition
class MyTestCase : public CppUnit::TestFixture

{
  /// registration of the test class and the specific(s) method to execute
  CPPUNIT_TEST_SUITE( MyTestCase );
  CPPUNIT_TEST( doJob );
  CPPUNIT_TEST_SUITE_END();

   public:

   void setUp() { ;; //empty }
   void tearDown() { ;; //empty }
   void doJob() {

       int myId=0;
       int theId=0;
       int anId=0;

       // do something assigning values to myId,theId
       //...

       // Cppunit assertion: checking if myId == theId
       CPPUNIT_ASSERT_EQUAL_MESSAGE("Error found checking myId",myId,theId);

       // second assertion: tested only if the previous is true
       CPPUNIT_ASSERT_EQUAL_MESSAGE("Error found checking myId",myId,anId);
   }
};

/// class registration on cppunit framework
CPPUNIT_TEST_SUITE_REGISTRATION(MyTestCase);

    



8.1.5 -  To compile and run your tests you simply have to proced as usual in CMT. This is  (if you are in ../test/src/ directory)

$ source ../../cmt/setup.csh 
$ make -C ../../cmt 
$ ../../rh72_gcc2952/DataSvcTest_CppUnit.exe



 
 return to top of page



    

    
    
8.2.- HowTo integrate Oval and  CMT 
                configuration management environment (Linux)


 8.2.1 -  Check if oval is already in your path, if not you should add the 
following line to your initial setup project script (ex: setupPool.csh ...). 

# To set the path for LCG stuff (CppUnit, Oval, ...)
 setenv PATH /afs/cern.ch/sw/lcg/external/packages/0.0.1/rh72_gcc2952/bin:$PATH



 After this you will have access to the tool and to ask for a short on-line help about oval (see section 2)

 


  8.2.2 -  To start your tests copy a basic OvalFile in your test directory. The following 
  one was tunned for POOL project under CMT:
  
=================================================
 OvalFile: basic configuration file to run OVAL
           validation and regression tool and/or
           CppUnit unit-test tool.

 SPI project ->Software Development Infrastructure
               LCG project
=================================================
=================================================


=================================================
 Error report concerning these tests will be send
 to the following e-mail if -w option is used
 Write here your e-mail.
=================================================
<watcher mail=your_mail_address@cern.ch>


=================================================
 Std OVAL environment to run POOL test programs
=================================================
 Inside an environment you can define the build, run and diff 
 instructions, the tag which oval will look for at diff procedure 
 You can also set environment variables and decide the order 
 in which you would like to run your tests.
 
<environment name=DataSvcTest>
<instructions build="make -C ../../cmt %s" cshruntime="echo 'source ../../cmt/setup.csh' " shruntime="echo">
<diffline expr="^\[OVAL">
<bin name="CacheMgrTest.cpp">  Write here your comments  </bin>
<bin name="DataSvcTest.cpp">    </bin>
<bin name="SmartRefTest.cpp">    </bin>
<bin name="DataSvcTest_CppUnit.cpp">  </bin>
</environment>
=================================================
  
 
  In this specific case:

   - We defined an environment called "DataSvcTest", in such a way that the build procedure will be driven by the Makefile  under "../../cmt" directory.

   - Before run the tests inside this environment we will set the environment variables with the "../../cmt/setup.csh" script. 

   - At the diff procedure we will look for the "OVAL" tag. 

   - The name and the order of the tests we would like to run are defined by the "<bin name="NameTest.cpp">  Write here your comments  </bin>" lines. 

   
   For another options in the customization of this file you can see OVAL project pages. 
  
  
 
  

  8.2.3 -   Create now your test sources (in this example: DataSvcTest.cpp, CacheMgrTest.cpp, SmartRefTest.cpp, DataSvcTest_CppUnit.cpp) including the ouput of that you would like to compare. 
 
 
 
  

  8.2.4 -   Make the needed modifications in your "../../cmt/requirements" file in order to build your sources. And after these use oval to build the executables:
 
 $ oval build DataSvcTest
 

 or if you want to build all the executable at once you can simply type 
 
 $ oval build
 

 
 
    
 

 8.2.5 -  Run your executables with oval to produce a log file that later you will copy into a reference file

$ oval run DataSvcTest
$ mv DataSvcTest.log DataSvcTest.ref


 
   
    
  

  8.2.6 -  At this point you can launch "oval diff" command or "oval prod". The later will 
   build, run and make diff. 
   
$ oval prod 
CacheMgrTest: build, run, diff.
DataSvcTest: build, run, diff.
SmartRefTest: build, run, diff 
DataSvcTest_CppUnit: build, run, diff.
   
 
 and it will complain if something is going wrong in any of these steps.
  
  
    
 return to top of page
 
    

    
LCG AppArea SPI support

    
    
Last modified: Tue Mar  4 11:44:49 CET 2003

    

    $Revision: 1.6 $
    $Author: mgallas $