Difference between revisions of "VCLTesttool/Introduction"

From Apache OpenOffice Wiki
Jump to: navigation, search
(VCL TestTool has been obsoleted)
 
(17 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{Historical}}
 +
 +
[[ru:RU/qa/vcltesttool/introduction]]
 
{{VCLTesttool/VCLTesttoolTOC
 
{{VCLTesttool/VCLTesttoolTOC
 
|ShowPrevNext=none
 
|ShowPrevNext=none
 
|ShowPrevPage=none
 
|ShowPrevPage=none
|ShowNextPage=none
+
|ShowNextPage=VCLTesttool/Developer
 +
|NextPage=VCLTesttool/Developer
 
|ShowNextPart=none
 
|ShowNextPart=none
 
|ShowPrevPart=none
 
|ShowPrevPart=none
Line 8: Line 12:
 
=Tool for Automated Testing of OpenOffice.org=
 
=Tool for Automated Testing of OpenOffice.org=
 
= About the TestTool =
 
= About the TestTool =
The TestTool is a standalone program that is used for the automated testing of OpenOffice.org. The TestTool communicates with the TCP/IP-Interface of OpenOffice.org and can test each installation of OpenOffice.org on a PC or in a local area network (LAN). The current TestTool can be used on OpenOffice.org 1.1beta and higher. However, as there can be some incompatible changes in future OpenOffice.org builds you may need to use a newer version of the TestTool.
+
The TestTool is a program that is used for the automated testing of OpenOffice.org. The TestTool communicates with the TCP/IP-Interface of OpenOffice.org and can test each installation of OpenOffice.org on a PC. The current TestTool can be used on OpenOffice.org 3.0 and higher. However, as there can be some incompatible changes in future OpenOffice.org builds you may need to use a newer version of the TestTool.
 
+
You do not need a complete OpenOffice.org installation to run the TestTool so long as the OpenOffice.org libraries that are required by the tool are in the same directory as the ''''''testtool''''''-script on UNIX or the '''''testtool.exe''''' application on Win32. Furthermore, the '''''.testtoolrc '''''(UNIX) and the '''testtool.ini '''(Win32) files must be in the ''home'' directory of the user (UNIX) or in the ''profile'' directory of a user on Win32.
+
  
 
The TestTool communicates with OpenOffice.org using SlotIDs, UniqueIDs, and HelpIDs that are associated with each menu item, window, dialog, and window or dialog control in OpenOffice.org. The IDs are automatically generated during the OpenOffice.org build-process or are assigned by developers.
 
The TestTool communicates with OpenOffice.org using SlotIDs, UniqueIDs, and HelpIDs that are associated with each menu item, window, dialog, and window or dialog control in OpenOffice.org. The IDs are automatically generated during the OpenOffice.org build-process or are assigned by developers.
Line 16: Line 18:
  
 
{| class="prettytable"
 
{| class="prettytable"
! SlotIDs: Each menu item has a SlotID that is used, for example, to open a dialog or perform an action.
+
| '''SlotIDs:''' Each menu item has a SlotID that is used, for example, to open a dialog or perform an action.
  
 
|-
 
|-
Line 25: Line 27:
  
 
|}
 
|}
You can create test scripts for the TestTool in the programing language StarBasic (like VisualBasic). Several commands are available for the TestTool, for example, to get or put information into OpenOffice.org controls. For a complete list of the available commands, see chapter 5: ''Internal Commands, Methods and Functions for TestTool''.You can simulate most mouse or keyboard actions with the TestTool as well as gather information from controls or change the default settings in OpenOffice.org. In other words,.the TestTool lets you simulate an OpenOffice.org user.
+
You can create test scripts for the TestTool in the programing language StarBasic (like VisualBasic). Several commands are available for the TestTool, for example, to get or put information into OpenOffice.org controls. For a complete list of the available commands, see: [[VCLTesttool/API_reference|Internal Commands, Methods and Functions for TestTool]].You can simulate most mouse or keyboard actions with the TestTool as well as gather information from controls or change the default settings in OpenOffice.org. In other words, the TestTool lets you simulate an OpenOffice.org user.
  
 
= Location of the TestTool =
 
= Location of the TestTool =
The ''TestTool Environment'' can be checked out via CVS on openoffice.org.The command line looks like:
+
The ''TestTool Environment'' can be checked out via [[Mercurial/Getting_Started#Creating_a_local_copy|Mercurial]] on openoffice.org,
 
+
and will be found in the directory: testautomation;
cvs -d :pserver:anoncvs@anoncvs.services.openoffice.org:/cvs login
+
Or it is available as a seperate package at: http://ooopackages.good-day.net/pub/OpenOffice.org/qa/testautomation/
+
 
+
<nowiki>Just press <return>, when asked for a password.</nowiki>
+
 
+
cvs -d :pserver:anoncvs@anoncvs.services.openoffice.org:/cvs -z3 co -P qa/qatesttool
+
  
The ''TestTool'' application is available on other platforms in the installation set (see setup). Just use the binary from the .''../{officepath}/program''-directory. Start the program t''esttool.bin'' respectively ''testtool.exe''. Just for Mac OS X: Copy the script ''soffice'' to ''testtool'' and run it.
+
The ''TestTool'' application is available at http://qa.openoffice.org/ooQAReloaded/AutomationTeamsite/ooQA-TeamAutomationBin.html ; On other platforms in the installation set (see setup). Just use the binary from the .''../{officepath}/program''-directory. Start the program t''esttool.bin'' respectively ''testtool.exe''. Just for Mac OS X: Copy the script ''soffice'' to ''testtool'' and run it.
  
 
= Installing the TestTool =
 
= Installing the TestTool =
Line 53: Line 50:
  
 
{| class="prettytable"
 
{| class="prettytable"
! [[Image:settings_special.png]]
+
! [[Image: Testtool-option-profile.jpg]]
  
 
|-
 
|-
| # '''''Profile:''' ''Enter a name for your profile.
+
|
 +
# '''''Profile:''' ''Enter a name for your profile.
 
# '''''Log base directory''''': Enter the path where you want to save the test results.For example: ''/opt/qa/qatesttool/errorlog/mymachine'' or ''d:\qa\qatesttool\errorlog\mymachine''
 
# '''''Log base directory''''': Enter the path where you want to save the test results.For example: ''/opt/qa/qatesttool/errorlog/mymachine'' or ''d:\qa\qatesttool\errorlog\mymachine''
 
# '''''Base directory''''': Enter the ''root path'' for the ''TestTool Environment'' ('''without a slash at the end!''')For example: ''/opt/qa/qatesttool'' or ''d:\qa\qatesttool''
 
# '''''Base directory''''': Enter the ''root path'' for the ''TestTool Environment'' ('''without a slash at the end!''')For example: ''/opt/qa/qatesttool'' or ''d:\qa\qatesttool''
Line 79: Line 77:
  
 
= Automated Crashreports =
 
= Automated Crashreports =
: Choose '''Extra''' - '''Settings''' – '''Crashreport.'''
+
: Choose '''Extra''' - '''Settings''' - '''Crashreport.'''
  
  
 
{| class="prettytable"
 
{| class="prettytable"
! [[Image:settings_crashreport.png]]
+
! [[Image: Testtool-option-crashreport.jpg]]
  
 
|-
 
|-
| # '''''Use Proxy:''''' Check if you need to use a proxy to access the internet.
+
|  
 +
# '''''Use Proxy:''''' Check if you need to use a proxy to access the internet.
 
# '''''Host''''': Address of the proy.For example: ''proxy.somewhere.de''
 
# '''''Host''''': Address of the proy.For example: ''proxy.somewhere.de''
 
# '''''Port''''': Enter the port number of the proxy.For example: ''8080''
 
# '''''Port''''': Enter the port number of the proxy.For example: ''8080''
Line 98: Line 97:
 
:  
 
:  
  
: <nowiki>[</nowiki>Crashreporter]'''UseProxy'''<nowiki>=true</nowiki>'''ProxyServer'''<nowiki>=proxy.somewhere.de</nowiki>'''ProxyPort'''<nowiki>=8080</nowiki>'''AllowContact'''<nowiki>=true</nowiki>'''ReturnAddress'''<nowiki>=you@somewhere.de</nowiki>
+
: <nowiki>[</nowiki>Crashreporter]
 +
:'''UseProxy'''<nowiki>=true</nowiki>
 +
:'''ProxyServer'''<nowiki>=proxy.somewhere.de</nowiki>
 +
:'''ProxyPort'''<nowiki>=8080</nowiki>
 +
:'''AllowContact'''<nowiki>=true</nowiki>
 +
:'''ReturnAddress'''<nowiki>=you@somewhere.de</nowiki>
  
 
= Selecting the OpenOffice.org Installation =
 
= Selecting the OpenOffice.org Installation =
 
There are two choices:
 
There are two choices:
  
Choose '''Extra''' - '''Settings''' – '''Misc.'''
+
Choose '''Extra''' - '''Settings''' - '''Misc.'''
  
  
 
{| class="prettytable"
 
{| class="prettytable"
| <center>[[Image:settings_misc.png]] </center>
+
| <center>[[Image: Testtool-option-misc.jpg]] </center>
  
 
# '''''OOo Program Dir:''' ''Select the OpenOffice.org installation to test.For example: ''/Volumes/OpenOffice.org 2.0.app/Contents/MacOS ''
 
# '''''OOo Program Dir:''' ''Select the OpenOffice.org installation to test.For example: ''/Volumes/OpenOffice.org 2.0.app/Contents/MacOS ''
Line 114: Line 118:
  
 
|}
 
|}
Choose '''Extra - Settings – Generic.'''
 
  
  
{| class="prettytable"
 
| <center>[[Image:tt_oooprogramdir.png]] </center>
 
 
# '''''Area:''' ''Select ''OooProgramDir''.
 
# '''''Path''''': Select the OpenOffice.org installation to test.
 
 
 
 
|}
 
 
If you want, you can also edit these settings in the .testtoolrc / testtool.ini files.
 
If you want, you can also edit these settings in the .testtoolrc / testtool.ini files.
  
 
<nowiki>[</nowiki>OOoProgramDir]
 
<nowiki>[</nowiki>OOoProgramDir]
  
Type=Path
+
'''Type'''=Path
  
Current=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS
+
'''Current'''=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS
  
All=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS
+
'''All'''=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS
  
 
= TestTool Editor =
 
= TestTool Editor =
The TestTool uses syntax highlighting for all BASIC, StarBasic, and internal TestTool commands (For more information, see chapter 5: ''Internal Commands, Methods and Functions for TestTool''). Highlighting can increase the load time for large files.
+
The TestTool uses syntax highlighting for all BASIC, StarBasic, and internal TestTool commands (For more information, see: [[VCLTesttool/API_reference|Internal Commands, Methods and Functions for TestTool]]). Highlighting can increase the load time for large files.
  
 
= Starting a TestTool Script =
 
= Starting a TestTool Script =
Line 147: Line 141:
  
 
|-
 
|-
| [[Image:]]''' Start''' Starts the script, opens a new or old result file, and inserts the result of the test in the file.
+
| ''' F5 Start'''  
 +
:Starts the script, opens a new or old result file, and inserts the result of the test in the file.
  
[[Image:]]''' Single stepping'''Runs the script by single steps.
+
''' F8 Single stepping'''
 +
: Runs the script by single steps.
  
[[Image:]] '''Single step over procedures'''Runs the script one sub routine at a time.
+
''' F10 Single step over procedures'''
 +
: Runs the script one sub routine at a time.
  
[[Image:]]''' Set / Select a break point'''Creates a breakpoint (red circle) in the script on the blue border on the left side.
+
''' F9 Set / Select a break point'''
 +
: Creates a breakpoint (red circle) in the script on the blue border on the left side.
  
[[Image:]][[Image:]]''' Cancel '''Cancels a running script.
+
|''' Shift+F5 Cancel '''
| [[Image:]]'''+[[Image:]] Interrupt'''Interrupts a running script.
+
: Cancels a running script.
 +
'''Ctrl+F5 Interrupt'''
 +
:Interrupts a running script.
  
[[Image:]]+[[Image:]] '''Next Error'''Jumps to the next error after a syntax error in a script.
+
'''Shift+F8 Next Error'''
 +
:Jumps to the next error after a syntax error in a script.
  
[[Image:]]'''+[[Image:]] Previous Error'''Jumps to the previous error in a script.
+
''' Shift+F7 Previous Error'''
 +
:Jumps to the previous error in a script.
  
 
|}
 
|}
 +
 
= Result File =
 
= Result File =
The TestTool automatically saves a result file during a test script is run. The name of the result file is the name of the script *.bas-file that generated the result, but with the *.res extension (For example: ''first.bas ‚Üí'' ''first.res''). To set the path for saving the result file, choose '''Extra - Settings - Profile''' and enter the path in the '''''Log Base directory''''' box.
+
The TestTool automatically saves a result file during a test script is run. The name of the result file is the name of the script *.bas-file that generated the result, but with the *.res extension (For example: ''first.bas -'' ''first.res''). To set the path for saving the result file, choose '''Extra - Settings - Profile''' and enter the path in the '''''Log Base directory''''' box.
  
 
If a result file already exists, the file is opened and the new result is added at the beginning of the file.
 
If a result file already exists, the file is opened and the new result is added at the beginning of the file.
Line 170: Line 173:
 
The results are presented as a hierarchical tree list where you can click plus sign (+) or minus sign (-) to expand or collapse the outputs.
 
The results are presented as a hierarchical tree list where you can click plus sign (+) or minus sign (-) to expand or collapse the outputs.
  
* [[Image:tt_result.png]]The first output is ''Reading the files''. Expand this entry to see the declaration files. If the entry is orange, the declaration files (*.win / *.sid) contain errors. The sum of the errors is displayed at the end of a test as Warnings occurred during initialization.
+
[[Image:tt_result.png]]
 +
*The first output is ''Reading the files''. Expand this entry to see the declaration files. If the entry is orange, the declaration files (*.win / *.sid) contain errors. The sum of the errors is displayed at the end of a test as Warnings occurred during initialization.
 
* The next output is a short entry that is generated automatically by the ''start up routines''. The entry displays information about the TestTool version, the paths where OpenOffice.org is installed, and where the application was started. The office- and the system language is also displayed.
 
* The next output is a short entry that is generated automatically by the ''start up routines''. The entry displays information about the TestTool version, the paths where OpenOffice.org is installed, and where the application was started. The office- and the system language is also displayed.
 
* <nowiki>Each entry that has a [+] in front of it is a testcase. If the testcase entry is a color other than black, the test contains errors.</nowiki>
 
* <nowiki>Each entry that has a [+] in front of it is a testcase. If the testcase entry is a color other than black, the test contains errors.</nowiki>
Line 177: Line 181:
 
* A light blue testcase indicates an Assertion that gives selectable informatoins when testing a debug build of OpenOffice.org.
 
* A light blue testcase indicates an Assertion that gives selectable informatoins when testing a debug build of OpenOffice.org.
 
* A red testcase indicates that one or more errors are present. The tree is expanded to display the error(s). In the example result file, the error occurred when the TestTool did not find the slot. That is because there is a typo in it – it should be ToolsOptions.
 
* A red testcase indicates that one or more errors are present. The tree is expanded to display the error(s). In the example result file, the error occurred when the TestTool did not find the slot. That is because there is a typo in it – it should be ToolsOptions.
* When you expand a red entry by clicking the plus sign (+) in front of the entry, a level by level (sub main ‚Üí testcase ... ‚Üí subroutine ... ‚Üí .....) view of the error is displayed. You can also see what the TestTool did to return OpenOffice.org to a defined base state so that the TestTool can run the next testcase without any open windows or dialogs.For example:  
+
* When you expand a red entry by clicking the plus sign (+) in front of the entry, a level by level (sub main - testcase ... - subroutine ... - .....) view of the error is displayed. You can also see what the TestTool did to return OpenOffice.org to a defined base state so that the TestTool can run the next testcase without any open windows or dialogs.For example:  
* the WORKINGWINDOW "Untitled2 - OpenOffice.org Writer" is closed (geschlossen = closed)
+
** the WORKINGWINDOW "Untitled2 - OpenOffice.org Writer" is closed (geschlossen = closed)
* Explanation:
+
** Explanation:
* the working document is closed
+
** the working document is closed
* The TestTool then started the next testcase at the base state of OpenOffice.org (that is, with one open document and no open dialog).
+
** The TestTool then started the next testcase at the base state of OpenOffice.org (that is, with one open document and no open dialog).
 
* At the end of a test, the number of errors and warnings is displayed
 
* At the end of a test, the number of errors and warnings is displayed
  
Line 190: Line 194:
 
{| class="prettytable"
 
{| class="prettytable"
 
|  
 
|  
| '''<nowiki>testtool.exe [/port=xxx /host=xxx] startprogram /run</nowiki>''' '''<nowiki>[/result xxx]</nowiki>'''
+
| '''<nowiki>testtool.exe [/port=xxx /host=xxx] startprogram /run</nowiki>''' '''<nowiki>[/result xxx]</nowiki>''' '''<nowiki>[/printlog]</nowiki>'''
  
 
|-
 
|-
Line 211: Line 215:
 
| <div align="right">'''/result'''</div>
 
| <div align="right">'''/result'''</div>
 
| Limits the number of testruns to xxx excluding the current one in the resultfile (0,1,2,...). The action gets executed on running the testcase.
 
| Limits the number of testruns to xxx excluding the current one in the resultfile (0,1,2,...). The action gets executed on running the testcase.
 +
|-
 +
|<div align="right">'''/printlog'''</div>
 +
|write the information written to logs on stdout *****  UNIX only ! *****
  
 
|}
 
|}
Line 223: Line 230:
  
 
|}
 
|}
 +
 
= Run a Batch with Testcases =
 
= Run a Batch with Testcases =
  
: There are examples for Win32 and Unix at '''http://qa.openoffice.org/source/browse/qa/qatesttool/script/'''.
+
: There are examples for Win32 and Unix at http://hg.services.openoffice.org/DEV300/file/1467f46f8817/testautomation/tools/run_tests.
 +
 
 
= Testing a Non-Product Version  =
 
= Testing a Non-Product Version  =
 
+
* Define the files where you want to store the debug-settings for OpenOffice.org by inserting the following section into the '''''win.ini''''' file (If you don't have access to the file, you may use environment variables.):
 
+
  <nowiki>[SV]</nowiki>
# Define the files where you want to store the debug-settings for OpenOffice.org by inserting the following section into the '''''win.ini''''' file (If you don't have access to the file, you may use environment variables.):
+
  dbgsv<nowiki>=c:\</nowiki>dbgsv.ini
 
+
  dbgsvlog<nowiki>=c:\</nowiki>dbgsv.log  
<nowiki>[SV]</nowiki>
+
* On Linux/Unix/Win32 Platforms you have to set an environment variable to control where the debug-settings are stored:
 
+
  DBGSV_INIT=/some/path/dbgsv.ini
dbgsv<nowiki>=c:\</nowiki>dbgsv.ini
+
  DBGSV_LOG=/some/path/dbgsv.log
 
+
* In OpenOffice.org, press '''Shift+Ctrl+Alt+D''', and then select ''TestTool'' in the '''Error '''list box.
dbgsvlog<nowiki>=c:\</nowiki>dbgsv.log # On Linux/Unix/Win32 Platforms you have to set an environment variable to control where the debug-settings are stored:<br/> DBGSV_INIT=/some/path/dbgsv.ini DBGSV_LOG=/some/path/dbgsv.log<br/>
+
* In the TestTool, assertions are indicated by light blue. To manually reproduce an assertion, select ''MessageBox'' in the '''Error '''list box''.''
 
+
# In OpenOffice.org, press [[Image:]][[Image:]][[Image:]][[Image:]], and then select “TestTool” in the '''Error '''list box.
+
 
+
In the TestTool, assertions are indicated by light blue. The TestTool cannot handle some assertions (for example, OSL-assertions). Instead, you must manually reproduce the assertion, or use a ''product version'' to test OpenOffice.org. To manually reproduce an assertion, select “MessageBox” in the '''Error '''list box''.''
+
 
+
 
+
 
+
  
 
=The TestTool Environment=
 
=The TestTool Environment=
= Introduction =
 
The structure of the TestTool Environment was changed after the release of source SRC641 (OpenOffice.org 1.0, StarOffice 6.0 / StarSuite 6.0). The following description of paths is only valid for source versions SRX641 and higher.
 
 
The TestTool Environment contains the start scripts, the test scripts, and the libraries (''include files'') that are required to automatically test OpenOffice.org. The TestTool Environment is modular, so that a module exits for each application or area that you want to test in OpenOffice.org. To test a single OpenOffice.org application, such as Writer, you only need the “application module” and the “global” module.
 
 
<center>When you export modules from the cvs, the module names and the path names are the same.</center>
 
  
 +
The TestTool Environment contains the start scripts, the test scripts, and the libraries (''include files'') that are required to automatically test OpenOffice.org. The TestTool Environment is modular, so that a module exits for each application or area that you want to test in OpenOffice.org. To test a single OpenOffice.org application, such as Writer, you only need the ''application module'' and the ''global'' module.
  
 
{| class="prettytable"
 
{| class="prettytable"
Line 258: Line 255:
  
 
|-
 
|-
| colspan="2" | ''application modules''
+
| colspan="2" |  
 +
==application modules==
  
 
|-
 
|-
| '''base'''
+
| '''dbaccess'''
 
| Database / Data source functionality in OpenOffice.org
 
| Database / Data source functionality in OpenOffice.org
  
 
|-
 
|-
| '''calc'''
+
| '''spreadsheet'''
 
| Calc (spreadsheet)
 
| Calc (spreadsheet)
  
 
|-
 
|-
| '''chart'''
+
| '''chart2'''
 
| Chart, the functionality of charts in a spreadsheet
 
| Chart, the functionality of charts in a spreadsheet
  
Line 281: Line 279:
  
 
|-
 
|-
| '''setup'''
+
| '''extensions'''
| Installation
+
| Extensions functionalities
  
 
|-
 
|-
Line 293: Line 291:
  
 
|-
 
|-
| colspan="2" | ''general modules''
+
| colspan="2" |  
 +
==general modules==
  
 
|-
 
|-
Line 302: Line 301:
 
| '''global'''
 
| '''global'''
 
| All general routines (for example, startup, tooling, declaration files)
 
| All general routines (for example, startup, tooling, declaration files)
 
|-
 
| '''errorlog'''
 
| Output directory for all result files
 
  
 
|-
 
|-
 
| '''tools'''
 
| '''tools'''
 
| Useful standalone applications (hid.lst perl script)
 
| Useful standalone applications (hid.lst perl script)
 
|-
 
| '''documentation'''
 
| The files you just read
 
 
|-
 
| '''scripts'''
 
| Batch files to run all available test scripts
 
  
 
|}
 
|}
Line 328: Line 315:
  
 
|-
 
|-
| '''update'''
+
| '''required'''
 
| Resource test: Activates all menu items and opens all dialogs of the tested application.
 
| Resource test: Activates all menu items and opens all dialogs of the tested application.
  
 
|-
 
|-
| '''loadsave'''
+
| '''optional'''
| Loads, saves, or exports tests
+
 
+
|-
+
| '''options'''
+
| Tests all of the settings in the Tools - Options area of the tested application.
+
 
+
|-
+
| '''level1'''
+
 
| Performs a general functionality test for each feature in the tested application, including each menu item.
 
| Performs a general functionality test for each feature in the tested application, including each menu item.
 
|-
 
| '''level2'''
 
| Performs intensive tests for separate areas in the tested application
 
 
|-
 
| ''special modules''
 
|
 
 
|-
 
| '''input'''
 
| Includes general input files that are required by all tests
 
  
 
|-
 
|-
Line 371: Line 338:
 
|-
 
|-
 
| '''<nowiki>*</nowiki>.bas'''
 
| '''<nowiki>*</nowiki>.bas'''
| Indicates executable test scripts. To run the script, open the *.bas file in TestTool, and then press [[Image:]] or choose '''Program - Start'''.
+
| Indicates executable test scripts. To run the script, open the *.bas file in TestTool, and then press F5 or choose '''Program - Start'''.
  
 
|-
 
|-
Line 386: Line 353:
  
 
|}
 
|}
= global-Module =
+
== global-Module ==
 
The ''global-directory'' is required for each test. The directory contains the main routines for running a test script, including the ''hid.lst'', the declaration files that identify menu items, dialogs and controls in OpenOffice.org, and general tooling routines.
 
The ''global-directory'' is required for each test. The directory contains the main routines for running a test script, including the ''hid.lst'', the declaration files that identify menu items, dialogs and controls in OpenOffice.org, and general tooling routines.
  
Line 429: Line 396:
  
 
|}
 
|}
= CVS information =
 
The complete TestTool Environment is checked into a CVS-tree under the module name qa/'''''qatesttool'''''. You can then check out the module or the ''global''-module along with the application or module that you want to test. Since the content (such as global routines and the ''hid.lst'') of the ''global''-module can change, update the ''global''-module regularly.
 
  
The commandline looks like:
+
=Configuration file entries=
cvs -d :pserver:anoncvs@anoncvs.services.openoffice.org:/cvs -z3 co -P qa/qatesttool
+
 
+
 
+
 
+
Win XP501Win Server 2003502
+
{| class="prettytable"
+
| <center>'''Configuration file entries'''</center>
+
 
+
|}
+
 
The configuration file (UNIX: ''.testtoolrc'', Win32: ''testtool.ini'') contains several sections.<nowiki>The section name is surounded by '[', ']' and contains several entries like: EntryName=Value.</nowiki>
 
The configuration file (UNIX: ''.testtoolrc'', Win32: ''testtool.ini'') contains several sections.<nowiki>The section name is surounded by '[', ']' and contains several entries like: EntryName=Value.</nowiki>
  
= GUI Platform =
+
== GUI Platform ==
 
''Current='' depends on the Platform.
 
''Current='' depends on the Platform.
  
Line 543: Line 499:
  
 
|}
 
|}
 +
 +
[[Category:TestAutomation]]

Latest revision as of 12:14, 10 October 2012

This page is archived for historical reasons only. It is no longer maintained and information may not be current.

[[{{{PrevPage}}}|< Previous Page

]]

Next Page >


Tool for Automated Testing of OpenOffice.org

About the TestTool

The TestTool is a program that is used for the automated testing of OpenOffice.org. The TestTool communicates with the TCP/IP-Interface of OpenOffice.org and can test each installation of OpenOffice.org on a PC. The current TestTool can be used on OpenOffice.org 3.0 and higher. However, as there can be some incompatible changes in future OpenOffice.org builds you may need to use a newer version of the TestTool.

The TestTool communicates with OpenOffice.org using SlotIDs, UniqueIDs, and HelpIDs that are associated with each menu item, window, dialog, and window or dialog control in OpenOffice.org. The IDs are automatically generated during the OpenOffice.org build-process or are assigned by developers.


SlotIDs: Each menu item has a SlotID that is used, for example, to open a dialog or perform an action.
HelpIDs: Each control, window, or dialog automatically receives a HelpID for the internal Help-System. The TestTool uses HelpIDs to identify specific controls, windows, or dialogs.
UniqueIDs: A developer can assign a UniqueID to a control that does not have a HelpID so that the TestTool can identify the control.

You can create test scripts for the TestTool in the programing language StarBasic (like VisualBasic). Several commands are available for the TestTool, for example, to get or put information into OpenOffice.org controls. For a complete list of the available commands, see: Internal Commands, Methods and Functions for TestTool.You can simulate most mouse or keyboard actions with the TestTool as well as gather information from controls or change the default settings in OpenOffice.org. In other words, the TestTool lets you simulate an OpenOffice.org user.

Location of the TestTool

The TestTool Environment can be checked out via Mercurial on openoffice.org, and will be found in the directory: testautomation; Or it is available as a seperate package at: http://ooopackages.good-day.net/pub/OpenOffice.org/qa/testautomation/

The TestTool application is available at http://qa.openoffice.org/ooQAReloaded/AutomationTeamsite/ooQA-TeamAutomationBin.html ; On other platforms in the installation set (see setup). Just use the binary from the .../{officepath}/program-directory. Start the program testtool.bin respectively testtool.exe. Just for Mac OS X: Copy the script soffice to testtool and run it.

Installing the TestTool

To install the TestTool, extract the contents of the downloaded TestTool archive to your local disk. If more than one user will use the TestTool, copy the contents of the extracted archive to a network drive.

When you run the TestTool executable file, the testtool.ini and .testtoolrc control files are created on Win32 and UNIX, respectively.

If you place a TestTool control file in the directory that contains the TestTool executable file, the control file is automatically copied to the profile directory on Win32, and the user's home directory on UNIX.

Linux only: If you get a message similar to "Couldn't bootstrap uno servicemanager for reason : Couldn't create registry file ...services.rdb for writing", ensure that you are using 'nfslock'. For example, on Linux run /etc/init.d/nfslock status. If it is not running, switch to root and run '/etc/init.d/nfslock start' and 'chkconfig nfslock on'. This is necessary for the TurboLinux and the SuSe 8.1 Linux distributions.

Setting up the TestTool

Choose Extra - Settings - Profile.


Testtool-option-profile.jpg
  1. Profile: Enter a name for your profile.
  2. Log base directory: Enter the path where you want to save the test results.For example: /opt/qa/qatesttool/errorlog/mymachine or d:\qa\qatesttool\errorlog\mymachine
  3. Base directory: Enter the root path for the TestTool Environment (without a slash at the end!)For example: /opt/qa/qatesttool or d:\qa\qatesttool
  4. Default HID directory: Mark the checkbox.


If you want, you can also edit these settings in the .testtoolrc / testtool.ini files.

CurrentProfile=_profile_Default

(...)

[_profile_Default]

LogBaseDir=/opt/qa/qatesttool/errorlog/mymachine

BaseDir=/opt/qa/qatesttool

HIDDir=/opt/qa/qatesttool/global/hid

Automated Crashreports

Choose Extra - Settings - Crashreport.


Testtool-option-crashreport.jpg
  1. Use Proxy: Check if you need to use a proxy to access the internet.
  2. Host: Address of the proy.For example: proxy.somewhere.de
  3. Port: Enter the port number of the proxy.For example: 8080
  4. Allow Contact: Mark the checkbox if you want to get an Email regarding your crashreport.
  5. Email: Your Email address.For example: you@somewhere.de


If you want, you can also edit these settings in the .testtoolrc / testtool.ini files.
[Crashreporter]
UseProxy=true
ProxyServer=proxy.somewhere.de
ProxyPort=8080
AllowContact=true
ReturnAddress=you@somewhere.de

Selecting the OpenOffice.org Installation

There are two choices:

Choose Extra - Settings - Misc.


Testtool-option-misc.jpg
  1. OOo Program Dir: Select the OpenOffice.org installation to test.For example: /Volumes/OpenOffice.org 2.0.app/Contents/MacOS



If you want, you can also edit these settings in the .testtoolrc / testtool.ini files.

[OOoProgramDir]

Type=Path

Current=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS

All=/Volumes/OpenOffice.org 2.0.app/Contents/MacOS

TestTool Editor

The TestTool uses syntax highlighting for all BASIC, StarBasic, and internal TestTool commands (For more information, see: Internal Commands, Methods and Functions for TestTool). Highlighting can increase the load time for large files.

Starting a TestTool Script

Start local OpenOffice.org installations using the TestTool command line parameter -enableautomation. In the TestTool Environment, executable scripts use the *.bas extension. To run a script, load a *.bas-file, and then run the script from the menu or with a shortcut.


Tt run.jpg
F5 Start
Starts the script, opens a new or old result file, and inserts the result of the test in the file.

F8 Single stepping

Runs the script by single steps.

F10 Single step over procedures

Runs the script one sub routine at a time.

F9 Set / Select a break point

Creates a breakpoint (red circle) in the script on the blue border on the left side.
Shift+F5 Cancel
Cancels a running script.

Ctrl+F5 Interrupt

Interrupts a running script.

Shift+F8 Next Error

Jumps to the next error after a syntax error in a script.

Shift+F7 Previous Error

Jumps to the previous error in a script.

Result File

The TestTool automatically saves a result file during a test script is run. The name of the result file is the name of the script *.bas-file that generated the result, but with the *.res extension (For example: first.bas - first.res). To set the path for saving the result file, choose Extra - Settings - Profile and enter the path in the Log Base directory box.

If a result file already exists, the file is opened and the new result is added at the beginning of the file.

The results are presented as a hierarchical tree list where you can click plus sign (+) or minus sign (-) to expand or collapse the outputs.

Tt result.png

  • The first output is Reading the files. Expand this entry to see the declaration files. If the entry is orange, the declaration files (*.win / *.sid) contain errors. The sum of the errors is displayed at the end of a test as Warnings occurred during initialization.
  • The next output is a short entry that is generated automatically by the start up routines. The entry displays information about the TestTool version, the paths where OpenOffice.org is installed, and where the application was started. The office- and the system language is also displayed.
  • Each entry that has a [+] in front of it is a testcase. If the testcase entry is a color other than black, the test contains errors.
  • An orange testcase indicates that one or more warnings are present. To see where the error occurred, expand the entry and examine the warnlogs, which are the outputs from the developer of the test script. To jump directly to the place in the test script where a warning occurred, double-click the warning.
  • A light green testcase indicates a QAErrorLog entry that was inserted by the developer of the testcase to give the tester some information (for example, about an existing bug or workaround)
  • A light blue testcase indicates an Assertion that gives selectable informatoins when testing a debug build of OpenOffice.org.
  • A red testcase indicates that one or more errors are present. The tree is expanded to display the error(s). In the example result file, the error occurred when the TestTool did not find the slot. That is because there is a typo in it ‚Äì it should be ToolsOptions.
  • When you expand a red entry by clicking the plus sign (+) in front of the entry, a level by level (sub main - testcase ... - subroutine ... - .....) view of the error is displayed. You can also see what the TestTool did to return OpenOffice.org to a defined base state so that the TestTool can run the next testcase without any open windows or dialogs.For example:
    • the WORKINGWINDOW "Untitled2 - OpenOffice.org Writer" is closed (geschlossen = closed)
    • Explanation:
    • the working document is closed
    • The TestTool then started the next testcase at the base state of OpenOffice.org (that is, with one open document and no open dialog).
  • At the end of a test, the number of errors and warnings is displayed

Starting a Test Script From the Command Line

To start a test-script from the command line, use the following syntax:


testtool.exe [/port=xxx /host=xxx] startprogram /run [/result xxx] [/printlog]
/Port
The com port on the machine that the TestTool uses to communicate with OpenOffice.org. If you do not include this parameter, the TestTool looks for relevant information in the .testtoolrc or testtool.ini files.
/Host
The hostname of the machine that contains the OpenOffice.org installation that you want to test. If the installation is on the same machine as the TestTool, enter localhost. If you do not include this parameter, the TestTool looks for relevant information in the .testtoolrc or testtool.ini files.(only localhost is supported)
Startprogram
The *.bas-file that contains the main-routine.
/run
Runs the test script, writes the result to the result file, and exits the TestTool at the end of the test.
/result
Limits the number of testruns to xxx excluding the current one in the resultfile (0,1,2,...). The action gets executed on running the testcase.
/printlog
write the information written to logs on stdout ***** UNIX only ! *****
Example: testtool.exe /port=12481 /host=localhost /opt/qatesttool/writer/update/w_update.bas /run
The TestTool runs the w_update.bas-test on the local machine through port 12481, and exits the TestTool when the test is done.

Run a Batch with Testcases

There are examples for Win32 and Unix at http://hg.services.openoffice.org/DEV300/file/1467f46f8817/testautomation/tools/run_tests.

Testing a Non-Product Version

  • Define the files where you want to store the debug-settings for OpenOffice.org by inserting the following section into the win.ini file (If you don't have access to the file, you may use environment variables.):
 [SV]
 dbgsv=c:\dbgsv.ini
 dbgsvlog=c:\dbgsv.log 
  • On Linux/Unix/Win32 Platforms you have to set an environment variable to control where the debug-settings are stored:
 DBGSV_INIT=/some/path/dbgsv.ini
 DBGSV_LOG=/some/path/dbgsv.log
  • In OpenOffice.org, press Shift+Ctrl+Alt+D, and then select TestTool in the Error list box.
  • In the TestTool, assertions are indicated by light blue. To manually reproduce an assertion, select MessageBox in the Error list box.

The TestTool Environment

The TestTool Environment contains the start scripts, the test scripts, and the libraries (include files) that are required to automatically test OpenOffice.org. The TestTool Environment is modular, so that a module exits for each application or area that you want to test in OpenOffice.org. To test a single OpenOffice.org application, such as Writer, you only need the application module and the global module.

module name / path name Description

application modules

dbaccess Database / Data source functionality in OpenOffice.org
spreadsheet Calc (spreadsheet)
chart2 Chart, the functionality of charts in a spreadsheet
graphics Impress (presentation application) and Draw (drawing application)
math Math (formula)
extensions Extensions functionalities
writer Writer (text document, HTML document, master document)
xml XML file format for all of the OpenOffice.org applications

general modules

framework General functionality for all applications (for example, galleries and extras)
global All general routines (for example, startup, tooling, declaration files)
tools Useful standalone applications (hid.lst perl script)

The following subcategories are defined for the TestTool Environment modules:


module name / path name Description
required Resource test: Activates all menu items and opens all dialogs of the tested application.
optional Performs a general functionality test for each feature in the tested application, including each menu item.
tools Includes libraries that contain the general subroutines that are required by all test scripts in the module or application.

These are only the defined name, you can also find other subcategories in the paths. Mostly the named of the directory mirror the tested area.

If you find a *.bas-file in one of these directories you are in a test-module. Those module should include the following directories/files:


module name / path name Description
*.bas Indicates executable test scripts. To run the script, open the *.bas file in TestTool, and then press F5 or choose Program - Start.
inc Includes the libraries and any associated files that are required for a test (included by use-method in the *.bas-file)
input Inputs the files that are required for the test.
tools Includes libraries that have general subroutines that are required by test scripts (*.bas). The libraries must be in the same directory as the test script.

global-Module

The global-directory is required for each test. The directory contains the main routines for running a test script, including the hid.lst, the declaration files that identify menu items, dialogs and controls in OpenOffice.org, and general tooling routines.


directory name description
hid Contains the hid.lst file.
input Contains common files that are required by the TestTool.
  • translated, office language dependent default filter names for each application (filters-directory)
  • translated, office language dependent OLE-object names (olenames-dir)
  • files for each graphic format that OpenOffice.org can read (graf_inp-dir)


sid Includes all SlotID declarations.
system Contains the routines to start a test script in these inc-files.
tools Contains general tooling routines in these inc-files (for example, declare.bas).
update Contains general resource-test routines in these files (options-dialog and autopilots)
win Contains all HelpID declarations.

Configuration file entries

The configuration file (UNIX: .testtoolrc, Win32: testtool.ini) contains several sections.The section name is surounded by '[', ']' and contains several entries like: EntryName=Value.

GUI Platform

Current= depends on the Platform.


UNIX Win32
Subsystem Current Subsystem Current
Solaris SPARC 01 Win 95 100
SCO UNIX 02 Win 98 395
Linux 03 Win NT 3 351
AIX 04 Win NT 4 400
Solaris x86 05 Win SE 410
Irix 06 Win ME 490
HP 07 Win 2000 500
FreeBSD 08 Win XP 501
Mac OS X 12 Win Server 2003 502
Linux PPC 13
NetBSD 14
Linux 64 bit 15
Linux SPARC 16
eComStation 17
Personal tools
In other languages