Difference between revisions of "QA/test automation guide"

From Apache OpenOffice Wiki
< QA
Jump to: navigation, search
(Setup project)
 
(20 intermediate revisions by 3 users not shown)
Line 11: Line 11:
  
 
== Notice ==
 
== Notice ==
 
+
There are some reasons for test failures, so please [[QA/test_env|prepare your testing machine]] before running test.
TCP/IP will be used during testing. It may lead your firewall to show some warning messages.
+
  
 
== Getting the source code ==
 
== Getting the source code ==
 
Anyone can checkout source code from our Subversion repository. Run the following command.
 
Anyone can checkout source code from our Subversion repository. Run the following command.
svn co https://svn.apache.org/repos/asf/incubator/ooo/trunk/main/test/ test
+
<source lang="bash">
If you are not familiar with Subversion, see our [http://incubator.apache.org/openofficeorg/svn-basics.html Subversion Basics] for more information.
+
svn co https://svn.apache.org/repos/asf/openoffice/trunk/test/ test
 +
</source>
 +
If you are not familiar with Subversion, see our [http://openoffice.apache.org/svn-basics.html Subversion Basics] for more information.
 +
 
 
== Getting started with Eclipse ==
 
== Getting started with Eclipse ==
 
=== Prerequisites ===  
 
=== Prerequisites ===  
Line 33: Line 35:
 
'''Step 3.''' Click "Finish" to import the projects.
 
'''Step 3.''' Click "Finish" to import the projects.
  
=== Run testing ===
+
'''Step 4.''' Click menu "Window->Preferences" ("Eclipse->Preferences" on Mac) and then select "Java->Build Path->Classpath Variables" page.<br />
'''Step 1.''' Set AOO's installation location firstly. Copy the default directory
+
[[File:test_automation_guide_10.png]]<br />
<ref name="Foot1">
+
Click "New" to create one new variable "'''openoffice.home'''", and set its value to Apache OpenOffice installation directory.<br />
{{Template:Documentation/Note| '''The default value of openoffice.home'''<br />
+
[[File:test_automation_guide_11.png]]<br />
  Mac OS X: "''/Applications/OpenOffice.org.app/Contents"''<br />
+
{{Note| '''The default installation directory of Apache OpenOffice'''<br />
  Linux: "''/opt/openoffice.org3''"<br />
+
  Mac OS X: "''/Applications/OpenOffice.app/Contents"''<br />
  Windows: "''C:/Program Files/OpenOffice.org 3''"<br />
+
  Linux: "''/opt/openoffice3''"<br />
  Windows 64b: "''C:/Program Files (x86)/OpenOffice.org 3''"
+
  Windows: "''C:/Program Files/OpenOffice 3''"<br />
 +
  Windows 64b: "''C:/Program Files (x86)/OpenOffice 3''"
 
}}
 
}}
</ref> for convienence. <br />
+
Click "OK" to close "New Variable Entry"dialog. Then click "OK" button of preferences dialog. A message box pops up. Click "Yes".<br />
Otherwise, click menu "Window->Preferences" ("Eclipse->Preferences" on Mac), select "Java->Build Path->Classpath Variables" page and then create one new variable "openoffice.home".<br />
+
[[File:test_automation_guide_12.png]]<br />
[[File:vclauto_guide_10.png]]<br />
+
 
Click "Edit" button and then add the following line in "Default VM Arguments"<br />
+
=== Run testing ===
-Dopenoffice.home="Your OpenOffice installation directory which contains soffice.bin"
+
'''Step 1.''' Select one test class. e.g. testcase.gui.AOOTest, and then click "Run" on the toolbar to start testing. <br />
[[File:vclauto_guide_12.png]]<br />
+
[[File:test_automation_guide_13.png]]<br />
Click "Finish".<br />
+
'''Step 2.''' Select one test class. e.g. testcase.gui.AOOTest, and then click "Run" on the toolbar to start testing. <br />
+
[[File:vclauto_guide_14.png]]<br />
+
 
When testing is finished, JUnit view will show the result.<br />
 
When testing is finished, JUnit view will show the result.<br />
[[File:vclauto_guide_16.png]]<br />
+
[[File:test_automation_guide_14.png]]<br />
 
You also can select one package, click "Run", and then choose "JUnit test" to run all test classes under it.
 
You also can select one package, click "Run", and then choose "JUnit test" to run all test classes under it.
  
Line 63: Line 63:
  
 
=== Run testing  ===
 
=== Run testing  ===
'''How do I run testing on OpenOffice?'''<br />
+
'''As a developer, how to run testing against an OpenOffice built by me?''' <br />  
Run the following commands.
+
It's easy for developers to run testing after building OpenOffice. One thing you need to do is to make sure you run the commands in the build environment. The script will automatically install your build and then start testing. e.g.
 
<source lang="bash">
 
<source lang="bash">
cd test
+
cd main
ant -Dtest.classes="Test class list" -Dopenoffice.home="Your OpenOffice installation directory which contains soffice.bin"
+
source MacOSXX86Env.Set.sh
 +
cd ../test && ant
 
</source>
 
</source>
e.g. Run testcase.gui.AOOTest and testcase.gui.SmokeTest against OpenOffice installed in "/Applications/OpenOffice.org.app"
+
 
 +
'''As a normal user, how to run testing against an installed OpenOffice?'''<br />
 +
Firstly compile the project with the command under test module.
 
<source lang="bash">
 
<source lang="bash">
cd test
+
ant -Dopenoffice.home="OpenOffice installation directory" compile
ant -Dtest.classes="testcase/gui/AOOTest.class,testcase/gui/SmokeTest.class" -Dopenoffice.home="/Applications/OpenOffice.org.app/Contents/MacOS"
+
 
</source>
 
</source>
e.g. Run all test classes under package testcase.gui.sc
+
Then start testing with the command "run".
 +
e.g. Run all test classes under package bvt ("Build Verification Testing")
 
<source lang="bash">
 
<source lang="bash">
cd test
+
run -Dopenoffice.home="/Applications/OpenOffice.app/Contents" -tp bvt
ant -Dtest.classes="testcase/gui/sc/**/*.class" -Dopenoffice.home="/Applications/OpenOffice.org.app/Contents/MacOS"
+
 
</source>
 
</source>
 
+
e.g. Run the given test classes
If test.classes is not specified, [http://wiki.services.openoffice.org/wiki/QA/Build_Verification_Test%28BVT%29 BVT] will be executed. If openoffice.home is not specified,  it will try to test OpenOffice installed in the default directory<ref name="Foot1"></ref>. e.g. Run BVT against OpenOffice  in the default directory.
+
 
<source lang="bash">
 
<source lang="bash">
cd test
+
run -Dopenoffice.home="/Applications/OpenOffice.app/Contents" -tc bvt.gui.BasicFunctionTest
ant
+
 
</source>
 
</source>
 +
On Linux use e.g. -Dopenoffice.home="/opt/openoffice4/program" instead.
  
 
'''As a developer, how do I run testing against my own build?''' <br />
 
It's easy for developers to run testing after building OpenOffice. One thing you need to do is to make sure you run the commands in the build environment. The script will automatically install your build and then start testing. e.g.
 
<source lang="bash">
 
source MacOSXX86Env.Set.sh
 
cd test && ant
 
</source>
 
  
 
''' Where to get the testing result? '''
 
''' Where to get the testing result? '''
  
By default, the testing output is stored in "test/testspace/output". Open "test/testspace/output/report/index.html" in your browser to see the testing report.<br />
+
By default, the testing output is stored in "test/testspace/output***". Open "test/testspace/output/result.html" in your browser to see the testing report.<br />
[[File:vclautoreport.png]]<br />
+
[[File:test_automation_report.png]]<br />
  
There are several directories to store special stuff.
+
Special stuff in the output directory.
  report/: Test result in HTML.  
+
  result.xml: Test result in HTML.  
  result/: Test result in XML.
+
  result.html: Test result in XML.
 
  screenshot/: Screenshot pictures when test assert is failed.
 
  screenshot/: Screenshot pictures when test assert is failed.
logs/: The detail log.
 
 
== Development ==
 
Now, all things have been ready for developing new test code.  If you are interested, please read [[QA/dev_gui_testing|GUI Testing Development]] to get more information.
 
  
== Note ==
+
== Next ==
<references/>
+
Now, all things have been ready for developing new test code. If you are interested, please read [[QA/dev_gui_testing|GUI Testing Development]] and [[QA/dev_api_testing|UNO API Testing Development]] to get more information.

Latest revision as of 13:55, 7 July 2018


Introduction

Two kind of approaches is supported by Apache OpenOffice.

  • GUI Testing: Simulating a common user to perform testing. It depends on a Java library named VCLAuto, which does testing like the old VCLTesttool. It can easily integrates with JUnit. VCLAuto provides APIs to generate user interface events such as keystrokes and mouse clicks, do GUI actions and get information from the GUI to validate functions.
  • UNO API Testing: Use UNO API to verify functions.

Notice

There are some reasons for test failures, so please prepare your testing machine before running test.

Getting the source code

Anyone can checkout source code from our Subversion repository. Run the following command.

svn co https://svn.apache.org/repos/asf/openoffice/trunk/test/ test

If you are not familiar with Subversion, see our Subversion Basics for more information.

Getting started with Eclipse

Prerequisites

Setup project

Step 1. Open Eclipse, click menu "File->Import...", and then select "General->Existing Projects into Workspace".
Test automation guide 1.png
Step 2. Click next, set "Select root directory" to the source code directory, and then check the following projects.

testcommon: The project includes the common test utilities and low-level implementation to do GUI testing
testgui:  The project includes the GUI testing scripts. 
testuno:  The project includes the UNO API testing scripts.

Test automation guide 2.png
Step 3. Click "Finish" to import the projects.

Step 4. Click menu "Window->Preferences" ("Eclipse->Preferences" on Mac) and then select "Java->Build Path->Classpath Variables" page.
Test automation guide 10.png
Click "New" to create one new variable "openoffice.home", and set its value to Apache OpenOffice installation directory.
Test automation guide 11.png

Documentation note.png The default installation directory of Apache OpenOffice
Mac OS X: "/Applications/OpenOffice.app/Contents"
Linux: "/opt/openoffice3"
Windows: "C:/Program Files/OpenOffice 3"
Windows 64b: "C:/Program Files (x86)/OpenOffice 3"

Click "OK" to close "New Variable Entry"dialog. Then click "OK" button of preferences dialog. A message box pops up. Click "Yes".
Test automation guide 12.png

Run testing

Step 1. Select one test class. e.g. testcase.gui.AOOTest, and then click "Run" on the toolbar to start testing.
Test automation guide 13.png
When testing is finished, JUnit view will show the result.
Test automation guide 14.png
You also can select one package, click "Run", and then choose "JUnit test" to run all test classes under it.

Getting started with command line

Prerequisites

Run testing

As a developer, how to run testing against an OpenOffice built by me?
It's easy for developers to run testing after building OpenOffice. One thing you need to do is to make sure you run the commands in the build environment. The script will automatically install your build and then start testing. e.g.

cd main
source MacOSXX86Env.Set.sh
cd ../test && ant

As a normal user, how to run testing against an installed OpenOffice?
Firstly compile the project with the command under test module.

ant -Dopenoffice.home="OpenOffice installation directory" compile

Then start testing with the command "run". e.g. Run all test classes under package bvt ("Build Verification Testing")

run -Dopenoffice.home="/Applications/OpenOffice.app/Contents" -tp bvt

e.g. Run the given test classes

run -Dopenoffice.home="/Applications/OpenOffice.app/Contents" -tc bvt.gui.BasicFunctionTest

On Linux use e.g. -Dopenoffice.home="/opt/openoffice4/program" instead.


Where to get the testing result?

By default, the testing output is stored in "test/testspace/output***". Open "test/testspace/output/result.html" in your browser to see the testing report.
File:Test automation report.png

Special stuff in the output directory.

result.xml: Test result in HTML. 
result.html: Test result in XML.
screenshot/: Screenshot pictures when test assert is failed.

Next

Now, all things have been ready for developing new test code. If you are interested, please read GUI Testing Development and UNO API Testing Development to get more information.

Personal tools