Performance/Startup/Linux

From Apache OpenOffice Wiki
Jump to: navigation, search

Target

  • Collect cold startup values of OpenOffice with linux automatical
  • Show the evolvment of master build and send email to responsible person(s)

Request

  • only linux
  • automatically detect new available master build
  • use a 'slow' system to get high times and small error of measurement
  • use dedicated computer only for this purpose, because it restarts automatical
  • test only masters
  • put measurement values for history in database
  • send email notification of measurement

Current state

  • Cron job looks for new available master
  • perl script controls installation, 10 times reboot and measurement
  • send to database
  • e-mail notification to mailing list
  • graphic of history

This is done for every milestone in DEV300 and every released master. The system was setup and developed by thorsten.bosbach@sun.com

Implementation

Used system: Ubuntu 8.04  512Mb 1.8 GHz Intel Asus PB4 G550

Automated tasks

  • Ask EIS via SOAP interface which master is available, and wait until new one is there.
  • Install OpenOffice.org, start via VCL TestTool to get rid of First Start Wizard and other stuff.
  • Reboot

The following is done 10 times

  • System is configured to bring up X11 and login user
  • Wait 60 seconds which makes sure everything is up and running
  • Measuring starts
  • take time and start: soffice -env:RTL_LOGFILE=$pathlog -norestore
  • After 60 seconds the office is killed with signal 3: Quit
  • Time mark in logfile taken is: { PERFORMANCE - Desktop::loadComponentFromURL()
  • Reboot

Evaluation

After measurement phase is finished, evaluation starts:
Since we are measuring something, we have to think about about the measured value and the error this value might have.
To get an idea, the measurement for every testcase is done ten times;
For these ten values the median is computed:
Which means, the value is by 95% chance somewhere between median-trust/2 and median+trust/2, and if this area overlaps between two different testcase runs, the value is identical.
For these values an image is generated: current.png, which is added to the mail.
The 10 values, median, trust, date, minor, master are transmitted to a database.
To show the relation to other values, the last 10 medians are taken from the database and put into another image: history.png, which is also added to the mail.

Example pictures from mail

Performance of DEV300m41: 2136 ms ± 9 ms
Current.png
History.png

DEV300 Values

Image-error.png

Release values

Version Value
so8 2.0 SRC680 m124 6065 ± 35 ms
so8 2.1 OOE680 m6 5311 ± 27 ms
so8 2.2 OOF680 m14 5086 ± 99 ms
so8 2.3 OOG680 m5 5369 ± 21 ms
so8 2.4 OOH680 m12 5436 ± 33 ms
so8 2.4.1 OOH680 m17 5506 ± 48 ms
so9 3.0 OOO300 m9 2037 ± 30 ms

Source code

The code is currently polished for publication :-)

  • measurement.pl
  • evaluation.pl
  • history.php
Personal tools