User:Kr/Packaging

OpenOffice.org (OOo) and its derivatives are complex products. Many features, templates, configuration files, registry entries, binaries, localizations etc. need to be delivered and deployed in a reliable and platform compliant way, while giving the user broad choice regarding the particular features he wants to actually install.

OOos growing ecosystem brings the current approach to its limits, we are currently facing a set of problems, which the below proposed solution is going to address. OOo based products need to support different platforms (Operating System / Machine Architecture) as well as different deployment systems (e.g. RPM, Debian Packages, Ports, Solaris Packages, MS Windows Installer), localizations and feature sets, as well as they need to be easy to extend and maintain.

Overview

Some clarification is needed regarding the terms to describe how to deploy, update, maintain and configure a particular software program.

Products

A product consists of a particular set of features and localizations targeted to a particular set of platforms and deployment systems. A product is a set of bits provided by a medium such as a CD, a floppy disk or as a download. A product typically has a brand as well as predecessor and successor products and may require other products to be installed. Independent Products assembled of other (sub-) products may share these (sub-) products, even if these (sub-) products are implementation details only.

Deployment

As OOo is cross platform (Operating System / Machine Architecture), OOo installation sets need to integrate with different deployment systems.

These deployment systems can be categorized as to be

package oriented (such as RPM) and / or
product oriented (such as MS Windows Installer).

Package Support

Package oriented deployment systems typically lack product support, e.g. in a package oriented deployment system, the user typically only sees the deployed packages in the systems configuration and is neither able to directly see which products are represented by which packages, nor which additional features are available. To remove a particular product the user needs to find and to remove all belonging packages (sometimes even the indirectly installed packages) typically by searching the package database.

Package oriented deployment systems enable sharing of packages between products, allowing for re-usage of particular files or functionalities.

Packages can typically be set into relationship to one another. The following are typical relationships,

one package may require on one or many another packages,
one package may conflict with one or many other packages,
one package may replace one or many other packages,
one package may suggest one or many other packages.

Program updates may be deployed at least on a package granularity, while some deployment systems even support patch packages (packages which only contain the differences between a particular package and its successor).

Product Support

In a product oriented deployment system, the user sees the installed products in the systems configuration. Customization is typically supported by product, offering not yet installed features as well as allowing to remove the product as a whole.

Product based deployment systems may allow sharing of entities (e.g. files) on some level (e.g. "component", see below).

Product based deployment systems typically allow to set products into predecessor / successor relationship.

Product based deployment systems mostly only support product updates on a product level, e.g. as new products or as product patches, which may only be applied to one particular product.

Updates

Over time, new versions of products are released, providing bug fixes, additional features, usability improvements etc. During the deployment of product updates the older versions may be de-installed or altered respectively completed. Updates may be provided as dedicated products or as implicit downloads.

Requirements

Compatibility

As a product evolves, its interfaces may change in an incompatible fashion. For binary packages mostly interesting are

ABI (Application Binary Interface) incompatible changes, as well as
structural incompatible changes (removed / renamed files).

Some installation units try to stay compatible, expressing any change of compatibility in their version numbers, while others may change incompatible with every version.

Version numbers expressing compatibility are typically used as follows,

a change in the micro version signals for bug fixes only,
a change in a minor version signals additional features,
a change in the major signals an incompatible (API / ABI etc.) change.

The OOo productizer needs to support compatibility changes, such that different versions and derivatives may be installed without conflict side by side.

Deployment Systems

Deployment systems to be supported by OOo are at least

MS Windows Installer,
Red Hat Package Manager,
Debian Packages
Solaris Packages and
Mac OS X Packages.

Developers

Support for Changes:

updated package(s)
(automatically) remove package(s)
add package(s)
rename package(s)

remove file(s)
add file(s)
move file(s)

Program Management

In a perfect world, program management would be able to create any kind of product, only depending on business needs, not restricted by technical constraints.

Program Management basically mediates between marketing / market requirements and the pool of available technologies.

Program management requires the following product relationships to be supported:

No relationship -> self contained
Update relationship
- update only (e.g. add-ons, localizations)
- self contained (e.g. OOo 2.1 updating OOo 2)
- Multi-update relationship
  - Cross version (major / minor / micro)
  - Cross language
  - Cross brand
  - Cross variant
  - Cross platform

Wishful Thinking

What we need for OOo is partly already available, partly not. Ideally we would be able to represent OOo as well as derivatives by a product view on all our target platforms. This product view would than allow to update, extend and to (partly) de-install it. Getting the needed data from one or multiple media / repositories. This product view would need to be accessible from inside the product as well as from the platforms standard location (e.g. "Add or Remove Program" on MS Windows).

The "extension manager" (tools/Extension Manager) already offers some update / installation capabilities from inside OOo, restricted to extensions.

Some platforms also provide mechanisms for system update and maintenance, certainly the OOo product view respectively the OOo installation needs to integrate with this as well.

Combining these things gets kind of complicated, remembering that parts of OOo may be installed system wide (e.g. "core" components), while others may be installed per user only (e.g. extensions).

So, what actually needed is

product independent update, as supported by packages,
product support, to customize a particular installation,
product independent dependency support,
product versioning,
patch support,
virtual packages
system repository support, to enable product maintenance by standard system maintenance (e.g. apt-get update; apt-get upgrade),
user repository support, to enable product maintenance by user system maintenance,
system installation, to make the product available for multiple users,
user installation, to make the product available for single users,
advertisement.

Non of OOos target deployment systems support all of these features / requirements. Some deployment systems provide APIs, hopefully enabling the implementation of missing desired features. Systems not providing a way to realize a particular feature may be left without it.

Modelling

De-Composition

Feature
Brand
Operating System
(Machine) Architecture / Interpreter
Localization

Composition

Model products by setting them into

inheritance, respectively
instantiation (template)

relationship.

Inheritance

Inheritance models a "is a" relationship. In practice that would mean, that a StarOffice 8 update 7 is an OOo 2.2.1 (respectively its basis) adding something.

Instantiation

Example

Template Product OOo-Standard {
  Features: writer, calc, impress, draw
}


Abstract Product OOo2.4 {
  Name: OOo 2.4
  Code-Base: SRC680m236
  Implements: OOo-Standard

}

Product OOo2.4-ISO : OOo2.4 {
  Name: OpenOffice 2.4
  Format: ISO-750
  Platform: Linux-x86, Windows-x86, Mac OS X x86
}

Product OOo2.4-download-linux-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: Linux-x86
}

Product OOo2.4-download-windows-x86 : OOo2.4 {
  Name: OpenOffice 2.4
  Format: donwload
  Platform: windows-x86
}

Abstract Product SO8u9 : OOo2.4 {
  Name: StarOffice 8 update 9
  Features: so-templates, so-fonts, so-spellchecker, so-brand
  Updates: < StarOffice 8 u 9
}

Product SO8u9-ISO : SO8u9 {
  Name: StarOffice 8 update 9 ISO
  Format: ISO-750
  Platform: Linux-x86, Solaris-x86, Solaris-Sparc, Windows-x86, Mac OS X x86
}

Product SO8u9-donwload {
  Name: StarOffice 8 update 9 ISO
  Inherits: SO8u9
  Format: download
  Platform: Linux-x86
}


Abstract Product {
  Name: StarSuite 8 update 9
  Inherits: OOo 2.4
  Features: so-templates, so-fonts, so-spellchecker, ss-brand
  Updates: < StarSuite 8 u 9
}

Tooling

Comparison of Installation Sets
Check for conflicts
Creation of Installation Sets
Visualization

Product Pipeline

Source =compilation(switches)=> Binaries =packaging(version,...)=> Packages =productization(features)=> Products

.cxx => lib.so,lib.dll => lib.rpm,lib.msm => Our Product.sh, Our Product.msi

Implementations

Example Product creation for by target deployment system.

Capabilities

Name	Products	Packages	Patches	Dependencies	API	virtual Packages	customization	Installation on Demand	Advertisement	User Installation	Administrative Installation	Usage Metrics	Tagging	Repository	System Maintenance
MS Windows Installer	yes	no	yes	no	yes	no	yes	yes	yes	yes	yes	yes	no	no	system only
Red Hat Package Manager	no	yes	partly	yes: "provides", "requires"	?	yes	no	no	no	no (though workaround)	no (though workaround)	no	?	yes: yum, apt	yes: yum, apt
Solaris Packages	no	yes	yes	yes:	?	?	?	?	?	?	?	?	?	?	?
Debian Packages	no	yes	no	yes: "depends", "recommends", "conflicts", "suggests", "replaces", "pre-depends", "breaks"	yes: libapt-pkg	yes	no	no	no	no (though workaround)	no (though workaround)	no	yes: "priority", "section"	yes: apt	yes: apt
Mac OS X bundles	yes	yes	?	?	?	?	?	?	?	?	?	?	?	?	?

Debian Packages / APT

Red Hat Package Manager

Mac OS X

Solaris Packages

MS Windows Installer Service

Experiments under Windows with WiX (Windows Installer XML) show,

that any set of products may share any number of files,
that any one product may update any number of other products (basically leading to a de-installation of these other products), and
that any product sharing files with other products may update any number of these shared files.

This lets me think, that Windows is as dynamic and flexible as the classic package based platforms, while providing a simpler user interface.

Merge Modules: "In order to avoid versioning problems, you should use always merge modules for any component or file that will be shared by multiple applications." [1]