uPM – The micro Package Manager

Why uPM? Description of uPM
Downloads Format description
uPM commands Files and Directories
Packaging Software upm.conf Configuration
Bootstrapping a system Install from CD
Configuration File Management ChangeLog
Frequently asked Questions Porting to new architectures
Comparison of different package managers Building Debian Packages
SYSV under uPM uOS – The micro Operating System
Things to do How to contribute

Acknowledgments

The work here would not have been possible without access to the huge body of software, patches and scripts of the Debian Project. Many of the uPM package descriptions were done based on Debian packaging. I have learned quite a bit about package management during my years (since 1996) as a Debian developer through others in the project. In particular useful was the feedback on Linux software I created (debmake and alien as well as numerous others) and the observation on what other developments resulted from the ideas in those pieces of software. I would also like to thank the Gentoo Project which has recently broken new ground with their source based distribution. I had the opportunity to explore that distribution too when I was a Gentoo developer in the summer of 2002. uPM packaging has also relied on many of the scripts and patches done by that project. uPM was basically created because I saw that the architecture of portage had hit a wall in the same way that I felt the Debian packaging mechanism has come to the limit of what they can do. The uPM package manager itself was written from scratch in C and does not contain any software from the Gentoo or Debian project but it incorporates many insights that I was only able to gain because of the work of those two projects. Debian in particular has always been a source of knowlege, good friends and support throughout the years. It would be great if this would become part of Debian someday.

Christoph Lameter, October 17, 2002

Why uPM?

The number of packages in Open Source Software distributions is growing and growing. The Debian project has 10 thousand packages available today and those packages are maintained by 1000 maintainers. It is to be expected that the number of packages will continue to grow. I think it is realistic to expect that the number of packages will increase to 50-100 THOUSAND when we reach the years 2005/2006.

How can such a huge quantity of software be managed effectively? It would take 5-10 thousand(!) maintainers to do the job with the current approach. The inertia coming with such a big organization is already impacting the availability of newer releases today. A new X and a new GCC seem to require changes that affect many areas. Managing 10000 developers will even be a bigger headache and require processes and policies that will establish strict procedures to manage the project effectively. In turn those tight procedures will stifle the ability to react in a flexible way to new situations.

uPM is an attempt to react to that challenge with a new way of packaging. The technology as always will also impact the way things are done.

  1. Extreme reduction of the amount of information maintained on each package. Ideally procedures should be available that allow an automated retrieval from an upstream site with fully automated incorporation into the distribution. One of the greatest problems with maintaining package using dpkg or rpm is the need to keep the installation scripts current. Installation scripts of multiple packages mostly only do a limited number of tasks in a variety of ways. If changes are needed then all those scripts have to be changed. This is an error prone process and releases are very difficult because there is always something broken in the packages. We need to do a radical change on that front. The amount of scripting for each package needs to be reduced by a factor of 10 to 100. Similar functionality in scripts of multiple packages needs to be centralized.
  2. Separation of package integration issues that are distribution specific from development activities which need to be done by the open source project the source code is obtained from. Debian for example is frequently carrying what amounts to forks of upstream packages (X, Kernel etc). Other distributions encounter the same phenomenon. Often patches are not submitted and not integrated to open source projects. The package manager needs to support patch patch writing that is directly submittable to open source projects. On the other hand distribution specific installation does not need to be done with patches. In my opinion it would be best to form real upstream forks and not use the packaging to provide enhance functionality. Having an upstream fork eases access by others to the improvements made and puts pressure on the original upstream authors to integrate changes. It is the best way to avoid having to maintain a fork over a prolonged time period using packaging tools.
  3. Standardization of build procedures in open source projects. There are already established customs on how software is to be build. Automake is one example of an attempt to formalize the build process. The package manager needs to provide a clean interface so that ideally no patching of sources complying to standardized build procedures is necessary for package integration. Patches that have to be done are to be of a generally nature which will warrant their inclusion in standard sources and allow the package maintainer to use these patches unmodified as obtained from the other sites.
  4. Simplification of package management. Package managers should not work in a mysterious ways but be transparent and simple to operate. The meta data that the package generates needs to be easy to handle and change.
  5. Comprehensive Management of all files on a System. Current Linux system already need tens of thousands of files for a well build out system. It is to be expected that future complex Linux systems will contain hundreds of thousands of files. The package manager must provide facilities to monitor file usage, audit problems and be able to insure the integrity of software installed. The system administrator cannot be expected to know the role of each individual file.
  6. Upgrades must be fully automated. Prompting for user input on an upgrade problem is not very helpful these days since the complexity of Linux systems has increased. The user or administrator will not know what to do about a configuration issue. The package manager must provide intelligent schemes for unattended upgrades and refuse to install a new package if an upgrade cannot be performed without user intervention. If a user decision is forced, as reflected in the implementation of configuration file management in package managers today, then the user will make a decision which might be bad since insufficient knowledge about the issue is available. The current situation is that RPM simply moves the configuration file out of the way hoping for future user intervention. Gentoo does the same and provides a tool that asks lots of questions on how to treat these configuration files. Debian prompts when the config file was changed from the default. All these approaches will not work if a system is being run by a non-Unix person. There are a variety of approaches to work around these issues in the package managers (debconf, taking the management of configuration files out of the hands of the package managers etc). The problem though is an deficiency in package management and needs to be solved in the manager itself.
  7. Flexible Build procedures. It should be possible to automatically build packages from source automatically customizing the build procedure to the needs on a special system and the desired overall configuration of software on the system. Debian has the custom of offering a variety of packages based on a common open source upstream package to address that. But if the build process can be customized then the package variety is unnecessary because custom versions can be build as necessary. This will reduce the number of packages needed. Build procedures need to sense the environment and then do an automated efficient build for the machine. Essential build characteristics need to be tracked by the package management system. Gentoo is allowing the specification of build options but does not effectively track that information which results often in a very fragile system.

uPM – Micro Package Manager

uPM is a package manager that attempts to address these issues by offering an integration of a builder, a classic package manager and an advanced management of package relations (like known from apt).

  • Follows the Unix philosophy of keeping things simple. Reduce overhead typically associated with packaging software to a minimum possible. Very easy and fast packaging is possible. I was able to do more than 200 packages in the first month while developing the whole thing.
  • Clear and concise meta information in a single file with a very intuitive format allowing the total control over the build and installation process. Have a look at the over 200 uPM package descriptions already available here.
  • Machine processable package descriptions, The meta data is constructed in such a way as to easily processable by programs that can verify or enhance the package description as needed. The machine processable nature of uPM meta information is essential for the maintenance of a potential huge archive of software package descriptions.
  • Support for multiple stages of package building. uPM recognizes three different stages: First the RECIPE where just the highest level of meta information exists. This basically contains locations of sources, patches, the procedure to build and references to centrally stored procedures on package maintenance. The RECIPE is usually written by a package maintainer and then enhanced and checked by various tools. Then the BUILD which results in an installable package. It adds more meta information and makes the package specific to an architecture (such as Intel, Sparc, Alpha..). A package in build state is comparable to a .deb or .rpm package. Typically uPM stores these packages in a .tgz file with integrated meta information. The meta information can be also be stored somewhere else with a reference to the .tgz file so that the package manager can make an informed decision before choosing to download or unpack that .tgz file for installation. Finally the INSTALLED stage in which the package has been deployed on a system. Meta information has been added again to complete information about the state of the package on the system. In the installed state all files of the package are tracked.
  • Control of files overwriting. uPM tightly controls file overwriting (dpkg, rpm have various approaches to solve this issue. Portage’s(Gentoo package manager) has been designed to allow the overwriting of files…) Blacklists and package options can be used to control file file overwrites if are needed in special situations (See for example the manpages package). Two overwrite strategies can be selected if a package needs to be forced into the system. uPM will track these events and can also detect that an outside source has corrupted a file. Tripwire clone build in.
  • Secure build procedure. uPM uses fakeroot for all build operations. No real root access is needed or given during build operation. Builds essential run as a regular user that cannot impede the operation of the system as a whole. Packages can be build from a recipe by a simple Unix command.
  • Use of Triggers, events, blacklists and other meta tools instead of maintainer scripts that repeat the same code over and over. A package does not need to provide any script for example if a library is added or a font to X11. The fact that a new file is deposited in certain directories causes an action that will update meta information as necessary. The action can be defined by the package providing that service and does not need to be implemented by the packages using that service. A trigger is associated with the directory containing libraries by the package providing library services. If a new library is installed the trigger executes the necessary scripts in order to update the necessary meta files. Instead of having a call to ldconfig in lots of postinst scripts there will be only one trigger. Similar measure can be taken for other situations. This vastly increases the reliability of the system, reduces the complexity and error proneness of installation scripts and takes a lot of hassles off the package maintainer. uPM preserves the ability to script the installation process through events. Events are much more flexible than the classic maintainer/ebuild script since they allow reference to centrally stored information and can react in a custom way depending on the configuration of the system. See an example of a package description for GLIBC.
  • Meta-Packages allow the accumulation of package sets. A single uPM command can install complete subsystems. Meta-packages can also be used for building. They store common bits of the procedures needed to build packages. See for example the c-build or shell-build packages.
  • Inheritance of build instructions from libraries and meta packages Libraries and meta packages can generate instructions for packages that have build dependencies on them. A whole class of recipes is simplified by such an approach. The kdelibs package for example generates special instructions so that other KDE packages just need to have a very small recipe.
  • AdvancedConfiguration File Management. No prompting for changed configuration files anymore. uPM keeps a copy of the original configuration file around and allows the display of a diff to see what changes have been done. Configuration states of a package can be saved and retrieved. uPM can restore a package to its default configuration. On upgrades a diff is generated to the earlier config file. That diff is then applied to the new config fi. If that diff would fail then the upgrade aborts.
  • Flexible Package Attributes so that any desired information about a package (such as source configuration options) can be tracked and used for dependency or build time configuration. Attributes can be used to construct complex configuration and build instructions for the source while allowing for accurate tracking of that information. Attribute construction can be partially centralized and partially package based so that the customizability of the individual package is preserved while reducing the amount of work needed to customize. B Records track the state of all attributes actually used during a build and can be used to track all elements that have influenced the build of a package.
  • Blacklists control the installation of files to various areas of the file system. This is a way that files provided by multiple packages are handled. Files can simply vanish into a black hole or acted upon. The package manager can declare section of the file system tree to be off-limits for packages. Packages can lay claim sections of the file system so that proper cleanup can be done on removal.
  • Clean removes. uPM tracks directories created by packages. A package that is removed vanishes completely and does not leave junk floating around the file system. The same is true for an upgrade of a package.
  • Easy user environment settings control. Settings such a MANPATH, custom shell environment settings and so on can easily be set directly in the package description. V Records are used for that purpose. New hierarchies can be generated with ease.
  • Dependencies activated based on feature configuration. The build system allows to determine the configuration of the source code dynamically. That implies that the dependencies of a package can also change dynamically. uPM fully supports conditional dependencies.
  • Tracking of packages used to build other packages. Build dependency cause an automatic inclusion of the version number of the depended on package, so that information about tools used to build packages is readily available for dependency checks and other queries. A simple command can be given to rebuild all the items that have been generated with some build tool that has a known problem or to insure that all software has been build using the newest tools.
  • Simple realization using C and Shell scripts alone. No complex dependencies on other system components. uPM can be deployed in minimal environments and does not break when other components (like Perl(dpkg) or python(Gentoo)) are in “transitionary stages” or have been compromised in some way.
  • Recovery tools and system audit capabilities All significant activities are logged in /var/log/upm.log. The history of changes to packages is logged separately so that a history of the files of a package can be produced.
  • Extensibility uPM provides hooks so that add-ons can increase functionality which might implement arbitrary package file formats, manage available software and build packages from source.
  • No Base System requirement Most package managers cannot work without extensive functionality around them. uPM allows the building of small installation directories trees. For example an initrd environment of a few hundred k can be easily managed by uPM. A new root tree can be initialized from a single package. uPM can be run external to the root tree that it is supposed to manipulate. The initrd environment does not need to include uPM in it to be manageable. uPM has minimal overhead for meta information stored about packages so that it is feasible to use it in situations with very limited capacities.
  • Scalability While uPM will happily cope with minimal resources it is still able to scale very well. uPM uses hashed algorithms for all search operations which allows very fast operation even with hundreds of thousands of files.
  • Self-determination of Meta data If the build package provided to uPM does not provide dependency and other meta information then uPM will by itself generate dependency information from:
    1. The library dependencies of binaries.
    2. The script interpreter used by scripts.
    3. The directories owned by other packages that files are installed in.
  • Packages will be rejected that have unresolved dependencies. If meta-data is provided in the directory tree then dependencies are checked and validated against self-generated dependency information. The approach here is the same as the one originally taken by alien the Linux package converter I wrote in 1996 and debmake written in the same time frame.
  • Logging of actions uPM logs all significant events so that it is possible to find out what has happened to the system should something go wrong. Daily checkups verify that all files are still in the installed condition and logs the presence of new files. No Tripwire is necessary anymore. uPM stores comprehensive information about each item installed.
  • Priority Links support multiple packages providing similar functionality and allow the election of one of the packages to provide the standard binary.
  • Direct control over /etc/init.d scripts. uPM directly interfaces with the init.d subsystem. Configuration options to manage startup links are specified with uPM configuration commands so that no scripts are needed for initialization.
  • Multiple patch support. uPM can apply any number of patches to a source code archive. It can restore the source code in any arbitrary patch or processing stage with a simple command.
  • Easy source management. uPM can build a source code directory at a requested stage of the build process with a simple command. Another command can generate a usable diff to be integrated into the recipe after changes have been made to the source archive.

uPM Sources available

Note that the implementation is not complete nor is it tested too well. I expect the implementation to be complete by version 1.0. Available downloads for uPM:

Installation CD. ISO image for booting a uOS system. This is a LiveCD. The complete system is available by booting from CD without installation. Auto detects hardware and network. Boots Linux in graphics mode and comes up with a web browser. DHCP used to discover network configuration. MP3 player working. Installation routine available but not tested yet. Two videos included but the video player is not working yet.
The image can also be used to bootstrap a system via a chrooted environment (if you already have a Linux distro installed). Loop mount the ISO image and then copy the files out of the image. BEWARE: The files on the CD are compressed using ZISOFS. You either need to have ZISOFS support compiled into the kernel or use the mkzftree tool (zisofs-tools) to decompress the file.
mount -oloop /path/to/image/sputnik-0.25.iso /path/to/tree

mkzftree -u /path/to/tree /my/chroot

umount /path/to/image/sputnik-0.25.iso

mount -t proc proc /my/chroot/proc

mount -t devfs dev /my/chroot/dev

chroot /my/chroot


now play arond with the upm environment. You should do “upm update” to update the list of available recipes etc before doing any major build actions.
umount /my/chroot/dev

umount /my/chroot/proc




uPM Sources
Sources can now be found on IBiblio.
A list of available package descriptions is at ftp.ibiblio.org/pub/linux/distributions/uos/sputnik/recipe.
Christoph Lameter, August 8, 2002.

Packaging Software
This is a short tutorial on how to start writing a package description. Package descriptions control all aspects of binary tarball generation and the deployment of software. A package description is an ASCII files with lines called Records. A record begins with a capital letter and a blank. It is followed by parameters that are separated by blanks.

Starting a package description
In general it is a bad idea to work as the root. For the following exercise use a regular user id if at all possible. Run your favorite editor and then type a W and a blank on the first line followed by a one line description of what this package is all about. Then type a S and a blank on the next line followed by the URL where the source code for the software you are trying to package can be obtained. This could look for example this way if one would try to package sed:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/sed-3.58.tar.gz


Save this file named packagename-version. So this one should be called sed-3.58. With that you have created a fully functional package description with a W Record describing what the package is and a S Record to specify the source location.
The package description can now be used as a parameter to uPM. For starters lets simply build the package and not immediately install it. That would work using:
upm build sed-3.58


upm should now do something and produce sed-3.58_i486.tgz. Read the documentation on upm to find out what else can be done with that package description.
Making the package description version independent
One problem with the package description is that the version number is hard coded into the package description. If 3.59 has been released then one would need to edit the package description which is bad. It would be great to have package descriptions that are independent of version numbers. We already have the package name and the package version number in the filename for the package. uPM can extract the information from the filename and make that information available as {V} for VERSION and {P} for PACKAGE. The revised version independent package description is now:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{P}-{V}.tar.gz


  • Now the package description is version independent. The package description can be simply renamed to use new version. The maintenance scripts for the uOS distribution check periodically for updates and will rename the package description and attempt to build if new releases become available.
    With this we have used a string expansion. uPM has a quite sophisticated mechanism for managing a database of strings and string snippets that can be used to extensively customize the build process.
    Why did this work?
    It is kind of amazing that one was able to integrate a package by just writing two lines of a package description. The reason that this worked so well is that:

    • Sed uses the standard autoconf/configure/make mechanism for package building and uPM will default to an autoconf build mechanism.
    • We have relied on defaults for dependencies and other important aspects of building.

In the following text the package description will be made more complete. The default build mechanism can be seen (and modified if so desired) in the settings of upm.conf. The default action for configure is:
./configure –prefix=/usr –host={HOST}


The default build action is
make


And the default installation action is:
make prefix={B}/usr install


These settings are exactly fitting for a lot of packages that use the standard autoconf build mechanisms. The above examples include some additional string expansions. {HOST} is the name of the current build environment typically set in upm.conf and {B} is the temporary directory for installation. Note that uPM never allows direct installation into the system files. Building is done in a user environment and a special fakeroot daemon is used to simulate a root environment for permission setting and package generation. The building process is completely secured. Nothing can happen to the system since the build procedure never uses real root access. The build process will fail if an installation routine tries to install files into unauthorized areas. Feel free to experiment with package descriptions. As long as no installation is done no harm can come from it.
Setting the LICENSE attribute
It is useful to track the license that a package has. This is typically set using the LICENSE attribute. Attributes are package characteristics that are defined in the package description and in upm.conf. They can be used to attach arbitrary information to a package. The license for sed is GPL-2 and the attributes are set using an A Record. The package description with the LICENSE attribute:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{PV}.tar.gz

A LICENSE GPL-2


We have also changed the version reference from {P}-{V} to {PV}. PV is frequently used and its defined in /etc/upm.conf as {P}-{V}.

Setting the build type
The package does not have an explicit specification of a build type yet. This means it build without any build information. No tracking of the tools used to build sed is done. The tracking of such tools might be useful. If a bug has been found in the compiler we would like to be able to rebuild all affected packages. Or we might want to make sure that everything is build using the newest tools. Build types are used to set default build environments. R Records control package relationships. Sed is written in C and there is a c-build environment to be found under the meta section of the recipes that sets up such an environment. Sed with setting of the build environment:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{PV}.tar.gz

H http://www.gnu.org/software/sed/{P}.html

A LICENSE GPL-2

R BUILD c-build


The R Record takes three parameters. The first one is the type of relationship. Here is it is a build relationship to the c-build meta package. The c-build meta package will set other build dependencies and runtime dependencies that are used by all C applications. With the c-build dependency also the runtime dependency on GLIBC is established. Incidentally a H Record slipped in. H records are used to contain information about the homepage of packages. The H records are used by documentation tools to show where more information about a package can be found. The URLs of H records are also used to realize keyword searches for packages.

Conditional Dependencies
Reading the documentation we find that sed does support NLS (national language support) and includes lots of files that display messages in Hebrew, Greek, Spanish etc etc. But our system is only used in the United States so we really do not want to carry that baggage around. On the other hand people in other countries might need those languages. For that purpose a global package build directive can be set in /etc/upm.conf. if a NLS attribute with the content of the package providing the NLS support (typically gettext) is defined then sed should be build with NLS support. Otherwise NLS support should be skipped and all those foreign languages should not land on our system.
Typically this is done with a conditional dependency and conditional configure string generation. The R BUILD settings take a string parameter for the package name. If that string parameter is empty then no dependency will be generated. Instead of specifying the package name we can simply specify that the contents of the global NLS attribute should be used as the dependency. If NLS is not set then no dependency will be generated. If NLS is set then a dependency on the package that provides build/runtime support for national language support will be included. The augmented package description:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{PV}.tar.gz

H http://www.gnu.org/software/sed/{P}.html

A LICENSE GPL-2

R BUILD c-build

R BUILD {NLS}

A CONFOPT {NLS?:”–disable-nls”}


Also included is the necessary customization for the parameters to the ./configure script. The default definition for CONFIGURE expands CONFOPT after –prefix=/usr. The default is that CONFOPT is empty. By assigning something to it we can change the configuration of the package to be build. Here is another string expansion. This time it is a ternary operation like known from C. This means that if NLS is not set then set CONFOPT to –disable-nls.
Verification of the software build
Some programs like sed here support the checking of the binary for proper functioning. By default uPM does not generate any action for checkups but an attribute CHECK can be set to the command of the test to generate it. The resulting package description is then:
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{PV}.tar.gz

H http://www.gnu.org/software/sed/{P}.html

A LICENSE GPL-2

R BUILD c-build

R BUILD {NLS}

A CONFOPT {NLS?:”–disable-nls”}

A CHECK make check

A DOC COPYING NEWS README* THANKS TODO AUTHORS BUGS


And we also set the DOC attribute to the files we would like to put in /usr/doc/sed from the source archive. uPM also supports MAN for man pages and INFO for info files. These are only to be used if the package does not install documentation on its own using the make install target.

Moving files around after the install
In some situations we might want to change what install has done. This is possible using event scripts. One such event is the POSTINST. It is happening after the installation (into the temporary tree). A shell command can be given that is running with the current directory set to the root of the temporary tree. We will be using that to allow for the possibility of alternate implementations of sed using the Y records.
W Sed – stream editor

S http://queen.rett.polimi.it/~paolob/seders/ssed/{PV}.tar.gz

H http://www.gnu.org/software/sed/{P}.html

A LICENSE GPL-2

R BUILD c-build

R BUILD {NLS}

A CONFOPT {NLS?:”–disable-nls”}

A CHECK make check

A DOC COPYING NEWS README* THANKS TODO AUTHORS BUGS

E POSTINST mv usr/bin/sed bin/sed-{V}

Y bin/sed sed-{V} 50


Sed is important for us and it should even be there when /usr has not been mounted yet. That is why we move sed to the /bin directory. Note that no leading / are used here. A leading / would cause the script to fail since it would attempt to access the real /. Sed is also renamed to sed-3.58 to make the name unique in case we have multiple Seds on the machine.
The Y Record is used to allow one name for multiple binaries. An Y record specifies the common name and then the special name and a priority. uPM will pick the one with the lowest priority from the available sed. A more powerful sed could be installed and have a lower priority. Then the /bin/sed link would point to the new sed. If that more powerful sed is later removed (naturally it had some strange behavior) then the /bin/sed link will be re-pointed to the old sed. This insures that a sed version is always available.
Conclusion
There are many other lines for other special effects that can be used with uPM. This was just a short introduction covering the development of a package description. Please read the rest of this HTML file for more details. Study the recipes on the uOS site for examples.
uPM Sample Package Description

GLIBC Package Description
Here is an example of a more extensive package description for GLIBC. This is also used to illustrate the various phases of supplementing information in the package description by the uPM system. The first listing is of the RECIPE state. This is the pre-build stage. The Recipe can contain MD5Sums for all sources and patches. Those have not been included.
W The GNU C Library

H http://www.gnu.org/software/libc/libc.html

M christoph@lameter.com

S ftp://sources.redhat.com/pub/glibc/releases/{PV}.tar.bz2

P ftp://sources.redhat.com/pub/glibc/releases/{P}-linuxthreads-{V}.tar.bz2

P {P}-manpages-2.2.5.tar.bz2@man

P {P}-2.2.4-string2.h.diff

P {P}-2.2.2-test-lfs-timeout.patch@io 

P {P}-2.2.5-dns-network-overflow.diff

P {P}-2.2.5-sunrpc-overflow.diff

# glibc mucks around with the library path breaking fakeroot

O REALROOT

R BUILD linux-headers

R BUILD gcc

R BUILD make

R BUILD binutils

R BUILD texinfo

R BUILD gawk

R BUILD sed

R BUILD {NLS}

R DEPEND base

G R BUILD gcc

G R DEPEND glibc V>=2.3

# O2 seems to be good for fragile glibc….

A O 2

# New glibc requires a framepointer for errno handling.

A OMITFRAMEPOINTER

A DESTSPEC install_root={B} {NLS?”localedata/install-locales”}

A CONFOPT –with-gd=no –without-cvs –enable-add-ons=linuxthreads –disable-profile 

#A CHECK make check

E POSTINST install -m644 {S}/nscd/nscd.conf etc

E POSTINST rm -rf etc/ld.so.cache etc/localtime etc/rpc usr/libexec

V PATH LDPATH /lib:/usr/lib:/usr/local/lib

T lib ldconfig #

T usr/lib ldconfig #

T usr/local/lib ldconfig #

A DOC BUGS CONFORMANCE COPYING* ChangeLog* FAQ INSTALL INTERFACE NAMESPACE NEWS NOTES PROJECTS README* 

A MAN man/*.3thr

V ENV LANGUAGE C

V ENV LC_ALL C

V ENV TZ UTC

X usr/share/locale MINE

# Loader must be executable

E PERM chmod a+x lib/ld-*.so


The first line is a W Record. The line sets the Title of the package as displayed using various tools.
The second line is a H Record which is used to give a hint where more information about the package can be found.
Third line is a M mail address where bug reports and patches might be submitted to.
Fourth line is the Source Specification. This is the location of the source code of the package.
The next lines are all Patch Specifications. These are patches to be applied to the source package. Patches can be either tarballs which are untarred into the source code directory or real patch files. The first two patches here are tarballs. The second one has a patch fix. This is the @ and the man name. This tarball should not be untarred into the main source code archive but into a subdirectory thereof.
The patch lines also use Macro-Expansion features of uPM. Any text surrounded by { } are variable substitutions and expansions to be done by uPM. {P} {V} are short hand references to the package name and the version. Use of these macros allows the reuse of the same package description for future releases. {P} and {V} are extracted from the filename. Simply renaming the package description often allows the installation of newer releases. The GLIBC patches are a mess though. Usually this list is much leaner and easier to maintain. {PV} is a default attribute defined in upm.conf.
Note also the @io at the test-lfs-timeout patch. That patch has to be applied to a subdirectory indicated by @.
The R Records are used to specify relationships to other package. First there are two BUILD time dependency. A build time dependency makes sure that this package is installed at build time and it tracks the versions of the dependent package in the GLIBC package. A B record will be generated indicating the version used to build GLIBC.
Then some runtime dependencies follow. When the package is installed a verification is done that these packages are installed. The last depend line is a conditional dependency. It checks if the NLS attribute is defined. If so then the gettext package is also needed at runtime.
Then some strange G R records appear. These generate dependencies for packages having a build dependency on GLIBC. If a package uses GLIBC for building then it also needs GLIBC for a runtime dependency (G R DEPEND). If a package used GLIBC for building then it will also need GCC during build time (G R BUILD).
The A Records are used to override attribute settings in upm.conf. They only make sense when viewed in the context of the attributes define there which control the overall build procedure. Attribute records in package descriptions are used to override elements of the build process. Here the O attribute is set to 2 which forces the optimization during build to -O2. The DESTSPEC sets the way how the install target is specified during “make … install”. There is a CHECK of the installation possible. The CHECK definition sets up the build engine to run a command and verify that there is no error before building the binary package.
The CONFOPT settings are the options set when invoking ./configure. The example here shows a conditional generation of these options depending on global settings in upm.conf about how the software on a machine should be configured.
The Events for POSTINST perform changes in the binary tree after the INSTALL action has been run. Here another file is extracted from the source archive into the binary tree and then some of the directories are removed.
Triggers are set for common library directories. If other packages deploy new libraries the ldconfig script will be run without these packages needing any special scripts.
The MAN and DOC attributes result in appropriate actions to deposit and manage both documentation and man pages. The V records cause the setting of environmental variables or search paths. Here a search path is set for the typical locations of libraries.

BUILD Stage

The package description is equipped with additional information via B records and filling out of missing MD5 values. It looks something like this after the build stage:
W The GNU C Library

H http://www.gnu.org/software/libc/libc.html

M christoph@lameter.com

S ftp://sources.redhat.com/pub/glibc/releases/{PV}.tar.bz2 5c646f67b0499b0c164b3332e9f58cf0

P ftp://sources.redhat.com/pub/glibc/releases/{P}-linuxthreads-{V}.tar.bz2 542db999866a993b015db74073430aa7

P {P}-manpages-2.2.5.tar.bz2@man b712a49b5113fccb4c8b0ada2a30d390

P {P}-2.2.4-string2.h.diff 135f8145885a2f4f9876fe973f33ddf6

P {P}-2.2.2-test-lfs-timeout.patch@io 52cfc7627fc62dfb26d8d163aac361f6

P {P}-2.2.5-dns-network-overflow.diff b12e1a7de85cd82bcb341863643c9ffd

P {P}-2.2.5-sunrpc-overflow.diff d28a4b6eb36a891a259806430a612de5

# glibc mucks around with the library path breaking fakeroot

O REALROOT

R BUILD linux-headers

R BUILD gcc

R BUILD make

R BUILD binutils

R BUILD texinfo

R BUILD gawk

R BUILD sed

R BUILD {NLS}

R DEPEND base

G R BUILD gcc

G R DEPEND glibc V>=2.3

# O2 seems to be good for fragile glibc….

A O 2

# New glibc requires a framepointer for errno handling.

A OMITFRAMEPOINTER

A DESTSPEC install_root={B} {NLS?”localedata/install-locales”}

A CONFOPT –with-gd=no –without-cvs –enable-add-ons=linuxthreads –disable-profile 

# A CHECK make check

E POSTINST install -m644 {S}/nscd/nscd.conf etc

E POSTINST rm -rf etc/ld.so.cache etc/localtime etc/rpc usr/libexec

V PATH LDPATH /lib:/usr/lib:/usr/local/lib

T lib ldconfig #

T usr/lib ldconfig #

T usr/local/lib ldconfig #

A DOC BUGS CONFORMANCE COPYING* ChangeLog* FAQ INSTALL INTERFACE NAMESPACE NEWS NOTES PROJECTS README* 

A MAN man/*.3thr

V ENV LANGUAGE C

V ENV LC_ALL C

V ENV TZ UTC

X usr/share/locale MINE

# Loader must be executable

E PERM chmod a+x lib/ld-*.so

# The following information was added during build/install

B PACKAGE glibc

B VERSION 2.3.1

B BUILD_linux-headers 2.4.19

B BUILD_gcc 3.2-d1

B BUILD_make 3.79.1

B BUILD_binutils 2.13.90.0.6

B BUILD_texinfo 4.2

B BUILD_gawk 3.1.1

B BUILD_sed 3.95

B BUILD_none 

B BUILD_upm 0.77

B PV glibc-2.3.1

B B /var/tmp/upm/build-glibc-2.3.1

B S /var/tmp/upm/glibc-2.3.1/glibc-2.3.1

B CPU k6

B O 2

B ARCH k6

B OMITFRAMEPOINTER

B CFLAGS -mcpu=k6 -O2 -march=k6   

B CONFOPT –with-gd=no –without-cvs –enable-add-ons=linuxthreads –disable-profile 

B HOST k6-pc-linux-gnu

B CONFIGURE ./configure –prefix=/usr –with-gd=no –without-cvs –enable-add-ons=linuxthreads –disable-profile  –host=k6-pc-linux-gnu

B BUILD make

B CHECK

B DESTSPEC install_root=/var/tmp/upm/build-glibc-2.3.1 

B INSTALL make install_root=/var/tmp/upm/build-glibc-2.3.1  install

B MAN man/*.3thr

B MAN_HANDLER man_handler /var/tmp/upm/build-glibc-2.3.1 man/*.3thr

B DOC BUGS CONFORMANCE COPYING* ChangeLog* FAQ INSTALL INTERFACE NAMESPACE NEWS NOTES PROJECTS README* 

B DOC_HANDLER doc_handler /var/tmp/upm/build-glibc-2.3.1 glibc BUGS CONFORMANCE COPYING* ChangeLog* FAQ INSTALL INTERFACE NAMESPACE NEWS NOTES PROJECTS README* 

B INFO_HANDLER

B FINAL_HANDLER final_handler

T lib ldconfig

T usr/lib ldconfig

T usr/local/lib ldconfig


Note the addition of B records to track all the attribute information referenced.
The installation adds more information about files. Just a few excertps:
B DESTSPEC install_root=/var/tmp/upm-binary/glibc-2.2.5

B INSTALL make install_root=/var/tmp/upm-binary/glibc-2.2.5 install

D usr/lib/gconv 755 0 0

D usr/include/sys 755 0 0

D usr/include/bits 755 0 0

 

 

L usr/lib/libnss_compat.so ../../lib/libnss_compat.so.2

L usr/lib/libutil.so ../../lib/libutil.so.1

F etc/rpc 644 0 0 829918f13dcb14fdeab47541ff658cfa 1615

F etc/ld.so.cache 644 0 0 18aa7b9db00e80a90c825bdaa9473e5e 5630

F lib/libc-2.2.5.so 755 0 0 046d279de1888ee35f81bc306b6927cc 1355938

 

 

F usr/info/libc.info-9 644 0 0 e5b2e9de70017dd31c98532e3671f4a8 49111

F usr/info/dir 644 0 0 feae9e11b3fd7bf07613d124b56de336 64950

F usr/libexec/pt_chown 4755 0 0 fe81d227c68162eb57cbf2b129a76065 8620

F usr/upm/in/glibc-2.2.5 644 0 0 ddc3b478ccbdea7184af9e84e1aadc31 1832


  • Well enough for now…

    Release names for uPM distribution

Name Description Date of release
Start Beginning of development August 7, 2002
Sputnik Initial release to establish baseline functionality.

  • Fully functional package manager
  • Ability to bootstrap the system
  • Ability to install from CD media
  • Core functionality set of 300 packages
  • Basic X functionality
  • Hardware autodetection
  • Basic archive functionality
Oct 2002
Explorer Release to get full functionality. Goals are:

  • Build core development team (10 max)
  • A large set of software (>1000 packages)
  • Full support for KDE and Gnome desktops and all major browsers
  • Enhanced Documentation
  • Fully debugged package manager
  • Library of metapackages and build-types supported that standardize packaging
  • Basic packaging philosophy documented
  • Functionality of the Debian based TelemetryBox releases available.
  • Enhanced Hardware support
  • Ability to import .deb , .rpm and .ebuilds
Estimated March 2003
Vanguard Release to be able to compete with other distributions. Goals:

  • Large development team (50 max) Comprehensive coverage with package descriptions (~ 10000 packages)
  • Very good documentation and a well-maintained website
  • Support for multiple arches and established porting procedures
  • Installation via the Network using a floppy disk.
  • Customized installs for Desktop use, Server deployment etc.
Estimated at the end of 2003

  • Package Description File Format

Format overview Overview of all record types
String Expansion Filesystem Entities
Header records Special Features
Special variables

  • Reference of all Record Types Supported
    This is an overview over all record types supported in all stages. See information following the table for more explanations. The list covers all the elements used for the specification of recipies as well as the metainformation structures used by uPM to keep track of packages and their content.
R Stages Syntax Purpose
A RBI A variable [value] Set attributes to parameterize builds
B BI B variable [value] Record values of attributes during builds.
C B C arch location Shows location where binary tarballs of the package are available
D I D directoryname uid gid permissions Information about a directory
E RB E event shell-code Modifications on certain types of events
F I F filename uid gid permisson md5sum length Information about installed file
G R G upm-command Allows the insertion of commands into package descriptions depending on this one
H RBI H urls URLs to find out more information about the package
I I I variable [value] Record important values during installation.
L I L filename link-target Information about an installed symlink
M RBI M email-addresses Email addresses to contact regarding package maintenance
N I N filename uid gid permission devinfo Information about special node installed
O RBI O option Sets special processing options for a package
P R P url/filename[@patchfix]i [md5sum] Apply patch during build process
Q RBI Q action letter init.d-script update-rc.d-opts Set up startup for scripts
R RBI R relationship package [condition] Define a relationship to another package
S RBI S url/filename [md5sum] Where to get the source code from
T RBI T directory action Actions to execute on modifications of files in directories
U RBI U user name uid gid [Name] Specify users necessary for a package
V RBI V type envv [value] Manager environment and paths
W RBI W text Description of the package
X RBI X blacklist-type file/dir Define Blacklist
Y RBI Y symlink file priority Realize prioritylinks
Z I I file package Information about a file that used to belong to a package

  • The Stages mentioned refer to R=Recipe, B=Build, I=Installed. If a letter is provide then the record type can be present in that stage. Filenames are always specified without a leading / since the package can be build in any directory and a relative filenames makes it easier to work with.
    Package description files are used to store meta data about a package. This includes information on how to retrieve/build the package and what files are used by the package. Package description go through various phases until they are ready to be used:

    Recipe
    In this stage the package description is all there is. It is an instruction as to where to get the files and how to build a binary from the source code. Ideally the instructions are platform independent as well as version independent. The version number is derived from the recipe name so that the recipe can be simply renamed for newer releases. If no changes to the build process have been done then the recipe can simply be used for a newer release. Attributes are used to specify various flexible characteristics of the build environment. Recipes usually exist in /var/upm/recipe and its subdirectories. A single files contains all the build instructions. Examples can be found here.
    Build
    In the build stage B records have been added and the variable nature of the attributes has become fixed by commitments to a certain style of build. The binary package is bound to a CPU architecture and the features are fixed now. A binary package might be stored as a .tgz tarball somewhere or just be a temporary stage in a directory generated by uPM. The meta data of BUILD package can be stored separately to allow the package manager to search for fitting binaries. See examples here The metadata is usually stored on system in /var/upm/build. The bi. The bi. The bi. The bi in /var/upm/binary/ if the upm is configured to save binary tarballs.
    Installed
    In the installed stage the package description contains records of files, directories and symlinks installed and exact records on the meta information of all packages so that the current state of the package can be verified. The package descriptions of installed packages are to be found in /usr/upm/installed.


    Package description files are simple lines of text with fields separated by blanks. The first parameter is a letter indicating the type of record. The last parameter might be a string containing blanks. No quoting is necessary.

    String Expansion

    Some records can contain references to variables implemented by String Expansion which might refer to the attribute definition in /etc/upm.conf and the package description.. String expansion is done when content is contained in { }. The following expansions are supported:

    exp ? e1 : e2
    Conditional. If exp is true/defined then e1 is used as a value otherwise e2
    exp : pattern
    Pattern matching. If pattern does not match return “” or false
    e1 & e2
    Boolean logic & and | are supported.
    e2 = e2
    Comparison’s. All typical forms of comparison in C are supported.
    e1 + e2
    String concatenation.
    VARIABLE
    Reference to a variable. Looked up in B records A records and attribute section of upm.conf
    “string”
    String literal
    123.xx.44
    String literal
    ( e )
    Subexpression
    ! e
    NOT expression
    The following fixed variables are available during the build process:

    • B The target directory where the install should place the binaries
    • S The source directory where the build process takes place
    • V The upstream version number of the package (VERSION without -dx)
    • D The delta version of the package or nothing if no delta (-dx part of VERSION).
    • P The name of the package (Shorthand for PACKAGE)
    • UPM_VERSION The version of uPM running

File system Entities Definition records
These lines are used to track the files owned by a package. They only exist during the installed phase of the package. Upon installation these lines are derived from the contents of the source tree.
Directories
D <FN> <PERM> <OWNER> <GROUP>
Files
F <FN> <PERM> <OWNER> <GROUP> <MD5SUM> <SIZE>
Links
L <FN> <TARGET>
Nodes (special devices, pipes and such)
N <FN> <PERM> <OWNER> <GROUP> <DEV>
Entities commandeered by other packages
Z <FN> <Package>
Z records will be generated when another package has replaced files of the package. They are a memory of file that have been there and a record of who the villain was.

Header Records


Description of the Package
W <descriptive text>
This contains the text displayed when an overview of packages is requested.
Options
O <option>
Is used to set up special behavior of the packaging system. Supported are


OVERWRITE
Allows other packages to overwrite files in this package. Files will vanish as other files provide them. This is used for busybox to gradually relinquish its commands to the “real” thing. A package with the OVERWRITE package will not install files that are already provided by other packages.
BASE
A base package owns all directories it includes. No Husk will be deployed during the installation phase.
REALROOT
Package cannot be build unless the building process is run as root. This means that the package cannot be build by a regular user using fakeroot.
INSANE
The package does strange things like putting files into the / directory. Switches off all sanity checks and lets the package overwrite files of other packages at will.
NOBINARY
A binary package is not generated or stored for this package. Setting NOBINARY is useful for packages that have a fast build or for packages with License restrictions. If a package has a license that prohibits redistribution of binaries then this option can make sure that redistribution cannot happen.
HOLD
The package is on hold. uPM will not automatically upgrade this package.

NOAUTODEP
Do not add dependencies generated from binaries and scripts during the build process. This insures that the manual configuration of dependencies is possible.

Source specification
S <url> <md5sum>
Retrieve and unpack source code from the indicated location. Contents retrieved are checked against the (optional) md5sum. All the classic tar compressed formats are supported. If multiple lines of source specifications are given then these are understood to be alternate sources for the same information. The S records will be tried until one is successful. If a simple file is specified then the file will be fetches from the configured source location in upm.conf. By default this is http://ftp.ibiblio.org/pub/linux/distributions/uos/sputnik/sources. If the first character of the URL is a @ then the mirror list in upm.conf will be consulted and all mirror sites will be tried to obtain the source code.
Patch Specifications
P <patch-url> <md5sum>
The patch specifications works in the same way as the Source specification. No multiple sources are supported though. Patches will be applied in the sequence as the P records are encountered in the package description. Patch-Fitting allows hinting to uPM how the patch has to be applied. Without the fitting specification uPM will try to do patch-magic by trying various way of applying the patch until it fits. Patch fitting is done by appending @ to the end of the patch. If a number follows then that is the -p parameter value usually given to patch. If some text follows then that is the directory to CD to before doing a patch -p0. Tarballs can also be specified. They will simply be unpackaged into the main source directory or into the location specified by the @ hint.
Binary specification
C <arch< <binary-url> <md5sum>
C lines specify where binaries with default configuration are available. The arch specifies the basic architecture. Binaries are not as flexible as recipes. The binary will not install if there is a basic incompatibility of attribute settings.
Homepage specification
H <homepage-url>
Relationship specification
R <relationship> <package> <condition>
How does a package relate to other packages. Relationships allowed are DEPEND, CONFLICT, REPLACES, PROVIDE and BUILD. In the case of PROVIDE the condition is the version of the virtual package and not a condition. REPLACE means that the package referred to is completely removed on installation of this package. A BUILD reference causes version information from the depended on package (plus all the G records of that package) to be imported. G records can be used to generate necessary dependencies for packages using libraries or to generate types of build environments.
Event specifications
E <Event> line of shell command
Shell commands are run at specific events. All shell commands undergo string expansion first. It is possible to have these shell commands customized by the attributes of the package or of the system. It is strongly suggested NOT to use these event scripts but rather to rely on the attributes controlling the build engine, triggers, Blacklists and system wide scripts as much as possible. Package descriptions containing *MERGE events might be considered unsafe. All other events can be executed in a fakeroot or user environment during package build. Do not write maintainer scripts that are package specific If you need some functionality then please generalize it and put it into some kind of utility package. The event handlers can then invoke those scripts. Maintainer scripts tend to rot since no one bothers with them. Tools that can be useful for multiple packages will usually be maintained well.
Possible events are (shorthand notations in parentheses):

POSTINST (>I)
Execute after the package has been installed into the temporary binary tree (NOT into the / tree). Executed as the root user in a fakeroot environment. The current directory is the binary directory. This is typically used to clean up the binary tree, move files around the target tree and provide other finishing touches. Note that the base husk (directory and links from the base package) is removed automatically before the POSTINST is run.
PREINST (<I)
Execute before the package files are moved into the temporary binary tree (NOT into the / tree). Executed as the root user in a fake root environment. Current directory is the binary directory. This is typically used to setup directories that the installation stage expects. Note that the husk (directories and symlinks) of the base package is always provided in temporary binary trees. It is only necessary to create directories that are unique to the package.
PERM (<P>)
Run as fake root after all permissions and ownerships have been normalized and after all other scripts have been run on the binary tree. The native permissions are typically overwritten when the final script is run. Allows customization of permissions when needed.
PRECONFIGURE (<C)
Run as build user. Directory is source directory. Before configure.
POSTCONFIGURE (>C)
Run as build user. Directory is source directory. After configure.
PREBUILD (<B)
Run as build user. Directory is source directory. Before build.
POSTBUILD (>B)
Run as build user. Directory is source directory. After build.
PREMERGE (<M)
Executed before the package files are moved into the / tree. {B} is the directory of the binary tree to be merged in. This can be used to simulate Debian preinst scripts. Simply invoke a script from the {B} tree… maybe {B}/var/lib/dpkg/info/mypackage.preinst ?? I would recommend not to use such a script since this script has to run as real root.
POSTMERGE (>M)
Execute after all the package files have been moved into the / tree as real root. Current directory is the / directory.
PRERM (<R)
Run as root before the package is removed.
POSTRM (>R)
Run as root after the package is removed. There are a couple of shorthand notations for frequently used actions. These are:

>file text
Write the text to the file specified
>>file text
Append the text to the file specified
@file sed-command
Execute a sed command on the file. like cp file file.new;sed -e ‘sedcommand’ file.new >file;rm file.new

Attribute specification
A <Attribute> <contents>
Attributes allow to specify various information that might be used by dependency conditions and to control and track the build process. An important attribute is VERSION which is the version of the package installed. VERSION is generated from the version provided at install time. It might contain a delta which is a suffix -d followed by the number of the delta.
Build Attributes
B <Attribute> <contents>
Build records are generated during the building of binaries from source. They track the values attributes had when the binary was build. Build records are also used to track the versions of all packages mentioned in build depends. This enables a forced rebuild of all packages that have been generated with a broken version. If dependency checks are being done and references to attributes of packages are made then the build attributes are the significant values..
Installation Attributes
I <Attribute> <contents>
Installation attributes track what happened during the merging of the package into the real tree. Currently that info includes only version of packages replaced.
Environment Variables
V <type> <variable> <value>
Environment variables allow the control of paths such as MANPATH, PATH and the paths used for ld.so as well as default settings in /etc/profile. The V records are processed in order of priority. The packages with the most depth of dependencies are processed first and the packages depending on them later. Therefore packages depending on others can override settings as needed.
Types of variables:

PATH
Multiple specifications of the variable result in concatenation of the strings specified by :.
ENV
Simply set environment variables. Multiple occurrences cause an override.
SET
Multiple occurrences are not allowed. Fail if multiple settings exist.
The following files are generated from the V settings:

/usr/upm/env/environment
Raw list of the variables and their effective settings
/usr/upm/env/profile.env
File with a suitable format to be sourced by bash and others.
/usr/upm/env/csh.evn
File with a suitable format to be sourced by csh and others.
/usr/upom/env/ld.so.conf
Configuration file for ld.so suitable to generate a list of all directories to be processed by ld.so

Priority Links
Y <FN> <TARGET> <Priority>
Priority Links are used to manage multiple programs providing the same functionality. For example there are multiple ways to provide a vi editor. Packages can define a priority and the system will point the link to the lowest priority link. The user can display priorities and fix these pointers to a favorite implementation.
Blacklist
X <File> ABORT/WARN/IGNORE
Blacklisting is used to prevent packages from installing files. Typically there are files that are directories of information such as /usr/info/dir. Many packages provide these files. uPM enforces that no packages should overwrite other packages files and therefore those installs will fail. Blacklisting the file will make it possible for uPM to handle the situation intelligently. ABORT means that packages providing that packages providing that file will not install. WARN means allow the installation but warn about the situation. IGNORE means to simply silently ignore the garbage.
Blacklisting can also be used for Directories. Some major directories can be taken offline. The base package does that extensively to prohibit accidents.
Trigger
T <Dir> <command<
Execute the script when a package writes new contents to the directory or removes files from the directory. Invocations are
command add list of files


if new files are installed and as
command remove list of files


when files are removed from the directory. A simple example for the use of a trigger is the invocation of ldconfig:
T lib ldconfig #

T usr/lib ldconfig #


  • will insure that ldconfig is run whenever there is a new file in those directories. The # results in the parameters being ignored and not passed to ldconfig.
    Start Stop Service support
    Q <mode> <scriptname> <update-rc.d params>
    Allows the control of services provided by the package. The script name is a referring to a script which must be provided in the /etc/init.d directory of the generated binary tree of the package. Upon initial installation the service is installed. Service is started after installation of the package is complete. Upon upgrade the service is stopped. Script removed and then reinstalled and started when the new files have been installed. The mode specifies what exactly to do and is a combination of characters that enable specific actions of BUE. B (Begin) means run the service upon installation of the package. U means take service down upon upgrade and start again after upgrade. E (End) means take service down on package removal.
    User/group modifications (/etc/passwd,/etc/group)
    U <username> <uid> <gid;< <long name>
    U records allow the adding of new user and group ids as needed by programs. U records will fail if the user-id already exists. If the group id does not exist then an entry in /etc/group is also generated with the same user name.
    Inheritance from BUILD dependencies
    G <line to put into package depending on this one>
    G records allow the setting of additional records in package descriptions that depend on another. This allows the setup of build types and by a simple R BUILD line a package can use generic build instructions. This is very useful if many packages have always the same settings for building.
    Frequently Asked Questions

    • Why was uPM not written in C++?
    • Why is uPM meta data stored in /usr
  • Why was uPM not written in C++?


    Unclear Language Design
    At this point I just put a couple of days into trying to switch uPM from C to C++. I am teaching classes in C++ so I am familiar with the concepts of C++ and I have done stuff before in C++. I was again surprised by the wasting-time-on-what-in-the-world-is-going-on-with-the-compiler factor that crept in and began to consume more and more of my time as I used more and more advanced OO concepts of C++. C++ seems to be always trying to do something in the background that one has to imagine to properly construct a program. Such issues are plainly unacceptable for programming and in particular for large programs. The model in which programming takes places must be comprehensible and self-consistent. That is simply not the case with C++. One might argue that all high-level languages do that. But that is not the case. Programming in Ruby, Python, Perl or PHP is possible without having the constant feeling that someone in the back is trying to interfere with what you are trying to accomplish.
    Object Orientedness
    C++ has some elements attempting to support polymorphism. In the implementation of uPM I had to use that polymorphism for the first time to the fullest extend with multiple inheritance structures. I am used to clean and easy implementations of polymorphism from Ruby. C++ was a rude awakening. Seems that C++ does some tricks with pointers to classes that one needs to be aware of in order to write correct programs. A pointer to a pointer of a derived class that I needed to convert to a pointer to a pointer of base class was impossible to do whereas a simple pointer conversion would work. Surprise… I was able to override C++ by using void** instead but then why not use C?
    Constructors
    have severe limitations in C++. I had to rewrite my APIs several times because of the inability of constructors inherited to a derived class to call a method in that derived class through a virtual function. A constructor therefore cannot call a helper function to analyze say a string passed to it into the individual properties of the object.
    Strings
    C++ requires the use of a mixture of C and C++ in order to interface with the operating system. There is no standard API even for managing file like renaming or deleting files. This requires frequent conversions between the C and C ++ representations of strings. The function provided for that purpose is c_str() which provides a C string “as long as the C++ string was not changed”. A very vague notion and rather risky in view of the evil C++ compiler in the background doing things that you have to mind read.
    “Dynamic” allocation of derived types
    Does not seem to be implemented. An array of types cannot be build and used for allocation. That handicap makes it impossible to manage a set of objects with a set of derived types from a common base types. The generation of that object has to explicitly specify the derived type. This means that a case statement is needed to allocate each derived type individually. This results in the inability to easily expand the program by adding new derived types. Fixes have to be made all over the program if a new type is needed. Generic frameworks are difficult to construct.
    Excessive overloading
    The standard C++ libraries do excessive overloading. As a result the compiler cannot really give too much help if it cannot properly “match” a function. Additional in-depth study of the library implementation and the way overloading is being used is necessary to fix these things. The C++ libraries follow the C++ pattern of doing something sophisticated that one has to fully understand in order to be able to use it. One gets the feeling that the C++ libraries are full of booby traps for the uninitiated. Surprise after surprise. Count on wasting lots of time on C++ language research and the finer points of C++s way of implementing objects.
    Library dependencies
    The use of C++ would have meant additional use of libraries. libstdc++ is mandatory. In order to use C++ strings for file management I could have used commonc++ for example. The STL functions might have required additional libraries too. C allows the writing of a program by just using the standard c library of the system.


    Conclusion
    C++ is a hybrid of object oriented concepts and C that went awry. Developers are wasting lots of their time to acquire language and tool know how instead of producing code. It is probably better to use C (more transparent) for low level tasks and use fully object capable languages such as Ruby for high level components.
    Why is package meta data stored in /usr?”
    The /usr partition contains the majority of all the applications residing on the system. It is natural to store meta data on the same medium than the data that the meta data is about. The meta data in /usr/upm only changes when files are removed and added to the system. That is when also access to /usr must be read-write. If /usr has been mounted readonly then the package meta data is also protected. uPM uses a special directory /usr/upm/froot to stage a newly installed package. Presumably the staging area is on the same volume as the final resting place of the data. This means that uPM can use “mv” to move files into the final locations rather than a costly copy operation. It also insures that space problems become evident during the staging phase and not during the move of files into the final locations.

    Bootstrapping a system
    There is a “bootstrap” script included in the upm tarball. That script generates a directory with a system that is self-building. This means that it only includes components necessary for the building of the core components. The chroot command can be used to enter that build environment to build things. A ready made build-base system is available from the Download Section. It take about a day to build a system from scratch. Note that the build system requires a host Linux system with all the build tools installed (can be any Linux distribution) to bootstrap. You need a recent gcc compiler to build upm. uPM is following C99 which is only supported by gcc 3.X versions. The bootstrap will not work on systems with gcc 2.95.x (Debian 3.0 (woody) comes with gcc 2.95.x by default!). Most of the build tools in that build environment also should be of a more recent vintage.
    If the users build and upm are not defined on the host system then comment them out in /usr/upm/standard.def.

    Step by Step instructions for Bootstrap

    • Download, build and install upm on the host system if it is not a uOS system.
    • Make sure you have current recipes by running “upm update”.
    • Run the bootstrap script to build the key components using the foreign environment.
    • copy /etc/resolv.conf to the etc directory of the chroot environment.

Enter the chroot by using
chroot bootstrapdir

    • Customize the settings in /etc/upm.conf as desired.

Rebuild all components in new environment using
upm -d syncbuild


    • Syncbuild will insure that all components are build with the newest software installed.

Build out the system to be a full uOS system. Use the following command:
upm -d install system


    • This will compile and chug along until a complete system is setup. The system can now be made bootable by configuring grub to load a kernel from /boot/vmlinuz. The chroot environment can be used to debug the new distribution. Software can be added and/or removed as necessary using uPM commands.
  • Porting uOS to a New architecture

    WARNING: No one has tried this yet. This is how it SHOULD THEORETICALLY work.
    Basically the same steps as before have to be followed if you already have another Linux distribution running on the hardware but there is no uOS version for that hardware.

    • Download upm sources and customize the build process for the new arch as needed. Build and install upm in the foreign environment.
    • Customize /etc/upm.conf. In particular change the ARCH and the other compiler settings to fit the new architecture.
    • Run the bootstrap script to generate a chrooted uPM environment.
    • Copy the custom upm.conf to the build environment.
    • copy /etc/resolv.conf to the etc directory of the chroot environment
    • Customize settings in /etc/upm.conf in the build environment as needed.

Rebuild all components using the new architecture by:
upm -d syncbuild

    • Follow the instructions for bootstrap. There might be issues that need to be addressed in the build process or in package selections. Add conditionals to the package descriptions as needed using the {ARCH=’xxx’? … } string expansions. Submit the changes to the package descriptions when you have a working system.
  • Files and directories used by uPM
    UPN uses a variety of files in /etc, /var and /usr.
Directory Purpose
/usr/upm Contains all the information that is only changing when packages are installed and removed from the system
/usr/upm/installed State information about all currently installed packages and files. This is read on uPM startup and used on most operations of uPM.
/usr/upm/in The state of a package before it entered the system. This is a file included in the original tarball. It is used to check that a tarball has been generated for the uPM packaging system.
/usr/upm/env The environment settings (search paths, shell environment, ldconfig paths etc) as configured by the installed packages. Some files from /etc are linked to /usr/upm/env.
/usr/upm/etc All files in /etc in their originally installed state.
/usr/upm/scripts Scripts that are executed by uPM for various purposes.
/usr/upm/standard.def Default file locations, URLS and attributes. Can be overridden in /etc/upm.conf
/usr/upm/froot Temporary staging directory used to unpack tarballs during package installation. This directory might exist after upm was aborted. It will be erased by the next install action. The directory might be safely removed when uPM is not running.
/usr/upm/staging-xxx Temporary staging directories created when uPM realizes after unpacking of a tarball that additional software needs to be installed first. The directories might not be erased if uPM terminated abnormally. The directories can be safely removed if uPM is not running.
/etc/ld.so.conf points to /usr/upm/env/ld.so.conf. The file in env is generated from V records of package descriptions
/etc/profile Typically sources /usr/upm/env/profile.env to get default environment variable settings set by V records.
/etc/upm.conf Master configuration file for uPM
/var/upm Information by uPM that is subject to frequent change without necessarily affecting packages installed
/var/upm/binary Binaries build on the system and saved here. Typically the setting of save-binaries in /etc/upm.conf causes these files to be written
/var/upm/build Package descriptions of binaries available anywhere. Usually the contents are mirrored off a uOS master site. The setting of save-binaries writes new package descriptions for binaries to this directory if enabled.
/var/upm/cache All files retrieved from the INTERNET are staged in this directory. The contents should be erased once in awhile. If a file is in here then the files will not be downloaded from the network.
/var/upm/descr A directory with descriptions for each individual package. This directory is also mirrored off a master site. It is used for keyword searches.
/var/upm/recipe Recipes on how to build all the packages on the system. uPM will get instructions from here on how to build software. Usually the contents are mirrored off a master site.
/var/upm/extra The directory might exist if there is any information that does not fit into the other directories


  • uPM Commands
    The upm command is used to manage software on a uOS system. The basic syntax is:
    upm options command list-of-packages
    Summary of commands:
Command Parameter Purpose
install package-descr / tarball Install a package from a variety of sources
remove package-name Remove a package from the system
list pattern List installed packages
search pattern Search for a term in all packages and descriptions
report type Show suid files, broken links and verify the integrity of all files.
build package-name Generate binary tarball from sourcecode
update Synchronize list of recipes, builds and descriptions
upgrade Upgrade packages where new versions are available
check package-name Check actual state of files against meta data
repair package-name Synchronize meta data to actual files
info package-name Print information about package files
config-save package-name Generate -config.tgz tarball with config files
config-load package-name Load -config.tgz tarball with config files
config-default package-name Reset configuration to state at install time
config-diff package-name Build diff of actual config files against original config files
extract package-name Extract package as a binary package from the installed files on the system
findfile pattern/file Find uPM meta data about file
track directory Check part of the file systems for conformity with meta data. Finds extra files and lost files
directory directory Print directory contents with a list of package ownership for each file
cleanup Clean out temporary files
syncbuild build-dependencies-to-ignore Make sure that all packages are build with installed tools
syncattr attributes-to-ignore Make sure that the way all packages were build reflects the current attribute settings in /etc/upm.conf
deprebuild packagenamnes Rebuild all packages depending on the packages listed
source package [level] Retrieve source and go up to in the build stage
diff package [level] Build diff against the source build up to
hold packages Put a list of packages on hold. Lists all holds if no parameter given.
unhold packages Release a list of packages. Release all holds if no parameter given.
rcdupdate initscript rc.d-spec Reconfigure SYSV Init configuraton for an init script
stop initscript Start service if it is supposed to be active in the current initlevel
start initscript Stop service if it is supposed to be active in the current initlevel
Level What was done
0 Source and Patches retrieved into cache
1 Source unpacked
2 First patch applied
3 Second patch applied
n n-1 patches applied
45 All patches applied
49 PRECONFIGURE events run
50 CONFIGURE action performed
51 POSTCONFIGURE events run
59 PREBUILD events run
60 BUILD action performed
61 POSTBUILD events run
70 CHECK action performed
78 Husk deployed
79 PREINST events run
80 INSTALL action run
81 MAN_HANDLER run
82 DOC_HANDLER run
83 INFO_HANDLER run
84 POSTINST events run
85 Husk removed
86 FINAL run
90 PERM events run
99 Complete build performed


  • Summary of options

Long Option Short Purpose
–version -v Print uPM version
–help -h Print short description of options and commands
–keepreal -k When doing repair work keep the settings of the files and adjust meta data
–deps -d Automatically handle dependencies. uPM will only display what packages would be installed/handled without this option.
–fetchonly -f Only fetch the necessary files for the operation. Do not perform the operation itself.
-a attribute-value Override attribute setting for building
–root dir -r dir Install to a directory rather than /. This disables dependency checking and marks packages in a special way so that they can later be rebuild using upm syncbuild
–verbose -v Increase verbosity of processing. The option can be given multiple times to increase verbosity even more.
–quiet -q Decrease verbosity. The option can be given multiple times to decrease verbosity even more.
–generate -g Generate package description on the fly
–no-depends -D Generate warnings instead of errors on dependency problems
–no-protect -P Overwrite files of other packages installed.
–nice -n Omit files already installed by other packages.
–log -L Log build output for each build instead of displaying it on the console
–fakeroot -F Do not build using fakeroot but use real root (only possible as root)
–keepdirs -A Do not remove temporary build and binary directories
–match -m Do pattern matching instead of looking for exact match during find file or package
–thorough -t Be more thorough during track command. Check perms and md5sums.
–source -s Build and install from source, Ignore upm.conf settings.
–binary -b Install from a binary only. Ignore upm.conf settings.
–config type -c Do not abort in case of an unupgradable config file but do the specified action
type Action
overwrite Simply overwrite the old configuration file with the new one.
keep Keep the old configuration file. Discard the new one.
rename Rename the old configuration file to name.upm.old and install the new one


  • Configuration File Management

    Files in the /etc directory are managed in a special way. In order to allow the restoration of a package to the original condition all original unmodified /etc files are stored in an alternate location (/usr/upm/etc). uPM can display the modifications made to the original (upm config-diff) as well as restore the original configuration of each package (upm config-default).
    If new configuration files are provided during an upgrade then uPM will build a diff from the original old configuration file to the new configuration file and apply that diff to the user modified configuration file. In most cases this will accurately upgrade the configuration file without the need for manual intervention. If this process would fail then uPM will refuse to install the package. The situation can be cleared by first saving the configuration files using upm config-save. Then restoring the configuration files using upm config-default and then performing the upgrade. Then the old configuration may be restored using upm config-load or the user might choose another course of action to integrate the changes.
    The default way of treating config files can be overridden with the -c option.
    Reference of Attributes used by uPM code
    (Note that more attributes are defined in /etc/upm.conf. These are attributes that are directly used by uPM code.)
Attribute Purpose
UPM_VERSION The version of uPM running
PACKAGE The name of the package
VERSION The version of the package
P Shorthand for PACKAGE
D Delta of the version or empty
V Upstream version of the package (VERSION without delta)
S Build directory with Makefile configure script etc
B Temporary target directory
CFLAGS Value of CFLAGS during configure and make. Set up in [env] section of upm.conf
LDFLAGS Value of LDFLAGS during configure and make. Set up in [env] section of upm.conf
CONFIGURE Shell code to execute to configure the package. Not executed if empty. Current directory sourcedir
BUILD Shell code to execute to build the package. Not executed if empty. Current directory sourcedir
CHECK Shell code to execute for testing. Not executed if empty. Current directory sourcedir
INSTALL Shell code to perform the installation into the temporary {B} directory. Must be set. Current directory is sourcedir
MAN_HANDLER Shell code to check for man pages. Current directory targetdir.
INFO_HANDLER Shell code to check and install info pages. Current directory targetdir.
DOC_HANDLER Shell code to check and install documentation pages. Current directory targetdir.
FINAL_HANDLER Shell code to perform final adjustments in the archive. Current directory targetdir.
ARCH Architecture. Used for architecture suffixes.
BUILDDIR The build directory relative to the source directory. If not set uPM will look for typical files indicating buildable software in the main directory of the source archive and in any directory below it. If BUILDDIR is set then this search will not be done but the setting will be accepted as the build directory. This build directory will be accessible via {S}. Note that this is also the directory relative to which patches are applied. Using BUILDDIR is a mixed blessing because it can complicate access to other elements of the sourcecode archive that are outside of the build directory. In a complicated situation with multiple build directories it is better to avoid using BUILDDIR. Instead use cd shell commands on events and actions.


uPM Configuration files

uPM standard definitions in /usr/upm/standard.def

# Standard definitions for UPM

#

# Christoph Lameter, , Oct 24, 2002

#

# Please do not modify this file. Add settings you would like to override

# to  /etc/upm.conf . If you feel that the values set here

# need fixing then please email me.

#

 

# Mirror to use to retrieve new package information

mirror=http://ftp.ibiblio.org/pub/linux/distributions/uos/mirror

 

# RSYNC server to synchronize availablity lists

sync=ibiblio.org::linux_distros/uos/sputnik

 

# Default patch and source location (used if mirror fails)

patch_loc=http://ftp.ibiblio.org/pub/linux/distributions/uos/sputnik/patches

source_loc=http://ftp.ibiblio.org/pub/linux/distributions/uos/sputnik/sources

 

# For some packages sources and binaries are available.

# For importang big packages binaries are usually available.

# Binaries are also locally generated and cached.

# The following setting indicates to uPM how binaries should be used

#

# binaries=always|never|ifavail|iflocal

 

# Only install binary packages if they were locally generated

binaries=iflocal

 

# Always use binary packages. Never build from source

#binaries=always

 

# Never use binaries. Always build from source

#binaries=never

 

# Use binaries if they are available

#binaries=ifavail

 

# The binaries generated during the build phase can be automatically

# saved if a path is specified here.

save_binaries=/var/upm/binary

 

# UserIDs for access to metadata and building

# uPM will build as root if these are commented out.

upmuser=upm

builduser=build

 

[mirror]

# List of mirrors to be used using S @/path P @/path

# Resorting this so that the nearest mirror comes first improves fetch performance.

uos=http://ibiblio.org/uos https://u-os.org

debian=ftp://ftp.debian.org/debian

kernel=ftp://mirrors.kernel.org/pub/linux

sourceforge=http://unc.dl.sourceforge.net/sourceforge http://belnet.dl.sourceforge.net/sourceforge ftp://telia.dl.sourceforge.net/pub/sourceforge

gentoo=http://www.ibiblio.org/pub/linux/distributions/gentoo/distfiles

gnome=ftp://ftp.gnome.org/pub/gnome ftp://archive.progeny.com/GNOME ftp://ftp.sunet.se/pub/X11/GNOME ftp://ftp.no.gnome.org/pub/GNOME ftp://ftp.gnome.org/pub/gnome/2.0.0

kde=http://ibiblio.org/pub/packages/desktops/kde/ ftp://ftp.kde.org/pub/kde/ ftp://download.us.kde.org/pub/kde/ ftp://ftp.gtlib.cc.gatech.edu/pub/kde ftp://download.uk.kde.org/pub/kde/ ftp://download.au.kde.org/pub/kde/ ftp://download.at.kde.org/pub/kde/ ftp://download.tw.kde.org/pub/kde/

gnu=http://www.ibiblio.org/pub/gnu ftp://mirrors.kernel.org/gnu ftp://aeneas.mit.edu/pub/gnu

 

[env]

#

# Environment settings for the build process.

# These are expanded using values from the attribute

# sections of the package and those defined here.

 

# Note that these are NOT attributes to be tracked

# or to be used in regular attribute expansion

# If you want to do that then define such attribute

# in the attributes section

CFLAGS={CFLAGS}

CXXFLAGS={CFLAGS}

CC=gcc

 

[attributes]

# These are lists of default attributes to be used for

# building programs. 

# Attributes defined here can be overridden by attribute

# specifications in the package or by package specific

# overrides later or by attributes specified on the command

# line of upm.

 

# The following attributes are always predefined by upm

# P The packagename (same as PACKAGE)

# V The upstream version number

# VERSION the full version number including delta

# D The delta

# S Source directory (Only available during source building and install )

# B Binary directory (Only available during installation phases )

# UPM_VERSION Version of UPM processing the current recipe

 

# Specification what to do in each of the build stages

# The shell expansions here are deferred until the time

# just before the corresponding build stage is run

 

# These are preset for autoconf sources.

 

# CONFIGURATION STAGE

# The configuration stage. If this package does not

# support autoconf override this setting using

# A CONFIGURE

# CONFOPT can be set with additional options

CONFIGURE=./configure –prefix=/usr {CONFOPT} –host={HOST}

 

# BUILD STAGE

# This works with most packages

# Override BUILD if additional targets are needed

BUILD=make

 

# VERIFICATION STAGE

# The check stage is typically unused. Define this to override and actually

# perform a regression test or so.

CHECK=

 

# INSTALLATION STAGE

# Installation. Typically the specification of the destination

# varies. Define DESTSPEC attribute to fill that one out if the

# prefix default will not work.

INSTALL=make {DESTSPEC} install

 

# POST Installation management

# Installation of manpages via the A MAN attribute

MAN_HANDLER={MAN?”man_handler {B} {MAN}”}

# Installation of documentation via the DOC attribute

DOC_HANDLER={DOC?”doc_handler {B} {P} {DOC}”}

# Installation of documentation via the INFO attribute

INFO_HANDLER={INFO?”info_handler {B} {INFO}”}

# Final handler run unconditionally

FINAL_HANDLER=final_handler

 

# Override this to specify your way of setting the binary

# destination to install into in “make install”

DESTSPEC=prefix={B}/usr

 

# The default format for the CFLAGS passed to programs.

# Other archs than Intel need to have this setup differently.

CFLAGS=-mcpu={CPU} -O{O} -march={ARCH} {MMX?”-mmmx”} {NOEXCEPTIONS?”-fno-exceptions”} {OMITFRAMEPOINTER?”-fomit-frame-pointer”}

HOST={ARCH}-pc-linux-gnu

 

# Frequently used shorthands

PV={P}-{V}

# Someone screwed up regexp processing in glib 2.3 \w does not match numbers anymore… sigh.

Vm={V:”[0-9A-Za-z]+”}

Vmm={V:”[0-9A-Za-z]+\.[0-9A-Za-z]+”}

 

# Can the platform support X86 assembly? Might have to be extended for future compatible archs.

# Typically this is used to generate specially optimized code. If this is not set C code is substituted.

X86={(ARCH:”i.86″)|(ARCH:”k6.*”)|(ARCH:”pentium.”)|ARCH=”athlon”|ARCH=”duron”}

 

# CPAN is organized by sections. They are separated by a – in the packagename.

PERLSECT={(P:”.+-.+”)?(P:”(.+)-.+”):P}

 

# Common references to upstream sites

GET_GNOME=@gnome/sources/{P}/{Vmm}/{PV}.tar.bz2

GET_KDE=@kde/stable/{V}/src/{PV}.tar.bz2

GET_SF=@sourceforge/{P}/{PV}.tar.gz

GET_DEBIAN=@debian/pool/main/{P:”(.)”}/{P}/{P}_{V}.orig.tar.gz

GET_GNU=@gnu/{P}/{PV}.tar.gz

GET_DDIFF=@debian/pool/main/{P:”(.)”}/{P}/{P}_{V}-{DD}.diff.gz

GET_CPAN=http://cpan.org/modules/by-module/{PERLSECT}/{PV}.tar.gz

README_CPAN=http://cpan.org/modules/by-module/{PERLSECT}/{PV}.readme



Standard /etc/upm.conf file
This is the standard upm.conf for refrence:
# UPM Configuration File

# /etc/upm.conf

#

# Christoph Lameter, , August 21, 2002

#

 

[attributes]

 

##########################################

# Set up of global build options

##########################################

 

# Optimization. O=2 is typically safe.

O=2

# For best performance set the following to the CPU type you actually have.

# The CPU setting optimizes the scheduling of the instructions for the specific CPU

# The ARCH setting makes the compiler use a specific instruction set.

CPU=i686

ARCH=i486

#MMX=Y

#NOEXCEPTIONS=Y

#OMITFRAMEPOINTER=Y

 

########################################

# Set up of global switches for packages

# Empty means off, “Y” means on 

########################################

 

# Languages to support. If a language is disabled

# and a package requires it then it will be impossible

# to install the package. Notably C++ is used for 

# a large selection of packages and should

# not be disabled unless you want 

# a very minimal system.

 

CXX=Y

#JAVA=Y

#FORTRAN=Y

#OBJC=Y

 

# Debugging support

#DEBUG=y

FBCON=y

#DOCUMENTATION=y

#MTA_TLS=y

#IPV6=y

 

################################################

# Global Configuration options for packages

################################################

 

# Interpreted languages. Uncommenting will cause all packages that can

# support that language be build with that language support

#PERL=perl

#PYTHON=python

RUBY=ruby

 

# National Language Support. This will include all other languages.

#NLS=gettext

 

# Databases

BERKDB=db

GDBM=gdbm

# MYSQL=mysql

# POSTGRES=postgresql

 

READLINE=readline

GPM=gpm

GTK1=gtk+1.2

GTK=gtk+

X=xfree

SSL=openssl

PAM=pam

JPEG=jpeg

PNG=libpng

TIFF=tiff

OGGVORBIS=libvorbis

 

# ACL=

# ALSA=

# ARTS=arts

# CRYPT=

# CUPS=cups

# CURL=curl

# DIRECTFB=directfb

# ENCODE=lame

# ESD=esound

# FIREBIRD=

# FLASH=

# FREETDS=

# GD=gd

# GGI=libggi

# GNOME=gnome-libs

# GPHOTO=

# GTK=gtk+

# GTK1=gtk+1.2

# LCMS=lcms

# LDAP=openldap

# LIBWWW=libwww

# MIKMOD=mikmod

# MOTIF=openMotif

# MTA_LDAP=openldap

# MTA_MYSQL=mysql

# MTA_POSTGRESQL=postgresql

# NAS=nas

# ODBC=odbc

# OPENGL=opengl

# PDA=

# PDFLIB=pdflib

# QT=qt

# SAMBA=samba

# SCANNER=

# SDL=SDL

# SLANG=slang

# SNMP=snmp

# SOCKS=socks

# SVGA=svga

# TCLTK=tcltk

# TCPD=tcp-wrappers

# TETEX=tetex

# XML=libxml2

# XML2=libxml2

 

###############################

# Package specific overrides

###############################

[rocksndiamonds]

# This package is to be compiled with maximum optimization

O=6




  • Comparison of different package managers

Issue uPM RPM DPKG(Debian) Portage(Gentoo)
Meta data All machine processable spec file with partially machine processable parts. control file machine processable. Extensive use of scripts otherwise ebuilds are scripts and recently Gentoo has attempted to extract information from scripts which is rather difficult. Attempts to structure the scripts are visible
Overwriting files of other packages Off by default. Controllable via Blacklists, Package options and command line parameters Do not know Overwrites files by default for stable releases. Off in the unstable release (!). Controllable via command line option. Design allows overwriting files (source: Daniel Robbins)
Build procedure Building and installation as a regular user (fakeroot environment) Typically as root (build can be done as user, install for rpm generation requires root) fakeroot builds are default Uses sandbox which filters file accesses(!) while build is being done as root (insecure)
Integration Triggers, Blacklists and Meta data. Scripts as a fall back Simple script scheme Sophisticated scripting scheme that has proven itself over years Simple script scheme
Binary packages Yes. Full support Yes Yes. Advanced package management Limited support. Impossible to generate a package without installing it.
Customizable build procedure Yes Source config fixed Source config fixed by maintainer Flexible build procedure
Tracking of build characteristics Yes N/A N/A Some support through “USE” flags. Fragile because changing of build characteristics cannot be tracked. Allows rebuild of all system components to fix that.
System wide package configuration Attributes allow detailed configuration No No USE flags allow simple on/off configuration
Language base C+Shell C+Shell C+Perl+Shell Python based +Shell+Some C.
Rebuild packages affected by global config change Yes No No No
Rebuild packages to reflect new build tools Yes No No No
Rebuild all buildable packages skipping broken builds Yes No No No
User can build package Yes Yes Yes No
Configuration file Management Attempts to patch config file. Can show diff to default config. Can restore to default config. Strategies for resolution selectable on the command line. Renames config file (can be configued in spec file to replace or not replace existing config file) Updates if config file was not changed. Prompts for conflicts. Renames config file. Newer version check CVS id for changes (J) and update unchanged config files
Package removal Removes everything associated with package Macros allow the removal of everything Typically done by script PostRM script can remove leftover files (usually not done).
Package upgrades Removes all traces of old files Removes all traces of old files Removes all traces of old files Leaves rests of old files not replaced by the upgrade. Recent version remove leftovers(J). Experienced some breakage due to libraries vanishing. That might be fixed now.
Alternatives (multiple files with the same name) Yes Yes in Mandrake Yes No
SYSV init scripts Integrated / Script generator Maintainer scripts Maintainer scripts Uses its own scheme of dependencies between init.d scripts. Not compatible with SYSV init scripts.
Generalization of build procedures BUILD/library dependencies rewrite RECIPE N/A N/A Eclass (shell scripts overriding shell functions in ebuilds).
Multiple patches Yes Source archive one patch (diff) Yes
Guestimate of lines of text to write/generate to get a source package integrated 10 (current average is 16 lines/package) ~200 ~500 ~100
Analysis of library dependencies to insure accuracy of runtime dependencies Yes In Mandrake Yes No
Complexity Simple single C binary plus a few shell scripts Simple single C binary + scripts Difficult to comprehend suite of C, Shell and perl scripts that has been added to for almost 10 years Difficult to comprehend mixture of Python scripts, Shell scripts and some C. Lots of scripts that do not add significant functionality (doman,doins,doexe,insinto and so on).
Number of available packages 350 Most widely used distribution. But various flavors with incompatible .rpm packages 9000 2500 (Maik Schreiber)
Homepage http://ibiblio.org/uos http://www.redhat.com http://www.debian.org http://gentoo.org


Please email me should there be any inaccuracies in the comparison above or if you could contribute more helpful information.
Building Debian Packages with uPM

uPM can use debian sources in a variety of ways to build packages. The uPM framework can be used to simulate a Debian packaging environment using debian/rules. In order to have that environment available a package needs to have a build dependency on deb-build-compat. Note that the use of debian/rules is often unnecessary. Debian packages often build fine with the standard R BUILD c-build environment. debian/rules is only necessary if things in addition to the standard upstream install action are needed.
Retrieving Debian Sources
The following lines will retrieve debian sourcecode and the debian diff if the source archive is named the same way as a Debian package and if the version number is the same. The Debian delta must be set using the DD attribute::
A DD <debian-delta>

S {GET_DEBIAN}

P {GET_DDIFF}


  • For details on how these attributes work have a look at /etc/upm.conf.
    Dependencies
    A way to process the Debian metadata in the debian/control file is not yet available. Until we have that working it is necessary to declare at least BUILD time dependencies using R BUILD lines.
    Building the software
    If the package contains only a single binary .deb target then a R BUILD deb-single-compat will set up a complete build environment and install the package as necessary using debian/rules.
    Packages with multiple .deb targets need individual treatment. The cron package is an example of that. Basically a R BUILD deb-build-compat is established. The package provides a set of scripts that simulate typical scripts invoked from debian/rules. uPM will then be instructed to invoke debian/rules to generate the target tree.

    Limitations

    • The script emulation is not complete. Some scripts might not have been provided and debian/rules might fail.
    • Maintainer scripts are not invoked by default. Typically the trigger mechanisms of upm take care of all necessary actions. If you need to execute the scripts put them into /usr/share/packagename/postinst etc and then execute them from the corresponding E POSTMERGE etc events.
    • The software infrastructure that uPM provides is not complete yet. It would be great if you could help to bring more debian infrastructure into the distribution.
  • TODO LIST

    • Find and fix bugs in uPM, the bootup and the installation process.
    • Fix bugs and enhance recipes that exist.
    • Contribute more Recipes for more software. See the available recipes for examples on how to do this.
    • Generate a logo and some design to produce nice looking web content.
    • Create a web portal for uPM packaging with a mailing list for each recipe that exists.
    • Create a tool that allows online editing of recipes.
    • We need a volunteer to do the administration of the web and the mailing lists.
    • We are looking for people responsible for subsystems (like networking, X etc)
    • If you have any other gifts and talents that you might want to contribute ask us.
    • Port to other architectures. We only have intel hardware right now. uPM has never been ported to another architecture so surprises are to be expected when that will be attempted.
    • Enhance the ability to directly build source code from Debian, RedHat and others.
    • Write tools that process recipes and allow global changes to metadata.
  • Contact Christoph if you would like to help.
    Contributing
    I surely wish that uPM would be part of a larger distribution but I have to realize after an experience with another distribution that this might not be realistic. Maybe this could one day be a real Linux distribution. It might have to prove itself before becoming part of something larger.
    If you want to help out you can find me on IRC (OPN err freenode network. See http://freenode.net). Talk to o-o. There is a channel for matters related to uPM and uOS called #uOS.
    The best contribution right now is to just simply download the image and find out all that is wrong with it. Give us feedback (use the talk at u-os.org. See http://u-OS.org) and if you can fix things then send patches. Look at the Todo List for things that we know need help with.

    SYSV Init scripts with uPM

    The way uPM handles init scripts is significantly different from other what distributions do. Scripts are typically used that run on install to generate links in /etc/rcX.d. uPM generates the links instead a result of Q Records in recipe during build time. This means that no script needs to be run at installation time, nor when the package is to be removed. RC links are treated like any other link in a binary tarball. Symlinks are managed as configuraton files since they are in the /etc directory.

    Automatic start and stop of services
    uPM starts and terminates services on install/removal/upgrade if the Q record specifies that this should be done. uPM also verifies that the service is supposed to be active in the currently active runlevel during the installation. If symlinks for the runlevel do not exist or if the runlevel cannot be determined (chroot environment) then no startup or termination will be performed.

    Customizing SYSV configuration for a init-script
    Since uPM handles the /etc/rcX.d directory it is not advisable for the system administrator to change symlinks after install or run rc.d-update to change symlinks. uPM will change them back on the next upgrade. uPM has a special command upm rcdupdate that can be used to change the SYSV symlinks for an init.d script.
    Definition of Runlevels
    uPMs definition of runlevels is different. It is recognized that todays machines are typically connected to the Internet. One major characteristic that needs to be controlled is the connectivity to the Internet. It must be possible to switch machines between different service levels and react to threads or vulnerabilities. With that in mind uOS defines the following runlevels:

    • 0 HALT
    • 1 Single User. Network might be used as a client only.
    • 2 Safe Mode. Multi-user mode. All network services down. Console access only.
    • 3 Diagnostic Mode. All network services down except for remote secure system administration (SSH, VNC etc)
    • 4 Infrastructure Mode. Network Infrastructure Services up (DHCP, Routing, DNS etc)
    • 5 Full Service Mode. All Internet Services up (httpd, rsync, ftp etc)
    • 6 Reboot
  • The default runlevel to boot into is 3. In order to run a server or a router ./etc/inttab must be modified. uPM will not automatically offer services to the outside world. If apache is installed in runlevel 3 then it will not be automatically started! 

 

Author

Write A Comment