The APT project design document


Manoj Srivastava

Version 3.1.3

Copyright © 1997 Manoj Srivastava

License Notice

   

APT, including this document, is free software; you may redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.

   

This is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

   

You should have received a copy of the GNU General Public License with your Debian system, in /usr/share/common-licenses/GPL, or with the debiandoc-sgml source package as the file COPYING. If not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Abstract

This document is an overview of the specifications and design goals of the APT project. It also attempts to give a broad description of the implementation as well.

Table of Contents

1. Introduction
2. Requirements
3. Procedural description
4. Modules and interfaces
5. Data flow and conversions analysis.

Chapter 1. Introduction

   

APT is supposed to be a replacement for dselect, and not a replacement for dpkg. However, since addition functionality has been required for APT, and given the fact that this is very closely related to dpkg, it is not unreasonable to expect that additional functionality in the underlying dpkg would also be requested.

   

Deity/dselect are the first introduction that people have to Debian, and unfortunately this first impression contributes greatly to the public perception of the distribution. It is imperative that this be a showcase for Debian, rather than frighten novices away (which has been an accusation often levelled at the current system)

Chapter 2. Requirements

   

  1. APT should be a replacement for dselect. Therefore it should have all the functionality that dselect has currently. This is the primary means of interaction between the user and the package management system, and it should be able to handle all tasks involved in installing, upgrading, and routine management without having the users take recourse to the underlying management system.

  2. It should be easier to use and less confusing for novice users. The primary stimulus for the creation of APT was the perceived intractability, complexity, and non-intuitive behavior of the existing user interface, and as such, human factors must be a primary mandate of APT.

  3. It should be able to group packages more flexibly, and possibly allow operations based on a group. One should be able to select, or deselect, a coherent group of related packages simultaneously, allowing one to add, remove, or upgrade functionality to a machine as one step.

  4. This would allow APT to handle standard installations, namely, one could then install a set of packages to enable a machine to fulfill specific tasks. Define a few standard installations, and which packages are included therein. The packages should be internally consistent.

  5. Make use of a keywords field in package headers; provide a standard list of keywords for people to use. This could be the underpinning to allow the previous two requirements to work (though the developers are not constrained to implement the previous requirements using keywords)

  6. Use dependencies, conflicts, and reverse dependencies to properly order packages for installation and removal. This has been a complaint in the past that the installation methods do not really understand dependencies, causing the upgrade process to break, or allowing the removal of packages that left the system in an untenable state by breaking the dependencies on packages that were dependent on the package being removed. A special emphasis is placed on handling pre-dependencies correctly; the target of a predependency has to be fully configured before attempting to install the pre-dependent package. Also, configure immediately requests mentioned below should be handled.

  7. Handle replacement of a package providing a virtual package with another (for example, it has been very difficult replacing sendmail with smail, or vice versa), making sure that the dependencies are still satisfied.

  8. Handle source lists for updates from multiple sources. APT should also be able to handle diverse methods of acquiring new packages; local filesystem, mountable CD-ROM drives, HTTP accessible repositories are some of the methods that come to mind. Also, the source lists can be separated into categories, such as main, contrib, non-local, non-free, my-very-own, etc. APT should be set up to retrieve the Packages files from these multiple source lists, as well as retrieving the packages themselves.

  9. Handle base of source and acquire all Packages files underneath. (possibly select based on architecture), this should be a simple extension of the previous requirement.

  10. Handle remote installation (to be implemented maybe in a future version, it still needs to be designed). This would ease the burden of maintaining multiple Debian machines on a site. In the authors opinion this is a killer difference for the distribution, though it may be too hard a problem to be implemented with the initial version of APT. However, some thought must be given to this to enable APT to retain hooks for future functionality, or at least to refrain from methods that may preclude remote activity. It is desirable that adding remote installation not require a redesign of APT from the ground up.

  11. Be scalable. Dselect worked a lot better with 400 packages, but at last count the number of packages was around twelve hundred and climbing. This also requires APT to pay attention to the needs of small machines which are low on memory (though this requirement shall diminish as we move towards bigger machines, it would still be nice if Debian worked on all old machines where Linux itself would work).

  12. Handle install immediately requests. Some packages, like watchdog, are required to be working for the stability of the machine itself. There are others which may be required for the correct functioning of a production machine, or which are mission critical applications. APT should, in these cases, upgrade the packages with minimal downtime; allowing these packages to be one of potentially hundreds of packages being upgraded concurrently may not satisfy the requirements of the package or the site. (Watchdog, for example, if not restarted quickly, may cause the machine to reboot in the midst of installation, which may cause havoc on the machine)

Chapter 3. Procedural description

   
Set Options

This process handles setting of user or site options, and configuration of all aspects of APT. It allows the user to set the location and order of package sources, allowing them to set up source list details, like http site locations, passwords, etc. Display options may also be set.

Updates

Build a list of available packages, using source lists or a base location and trawling for Packages files (needs to be aware of architecture). This may involve finding and retrieving Packages files, storing them locally for efficiency, and parsing the data for later use. This would entail contacting various underlying access modules (http, cdrom mounts, etc) Use a backing store for speed. This may also require downloading the actual package files locally for speed.

Local status

Build up a list of packages already installed. This requires reading and writing the local?? status file. For remote installation, this should probably use similar mechanisms as the Packages file retrieval does. Use the backing store for speed. One should consider multiple backing stores, one for each machine.

Relationship determination

Determine forward and reverse dependencies. All known dependency fields should be acted upon, since it is fairly cheap to do so. Update the backing store with this information.

Selection

Present the data to the user. Look at Behan Webster's documentation for the user interface procedures. (Note: In the authors opinion deletions and reverse dependencies should also be presented to the user, in a strictly symmetric fashion; this may make it easier to prevent a package being removed that breaks dependencies)

Ordering of package installations and configuration

Build a list of events. Simple topological sorting gives order of packages in dependency order. At certain points in this ordering, predependencies/immediate configure directives cause a break in normal ordering. We need to insert the uninstall/purge directive in the stream (default: as early as possible).

Action

Take the order of installations and removals and build up a stream of events to send to the packaging system (dpkg). Execute the list of events if successful. Do not partially install packages and leave system in broken state. Go to The Selection step as needed.

Chapter 4. Modules and interfaces

   
The user interface module

Look at Behan Webster's documentation.

Widget set

Related closely to above Could some one present design decisions of the widget set here?

pdate Module

Distinct versions of the same package are recorded separately, but if multiple Packages files contain the same version of a package, then only the first one is recorded. For this reason, the least expensive update source should be listed first (local file system is better than a remote http site)

This module should interact with the user interface module to set and change configuration parameters for the modules listed below. It needs to record that information in an on disk data file, to be read on future invocations.

  1. HTTP methods

  2. mount and file traversal module(s)?

  3. Other methods ???

Status file parser/generator

The status file records the current state of the system, listing the packages installed, etc. The status file is also one method of communicating with dpkg, since it is perfectly permissible for the user to use APT to request packages be updated, put others on hold, mark other for removal, etc, and then run dpkg -BORGiE on a file system.

Package file parser/generator

Related to above. Handle multiple Packages files, from different sources. Each package contains a link back to the packages file structure that contains details about the origin of the data.

Dependency module

  • dependency/conflict determination and linking

  • reverse dependency generator. Maybe merged with above

Package ordering Module

Create an ordering of the actions to be taken.

Event generator

module to interact with dpkg

Chapter 5. Data flow and conversions analysis.

   
                                                          _____________
                                                       __\|http modules|
                                                      /  /|____________|
                                    _ ____________   /     ________________
                                    |    update   | /     |mount/local file|
        |==========================>|   module    |/_____\|  traversals    |
        |                           |_____________|      /|________________|
        |                             ^        ^
        |                             |        |               ______________
  ______|_______    _ _____ ______    |   _____v________      \|            |
 |Configuration |   |configuration|   |   |Packages Files|  ===|Status file |
 |  module      |<=>|    data     |   |   |______________| /  /|____________|
 |______________|   |_____________|   |        ^          /
         ^                            |        |         /
         |                            | _______v_______|/_
         |                            | |              |    ________________
         |                            | |              |/_\|   Dependency  |
         |                            | |backing store |\ /|     Module    |
         |                            | |______________|  _|_______________|
         |                             \       ^          /|       ^
         |                              \      |         /         |
         |                              _\|____v_______|/__    ____v_______
         |_____________________________\| User interaction|    |    dpkg   |
                                       /|_________________|<==>  Invoker  |
                                                               |___________|
   

dpkg also interacts with status and available files.

   

The backing store and the associated data structures are the core of APT. All modules essentially revolve around the backing store, feeding it data, adding and manipulating links and relationships between data in the backing store, allowing the user to interact with and modify the data in the backing store, and finally writing it out as the status file and possibly issuing directives to dpkg.

   

The other focal point for APT is the user interface.