Before we get started:
This presentation is focusing on
Linux
and
rpm
and
deb
based systems in particular.
Disclaimer: Since I haven't done much rpm based packaging for
the last 6+ months and have been concentrating solely on Debian, much
of the rpm info is from memory or man pages. Sorry if you are an
rpm user. :) There will hopefully be some useful resources
listed that I used regularly as an rpm developer.
What is a package?
A package is a software application that has been bundled together in
such a way that your favorite OS can do the following for you:
- Tell you if it is currently installed
- Tell you what files are in the package
- Tell you what other packages this package depends on to work
- Install the package and its dependencies for you
- Uninstall the package
There are 2 main types of packages:
source
and
binary
- Source packages are the
source code and files needed to build a binary package from.
Usually they are distributed as a tarball.
- Binary packages are the
compiled version of the source package that is then ready to be
installed on your computer and is usable with possibly minor
configuration. Some examples of binary package formats are rpm's and deb's.
Why do you need packages?
Have you ever tried to install
KDE,
Gnome,
Mozilla or
XFree86 from source?
If yes, then you most likely are using
gentoo
or really like suffering.
If no, then you don't want to!
Packages, and their management software, make installing complicated
software applications easier for the system administrator. You
don't have enough time in the day to do the general maintenance work,
plus keep all the security updates updated and try out new software
updates, if you are doing them all from source. Packages allow
you to build or get the software for one machine, and as long as the
rest of your machines are the same OS and cpu architecure, you can use
the initial copy to install or upgrade the software on all of
them. No more rebuilding from source on each machine.
How do you find a package to install?
For source and binary packages or programs not included in your
distribution:
If using an rpm based system, try the following sites:
If using a deb based system, try
the following sites:
What are some of the available package managers?
This presentation will focus on
apt-get.
apt-get gets the information about the available packages from
repositories you point it at. Use the
/etc/apt/sources.list config file
to configure apt-get.
For example, to get packages from Debians testing distribution, you
would have the following in your
/etc/apt/sources.list
file:
deb
http://ftp.us.debian.org/debian/ testing main non-free contrib
This points to the testing distribution
and requests the main, non-free and contrib components of the
repository.
The format of the sources.list entries
is as follows:
deb|deb-src uri distribution [component1] [component2] [...]
where uri must specify the base of the Debian distribution and is
usually one of the following:
To
make apt-get aware of any changes to the packages, you have to update
it.
apt-get update
Then when you want to apply the available updates to your distro do:
| apt-get upgrade |
to upgrade, but not handle any
dependencies not already resolved |
or
apt-get dist-upgrade
|
to upgrade and handle dependencies |
If you want to download all available updates, but not install them, do:
apt-get -u --download-only --yes dist-upgrade
You probably want to run that from a cronjob daily.
To find packages that apt-get is aware of, use:
apt-cache search package-pattern |
searches for all
packages that match the specified pattern as part of the package name
or info. Ex: apt-cache search quake
|
apt-cache show package |
shows the available package info
for the specified package. This includes if it is installed, how
big it is, what packages it depends on, conflicts with, and it's
comment.
|
To install/remove packages:
apt-get install package
|
installs the specified package and any dependancies
|
apt-get remove package | removes the specified package and any dependancies
|
Now behind the scenes, apt-get uses the actual package management
program of your OS. On an rpm based system the program is
rpm. On a deb based system the
program is
dpkg.
The basic commands for
rpm are:
rpm -ivh package.rpm |
installs the specified package,
verbosely with hash marks for progress.
|
rpm -Uvh package.rpm |
upgrades
the specified package, verbosely with hash marks for progress.
|
rpm -Fvh package.rpm
|
upgrades
the specified file, but only if it is currently installed and this is a
newer version.
|
rpm -e package |
uninstalls
the specified package. No .rpm extension allowed. Ex: rpm
-e apt-get to remove the apt-get package.
|
rpm -q package |
queries
the rpm database.
|
rpm -qa |
shows all packages in the rpm database.
|
rpm -V package
|
verifies
the package - checks md5sums, etc.
|
The basic commands for
dpkg
are:
dpkg -i package.deb |
installs the specified package.
|
dpkg --purge package |
removes
the specified package along with its config files.
|
dpkg -r package |
removes
the specified package, but not its config files.
|
dpkg-reconfigure package |
reconfigures
the installed package.
|
dpkg -C |
checks
the system for any partially installed packages.
|
dpkg -c package.deb |
lists
the contents of the uninstalled package.
|
dpkg -I package.deb |
shows
information about the uninstalled package.
|
dpkg -l package-name-pattern |
lists
installed packages matching the pattern.
|
dpkg -s package |
shows
the status of the specified package.
|
dpkg -L package |
lists
the files installed on your machine from package.
|
dpkg -S filename-search-pattern |
searches
all installed packages for the owner of the specified file.
Ex: dpkg -S /bin/bash would return the package bash.
|
dpkg -p package |
displays
the info about the installed package. Similiar to -I.
|
But I don't want to manage packages at the CLI!
Use
synaptic. It
provides a very nice GUI which doesn't take long for the beginner to
intermediate system administrator to feel at ease with. It is
available for both rpm and deb based systems as a front-end to apt-get.
How to speed up getting packages from remote repositories?
Use
apt-cacher.
apt-cacher is a perl cgi script that caches downloaded packages (rpms
or debs, though designed mainly for debian systems). Once
configured, which on a Debian system doesn't need any tweaking (by
default), you just add your webserver path into the http apt-get
entries which causes apt-get to use the apt-cacher system.
apt-cacher can not handle ftp sites at this time. :(
Using the example from above, where the apt-cacher has been configured
on a central webserver called
packages.your.domain
and the path to apt-cacher is
/apt-cacher/,
you would make the following entry in your
/etc/apt/sources.list file:
deb http://packages.your.domain/apt-cacher/ftp.us.debian.org/debian/
testing main non-free contrib
Alternatively, you can always mirror the remote repository using rsync,
but then you better be prepared to wait a very long time for the
initial Debian sync. It took me over 5 days to sync up the first
time, and that was only the
i386
and
all architectures for
testing and unstable, binary packages only! The resulting mirror
was about 20GB when I stopped mirroring one month ago.
mirrors.kernel.org is anonymous
rsync friendly if you want to try your hand at this.
What if I find a package but it isn't in my systems package format
(I'm using the other type)?
See
alien for your
converting needs.
alien allows you to convert a debian package to rpm and an rpm package
to debian. The only drawback is that any configuration scripts
that the package may have had to help do post/pre installation setup
will not be converted (at least the latest version of alien I used
didn't support it). The critical things, being the contents of
the package are converted to the package format your OS knows how to
install and thus be able to track! Package versions that the
other OS relied on are most likely not going to match, so you probably
will have to do some manual forcing and installing of dependencies if
it is a package that relies on lots of external packages.
Advanced Package Management Topics
So now you want to try your hand at creating a package. Well,
first you have to have a software application you want to package and
then you need to have a source tarball of that software application.
What files should be in a source tarball?
- README
- LICENSE
- CHANGELOG or CHANGES or Changelog, etc.
- INSTALL (optional, most packages use the README file)
- Makefile (if your application uses make to build itself with)
The rest of the files in your source tarball are going to be specific
to the type of application you are building. If a perl module,
you would have the perl files, along with the Makefile.PL script, test
scripts, etc. If building a custom kernel, you would have the
source code for the kernel.
If building a Debian package, you will have a debian directory with the
control files that specify how the debian package should be
built. If building an rpm package, you will have the spec file
that controls the rpm build process.
How do I name my package?
Name your package after the name of the application, of course.
rpms:
You can use upper or lowercase and there is no fixed naming convention
in regards to libraries, binary packages, kernels, or speciality
packages such as perl modules. The normal naming convention is
name-version-release.architecture.rpm.
where (taken from the online
Maximum RPM book
published by
Red Hat):
name is a name describing the packaged software.version is the version of the packaged software.release is the number of times this version of the
software has been packaged.architecure is a shorthand name describing the type
of computer hardware the packaged software is meant to run on. It
may also be the string src, or nosrc.
Both of these strings indicate the file is an RPM source package.
The nosrc string means that the file contains only
package building files, while the src string means the
file contains the necessary package building files and the softwares source code.
Go
here
for more details and all the caveats, etc. for naming rpms.
(This is probably slightly out of date in
regards to the latest versions of rpm, but does still provide a good
starting point to the beginner.)
debs:
Start out at the
Debian
Developers' Corner and read the
Debian Policy Manual
before trying to build a debian package from scratch.
Package names must consist only of lower case letters (
a-z),
digits (
0-9), plus (
+) and minus (
-)
signs, and periods (
.). They must be at least two
characters long
and must start with an alphanumeric character.
Binary packages are normally the name of the application (bash,
dvdauthor, etc.), while libraries must start with
lib and then the package name.
A perl module is considered to be a library and so must start with
lib, but must also end with
-perl to indicate it is a perl
package. Any
::'s in the
perl module name are converted to
-.
Ex: Video::ivtv -> libvideo-ivtv-perl.
architectures:
Are handled slightly differently in the rpm and debian world
when it comes to a package that is supposed to be able to run on any
hardware platform. In the rpm world, it is called
noarch,
in the debian world it is call
all. All other
architecures pretty much take their name from the class of cpu they are
running on. For example you have:
i386 (rpm),
386
(deb),
sparc,
ppc,
x86_64,
etc.
How do I debianize my application?
Make sure you have the following packages installed:
dpkg-dev - tools required to unpack, build and
upload Debian source packages.dh-make - provides dh_make, which is
used to generate Debian style source packages from regular source code
archives.dh-make-perl - (optional)
provides dh-make-perl, which is used to Debianize perl CPAN archives.devscripts - provides debuild, which
is used to build the Debian binary packages.
lintian - (optional)
checks your generated packages for bugs and policy violations.
dput - (optional)
uploads your debian package to the repository you specify.mini-dinstall - (optional)
provides a debian repository implementation.
gnupg - (optional)
needed if you want to sign your packages.
First, make sure it builds correctly and works as desired.
Then, based upon the type of package you are building you will use one
of the following commands to assist you in debianizing your
applications source tarball, after you cd into the directory containing
your extracted source code.
So you would:
tar xvzf application-source-tarball.tgz
cd application-version
dh_make - source directory must be named <packagename>-<version>.
The <packagename> must be all lowercase, digits and
dashes. This allows you to build single or multi binary packages
and libraries.
dh-make-perl - debianizes perl modules.
You should now have a
debian/ directory in your current
directory, with the original source tarball re-created in the parent
directory and a diff file, containing the changes that were made to
initially debianize your source application.
cd into the
debian directory and modify the files to
properly reflect the info about your application (License, Changelog,
Readme, Contact, Dependancies, How to build it, etc.).
The files you will have to modify in the
debian directory
are:
changelog - used to keep track of the changes made
to the debian package. This file determines the version of the
generated package.
Sample entry is:
mailadmin (1.5.2-1) unstable; urgency=low
* Initial Release.
-- James A. Pattie <james@pcxperience.com> Mon, 29 Mar 2004
14:56:39 -0600
Where mailadmin is the name of the debian package, 1.5.2
is the version, -1 is the current build number of this
package. unstable is the repository it will be
uploaded into, when you use dput to upload to your
repository. urgency=low, specifies it isn't a
mandatory upgrade, like a security update.
Comments start indented 2 spaces and each new comment begins with a *.
The -- line indicates the package creator and when it was
created. There must be a blank line after the --
line, otherwise debuild can't parse the file properly.control - specifies the type of package being
created, any build dependencies it may have, what architecture it is
for, what packages it depends on, replaces, conflicts, etc. The
Description of the package is also specified here. You can create
multiple binary packages by defining them in this file.copyright - specifies the copyright this package is
licensed under. If the dh_make or dh-make-perl
scripts can detect the LICENSE file and recognizes the license being
used, it will automatically populate the correct info, in regards to
pointing to the local copy of the GPL, LGPL, etc. that are included in
the base install of Debian. These files are in /usr/share/common-licenses/.rules - the makefile that debuild uses to build the
binary debian package.README.Debian - You don't absolutely need this file,
but it does allow you to make a separate README file that is specific
to the Debian package.
The process of debianizing your source archive will leave a bunch of
other files in the
debian directory with the extension
.ex.
These are example files you can use as a template if you need their
features. For example, you might need to do post installation
processing, so you would copy the
postinst.ex to
postinst
and then modify it to do what your package needs done as part of the
post install process.
After your debian directory is created and you have modified the
control files as needed, you should make any changes necessary to the
applications source so that it will work properly in the Debian
environment and framework. This may mean modifying programs to
look for their config files in
/etc/<program name>
or in
/etc/default/<program name>.
Remember, anytime a new version of the software comes out that you want
to Debianize, you will have to re-apply any of the changes you made to
make it work, unless of course you are the author and you make it
Debian native. Debian native just means the source tarball comes
with the
debian directory already included and ready to
build without any changes needed by the package builder.
How do I build my debian package?
Building the Debianized source package
In the extracted source directory after you have debianized the source
archive and would like to build the binary package, execute:
debuild -tc --lintian-opts -i
to build the debian package and run it through
lintian.
-tc causes the source tree to be cleaned by executing
debian/rules clean after the package
has been built. You can also add:
-pgpg -sgpg -k<GPG KEY> to cause the generated
package to be gpg signed with the key specified with
-k<GPG
KEY>.
The generated debian package(s) and their source files, etc. will be in
your parent directory (../).
To upload the newly generated package to your debian repository
identified by
local:
dput local ../<generated debian package>.changesBuilding a custom kernel (only 2.4.x tested - I haven't played with
the 2.6 kernels yet):
Install the following packages in addition to your gcc compiler
environment:
kernel-packagekernel-patch-scriptsinitrd-toolslibncurses5-dev
Extract the kernel-source package you are interested in into
/usr/src.
cd /usr/src/kernel-source-dir- copy any
.config file you want to start with into
this directory
- install any
kernel-patch-* packages you need.
preempt, debianlogo, freeswan,
etc.
- edit
/etc/kernel-pkg.conf to specify your
maintainer info, add:
patch_the_kernel := YES
config_target := menuconfig
If you want patches applied and to always run menuconfig when
the kernel needs configuring.
make-kpkg
--append-to-version=-<tag>-<version>-<processor>
--revision=10.00 --added-patches=<comma separated list of patches
here> --initrd kernel_image
where <tag> = your prefix tag (we
use pcx), <version> is the release number of this kernel build =
1, 2, 3, etc., <processor> = 686, 586, k7, sparc, etc., patches
could be preempt,debianlogo,freeswan.
This should apply the patches you specified and then run make
menuconfig for you or whatever config_target you
specified. Make sure you compile in cramfs, initial ramdisk support and devfs. You do not need to have
devfs auto-mount since devfs is only used in the boot process by the
initial ramdisk code.
If working with a vanilla kernel or a kernel that has not been patched
with the debian kernel patch, you need to apply the following patch so
the cramfs initial ramdisk will work properly.
diff -urN
linux-2.4.23-orig/fs/block_dev.c linux-2.4.23/fs/block_dev.c
--- linux-2.4.23-orig/fs/block_dev.c
2003-06-13 09:51:37.000000000 -0500
+++ linux-2.4.23/fs/block_dev.c 2003-12-08
16:47:42.000000000 -0600
@@ -95,7 +95,7 @@
sync_buffers(dev, 2);
blksize_size[MAJOR(dev)][MINOR(dev)] = size;
bdev->bd_inode->i_blkbits = blksize_bits(size);
- kill_bdev(bdev);
+
invalidate_bdev(bdev,1);
bdput(bdev);
return 0;
}
diff -urN linux-2.4.23-orig/init/do_mounts.c
linux-2.4.23/init/do_mounts.c
--- linux-2.4.23-orig/init/do_mounts.c 2003-11-28
12:26:21.000000000 -0600
+++
linux-2.4.23/init/do_mounts.c
2003-12-08 15:35:47.000000000 -0600
@@ -648,11 +648,11 @@
goto done;
}
- printk(KERN_NOTICE
"RAMDISK: Loading %d blocks [%ld disk%s] into ram disk...
",
+ printk(KERN_NOTICE
"RAMDISK: Loading %d blocks [%lu disk%s] into ram disk...
",
nblocks, ((nblocks-1)/devblocks)+1, nblocks>devblocks ? "s" : "");
for (i=0; i <
nblocks; i++) {
if (i && (i % devblocks == 0)) {
-
printk("done disk #%ld.\n", i/devblocks);
+
printk("done disk #%lu.\n", i/devblocks);
rotate = 0;
if (close(in_fd)) {
printk("Error closing the disk.\n");
@@ -664,7 +664,7 @@
printk("Error opening disk.\n");
goto noclose_input;
}
-
printk("Loading disk #%ld... ", i/devblocks+1);
+
printk("Loading disk #%lu... ", i/devblocks+1);
}
read(in_fd, buf, BLOCK_SIZE);
write(out_fd, buf, BLOCK_SIZE);
|
- If all goes well, there should be a
kernel-image-<kernelver>-<tag>-<version>-<processor>_10.00_i386.deb
in /usr/src.
- Install via:
dpkg -i
kernel-image-<kernelver>-<tag>-<version>-<processor>_10.00_i386.deb
If you do not want the kernel-image install process to ask
every time if you are setup properly for initrd kernels, edit /etc/kernel-img.conf
and add:
do_initrd = Yes
If you don't want the vmlinuz and initrd.img links in /, then set:
do_symlinks = No
- Update lilo.conf and
make sure the
initrd= option is specified for your kernel.
lilo -v
Notes:
If you run into issues in the make-kpkg part, issue a
make-kpkg
clean to undo the affects of trying to debianize the kernel
source tree (undoes patches, etc.)
You can also build external kernel modules for ATI and nVidia drivers
using the
--added-modules argument to make-kpkg and
specifying
modules_image as an action so it will also
build the kernel-module packages. nVidia drivers are in Debians
non-free tree. You can find
the debianized ATI drivers
here.
How do I build my rpm package?
See the
Maximum RPM book for a
more definitive overview on creating rpm packages.
rpms are built much the
same as a debian package, but there are major differences.
An rpm-alized source distribution contains a
<package
name>.spec file which tells the
rpmbuild
command how to go about building the specified package. It
contains the equivalent of the
debian/ control,
changelog and
rules files in itself.
The spec file tells the
rpmbuild command what the source
tarballs are and what patches and other source files are needed to
build the rpm. Then, when you are "building" the rpm, you have to
issue the commands to extract the source packages, patch the source,
etc. The only other major difference between the rpm spec file
and the debian control files is that in an rpm you have to specify all
the files that are to be included in the rpm package. In a debian
package, all the files being packaged are placed in the
debian/<package
name> directory and so the
debuild program
always knows what files are to be packaged.
Check out the
HTMLObject perl
project for an
OUT OF DATE
spec file that can help get you started.
If you have a source tarball that is rpmalized, just execute:
rpmbuild -tb <source tarball>
to build the binary rpm from the source without having to extract
it. The resulting binary will end up in your rpm build tree
(which used to be /usr/src/redhat/RPMS/<arch>) where arch is
i386, i586, i686, noarch.
The rpm building process is from
memory and RedHat 7.3 versions so things may be in different locations
and have changed since then. :)
Extra
Shameless plug: If anyone would like debian packages made,
Xperience, Inc. is providing a
Software
Subscription Package that gives you access to our private custom
debian repository. This repository contains debianized
applications we are custom packaging - ie, they are not in Debian
proper. We can build debian packages for you at a flat fee that
are then made available to all the subscribers of the repository or you
can pay an hourly rate to have a custom package made that is then not
put in the repository. We also have an announce list for the
repository to let our subscribers know when new packages are made
available, along with a brief changelog snippet and what the package
does.
The End