FreeBSD 5.4-RELEASE Migration Guide
The FreeBSD Release Engineering Team
Copyright (c) 2002, 2003, 2004, 2005 The FreeBSD Release
Engineering Team
$FreeBSD:
src/release/doc/en_US.ISO8859-1/early-adopter/article.sgml,v
1.21.2.7 2005/03/25 15:53:33 bmah Exp $
FreeBSD is a registered trademark of the FreeBSD Foundation.
Apple, FireWire, Mac, Macintosh, Mac OS, Quicktime, and TrueType
are trademarks of Apple Computer, Inc., registered in the United
States and other countries.
IBM, AIX, EtherJet, Netfinity, OS/2, PowerPC, PS/2, S/390, and
ThinkPad are trademarks of International Business Machines
Corporation in the United States, other countries, or both.
IEEE, POSIX, and 802 are registered trademarks of Institute of
Electrical and Electronics Engineers, Inc. in the United States.
Intel, Celeron, EtherExpress, i386, i486, Itanium, Pentium, and
Xeon are trademarks or registered trademarks of Intel Corporation
or its subsidiaries in the United States and other countries.
Microsoft, IntelliMouse, MS-DOS, Outlook, Windows, Windows Media
and Windows NT are either registered trademarks or trademarks of
Microsoft Corporation in the United States and/or other countries.
Sparc, Sparc64, SPARCEngine, and UltraSPARC are trademarks of
SPARC International, Inc in the United States and other countries.
Products bearing SPARC trademarks are based upon architecture
developed by Sun Microsystems, Inc.
Many of the designations used by manufacturers and sellers to
distinguish their products are claimed as trademarks. Where those
designations appear in this document, and the FreeBSD Project was
aware of the trademark claim, the designations have been followed
by the ``(TM)'' or the ``(R)'' symbol.
This article describes major differences between FreeBSD 4.X and
FreeBSD 5.4-RELEASE, from the viewpoint of a user or administrator
upgrading an existing system. It provides a brief overview of
FreeBSD release engineering. It then describes some of the new
features in FreeBSD 5.X, as well as some changes that might be of
particular interest to users accustomed to working with the 4.X
series of releases. This article also describes binary- and
source-based upgrading procedures for existing systems. Earlier
versions of this article appeared in prior FreeBSD 5.X releases.
Prior to FreeBSD 5.3-RELEASE, this article was known as the
``Early Adopters Guide''.
--------------------------------------------------------------
1 Introduction
FreeBSD 5.4-RELEASE is the latest version of FreeBSD in the
``FreeBSD-STABLE'' series of releases. This and future releases in
the 5.X series are targeted toward production usage in much the
same way as the prior 4.X series of releases.
This article addresses a number of topics and issues of interest
to users updating from a 4.X release to 5.4-RELEASE. It begins
with a brief overview of current release engineering practices and
then describes some of the new features available with the FreeBSD
5.X series. Perhaps the most important section lists issues where
major changes have taken place, user-visible behavior has changed,
or external software interfaces have been modified. Last are some
notes on upgrading existing FreeBSD 4.X systems to FreeBSD
5.4-RELEASE, from binaries or from source.
--------------------------------------------------------------
2 An Overview of the FreeBSD Release Process
FreeBSD employs a development model relying on multiple branches
within the source code repository. The main branch is called
``CURRENT'', and is referred to in the CVS repository with the
HEAD tag. New features are committed first to this branch;
although CURRENT is the first to see new functionality, it also
occasionally suffers from breakages as new features are added and
debugged.
Most FreeBSD releases are made from one of several ``STABLE''
branches. Features are added to these branches only after some
amount of testing in CURRENT. For the past two years, the only
STABLE branch under active development was known as ``4-STABLE'',
and all of the FreeBSD 4.X releases were based on it. This branch
has the tag RELENG_4 in the CVS repository.
The past two stable branches (3-STABLE and 4-STABLE) were created
very early in their development cycles (their branchpoints were
3.1 and 4.0, respectively). In hindsight, this practice did not
give sufficient time for either CURRENT to stabilize before the
new branches were created. This in turn resulted in wasted effort
porting bug fixes between branches, as well as creating some
architectural changes that could not be ported between branches at
all.
The 5.X releases have been handled slightly differently. FreeBSD
5.0, 5.1, and 5.2 were based on the CURRENT branch. The first of
these releases was made after over two years of development (prior
to these, the last release from HEAD was FreeBSD 4.0, in March
2000).
Unlike prior ``STABLE'' branches, the ``5-STABLE'' branch in CVS
(with the branch tag RELENG_5) was created more than a year after
the first 5.X release (specifically, during the 5.3 release
cycle). This delay gave time for the development team to complete
needed architectural changes, stabilize the system, finalize
various interfaces, and create a good starting point for the
remaining 5.X releases.
Aside from general improvements and bug fixes, a major priority
for development on the 5-STABLE branch is the preservation of
Application Binary Interface (ABI) and Application Program
Interface (API) compatibility. Any changes that could break
backward compatibility (including kernel or library interfaces)
are strongly discouraged, and will not be permitted except as a
last-resort solution to a critical problem.
The next release from CURRENT will likely be 6.0-RELEASE, created
from CVS HEAD. There is no firm date for 6.0, as of this writing,
although it is expected sometime in the third quarter of 2005.
Future release engineering plans call for new major releases
approximately every eighteen months, with minor releases
approximately every four months.
The last planned release along the 4-STABLE development branch was
4.11-RELEASE, which was announced in January 2005. No new major
features are planned for the 4-STABLE branch at this time, and any
future work will likely be limited to bug fixes and security
patches. For especially conservative users, it may be reasonable
to continue using the 4.X releases for a time. The Security
Officer Team <security-officer@FreeBSD.org> will continue to
support releases made from the 4-STABLE branch in accordance with
their published policies, which can be found on the Security page
on the FreeBSD web site.
More information on FreeBSD release engineering processes can be
found on the Release Engineering Web pages and in the ``FreeBSD
Release Engineering'' article.
--------------------------------------------------------------
3 New Features
A major attraction of FreeBSD 5.X is a number of new features,
generally involving large architectural changes that were not
feasible to port back to the FreeBSD 4-STABLE development branch.
(By contrast, many self-contained enhancements, such as new device
drivers or updates to userland utilities, have already been
ported.) A brief, but not exhaustive list includes:
* SMPng: The ``next generation'' support for Symmetric
MultiProcessor (SMP) machines. Ongoing work aims to perform
fine-grained locking of various kernel subsystems to increase
the number of threads of execution that can be running in the
kernel. Particular attention has been paid to the performance
of the network stack and storage drivers. More information can
be found on the FreeBSD SMP Project page.
* KSE: Kernel Scheduled Entities allow a single process to have
multiple kernel-level threads, similar to Scheduler
Activations. The libpthread and libthr threading libraries
make this feature available to multi-threaded userland
programs, using the pthread(3) API. libpthread is now the
default threading library.
* New architectures: Support for the amd64, ia64, pc98, and
sparc64 architectures, in addition to the i386 and alpha.
Other platforms under development include powerpc (which runs
on many PowerPC(R)-based Macintosh(R) platforms) and arm.
* GCC: The compiler toolchain is now based on GCC 3.4.X, rather
than GCC 2.95.X.
* MAC: Support for extensible, loadable Mandatory Access Control
policies.
* GEOM: A flexible, modular framework for transformation of disk
I/O requests. This system supports a number of features
related to disks and volumes, such as: recognition of disk
partitions, the gbde(4) disk encryption facility, various
levels of RAID functionality, network export of disk devices
(with ggated(8) and related utilities), and transparent disk
decompression. The geom_vinum system brings GEOM compatibility
to the VINUM volume manager.
* FFS: The FFS file system now supports background fsck(8)
operations (for faster crash recovery) and file system
snapshots.
* UFS2: A new UFS2 on-disk format has been added, which supports
extended per-file attributes and larger file sizes. UFS2 is
now the default format for newfs(8). On all platforms except
pc98, file systems created from within sysinstall(8) will use
UFS2 by default.
* New networking features: A number of new networking features
have made their debut in 5.X releases, including support for
TCP SACK (selective acknowledgements), the pf(4) packet filter
from OpenBSD, and the ALTQ packet queueing system.
* New hardware support: Support for more hardware devices, such
as Cardbus, Bluetooth devices, and IEEE 802.11a/b/g network
interfaces based on Atheros chipsets. Also, on the i386
architecture, some network devices not explicitly supported by
FreeBSD drivers may be supported using vendor drivers for
Microsoft(R) Windows(R) and the ndis(4) compatibility layer.
A more comprehensive list of new features can be found in the
release notes for the various FreeBSD 5.X releases.
--------------------------------------------------------------
4 Notable Changes
Some of the differences between FreeBSD 4.X and 5.X deserve
special mention because they involve major architectural changes,
or break backwards compatibility in some way. While these changes
are unlikely to cause a loss of data, they could cause some
confusion for the unwary. Some notable examples are:
* Several parts of FreeBSD's base system functionality have been
moved to the FreeBSD Ports Collection, usually because they
are easier to maintain in that form or because they were not
really essential parts of the system. The most noticeable
example of this is Perl (available in the FreeBSD Ports
Collection as lang/perl5.8), whose former location in the base
system made upgrades problematic. Utilities in the base system
that formerly relied on Perl have been either rewritten (if
still applicable) or removed (if obsolete).
Other examples include UUCP (available in the Ports Collection
as net/freebsd-uucp), the doscmd utility (emulators/doscmd),
and many of the traditional games (games/freebsd-games).
* Changes in kernel data structures and ABIs/APIs meant many
third-party binary device drivers had to be modified before
they would work correctly under FreeBSD 5.X. In some
(hopefully rare) cases, user-visible data structures have
changed, requiring recompilation of applications or
reinstallation of ports/packages. As with the 4.X series, the
FreeBSD development team has the goal of not allowing
incompatible changes in future releases on the 5.X branch.
* A shortage of users and maintainers for some parts of the base
system has caused them to fall into disrepair. These have been
removed. Examples include the generation of a.out-style
executables, [1] XNS networking support, and the X-10
controller driver. A number of other drivers for old ISA-based
devices have been discovered to be non-working and have been
removed. More details are given in the release notes.
* On FreeBSD/i386 4.X, the configuration of ISA devices was
initially specified in the kernel configuration. These could
be modified at boot-time with an interactive UserConfig
utility. On FreeBSD 5.X, both of these mechanisms have been
replaced by the device.hints(5) mechanism, which eliminates
the need to hard-code ISA configuration parameters in the
kernel. Boot-time ISA configuration is no longer menu-driven,
but is performed by entering new parameters to the boot loader
command line prompt.
* MAKEDEV is no longer available, nor is it required. FreeBSD
5.X uses a device file system, which automatically creates
device nodes on demand. Configuration of the entries in the
device filesystem can be performed with the devfs(8) utility
or with the /etc/devfs.conf mechanism. More information can be
found in the devfs(5) manual page.
* UFS2 is the default on-disk format for file systems created
using newfs(8). For all platforms except pc98, it is also the
default for file systems created using the disk labeling
screen within sysinstall(8). Because FreeBSD 4.X only
understands UFS1 (not UFS2), disk partitions that must be
accessed by both 5.X and 4.X must be created with UFS1. This
can be specified using the -O1 option to newfs(8), or on the
disk labeling screen in sysinstall(8). This situation most
often arises with a a single machine that dual-boots FreeBSD
4.X and FreeBSD 5.X. Note that there is no way to convert file
systems between the two on-disk formats (other than backing
up, re-creating the file system, and restoring).
* Due to the upgraded GCC compiler, C++ programs generally need
to be recompiled and reinstalled. This requirement comes from
changes in the C++ ABI.
* Running executables compiled for FreeBSD 4.X may be possible
with the aid of the compat4x distribution (or the equivalent
misc/compat4x package/port). However, if any of the ports or
packages on a system require upgrading, rebuilding and
reinstalling all of the installed ports on a system is
generally advised. This will prevent problems that can occur
with programs being linked against some combination of old and
new libraries. The sysutils/portupgrade utility may be helpful
in reinstalling ports.
Note that in any case, there are a number of known instances
of backward incompatibility. For example, the devel/gnomevfs2,
mail/postfix, and security/cfs ports need to be recompiled due
to changes in the statfs structure.
* The host.conf resolver services configuration file has been
replaced by the (much more general) nsswitch.conf(5)
name-service switch configuration mechanism.
* BIND has been updated from version 8 to version 9. There are
numerous differences between BIND 8 and 9. Users with critical
named(8) installations should read the migration documentation
in /usr/share/doc/bind9/misc/migration. There is also a new
instruction manual in /usr/share/doc/bind9/arm. On systems
that run named(8), it now runs by default in a chroot(8)
environment in /var/named. Users with existing named(8)
configurations in /var/named should back up the contents of
this directory before upgrading to avoid them being
overwritten by the named(8) startup script.
* The Xorg implementation of the X Window System is the default
for FreeBSD 5.X beginning with 5.3-RELEASE. As of this
writing, XFree86(TM) remains the default for FreeBSD 4.X. More
information on the differences between these versions, as well
as upgrade information for existing systems, can be found in
the X Window System chapter in the FreeBSD Handbook.
* Documentation (such as the FreeBSD Handbook and FAQ) is still
being updated to reflect changes recently made to FreeBSD 5.X.
More information can be found in the release notes for the various
FreeBSD 5.X releases, as well as the src/UPDATING file in the
FreeBSD source tree. Notable changes to the FreeBSD Ports
Collection are listed in the ports/UPDATING and ports/CHANGES
files in the ports tree.
--------------------------------------------------------------
5 Notes on Upgrading from FreeBSD 4.X
For those users with existing FreeBSD systems, this section offers
a few notes on upgrading a FreeBSD 4.X system to 5.X. As with any
FreeBSD upgrade, it is crucial to read the release notes and the
errata for the version in question, as well as src/UPDATING in the
case of source upgrades.
--------------------------------------------------------------
5.1 Binary Upgrades
Probably the most straightforward approach is that of ``backup
everything, reformat, reinstall, and restore user data''. This
eliminates problems of incompatible or obsolete executables and
configuration files polluting the new system. It allows new file
systems to be created to take advantage of new functionality (most
notably, the UFS2 defaults).
As of this writing, the binary upgrade option in sysinstall(8) has
not been well-tested for cross-major-version upgrades. Use of this
feature is not recommended.
Several changes may be of interest to those users accustomed to
the FreeBSD 4.X boot media. Installation floppies (on platforms
that support them, such as i386), are organized somewhat
differently than on prior releases. On 4.X releases, the floppy
set contained a stripped-down kernel with just enough
functionality to install the system. This arrangement allowed the
kernel to fit on a single floppy disk, but it lacked the device
drivers required by certain hardware configurations. Beginning
with FreeBSD 5.3-RELEASE, the installation floppies contain a
standard GENERIC kernel segmented across multiple disks, with a
much more complete set of drivers and features. The boot loader
prompts for the insertion of additional disks as required. Users
downloading floppy images (perhaps to perform a network-based
installation) should note that the floppy disk set now includes
three disks: boot.flp, kern1.flp, and kern2.flp.
The installation CDROMs for the i386 architecture now use a
``no-emulation'' boot loader. This allows, among other things, the
use of a GENERIC kernel, rather than the stripped-down kernels
used on the floppy images in previous versions. Generally, any
system capable of booting the Microsoft Windows NT(R) 4,
Windows 2000, or Windows XP installation CDROMs should be able to
boot the FreeBSD 5.X CDROMs.
The layout of CDROMs has changed from the 4.X series. Beginning
with 5.4-RELEASE, the installation image and the ``live
filesystem'' are now contained on the first disk on all
architectures except for ia64 (ia64 releases use separate
installation and live filesystem disks due to disk space
constraints). Common packages are now contained on the second and
succeeding disks of a release.
--------------------------------------------------------------
5.2 Source Upgrades
Note: Many users and developers have found it easier to backup
all their data and configuration files (a wise precaution in any
case), perform a binary installation (such as from CDROM), and
restore their data. Compared to a source upgrade, the binary
upgrade removes the need to deal with leftover files and
programs on the disk, and allows the system to take advantage of
new file system features such as the UFS2 file system layout.
Users unfamiliar with the buildworld/installworld procedures for
updating FreeBSD from source should not attempt a source
upgrade, but should instead perform a binary installation after
backing up all data.
A source-based upgrade procedure builds and installs a set of
binaries compiled from source on the local machine. It is based on
the buildworld/installworld procedure often used by advanced
FreeBSD users to track changes along a development branch (such as
FreeBSD-STABLE or FreeBSD-CURRENT). In general, this procedure
involves more effort than the binary upgrade procedure, but may be
useful when a system's configuration files are complex or have
been highly customized. A source upgrade can also be useful for a
remote system where an administrator has remote console access but
no physical access (and therefore cannot insert installation
media).
Reading src/UPDATING is absolutely essential. The section entitled
``To upgrade in-place from 4.x-stable to current'' contains a
step-by-step update procedure, which must be followed exactly,
without making use of the ``shortcuts'' that some users
occasionally employ. An annotated list of these steps is presented
below:
1. Make backups.
The importance of this step cannot be overstated. It is
important to make backups of all user data and configuration
files. Level zero dumps with dump(8) are an excellent way to
do this, although there are of course many workable
alternatives.
2. Ensure that there is about 30MB of free disk space on the /
file system.
FreeBSD 5.X uses more space than FreeBSD 4.X. If the /tmp
directory resides on the / file system (as is frequently the
case), deleting all of the content from this directory may
help to free up needed space.
3. Fix /etc/fstab if required.
This item probably only affects older FreeBSD/i386 systems. On
systems that use MBR-style disk slices, FreeBSD used to
support ``compatibility slices'', where disk partition names
could take the form /dev/ad0a (without specifying a slice name
explicitly). These are no longer supported; disk partitions
must be named according to the form /dev/ad0s1a. In other
words, disk partitions must now completely specify a disk
device, a slice number, and a partition letter.
Note that ``compatibility slices'' have generally not been
used by default since FreeBSD 2.2.6-RELEASE. This item does
not apply to FreeBSD/alpha at all, or to systems using
``dangerously dediated'' mode.
4. Using cvs(1) or other means, obtain a copy of the FreeBSD base
system sources (the src/ module). To check out 5.4-RELEASE
from the FreeBSD CVS repository, use the RELENG_5_4_0_RELEASE
tag. For the 5.4 release and errata branch, use the RELENG_5_4
tag. To track the FreeBSD 5-STABLE development branch, use the
RELENG_5 branch tag. When using CVS to check out the source
tree, it is important to pass the -P flag to CVS so that it
prunes away empty directories.
5. FreeBSD 5.X requires some new entries in the system password
and group files, corresponding to various system services.
They should be installed by running:
# mergemaster -p
This step must be performed to give some new files the correct
usernames and groupnames.
Note that in FreeBSD 5.X, the location of the nologin(8)
utility has moved from /sbin/nologin to /usr/sbin/nologin.
Because a number of pseudo-users have nologin(8) as their
login shell, this change generates additional differences in
/etc/passwd.
6. Build the new userland using:
# cd /usr/src
# make buildworld
If CPUTYPE is defined in /etc/make.conf, it should be defined
using the ?= operator, so the buildworld process can override
this variable if necessary.
Note that the MAKEOBJDIRPREFIX must not be defined in
/etc/make.conf.
In general, most of the build ``knobs'' defined in
/etc/make.conf should be commented out or removed. This
statement especially applies to those that directly affect the
build or basic operation of FreeBSD, such as NO_TOOLCHAIN.
7. Build and install a new kernel using:
# make buildkernel
Note that the buildkernel make(1) target must be used in order
to ensure that the resulting kernel is compiled with the
toolchain built in the buildworld step above. Manually using
config(8) to set up a kernel build area and attempting to
build a kernel will not work.
Although building (and later installing) a custom kernel at
this point is feasible, upgrading using the GENERIC kernel and
installing a custom kernel configuration later may be less
error-prone. When trying to build a custom kernel for the
first time, using the GENERIC kernel from FreeBSD 5.X as a
template is highly recommended, due to the number of device
and options that have been added or changed since 4.X. In any
case, including the COMPAT_FREEBSD4 kernel configuration
option is crucial for a successful upgrade.
8. Set up a device hints file for ISA devices with a variant of
the following command. Substitute the appropriate architecture
name (e.g. i386) for MACHINE.
# cp sys/MACHINE/conf/GENERIC.hints /boot/device.hints
While an empty device hint file may be sufficient for modern
hardware, ISA hints are needed for systems with custom ISA
non-PNP cards (with custom hints being needed), for
non-PNPBIOS systems, or for floppy disks to work correctly on
PNPBIOS systems.
9. Install the new kernel with:
# make installkernel
Note that while the FreeBSD 4.X kernel installs to /kernel by
default, the FreeBSD 5.X kernel installs to
/boot/kernel/kernel. Kernel modules under 4.X normally install
to the /modules directory, whereas under 5.X they install to
/boot/kernel.
10. Install the FreeBSD 5.X boot loader with:
# cd /usr/src/sys/boot
# make STRIP="" install
This step, though optional, is highly recommended.
11. Disable third-party modules (such as those for VMware) to
prevent crashes caused by changes in kernel ABIs or other
incompatibilities.
12. Reboot to single-user mode. Rebooting at this point is
absolutely required because the new kernel must be running to
install the new userland. Being in single-user mode will
drastically decrease the potential for unexpected things to
happen during the upgrade because there will be no other
programs running.
13. Mount the required file systems using:
# fsck -p
# mount -uw /
# mount -at ufs
Note that the fsck(8) invocation is necessary to update
certain fields in the file system superblocks for FreeBSD 5.X.
If the system clock tracks local time rather than UTC time,
also perform:
# adjkerntz -i
During this step, the following warning message may appear on
the console (numerous times).
WARNING: userland calling deprecated sysctl, please rebuild world
This message can be safely ignored.
14. Delete the C++ header files with:
# rm -rf /usr/include/g++
This step keeps future compilations from accidentally picking
up old header files from the GCC 2.95 C++ compiler.
15. Install the new userland utilities with:
# cd /usr/src
# make installworld
16. On systems running named(8), its configuration files need to
be moved into a chroot(8) area in /var/named. If any files
exist in /var/named, they should be backed up at this point.
# cd /etc
# mv named named.bak
# mkdir -p /var/named/etc/namedb
# cp -Rp named.bak/* /var/named/etc/namedb
If the configuration uses the generated localhost files, the
following steps may be needed to regenerate them:
# cd /var/named/etc/namedb
# /bin/sh make-localhost
# rm -f localhost-v6.rev localhost.rev
Finally, two obsolete binaries from the prior BIND need to be
removed:
# rm /usr/sbin/nslookup
# rm /usr/sbin/nsupdate
17. Update the system configuration files by running:
# mergemaster -i
This step is non-optional. It is required to make the startup
and configuration files in /etc consistent with the new kernel
and world.
After invoking mergemaster(8), it is a good idea to inspect
(and possibly modify) /etc/rc.conf. A number of default values
for the variables defined in this file have changed; in
particular, some services that were enabled by default in 4.X
are now disabled by default in 5.X.
18. Remove leftover BIND 8 files:
# rm /usr/bin/dnskeygen /usr/bin/dnsquery
# rm /usr/libexec/named-xfer
# rm /usr/sbin/named.restart /usr/sbin/ndc
19. Reboot.
After upgrading the base system, upgrades to some non-base-system
components are generally needed to restore normal functionality.
Perl is no longer a part of the base system and should be
installed from the Ports Collection (specifically, the
lang/perl5.8 port) or from a package. After this installation, all
ports and/or packages depending on Perl need to be reinstalled.
Running FreeBSD 4.X binaries requires a set of compatability
libraries. These are available by installing the misc/compat4x
package/port.
As mentioned in a prior section, Xorg is the default
implementation of the X Window System. The Ports Collection (as
well as packages) rely on this change to satisfy dependencies. To
convert the installed windowing system from XFree86 to Xorg, refer
to the Installing X11 section in the FreeBSD Handbook.
--------------------------------------------------------------
6 Summary
This article presents some of the more notable new features in
FreeBSD 5.X, and lists some areas of particular concern to those
users upgrading existing FreeBSD 4.X systems. It also presents two
sets of upgrade instructions, one using binaries from installation
media and one based on recompiling the base system from source
code.
Notes
[1] Note that execution of a.out executables on the i386 and pc98
is still supported with the COMPAT_AOUT kernel option. Some of
the compilers in the Ports Collection (for example,
lang/gcc28) are still capable of generating a.out format
executables.
--------------------------------------------------------------
This file, and other release-related documents, can be downloaded from
ftp://ftp.FreeBSD.org/.
For questions about FreeBSD, read the documentation before contacting
<questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.