Once the local source tree is synchronized against a particular version of FreeBSD such as FreeBSD-STABLE or FreeBSD-CURRENT, the source tree can be used to rebuild the system.
It cannot be stressed enough how important it is to make a backup of the system before rebuilding the system. While rebuilding the world is an easy task, there will inevitably be times when mistakes in the source tree render the system unbootable.
Create and verify a backup and have a bootable installation media at hand. You will probably never have to use it, but it is better to be safe than sorry!
The FreeBSD-STABLE and FreeBSD-CURRENT branches are, by their nature, in development. People that contribute to FreeBSD are human, and mistakes occasionally happen.
Sometimes these mistakes can be quite harmless, just causing the system to print a new diagnostic warning. Or the change may be catastrophic, and render the system unbootable or destroy file systems.
When problems occur, a “heads up” is posted to the appropriate mailing list, explaining the nature of the problem and which systems it affects. An “all clear” announcement is posted when the problem has been solved.
Users who track FreeBSD-STABLE or FreeBSD-CURRENT and do not read FreeBSD-STABLE mailing list or FreeBSD-CURRENT mailing list respectively, are asking for trouble.
make world
: Some older documentation recommends using
make world
. However, that command skips
some important steps and should only be used by experts. For
almost all circumstances make world
is the
wrong thing to do, and the procedure described here should be
used instead.
Before updating the system, read
/usr/src/UPDATING
for any pre-buildworld
steps necessary for that version of the sources. Then, use
the procedure outlined here.
These upgrade steps assume an upgrade from an older FreeBSD version, consisting of an old compiler, old kernel, old world, and old configuration files. “World” includes the core system binaries, libraries, and programming files. The compiler is part of “world”, but has a few special concerns.
These steps also assume that the sources to a newer version have already been obtained. If the sources are not up-to-date, refer to Section 24.6, “Synchronizing Source” for detailed help about synchronizing to a newer version.
Updating the system from source is a more subtle process than it might initially seem to be, and the FreeBSD developers have found it necessary over the years to change the recommended approach fairly dramatically as new kinds of unavoidable dependencies come to light. The rest of this section describes the rationale behind the currently recommended upgrade sequence.
Any successful update sequence must deal with the following issues:
The old compiler might have a bug and not be able to compile the new kernel. So, the new kernel should be built with the new compiler, meaning that the new compiler must be built before the new kernel is built. This does not necessarily mean that the new compiler must be installed before building the new kernel.
The new world might rely on new kernel features. So, the new kernel must be installed before the new world is installed.
These first two issues are the basis for the
core buildworld
,
buildkernel
,
installkernel
,
installworld
sequence described in
the following paragraphs. Other reasons for using these
steps are listed below:
The old world might not run correctly on the new kernel, so the new world must be installed immediately upon installing the new kernel.
Some configuration changes must be made before the new world is installed, but others might break the old world. Hence, two different configuration upgrade steps are generally needed.
For the most part, the update process only replaces or adds files and existing old files are not deleted. In a few cases, this can cause problems. As a result, the update procedure will sometimes specify certain files that should be manually deleted at certain steps. This may or may not be automated in the future.
These concerns have led to the following recommended sequence. Note that the detailed sequence for particular updates may require additional steps, but this core process should remain unchanged for some time:
make
buildworld
This first compiles the new compiler and a few related
tools, then uses the new compiler to compile the rest of
the new world. The result ends up in
/usr/obj
.
make
buildkernel
This uses the new compiler
residing in /usr/obj
in order to
protect against compiler-kernel mismatches.
make
installkernel
Place the new kernel and kernel modules onto the disk, making it possible to boot with the newly updated kernel.
Reboot into single user mode.
Single user mode minimizes problems from updating software that is already running. It also minimizes any problems from running the old world on a new kernel.
mergemaster
-p
This does some initial configuration file updates in
preparation for the new world. For instance, it may add
new user groups to the system, or new user names to the
password database. This is often necessary when new
groups or special system-user accounts have been added
since the last update, so that the
installworld
step will be able to
use the newly installed system user or system group names
without problems.
make
installworld
Copies the world
from /usr/obj
. The
new kernel and new world are now installed on disk.
mergemaster
Repeated to update the remaining configuration files, now that the new world is on disk.
make
delete-old
This target deletes old (obsolete) files. This is
important because sometimes they cause problems if left on
the disk, for example the presence of the old
utmp.h
causes problems in some ports
when the new utmpx.h
is
installed.
Reboot.
A full machine reboot is needed now to load the new kernel and new world with new configuration files.
make
delete-old-libs
Remove any obsolete libraries to avoid conflicts with newer ones. Make sure that all ports have been rebuilt before old libraries are removed.
Upgrades from one release of the same FreeBSD branch to a
more recent release of the same branch, such as from 9.0 to
9.1, may not need this procedure since it is less likely to
run into serious mismatches between compiler, kernel,
userland, and configuration files. The approach of
make
followed by building and installing a new kernel might work
well enough for minor updates.world
When upgrading across major releases, people who do not follow this procedure should expect some problems.
It is also worth noting that many upgrades may require
specific additional steps such as renaming or deleting
specific files prior to installworld. Read
/usr/src/UPDATING
carefully, especially
at the end, where the currently recommended upgrade sequence
is explicitly spelled out.
This procedure has evolved over time as the developers have found it impossible to completely prevent certain kinds of mismatch problems. Hopefully, the current procedure will remain stable for a long time.
To summarize, the currently recommended way of upgrading FreeBSD from sources is:
#
cd /usr/src
#
make buildworld
#
make buildkernel
#
make installkernel
#
shutdown -r now
There are a few rare cases when an extra run of
mergemaster -p
is needed before the
buildworld
step. These are
described in UPDATING
. In general,
though, this step can safely be omitted when not updating
across one or more major FreeBSD versions.
After installkernel
finishes
successfully, boot into single user mode using boot
-s
from the loader prompt. Then run:
#
mount -u /
#
mount -a -t ufs
#
adjkerntz -i
#
mergemaster -p
#
cd /usr/src
#
make installworld
#
mergemaster
#
make delete-old
#
reboot
#
make delete-old-libs
The following sections clearly describe each step, especially when using a custom kernel configuration.
Before updating, read
/usr/src/UPDATING
. This file contains
important information about potential problems and may specify
the order to run certain commands. If
UPDATING
contradicts the procedure in
this section, UPDATING
takes
precedence.
Reading UPDATING
is not an
acceptable substitute for subscribing to the correct mailing
list. The two requirements are complementary, not
exclusive.
Available make(1) options are shown in
make.conf(5) and
/usr/share/examples/etc/make.conf
. These
settings can be added to /etc/make.conf
to control the way make(1) runs and how it builds
programs. Changes to some settings can have far-reaching and
potentially surprising effects. Read the comments in both
locations and keep in mind that the defaults have been chosen
for a combination of performance and safety.
Options set in /etc/make.conf
take
effect every time make(1) is used, including compiling
applications from the Ports Collection or user-written C
programs, or building the FreeBSD operating system.
/etc/src.conf
controls the building
of the operating system from source code. Unlike
/etc/make.conf
, the contents of
/etc/src.conf
only take effect when the
FreeBSD operating system itself is being built. Descriptions of
the many options available for this file are shown in
src.conf(5). Be cautious about disabling seemingly
unneeded kernel modules and build options. Sometimes there
are unexpected or subtle interactions.
/etc
contains a
large part of the system's configuration information, as well
as scripts that are run at system startup. Some of these
scripts change between FreeBSD versions.
Some of the configuration files are used in the day to
day running of the system, such as
/etc/group
.
There have been occasions when the installation part of
make installworld
expected certain
usernames or groups to exist. When performing an upgrade, it
is likely that these users or groups do not yet exist. In
some cases make buildworld
will check to
see if these users or groups exist.
The solution is to run mergemaster(8) in
pre-buildworld mode with -p
. This compares
only those files that are essential for the success of
buildworld
or
installworld
.
To check which files are owned by the group being renamed or deleted:
#
find / -group GID
-print
This command will show all files owned by group
GID
, which can be either a group
name or a numeric group ID.
Consider compiling the system in single user mode. Reinstalling the system touches a lot of important system files, all the standard system binaries, libraries, and include files. Changing these on a running system, particularly one with active users, is asking for trouble.
Another method is to compile the system in multi-user
mode, and then drop into single user mode for the
installation. With this method, hold off on the following
steps until the build has completed. Drop to single user mode
in order to run installkernel
or
installworld
.
To enter single user mode from a running system:
#
shutdown now
Alternatively, reboot the system, and at the boot prompt, select the “single user” option. Once at the single user mode shell prompt, run:
#
fsck -p
#
mount -u /
#
mount -a -t ufs
#
swapon -a
This checks the file systems, remounts
/
read/write, mounts all the other UFS
file systems referenced in /etc/fstab
,
and turns swapping on.
If the CMOS clock is set to local time and not to GMT (this is true if the output of date(1) does not show the correct time and zone), run the following command:
#
adjkerntz -i
This ensures that the local time-zone settings get set up correctly.
As parts of the system are rebuilt, they are, by default,
placed in subdirectories of /usr/obj
.
The directories shadow those under
/usr/src
.
To speed up the make buildworld
process, and possibly save some dependency headaches,
remove this directory if it already exists.
Some files below /usr/obj
may have
the immutable flag set which must be removed first using
chflags(1).
#
cd /usr/obj
#
chflags -R noschg *
#
rm -rf *
It is a good idea to save the output from running make(1) to a file. If something goes wrong, a copy of the error message can be posted to one of the FreeBSD mailing lists.
The easiest way to do this is to use script(1)
with a parameter that specifies the name of the file to save
all output to. Run this command immediately before
rebuilding the world, and then type
exit
when the process has
finished:
#
script /var/tmp/mw.out
Script started, output file is /var/tmp/mw.out
#
make TARGET
… compile, compile, compile …
#
exit
Script done, …Do not save the output in /tmp
as this directory may be
cleared at next reboot. A better place to save the file is
/var/tmp
or in
root
's home directory.
While in /usr/src
type:
#
cd /usr/src
To rebuild the world, use make(1). This command
reads instructions from the Makefile
,
which describes how the programs that comprise FreeBSD should
be built and the order in which they should be built.
The general format of the command is as follows:
#
make -x
-DVARIABLE
target
In this example,
-
is an option
passed to make(1). Refer to make(1) for an
examples of available options.x
-D
passes a variable to the VARIABLE
Makefile
. The
behavior of the Makefile
is controlled
by these variables. These are the same variables as are set
in /etc/make.conf
, and this provides
another way of setting them. For example:
#
make -DNO_PROFILE target
is another way of specifying that profiled libraries should not be built, and corresponds with the
line in /etc/make.conf
.
target
tells make(1)
what to do. Each Makefile
defines a
number of different “targets”, and the choice
of target determines what happens.
Some targets listed in the
Makefile
are used by the build process
to break out the steps necessary to rebuild the system into
a number of sub-steps.
Most of the time, no parameters need to be passed to make(1) and the command looks like this:
#
make target
Where target
is one of many
build options. The first target should always be
buildworld
.
As the names imply, buildworld
builds a complete new tree under
/usr/obj
and
installworld
installs this tree on
the current machine.
Having separate options is useful for two reasons.
First, it allows for a “self hosted” build that
does not affect any components of a running system. Because
of this, buildworld
can be run on a
machine running in multi-user mode with no fear of
ill-effects. It is still recommended that
installworld
be run in part in
single user mode, though.
Secondly, it allows NFS mounts to be used to upgrade
multiple machines on a network. If order to upgrade three
machines, A
, B
and
C
, run make buildworld
and make installworld
on
A
. B
and
C
should then NFS mount
/usr/src
and
/usr/obj
from A
, and
run make installworld
to install the
results of the build on B
and
C
.
Although the world
target still
exists, users are strongly encouraged not to use it.
Instead, run:
#
make buildworld
It is possible to specify -j
which
will cause make
to spawn several
simultaneous processes. This is most useful on multi-CPU
machines. However, since much of the compiling process is
I/O bound rather than CPU bound, it is also useful on single
CPU machines.
On a typical single-CPU machine, run:
#
make -j4 buildworld
make(1) will then have up to 4 processes running at any one time. Empirical evidence posted to the mailing lists shows this generally gives the best performance benefit.
On a multi-CPU machine using an SMP configured kernel, try values between 6 and 10 and see how they speed things up.
To take full advantage of the new system, recompile the kernel. This is practically a necessity, as certain memory structures may have changed, and programs like ps(1) and top(1) will fail to work until the kernel and source code versions are the same.
The simplest, safest way to do this is to build and
install a kernel based on GENERIC
. While
GENERIC
may not have all the necessary
devices for the system, it should contain everything necessary
to boot the system back to single user mode. This is a good
test that the new system works properly. After booting from
GENERIC
and verifying that the system
works, a new kernel can be built based on a custom kernel
configuration file.
On FreeBSD it is important to build world before building a new kernel.
To build a custom kernel with an existing customized
configuration file, use
KERNCONF=
:MYKERNEL
#
cd /usr/src
#
make buildkernel KERNCONF=MYKERNEL
#
make installkernel KERNCONF=MYKERNEL
If kern.securelevel
has been raised
above 1 and noschg
or
similar flags have been set on the kernel binary, drop into
single user mode to use
installkernel
. Otherwise, both these
commands can be run from multi user mode without problems.
See init(8) for details about
kern.securelevel
and chflags(1) for
details about the various file flags.
Reboot into single user mode to test that the new kernel works using the instructions in Section 24.7.6, “Drop to Single User Mode”.
Next, use installworld
to install
the new system binaries:
#
cd /usr/src
#
make installworld
If variables were specified to
make buildworld
, specify the same
variables to make installworld
. However,
-j
must never be used with
installworld
.
For example, if you ran:
#
make -DNO_PROFILE buildworld
install the results with:
#
make -DNO_PROFILE installworld
otherwise, the command will try to install profiled
libraries that were not built during the
make buildworld
phase.
Remaking the world will not update certain directories,
such as /etc
,
/var
and
/usr
, with
new or changed configuration files.
The simplest way to update the files in these directories
is to use mergemaster(8). Be sure to first make a backup
of /etc
in case anything goes
wrong.
mergemaster(8) is a Bourne script to aid in
determining the differences between the configuration files
in /etc
, and the
configuration files in the source tree
/usr/src/etc
. This
is the recommended solution for keeping the system
configuration files up to date with those located in the
source tree.
To begin, type mergemaster
and it
will build a temporary root environment, from
/
down, and populate it with various
system configuration files. Those files are then compared
to the ones currently installed in the system. Files that
differ will be shown in diff(1) format, with the
+
sign representing added or modified
lines, and -
representing lines that will
be either removed completely, or replaced with a new file.
Refer to diff(1) for more information about the
diff(1) syntax and how file differences are
shown.
mergemaster(8) will then display each file that differs, and present the options of either deleting the new file, referred to as the temporary file, installing the temporary file in its unmodified state, merging the temporary file with the currently installed file, or viewing the diff(1) results again.
Choosing to delete the temporary file will tell mergemaster(8) to keep the current file unchanged and to delete the new version. This option is not recommended, unless there is no reason to change the current file. To get help at any time, type ? at the mergemaster(8) prompt. If the user chooses to skip a file, it will be presented again after all other files have been dealt with.
Choosing to install the unmodified temporary file will replace the current file with the new one. For most unmodified files, this is the best option.
Choosing to merge the file will present a text editor, and the contents of both files. The files can be merged by reviewing both files side by side on the screen, and choosing parts from both to create a finished product. When the files are compared side by side, l selects the left contents and r selects contents from the right. The final output will be a file consisting of both parts, which can then be installed. This option is customarily used for files where settings have been modified by the user.
Choosing to view the diff(1) results again will display the file differences just like mergemaster(8) did before prompting an option.
After mergemaster(8) is done with the system files, it will prompt for other options. mergemaster(8) may prompt to rebuild the password file and will finish up with an option to remove left-over temporary files.
To perform the update manually instead, do not just copy
over the files from
/usr/src/etc
to
/etc
and expect it to
work. Some files must be “installed” first as
/usr/src/etc
is not a copy of what
/etc
should look
like. In addition, some files that should be in
/etc
are not in
/usr/src/etc
.
If you are using mergemaster(8) (as recommended), you can skip forward to the next section.
The simplest way to merge files by hand is to install the files into a new directory, and then work through them looking for differences.
/etc
: It is recommended to first copy the existing
/etc
somewhere
safe, like so:
#
cp -Rp /etc /etc.old
where -R
does a recursive copy and
-p
preserves times and the ownerships on
files.
Build a temporary set of directories into which the new
/etc
and other files
can be installed:
#
mkdir /var/tmp/root
#
cd /usr/src/etc
#
make DESTDIR=/var/tmp/root distrib-dirs distribution
This will build the necessary directory structure and
install the files. A lot of the subdirectories that have
been created under /var/tmp/root
are empty and
should be deleted. The simplest way to do this is
to:
#
cd /var/tmp/root
#
find -d . -type d | xargs rmdir 2>/dev/null
This will remove all empty directories while redirecting
standard error to /dev/null
to prevent
the warnings about the directories that are not
empty.
/var/tmp/root
now
contains all the files that should be placed in appropriate
locations below /
.
Go through each of these files, determining how they differ
from the system's existing files.
Some of the files installed into /var/tmp/root
have a
leading “.”. Make sure to use ls
-a
in order to catch them.
The simplest way to compare files is to use diff(1):
#
diff /etc/shells /var/tmp/root/etc/shells
This command will show the differences between the
existing /etc/shells
and the new
/var/tmp/root/etc/shells
. Review the
differences to decide whether to merge in custom changes
or to replace the existing file with the new one.
/var/tmp/root
)
with a Time Stamp, so You Can Easily Compare Differences
Between Versions: Frequently rebuilding world entails frequently
updating /etc
as well, which can be a bit of a chore.
To speed up this process, use the following
procedure to keep a copy of the last set of changed files
that were merged into /etc
.
Make the world as normal. When updating
/etc
and the
other directories, give the target directory a name
based on the current date:
#
mkdir /var/tmp/root-20130214
#
cd /usr/src/etc
#
make DESTDIR=/var/tmp/root-20130214 \
distrib-dirs distribution
Merge in the changes from this directory as
outlined above. Do not remove
the /var/tmp/root-20130214
directory when you have finished.
After downloading the latest version of the
source and remaking it, follow step 1. Create a new
directory, which reflects the new date. This example
uses
/var/tmp/root-20130221
.
Use diff(1) to see the differences that have been made in the intervening week by creating a recursive diff between the two directories:
#
cd /var/tmp
#
diff -r root-20130214 root-20130221
Typically, this will be a much smaller set of
differences than those between /var/tmp/root-20130221/etc
and /etc
.
Because the set of differences is smaller, it is
easier to migrate those changes across into
/etc
.
When finished, remove the older of the two
/var/tmp/root-*
directories:
#
rm -rf /var/tmp/root-20130214
Repeat this process whenever merging
in changes to /etc
.
Use date(1) to automate the generation of the directory names:
#
mkdir /var/tmp/root-`date "+%Y%m%d"`
As a part of the FreeBSD development lifecycle, files and
their contents occasionally become obsolete. This may be
because functionality is implemented elsewhere, the version
number of the library has changed, or it was removed from the
system entirely. This includes old files, libraries, and
directories, which should be removed when updating the system.
The benefit is that the system is not cluttered with old files
which take up unnecessary space on the storage and backup
media. Additionally, if the old library has a security or
stability issue, the system should be updated to the newer
library to keep it safe and to prevent crashes caused by the
old library. Files, directories, and libraries which are
considered obsolete are listed in
/usr/src/ObsoleteFiles.inc
. The
following instructions should be used to remove obsolete files
during the system upgrade process.
After the
make
and the subsequent installworld
mergemaster
have
finished successfully, check for obsolete files and libraries
as follows:
#
cd /usr/src
#
make check-old
If any obsolete files are found, they can be deleted using the following command:
#
make delete-old
Refer to /usr/src/Makefile
for more targets of interest.
A prompt is displayed before deleting each obsolete file.
To skip the prompt and let the system remove these files
automatically, use
BATCH_DELETE_OLD_FILES
:
#
make -DBATCH_DELETE_OLD_FILES delete-old
The same goal can be achieved by piping these commands
through yes
:
#
yes|make delete-old
Verify that everything appears to be in the right place, then reboot the system using shutdown(8):
#
shutdown -r now
Deleting obsolete files will break applications that
still depend on those obsolete files. This is especially
true for old libraries. In most cases, the programs, ports,
or libraries that used the old library need to be recompiled
before make
is
executed.delete-old-libs
Utilities for checking shared library dependencies are
available from the Ports Collection in
sysutils/libchk
or
sysutils/bsdadminscripts
.
Obsolete shared libraries can conflict with newer libraries, causing messages like these:
To solve these problems, determine which port installed the library:
#
pkg_info -W /usr/local/lib/libtiff.so
/usr/local/lib/libtiff.so was installed by package tiff-3.9.4
#
pkg_info -W /usr/local/lib/libXext.so
/usr/local/lib/libXext.so was installed by package libXext-1.1.1,1Then deinstall, rebuild and reinstall the port.
ports-mgmt/portmaster
can
be used to automate this process. After all ports are rebuilt
and no longer use the old libraries, delete the old libraries
using the following command:
#
make delete-old-libs
You should now have successfully upgraded the FreeBSD system. Congratulations.
If things went slightly wrong, it is easy to rebuild a
particular piece of the system. For example, if
/etc/magic
was accidentally deleted as
part of the upgrade or merge of /etc
, file(1) will stop
working. To fix this, run:
#
cd /usr/src/usr.bin/file
#
make all install
24.7.16.1. | Do I need to re-make the world for every change? |
There is no easy answer, as it depends on the nature of the change. For example, if running svn only shows the following files as being updated: src/games/cribbage/instr.c
src/games/sail/pl_main.c
src/release/sysinstall/config.c
src/release/sysinstall/media.c
src/share/mk/bsd.port.mk it probably is not worth rebuilding the entire
world. Instead, go into the appropriate sub-directories
and run At the end of the day, it is your call. Some users re-make the world every fortnight and let changes accumulate over that fortnight. Others only re-make those things that have changed and are careful to spot all the dependencies. It all depends on how often a user wants to upgrade and whether they are tracking FreeBSD-STABLE or FreeBSD-CURRENT. | |
24.7.16.2. | My compile failed with lots of signal 11 (or other signal number) errors. What happened? |
This normally indicates hardware problems. (Re)making world is an effective way to stress test hardware, and will frequently throw up memory problems which normally manifest themselves as the compiler mysteriously aborts. A sure indicator of this occurs when make is restarted and it dies at a different point in the process. To resolve this error, start swapping around the components in the machine to determine which one is failing. | |
24.7.16.3. | Can |
The short answer is yes.
Advances users can instruct
| |
24.7.16.4. | Can interrupted builds be resumed? |
This depends on how far into the process the problem occurs. In general, During the last stage, it is fairly safe to: … fix the problem …
# cd /usr/src
# make -DNO_CLEAN all This will not undo the work of the previous
If you see the message: --------------------------------------------------------------
Building everything..
-------------------------------------------------------------- in the If that message is not displayed, or you are not sure, it is always better to be safe than sorry, and restart the build from scratch. | |
24.7.16.5. | How can I speed up making the world? |
| |
24.7.16.6. | What do I do if something goes wrong? |
Make absolutely sure that the environment has no extraneous cruft from earlier builds: # chflags -R noschg /usr/obj/usr
# rm -rf /usr/obj/usr
# cd /usr/src
# make cleandir
# make cleandir Yes, Then, restart the whole process, starting
with If problems persist, send the error and the
output of |
All FreeBSD documents are available for download at http://ftp.FreeBSD.org/pub/FreeBSD/doc/
Questions that are not answered by the
documentation may be
sent to <freebsd-questions@FreeBSD.org>.
Send questions about this document to <freebsd-doc@FreeBSD.org>.