9.2. Kernel Source Build

We now look at how the kernel is compiled into a binary image and gets loaded into memory prior to execution. As a kernel developer, you will be heavily involved with the source code. It is necessary to understand how to navigate the source code and how to edit the build system so you can add your changes.

This chapter is a roadmap to get you from downloading the source code to compiling a kernel image that loads. We cover how the kernel image is created. This is not a detailed, step-by-step instruction manual. There is much comprehensive documentation online about how to build a kernel image, such as the kernel HOWTO (www.tldp.org/HOWTO/Kernel-HOWTO/), which is currently under review. This is instead intended to provide the kind of information you need in order to incorporate changes into the build system.

Among developers, build systems or Makefiles are never a source of great interest, but as such, we need to understand the kernel build system and how to update it to illustrate changes to the source code. With the 2.6 kernel version, you now have more tools to help you understand all the options that surround the kernel build system. Also, the build system has been significantly cleaned up and redesigned as well as being more effectively documented.

This section covers how the source code is laid out and how the kernel build and Makefiles work. The first step is to get the source code. We start by describing the source code structure and where to get it.

9.2.1. Source Explained

The site for Linus's official code release is www.kernel.org. The source is available to download in a .tar.gz file with gzip compression or a .tar.bz2 file with bzip2 compression. These packages contain the source code to all the available architectures.

When a kernel developer makes a change to the kernel source, he submits it to the kernel maintainer. The maintainer then decides whether the changes get incorporated into the next stable tree. Cutting-edge PPC development used to occur in a separate tree maintained at www.penguinppc.org. The changes made in the PPC tree would then be submitted to the main tree into which they were (mostly) eventually incorporated. Currently, the Linux PPC community is moving toward working directly on the main tree.

The location of source code is somewhat dependent on your distribution. For example, in a Red Hat system, source code is placed (whether it is by default install or by an RPM) under /usr/src/linux-<version>/. If you are cross- compilingthat is, building the kernel for an architecture different to the one you are doing the actual compiling inthe location of the source code might be somewhere under /opt/<distribution name> on your host or alternatively in a root filesystem image the user chroots into. For example, Montavista, which is a distribution geared toward the embedded Linux market, stores the source code (and the cross compilers) under /opt/mvista/ by default.

In this section, the root of the source code filesystem is referred to simply as the root. In the Red Hat distribution, the root of the source code is located under /usr/src/linux-<version>. Figure 9.4 details the hierarchical layout of the source code.

Figure 9.4. Source Code Layout

The source code is divided into architecture-dependent and architecture- independent portions. The arch/ directory under the root holds all the code that is architecture dependent. Source code downloaded from a mirror of kernel.org contains all the supported architectures listed under this subdirectory. Every supported architecture has a directory under arch/ that contains a further breakdown of the architecture-dependent code. Figure 9.5 shows the supported architectures by means of displaying the listing under the arch/ directory.

We begin by looking at the structure of the architecture-independent portion of the source code to understand its breakdown. We then present an overview of the architecture-dependent portion of the source code, followed by a brief summary of miscellaneous files that pertain to neither category.

9.2.1.1. Architecture-Independent Code

The architecture-independent portion of the source code is divided into 11 subdirectories that follow a sensible categorization by functionality. Table 9.2 overviews these subdirectories.

Table 9.2. Architecture-Independent Subdirectories

Subdirectory	Description
crypto	Holds code for cryptographic API and various encrypting/decrypting algorithms.
drivers	Code for device drivers.
fs	Code for VFS and all the filesystems supported by Linux.
include	The header files. This directory has a series of subdirectories starting with the prefix asm. These directories hold the architecture-specific header files. The remaining directories hold architecture-independent header files.
init	The architecture-independent portion of the bootstrapping code and initialization code.
ipc	Code for interprocess communication (IPC) support.
kernel	Code for kernel space specific code.
lib	Code for helper functions.
mm	Code for the memory manager.
net	Code to support the various networking protocols.
sound	Code for sound system support.

Throughout the various chapters, we have been exploring source code that is located in one or more of these subdirectories. To put them in the proper context, the following sections provide a cursory look at some of the subdirectories. We leave out the ones we have not looked at in more detail.

fs/

The fs/ directory is further subdivided into C source files that support the VFS internals and subdirectories for each supported filesystem. As Chapter 7, "Scheduling and Kernel Synchronization," details, the VFS is the abstraction layer for the various types of filesystems. The code found in each of these subdirectories consists of the code bridging the gap between the storage device and the VFS abstraction layer.

init/

The init/ directory contains all the code necessary for system initialization. During the execution of this code, all the kernel subsystems are initialized and initial processes are created.

kernel/

The bulk of the architecture-independent kernel code is located in the kernel/ directory. Most of the kernel subsystems have their code under here. Some, such as filesystems and memory, have their own directories at the same level as kernel/. The filenames are fairly self-explanatory with respect to the code they contain.

mm/

The mm/ directory holds the memory-management code. We looked at examples of this code in Chapter 4, "Memory Management."

9.2.1.2. Architecture-Dependent Code

The architecture-dependent code is the portion of the kernel source that is directly tied to reference the actual hardware. One thing to remember in your travails through this portion of the code is that Linux was originally developed for the x86. To minimize the complexity of the porting efforts, some of the x86-centric terminology was retained in variable names and global kernel structures. If you look through the PPC code and see names that refer to address translation modes that don't exist in PPC, don't panic.

Doing a listing for both arch/i386/ and arch/ppc, you notice three files that they each have in common: defconfig, Kconfig, and Makefile. These files are tied into the infrastructure of the kernel build system. The purpose of these three files is made clear in Section 9.2.2, "Building the Kernel Image."

Table 9.3 gives an overview of the files and directories shown in a listing of arch/ppc. Once you have gone over the structure of Makefiles and Kconfig files, it is useful to browse through these files in each of the subdirectories to become familiar with where code is located.

Table 9.3. arch/ppc/ Source Code Listing

Subdirectory	Description
4xx_io	Source code for MPC4xx-specific I/O parts, in particular, the IBM STB3xxx SICC serial port.
8260_io	Source code for MPC8260-communication options.
8xx_io	Source code for the MPC8xx-communication options.
amiga	Source code for the PowerPC-equipped Amiga computers.
boot	Source code related to PPC bootstrapping. This directory also contains a subdirectory called images, which is where the compiled bootable image is stored.
config	Configuration files for the build of specific PPC platforms and architectures.
kernel	Source code for the kernel subsystem hardware dependencies.
lib	Source code for PPC specific library files.
math-emu	Source code for PPC math emulation.
mm	Source code for the PPC-specific parts of the memory manager. Chapter 6, "Filesystems," discusses this in detail.
platforms	Source code specific to platforms (boards) on which the PPC chips are mounted.
syslib	Part of the source code core for the general hardware-specific subsystems.
xmon	Source code of PPC-specific debugger.

The directories under arch/x86 hold a structure similar to that seen in the PPC architecture-dependent directory. Table 9.4 summarizes the various subdirectories.

Table 9.4. arch/x86 Source Code Listing

Subdirectory	Description
boot	Source code related to the x86 bootstrapping and install process.
kernel	Source code for the kernel subsystem hardware dependencies.
lib	Source code for x86-specific library files.
mach-x	Source code for the x86 subarchitectures.
math-emu	Source code for x86 math-emulation functions.
mm	Source code for the x86-specific parts of memory management. Chapter 6 discusses this in detail.
oprofile	Source code for the oprofile kernel profiling tool.
pci	x86 PCI drivers.
power	Source code for x86 power management.

You may be wondering why the two architecture-specific listings are not more similar. The reason is that functional breakdowns that work well in one architecture may not work well in the other. For example, in PPC, PCI drivers vary by platform and subarchitecture, making a simple PCI subdirectory less ideal than for x86.

9.2.1.3. Miscellaneous Files and Directories

In the source root, a few files are not necessarily pertinent either to the architecture-dependent code or the architecture-independent code. Table 9.5 lists these files.

Table 9.5. Miscellaneous Files

File/Directory	Description
COPYING	The GPL license under which Linux is licensed.
CREDITS	List of contributors to the Linux project.
MAINTAINERS	List of maintainers and instructions on submitting kernel changes.
README	Release notes.
REPORTING-BUGS	Describes the procedure for reporting bugs.
documentation/	Directory with partial documentation on various aspects of the Linux kernel and source code. Great source of information, if sometimes slightly out of date.
scripts/	Holds utilities and scripts used during the kernel build process.

9.2.2. Building the Kernel Image

The kernel build system, or kbuild, is the mechanism by which kernel configuration options can be selected when building the kernel. It has been updated for the 2.6 kernel tree. This new kbuild version is much faster than its predecessor and significantly better documented. The kbuild system is highly dependent on the hierarchical structure of the source code.

9.2.2.1. Kernel Configuration Tool

The kernel configuration tool automatically generates the kernel configuration file named .config. This is the first step of the kernel build. The .config file is placed in the source code root; it contains a description of all the kernel options that were selected with the configuration tool. Each kernel build option has a name and value associated with it. The name is in the form CONFIG_<NAME>, where <NAME> is the label with which the option is associated. This variable can hold one of three values: y, m, or n. The y stands for "yes" and indicates that the option should be compiled into the kernel source, or built in. The m stands for "module" and indicates that the option should be compiled as a module separate from the kernel source. If an option is not selected (or its value set to n for "no"), the .config file indicates this by having a comment of the form CONFIG_<NAME> is not set. The .config file options are ordered according to the way they appear in the kernel configuration tool and comments are provided that indicate under what menu the option is found. Let's look at an excerpt of a .config file:

 
 -----------------------------------------------------------------------
 .config
 1  #
 2  # Automatically generated make config: don't edit
 3  #
 4  CONFIG_X86=y
 5  CONFIG_MMU=y
 6  CONFIG_UID16=y
 7  CONFIG_GENERIC_ISA_DMA=y
 8
 9  #
 10  # Code maturity level options
 11  #
 12  CONFIG_EXPERIMENTAL=y
 13  CONFIG_CLEAN_COMPILE=
 14  CONFIG_STANDALONE=y
 15  CONFIG_BROKEN_ON_SMP=y
 16
 17  #
 18  # General setup
 19  #
 20  CONFIG_SWAP=y
 21  CONFIG_SYSVIPC=y
 22  #CONFIG_POSIX_MQUEUE is not set
 23  CONFIG_BSD_PROCESS_ACCT=y
 -----------------------------------------------------------------------
 
 

This .config file indicates that the options from lines 4 to 7 are located under the top level, the options from lines 12 to 15 are located under the Code Maturity Level Options menu, and the options from lines 20 to 23 are under the General Setup menu.

Looking at the menus made available through any of the configuration tools, you see that the first few options are at the root level along with the menu items Code Maturity Level Options and General Setup. The latter two get expanded into a submenu that holds those options listed underneath. This is shown in qconf, which is the configuration tool that executes when we issue a call to make xconfig. The menus the configuration tool shows default to x86. To have it show the PPC-related menus, as shown in Figure 9.6, the parameter ARCH=ppc must be appended at the end of the make xconfig call.

Figure 9.6. qconf Snapshot

The .config file generated by the configuration tool is read by the root Makefile when the image is to be built by the call to make bzImage. The root Makefile also pulls in information provided by the architecture-specific Makefile, which is located under arch/<arch>/. This is done by way of the include directive:

 -----------------------------------------------------------------------
 Makefile
 434  include .config
 ...
 450  include $(srctree)/arch/$(ARCH)/Makefile
 -----------------------------------------------------------------------
 

At this point, the Makefile has already determined what architecture it is compiling for. The root Makefile determines the architecture it is compiling for in three possible ways:

If the architecture being compiled for is different from the native host the compilation is executed on, the CROSS_COMPILE parameter has to be passed, which indicates the prefix of the cross compiler to be used. Alternatively, the Makefile itself can be edited and this variable is given a value. For example, if I compile for a PPC-based processor on an x86 host machine, I would execute the following commands:

 lkp:~#make xconfig ARCH=ppc
 lkp:~#make ARCH=ppc CROSS_COMPILE=ppc-linux-
 

The .config file also generates include/linux/autoconf.h, which #defines the CONFIG_<NAME> values that have been selected and #undefs those that were deselected.

9.2.2.2. Sub-Makefiles

The build system relies on sub-Makefiles that are located under each subdirectory. Each subdirectory's Makefile (called a sub-Makefile or kbuild Makefile) defines rules to build object files from source code files located in that subdirectory and only makes appropriate modifications in that directory. The call to each sub-Makefile is done recursively down the tree going into all subdirectories under init/, drivers/, sound/, net/, lib/, and usr/.

Before the beginning of the recursive make call, kbuild needs to make sure a few things are in place, including updating include/linux/version.h if necessary and setting the symbolic link include/asm to point at the architecture-specific files of the architecture for which we are compiling. For example, if we are compiling for PPC, include/asm points to include/asm-ppc. kbuild also builds include/linux/autoconf.h and include/linux/config. After this is done, kbuild begins to recursively descend down the tree.

If you are a kernel developer and you make an addition to a particular subsystem, you place your files or edits in a specific subdirectory and update the Makefile if necessary to incorporate your changes. If your code is embedded in a file that already existed, you can surround your code within an #ifdef(CONFIG_<NAME>) block. If this value is selected in the .config file, it is #defined in include/ linux/autoconf.h and your changes are included at compile time.

The sub-Makefile lines have a specific format that must be followed to indicate how the object file is to be built. These Makefiles are straightforward because information such as compiler name and libraries are already defined in the root Makefile and the architecture-specific root Makefile, and rules are defined in the scripts/Makefile.*s. The sub-Makefiles build three possible lists:

• $(obj-y) listing the object files that will be linked into built-in.o and later into vmlinux

• $(obj-m) listing the object files that will be built as a module

• $(lib-y) listing the object files that will be built into lib.a

In other words, when we issue a call to make of type make bzImage, kbuild builds all object files in obj-y and links them. The basic line in a sub-Makefile is of the type.

 obj-$(CONFIG_FOO) += foo.o
 

If CONFIG_FOO is set to y in the .config file read by the root Makefile, this line becomes equivalent to obj-y += foo.o. kbuild builds that object file from the corresponding foo.c or foo.S file in that directory according to rules defined in scripts/Makefile.build. (We see more about this file in a moment.) If foo.c or foo.S do not exist, make complaints with

 Make[1]: *** No rule to make target '<subdir>/foo.o', needed by '<subdir>/built-in.o'. Stop.
 

The way that kbuild knows to descend into directories is through explicit additions to obj-y or obj-m. You can add a directory to set obj-y, which indicates that it needs to descend into the specified directory:

 Obj-$(CONFIG_FOO) += /foo
 

If /foo does not exist, make complaints with the following:

 Make[2]: *** No rule to make target '<dir>/foo/Makefile'. Stop.
 

CML2 Where does the configuration program that you navigate when choosing kernel options get the information? The kbuild system depends on CML2, which is a domain-specific language designed for kernel configuration. CML2 creates a rulebase that an interpreter then reads and uses to generate the config file. This file covers the syntax and semantics of the language. The CML2 rulebase that is read by configuration programs is stored in files called defconfig and Kconfig. The defconfig files are found at the root of the architecture-specific directories, arch/*/. The Kconfig files are found in most other subdirectories. The Kconfig files hold information regarding the options created, such as the menu it should be listed under, the help information to provide, the config name value, and whether it can be built-in only or also compiled as a module. For more information about CML2 and Kconfig files, see Documentation/kbuild/kconfig-language.txt.

Let's review what we have seen of the kbuild process. The first step is to call the configuration tool with make xconfig or make xconfig ARCH=ppc, depending on the architecture we want to build for. The selection made in the tool is then stored in the .config file. The top Makefile reads .config when a call such as make bzImage is issued to build the kernel image. The top Makefile then performs the following before descending recursively down the subdirectories:

kbuild then descends the subdirectories, calling make on the sub-Makefiles and creating the object files in each one.

We have seen the structure of the sub-Makefiles. Now, we closely look at the top-level Makefiles and see how they are used to drive the kernel build system.

9.2.2.3. Linux Kernel Makefiles

Linux Makefiles are fairly complex. This section highlights the interrelationship between all the Makefiles in the source tree and explains the make particulars that are implemented in them. However, if you want to expand your knowledge of make, undertaking to understand all the specifics of the kbuild Makefiles is a fantastic way to get started. For more information on make, go to www.gnu.org/software/make/make.html.

In the source tree, virtually every directory has a Makefile. As mentioned in the previous section, the Makefiles in subtrees devoted to a particular category of the source code (or kernel subsystem) are fairly straightforward and merely define target source files to be added to the list that is then looked at to build them. Alongside these, five other Makefiles define rules and execute them. These include the source root Makefile, the arch/$(ARCH)/Makefile, scripts/Makefile.build, scripts/Makefile.clean, and scripts/Makefile. Figure 9.7 shows the relationship between the various Makefiles. We define the relationships to be of the "include" type or of the "execute" type. When we refer to an "include" type relationship, we mean that the Makefile pulls in the information from a file by using the rule include <filename>. When we refer to an "execute" type relationship, we mean that the original Makefile executes a make f call to the secondary Makefile.

Figure 9.7. Makefile Relationships

When we issue a make call at the root of the source tree, we call on the root Makefile. The root Makefile defines variables that are then exported to other Makefiles and issues further make calls in each of the root-level source subdirectories, passing off execution to them.

Calls to the compiler and linker are defined in scripts/Makefile.build. This means that when we descend into subdirectories and build the object by means of a call to make, we are somehow executing a rule defined in Makefile.build. This is done by way of the shorthand call $(Q) $(MAKE) $(build)=<dir>. This rule is the way make is invoked in each subdirectory. The build variable is shorthand for

 Makefile
 1157  build := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.build obj
 -----------------------------------------------------------------------
 

A call to $(Q) $(MAKE) $(build)=fs expands to

 "@ make f /path/to/source/scripts/Makefile.build obj=fs".
 

The scripts/Makefile.build then reads the Makefile of the directory it was passed as parameter (fs, in our example). This sub-Makefile has defined one or more of the lists obj-y, obj-m, lib-y, and others. The file scripts/Makefile.build, along with any definitions from the included scripts/ Makefile.lib, compiles the source files in the subdirectory and descends into any further subdirectories defined in the lists mentioned. The call is the same as what was just described.

Let's see how this works in an example. If, under the configuration tool, we go to the File Systems menu and select Ext3 journalling filesystem support, CONFIG_EXT3_FS will be set to y in the .config file. A snippet of the sub-Makefile corresponding to fs is shown here:

 Makefile
 49  obj-$(CONFIG_EXT3_FS)   += ext3/
 -----------------------------------------------------------------------
 

When make runs through this rule, it evaluates to obj-y += ext3/, making ext3/ one of the elements of obj-y. make, having recognized that this is a subdirectory, calls $(Q) $(MAKE) $(build)=ext3.

$(Q) The $(Q) variable prefixes all $(MAKE) calls. With the 2.6 kernel tree and the cleanup of the kbuild infrastructure, you can suppress the verbose mode of the make output. make prints the command line prior to executing it. When a line is prefixed with the @, the output (or echo) of that line is suppressed: -------------------------------------------------------------------- Makefile 254 ifeq ($(KBUILD_VERBOSE),1) 255 quiet = 256 Q = 257 else 258 quiet=quiet_ 259 Q = @ 260 endif -------------------------------------------------------------------- As we can see in these lines, Q is set to @ if KBUILD_VERBOSE is set to 0, which means that we do not want the compile to be verbose.

After the build process completes, we end up with a kernel image. This bootable, compressed kernel image is called zImage or vmlinuz because the kernel gets compressed with the zlib algorithm. Common Linux conventions also specify the location of the bootable image on the filesystem; the image must be placed in /boot or /. At this point, the kernel image is ready to be loaded into memory by a bootloader.