Compiling the Linux kernel. Linux kernel (receipt and control)
Building the Linux kernel
Hello, dear readers. Today I will tell you about such an interesting activity How to build Linux kernel. Why might you need to assemble the kernel yourself? In fact, there can be many reasons: the need to use additional features kernels, optimize the kernel for your computer, update the kernel to the latest version. In this article, I will demonstrate the process of obtaining source codes, configuring, compiling and installing the Linux kernel into the system, as part of solving the problem of including cryptoloop support (loopback encrypting devices) in the kernel.
Getting the source code
First of all we need to get the source code, this can be done different ways and from different sources. I propose to consider only two: system repositories, the official kernel website. The repositories will most likely contain versions of the kernel older than those on the official website, but these sources should include patches and fixes from the manufacturer of your Linux distribution. This approach is preferable if you do not need any new technology or a feature that is only supported by newer kernels. You can view all versions of the kernel sources that are contained in the repositories of your system by entering in the terminal (valid for Ubuntu Linux, in other distributions the package name may differ):
Apt-cache search linux-source
The command will list the available packages:
As you can see, I only have the current version package and the 3.5 version package (actually Current version kernels are also 3.5). To expand the list of kernels available in this way, it is worth connecting additional repositories. We can get the kernel with the command: sudo apt-get install linux-source
linux-source - the name of the source package, it may be different in your case.
After the command completes, a file will appear in the /usr/src directory, in my case - linux-source-3.5.0.tar.bz2. Let's go to the folder, unpack the archive and, for convenience, create a symbolic link:
Cd /usr/src sudo tar -xjvf linux-source-3.5.0.tar.bz2 sudo ln -s linux-source-3.5.0 linux
If you need the most latest version kernel, you can always download it from the website kernel.org. It is worth noting that the website lays out as stable versions kernels and versions intended for testing and modification (usually their name contains the abbreviation “RC” - Release candidate). If you don't want unnecessary problems with the system, I advise you to download the stable version:
Let's save the archive with the sources in the /usr/src folder. To unpack the resulting archive you may need to install additional utilities:
Sudo apt-get install xz-utils
Now, as in the case of downloading the kernel from the repositories, we must unpack the source archive and create a link:
Cd /usr/src sudo tar -xpJf linux-3.8.5.tar.xz sudo ln -s linux-3.8.5.tar.xz linux
Configuration and compilation.
Now we come to the most interesting part. Before we get started, let's install a few additional packages:
sudo apt-get install build-essential kernel-package libncurses-dev
We will create a new configuration based on the system used in this moment kernels:
Cd /usr/src/linux sudo make oldconfig
If you configure more new version kernels than are in the system, it is likely that parameters have appeared in it that are not in the configuration of the current kernel. In this case, the program will ask you to make a choice, you can leave the default values by simply clicking Enter key. In any case, the configuration is not yet complete. Now we can make the settings we need through the configuration creation menu:
Sudo make menuconfig
The configuration program will start in the terminal:
Here, the configuration options are divided into sections to make them easier to navigate. As I said above, I need to include cryptoloop support in the kernel. To do this, go to the “Device Drivers” section, and from there to the “Block Devices” subsection:
We find the “Cryptoloop Support” parameter, next to it there is the letter “M” which means that support for encryption devices will be added as a kernel module that can be enabled with the modprobe command. We need to include support for this technology directly in the kernel so that it is always supported. We move the focus to the “Cryptoloop Support” parameter and press the spacebar. The letter “M” should be replaced by the symbol “*”, this means that support for this technology will be “hardwired” into the kernel. Be careful, a space means that the technology will not be supported at all.
Press the “Tab” key and click on the “Exit” button until you are prompted to save the changes:
We answer “Yes”. We have successfully completed the configuration!
Let's start compiling. First we delete the files remaining from previous builds, if you are starting the build for the first time, it is not necessary to run this command: sudo make-kpkg clean
Let's start the compilation:
Sudo make-kpkg -j4 --initrd --append-to-version=-mykernel kernel_image kernel_headers
J4 - flag indicates how many threads to use for compilation. Will greatly speed up compilation multi-core processors. The number 4 here indicates 4 threads. Install as many threads as the number of cores of your processor that the system “sees”.
-mykernel - a prefix indicating that the kernel was built manually, you can change it, it actually doesn’t affect anything.
Now the compilation process has begun. It can take from 10 minutes to several hours depending on how powerful your computer is:
After compilation is complete, two files with the “deb” extension should appear in the /usr/src directory. They are installation packages of our new kernel and can be installed using the dpkg utility:
sudo dpkg -i linux-image-3.8.5-mykernel_3.8.5-mykernel-10.00.Custom_i386.deb
sudo dpkg -i linux-headers-3.8.5-mykernel_3.8.5-mykernel-10.00.Custom_i386.deb
Congratulations! The kernel is installed, now the system will boot with this kernel by default, but if you have problems with the new kernel, you can always boot with the old one by selecting it on the boot screen - Grub. With this, I conclude today’s article and wish you success, dear readers!
In a large family of operating systems based on GNU/Linux. Surely you have heard that these codes are open, freely distributed and free. They say, take whoever you want, but only the conditions GPL licenses observe, which is not at all difficult. However, few people explain clearly enough what is the essence of this phenomenon, what is its meaning. Therefore, we will attempt such an explanation.
The essence in brief
It all started in 1991, when a Finnish student Linus Torvalds released the kernel codes for the new operating system into the public domain Linux systems. Why in the open? Because he supported. But you probably already know this (or easily find out). We will pay attention to points that require clear classification.
Linux
Linux is not an operating system, but just a kernel. A set of software solutions necessary to start a computer and the functioning of its components (“hardware”), the basis for the functioning of other programs.
GNU
GNU - kit simple applications, which existed even before the appearance of the above kernel. These programs allow a person to carry out at least some interaction with the computer, and not just stare at the screen. The source code is also open, of course.
GNU/Linux is an OS, not just a kernel. Instead of GNU there may be something else, for example Dalvik in Android.
Drivers
Technology is developing, the number of computer hardware is growing, equipment is evolving. And every product needs a driver to work. So, some drivers are screwed directly to the core. If they are free (Free Software), like GNU and Linux, then the codes are certainly open.
Well, when there are no suitable free drivers, then nothing can be done, you have to install proprietary ones. Whether their codes are open depends only on the hardware manufacturers.
Applications
Custom apps related to category Open Source, are often manufactured in versions for different operating systems. They are not part of Linux. True, some are standard for a particular distribution or graphical shell, but are not included in the kernel.
Naturally, the codes for all options are open - for all supported operating systems. The same situation occurs with various utilities.
Who makes it
Linux kernel improved by a group of enthusiasts. Sometimes Linus Torvalds himself takes part. The kernel code, packed in an archive, can be downloaded from kernel.org for subsequent independent compilation.
Drivers, if they are free, are also often made by communities. For printer, scanner, video card, Wi-Fi adapter... In general, there is a lot for what.
For example, the Gutenprint package, which is a whole set of drivers for many printer models. Moreover, the print quality is often comparable to the indicators produced when using “native” drivers from manufacturers.
Sometimes hardware manufacturers themselves open the code under some suitable license, such as GPL or BSD. Such events usually cause indescribable joy among Open Source supporters.
As you may already guess, custom applications are also created either by communities or by individual enthusiasts. However, commercial companies also like to advertise themselves by giving the public part of their products in the form of Free Software. A striking example: office suite OpenOffice.org was published by Oracle for a long time.
Moreover, some companies even make entire distribution kits. Red Hat, SuSE, Xandros charge money for ready-to-use binary assemblies, but they do not have the right to hide the codes. That is, these codes, no matter how they are processed, must remain open. This is a requirement of the GPL license.
Who uses it
A programmer looks at the software and thinks: “It’s a good thing, but it can be done better!” Downloads an archive with code from the developer’s website and improves it. He is joined by a group of specialists who wish to participate, and a new project is born.
This is how “forks” appear (from the English “fork”, which in in this case translated as “branch”). New programs based on the code base of existing ones.
For example, from good audio player Amarok made even better - Clementine. And from the package office applications OpenOffice.org - LibreOffice, rapidly developing and very promising.
So, entire operating systems are cloned using this principle. The free CentOS OS is compiled from the source codes of the paid Red Hat Enterprise Linux. Of course, the bosses at Red Hat are probably biting their elbows in frustration, but they can’t do anything since they don’t own the source code.
However, in this case, the modification comes down mainly to cutting out registered logos, but without the mandatory openness of the code, the very existence of CentOS would be impossible in principle.
Conclusion
Open source is a fundamental concept of Linux in particular and of all Free Software in general. The codes can be used for your own projects, checked for harmlessness through the efforts of the community, studied, improved your skills by participating in the development, improved and assisted people in their noble cause.
There is no binary assembly of software that is important to you for a specific GNU/Linux distribution? Is the driver not included in the kernel? We took the archive with the source code, unpacked it, read the assembly instructions, compiled it, installed it - and use it. You do not depend on the manufacturer, are not tied to a specific operating system- this is real freedom.
Previous publications:
The Linux kernel contains over 13 million lines of code and is one of the most major projects open source in the world. So what is the Linux kernel and what is it used for?
The core is the most low level software that interacts with computer hardware. It is responsible for the interaction of all applications running in user space up to physical equipment. Also allows processes known as services to receive information from each other using the IPC system.
Types and versions of the kernel
You already know what the Linux kernel is, but what types of kernels are there? Eat various ways and architectural considerations when building kernels from scratch. Most kernels can be one of three types: monolithic kernel, microkernel, and hybrid. The Linux kernel is a monolithic kernel, while Windows kernels and OS X hybrid. Let's take an overview of these three types of kernels.
Microkernel
Microkernels implement an approach in which they only manage what they are supposed to: CPU, memory, and IPC. Almost everything else on the computer is treated as an accessory and handled in user mode. Microkernels have the advantage of portability; they can be used on other hardware, and even another operating system, as long as the OS tries to access hardware in a compatible manner.
Microkernels also have very small size and are more secure because most processes run in user mode with minimal privileges.
pros
- Portability
- Small size
- Low memory consumption
- Safety
Minuses
- Hardware accessible via drivers
- Hardware is slower because drivers run in user mode
- Processes must wait their turn to receive information
- Processes cannot access other processes without waiting
Monolithic core
Monolithic kernels are the opposite of microkernels because they cover not only the processor, memory and IPC, but also include things like device drivers, file system management, I/O system. Monolithic kernels give better access to the hardware and enable better multitasking because if a program needs to get information from memory or another process, it doesn't have to wait in a queue. But this can cause some problems, because many things are performed in superuser mode. And this can harm the system if done incorrectly.
Pros:
- More direct access to hardware
- Easier exchange of data between processes
- Processes respond faster
Minuses:
- Big size
- Takes up a lot of RAM
- Less secure
Hybrid core
Hybrid kernels can choose what to work with in user mode and what in kernel space. Often device drivers and file systems are in user space, and IPC and system calls are in kernel space. This solution takes the best of both previous ones, but requires more work from equipment manufacturers. Because all responsibility for the drivers now lies with them.
pros
- Ability to choose what will work in kernel and user space
- Smaller in size than a monolithic core
- More flexible
Minuses
- May work slower
- Device drivers are released by manufacturers
Where are the kernel files stored?
Where is the Linux kernel located? Files Ubuntu kernels or any other Linux distribution are located in the /boot folder and are called vmlinuz version. The name vmlinuz comes from the Unix era. In the sixties, kernels were usually called simply Unix, in the 90s years Linux The kernels were also called Linux.
When was it developed to make multitasking easier? virtual memory, the letters vm appeared before the file name to show that the kernel supports this technology. For some time the kernel was called vmlinux, but then the image no longer fit into memory bootstrap, and was compressed. After this, the last letter x was changed to z to indicate that zlib compression was used. This particular compression is not always used; sometimes you can find LZMA or BZIP2, so some kernels are simply called zImage.
The version number consists of three digits, the version number of the Linux kernel, your version number and patches or fixes.
The /boot package contains not only the Linux kernel, but also files such as initrd.img and system.map. Initrd is used as a small virtual disk, which extracts and executes the actual kernel file. The System.map file is used to manage memory while the kernel is not yet loaded, and configuration files can specify which kernel modules are included in the kernel image when built.
Linux kernel architecture
Since the Linux kernel is a monolithic structure, it is larger and much more complex than other types of kernels. This design feature attracted a lot of controversy in the early days of Linux and still carries some controversy today design flaws inherent in monolithic kernels.
But to get around these shortcomings, the Linux kernel developers did one thing - kernel modules that can be loaded at runtime. This means you can add and remove kernel components on the fly. Everything can go beyond adding hardware functionality, you can run server processes, enable virtualization, and completely replace the kernel without a reboot.
Imagine being able to install a package Windows updates without the need for constant reboots.
Kernel modules
What if Windows already had everything necessary drivers by default, but you could only enable the ones you need? This is exactly the principle that Linux kernel modules implement. Kernel modules, also known as loadable modules (LKMs), are essential for keeping the kernel running with all the hardware without using up all the RAM.
The module expands functionality basic kernel for devices, file systems, system calls. Loadable modules have a .ko extension and are usually stored in the /lib/modules/ directory. Thanks to its modular nature, you can customize the kernel very easily by installing and loading modules. Automatic download or module unloading can be configured in configuration files or upload and download on the fly using special commands.
Third-party, proprietary, closed-source modules are available on some distributions such as Ubuntu, but they are not shipped by default and must be installed manually. For example, the developers of the NVIDIA video driver do not provide source code, but instead they have compiled their own modules in the .ko format. Although these modules appear to be free, they are not free. That's why they are not included in many distributions by default. The developers believe that there is no need to pollute the kernel with proprietary software.
Now you are closer to answering the question what is the Linux kernel. The core is not magic. It is very necessary for the operation of any computer. The Linux kernel is different from OS X and Windows because it includes all the drivers and does a lot of things supported out of the box. Now you know a little more about how your software works and what files are used to do it.
In this step by step guide You will learn how to correctly build and install the branch kernel >2.6 on the Ubuntu OS family.
Step 1: Obtaining the kernel source code
The Ubuntu kernel sources can be obtained in two ways:
By installing the archive from the repository, with automatic application of the latest official patches. This will download a package of ~150 MB in size. current folder. To get the sources of the kernel, the version of which is installed on the computer, run the command: apt-get source linux-image-`uname -r`
Or instead of `uname -r` you can specify specific version from those available in the repository.
The list of versions available in the repository can be seen by typing the command: “apt-get source linux-image-” and, without pressing Enter, press twice Tab key.
Don't forget to turn on general access to the sources in the repository (System Settings → Programs and updates → Software Ubuntu → Source code). You can do this from the console by uncommenting the lines starting with deb-src in the /etc/apt/sources.list file, and then perform the update with the command: “sudo apt-get update”.
The latest version of the kernel is available via git. The size of the downloaded package is ~500-800 MB. git clone git://kernel.ubuntu.com/ubuntu/ubuntu-
Where
Git clone git://kernel.ubuntu.com/ubuntu/ubuntu-xenial.git
Other kernels
There are also kernels that are not guaranteed to work in Ubuntu. For example, there is a known problem with a number of popular system applications(in particular NVidia drivers, VirtualBox), which, when installed, are compiled for the installed kernel. Therefore, to install them on a kernel that is non-standard for this Ubuntu versions(for example, Ubuntu 16.04 comes with kernel 4.4.0), they may require separate manual compilation or special patches, and latest versions The application may not support kernels from kernel.org at all.
Archive with basic versions without patches, i.e. for example "4.8.0", "4.8.10": sudo apt-get install linux-source
Unpack the resulting archive using the commands:
Cd ~/ tar -xjf linux-2.6.x.y.tar.bz2
Or in the case of linux-source:
Cd /usr/src tar -xjf linux-source-2.6.x.y.tar.bz2
Step 2. Obtaining the necessary packages for assembly
This step must be performed only if the kernel is assembled on the computer for the first time
Run the following commands to install the core packages:
Sudo apt-get update sudo apt-get build-dep linux sudo apt-get install kernel-package
config- traditional configuration method. The program displays the configuration options one at a time, prompting you to set a different value for each of them. Not recommended for inexperienced users.
oldconfig- the configuration file is created automatically based on the current kernel configuration. Recommended for beginners.
defconfig- the configuration file is created automatically, based on the default values.
menuconfig- pseudo-graphical interface of manual configuration, does not require serial input parameter values. Recommended for use in a terminal.
xconfig- graphical (X) interface for manual configuration, does not require sequential input of parameter values.
gconfig- graphical (GTK+) interface for manual configuration, does not require sequential input of parameter values. Recommended for use in the GNOME environment.
localmodconfig- a configuration file that is created automatically, which includes only what is needed for a given specific device. When this command is called, most of the core will be modulated
In case you want to use config, oldconfig, defconfig, localmodconfig or localesconfig, you don't need any more additional packages. In the case of the remaining three options, you must also install additional packages.
menuconfig execute next command:
Sudo apt-get install libncurses5-dev
To install packages required for use gconfig run the following command:
Sudo apt-get install libgtk2.0-dev libglib2.0-dev libglade2-dev
To install packages required for use xconfig run the following command:
Before Ubuntu 12.04: sudo apt-get install qt3-dev-tools libqt3-mt-dev
Sudo apt-get install libqt4-dev
Step 3: Applying patches
This step is optional.
Official patches have already been applied to the sources if the kernel was obtained using the command described above:
Apt-get source linux-image-`uname -r`
If you have never applied patches to source code then run the following command:
Sudo apt-get install patch
This command will install the patch program, which is necessary for, as you might guess, applying patches. Now download the patch file to the folder where you extracted the kernel. It could be either archive file(eg Bzip2 or Gzip), or an uncompressed patch file.
At this point, it is assumed that you have already saved the file to the folder where you previously unpacked the kernel and installed the patch program.
If the file you downloaded was in Gzip (*.gz) format, then run the following command to extract the contents of the archive:
Gunzip patch-2.6.x.y.gz
If the file you downloaded was in Bzip2 (*.bz2) format, then run the following command to extract the contents of the archive:
Bunzip2 patch-2.6.x.y.bz2
where 2.6.x.y is the kernel patch version. The appropriate commands will extract the patch file into the kernel source folder. Before applying the patch, you need to make sure that it will work without errors. To do this, run the command:
Patch -p1 -i patch-2.6.x.y --dry-run
where 2.6.x.y is the kernel patch version. This command will simulate applying a patch without changing the files themselves.
If no errors occur during its execution, then the changes can be safely implemented into the files themselves. To do this, run the command:
Patch -p1 -i patch-2.6.x.y
where 2.6.x.y is the kernel patch version. If there were no errors, then the patch was successfully applied to the source code.
Attention! Before applying the patch, perform the following steps: 1. Download from http://www.kernel.org the patch of the same version as your sources. 2. Run the following command: patch -p1 -R where 2.6.x.y is the version of the patch and your sources Go to the folder where you extracted the kernel by running the command Cd ~/linux-2.6.x.y where 2.6.x.y is the version of the kernel you downloaded. At this point, you should have already decided on a kernel configuration method (if not, then check them out in the “Obtaining the packages needed to build” section). Depending on this, run the following command to run your chosen configuration method: config- traditional configuration method. The program displays the configuration options one at a time, prompting you to set a different value for each of them. Called by make config command oldconfig- the configuration file is created automatically based on the current kernel configuration. Recommended for beginners. Called by make oldconfig defconfig- the configuration file is created automatically, based on the default values for this particular architecture. Called by make defconfig menuconfig- pseudo-graphical interface for manual configuration, does not require sequential input of parameter values. Recommended for use in a terminal. Call: make menuconfig gconfig And xconfig- graphical configurators for manual configuration. Call: make gconfig Make xconfig respectively localmodconfig And localesconfig- automatic configurators. The config is created based on the currently called modules and the running kernel. The difference between these two configurators is the number of modules. In the first case, there will be at least 50% of the core, and in the second, no more than 2 modules. Call: make localmodconfig Make localesconfig respectively Once called, the corresponding configuration program will be launched. Make the necessary settings according to your needs, save the configuration file and proceed to the next step. So, the preparations are complete. Now you can start the kernel build process. To do this, run the command: Fakeroot make-kpkg -j 5 --initrd --append-to-version=-custom kernel_image kernel_headers #-j<количество ядер процессора>+1
Building the kernel can take from 20 minutes to several hours, depending on the kernel configuration and technical parameters of the computer. Assembly with a multi-core processor can be several times faster When the kernel build is complete, two deb packages will appear in your home folder. They need to be installed. To do this, run the commands: Cd ~/ sudo dpkg -i linux-image-2.6.x.y-custom_2.6.x.y-custom-10.00.Custom_arc.deb sudo dpkg -i linux-headers-2.6.x.y-custom_2.6.x.y-custom-10.00. Custom_arc.deb where 2.6.x.y is the version of the assembled kernel, arc is the processor architecture (i386 - 32-bit, amd64 - 64-bit). and find these same two packages. To work correctly, Ubuntu requires an initial RAM disk image. To create it, run the command: Sudo update-initramfs -c -k 2.6.x.y-custom where 2.6.x.y is the version of the compiled kernel. To ensure that the new kernel version is available for selection when the computer boots, run the following command: Sudo update-grub The file menu.lst (for GRUB version 1) or grub.cfg (for GRUB version 2) will be updated according to the availability of installed operating systems and kernel images. Kernel assembly and installation completed successfully! Now restart your computer and try to boot the system with the new kernel. To make sure the system is running with the new kernel, run the command Uname -r It will display the kernel version being used. If everything is done correctly, then you can delete the source code archives and the entire linux-2.6.x.y directory in your home folder. This will free up about 5 GB on your hard drive (the amount of space freed depends on your build settings). This completes the assembly and installation process, congratulations!Step 4. Configuration of the future kernel build
Step 5: Building the Kernel
Step 6: Installing Kernel Images and Headers
If you don't know the exact name of the package, list the files in your home directory with the commandStep 7. Generating the initial RAM disk
Step 8: Updating the GRUB Boot Loader Configuration
Step 9: Checking the Kernel
