Differences

This shows you the differences between two versions of the page.

--- cucumber_linux_from_scratch [2018/07/07 16:26]
+++ cucumber_linux_from_scratch [2018/09/15 13:52]
z5t1 [Chapter 1 - Introduction]
@@ Line 1: / Line 1: @@
+====== Cucumber Linux from Scratch ======
+====== Chapter 1 - Introduction ======
+This page details the procedure for building Cucumber Linux from scratch. This process was originally derived from Linux from Scratch and as such shares many similarities with it. The process is broken down into several "chapters" which mirror the chapters in the Linux from Scratch book.
+The build process is also broken down into several phases. Before we begin, here is a brief overview of the phases and chapters, how they line up and what their purposes are:
+^ Phase   ^ Chapter(s)     ^ Purpose      ^
+| "0"     | 1, 2, 3 & 4    | Set up the environment used to build Cucumber Linux. Note that nothing is actually built during this stage. As a result, this is not //technically// considered a phase, hence the quotes around 0. |
+| 1       | 5              | Construct a temporary system in /tools that will be used to build the final system. |
+| 2       | 6              | Build an initial bootstrap system. At the end of phase 2, the system will be capable of running itself (without the /tools directory), but will not be capable of compiling itself. |
+| 3       | 7              | Build the necessary packages for the /cucumber directory of the ports tree to become entirely self hosted. At the end of phase 3, the system will be capable of both running *and* compiling itself; however, it will not be complete yet. |
+| 4       | 8              | Build the remaining packages in the /cucumber directory of the ports tree that were not built in phases 2 or 3. At the end of phase 4, the system will be complete. |
+As a final note, some users may wish to use a different distro (besides Cucumber Linux) to build Cucumber Linux. If you do choose to do this, please know that this is not officially supported. Generally, the only distribution that is officially supported for building a given major version of Cucumber Linux (i.e. 2.x) is the previous stable major version (i.e. 1.x) and the version itself (i.e. 2.x).
+====== Chapter 2 - Preparing the Build Location ======
+===== Introduction =====
+In this chapter, the build environment is prepared. We will create two subdirectories: one for building the temporary system (Chapter 5) and another for building the final system (Chapters 6 & 7).
+===== Creating the Directories =====
+The Cucumber Linux from scratch build will reside in /opt/culfs. Start by creating that directory and changing the working directory to it:
+  mkdir -pv /opt/culfs
+  cd /opt/culfs
+The two subdirectories (lfscript and chroot) will be created when in subsequent chapters as they become necessary.
+====== Chapter 3 - Downloading the Package Sources ======
+===== Introduction =====
+There are two source trees that must be downloaded: the LFScript source tree (which will be used to build chapter 5) and the Cucumber Linux source tree (which will be used to build chapters 6 & 7).
+===== Downloading LFScript =====
+Clone the Git repository for the Cucumber Linux version of LFScript:
+  git clone https://github.com/cucumberlinux/lfscript.git
+If you are building the current development version of Cucumber Linux, then no further action is necessary for this step and you may move on to the Downloading the Cucumber Linux Source Tree section; however, if you are building a different version of Cucumber Linux, it will be necessary to switch to the Git branch for that version. For example, if you are building any version of Cucumber Linux 2.x, run:
+  cd lfscript
+  git checkout 2.x
+  cd ..
+===== Downloading the Cucumber Linux Source Tree =====
+Clone the Git repository for the official Cucumber Linux source tree:
+  install -m 755 -d chroot/usr
+  cd chroot/usr
+  git clone https://github.com/cucumberlinux/ports.git
+If you are building a version of Cucumber Linux other than the current development version, switch to the appropriate Git branch for it:
+  cd ports
+  git checkout 2.0
+  cd ..
+Temporarily change the portmake configuration so it expects to find the ports tree in /opt:
+  if [ -e /etc/portmake.conf ]; then
+    mv /etc/portmake.conf{,.orig}
+  fi
+  echo PORTS_TREE=/opt/culfs/chroot/usr/ports > /etc/portmake.conf
+Download the source tarballs for the phase 2 and 3 packages. This must be done now, as the chroot environment will not have internet access until after phase 3 is built. This step will also verify the integrity of all the downloaded tarballs via checksums and PGP signatures.
+  ports/utilities/tools/portmake download-recursive phase2
+  ports/utilities/tools/portmake download-recursive phase3
+Restore the original portmake.conf.
+  if [ -e /etc/portmake.conf.orig ]; then
+    mv /etc/portmake.conf{.orig,}
+  else
+    rm /etc/portmake.conf
+  fi
+===== Downloading a Couple of Additional Packages =====
+There are two additional packages that must be downloaded now so they are available immediately inside the chroot environment: pkgtools and tar 1.13. Create a directory to store them in:
+  cd ..
+  mkdir sources
+  cd sources
+Download the source archives for them:
+  wget https://github.com/cucumberlinux/pkgtools/archive/c2.0.0.tar.gz
+  wget https://ftp.gnu.org/gnu/tar/tar-1.13.tar.gz
+Note that you *must* download tar version 1.13 specifically. The version of pkgtools that is downloaded should be the latest patch release available for the version of Cucumber Linux you are building. For example, if you are building Cucumber Linux 3.0, then you should use the latest version of pkgtools 3.0.x.
+Finally cd back to /opt/culfs.
+  cd ../..
+====== Chapter 4 - Final Perparations ======
+===== Introduction =====
+In this chapter, we will perform a few additional tasks to prepare for building the temporary system.
+===== Adding the lfs user =====
+Create a directory to download the phase 1 source tarballs to:
+  mkdir /opt/culfs/lfscript/sources
+Ensure that the lfs user has full access to /opt/culfs/lfscript:
+  chown -Rv lfs /opt/culfs/lfscript
+====== Chapter 5 - Building Phase 1 ======
+===== Introduction =====
+This phase is akin to Chapter 5 of Linux from Scratch: [[http://www.linuxfromscratch.org/lfs/view/development/chapter05/introduction.html|Constructing the Temporary System]]. This step will be automated using a modified version of [[https://www.lfscript.org/wiki/index.php/Main_Page|LFScript]].
+This chapter has two parts: first, we will build a tarball for the [[http://www.linuxfromscratch.org/lfs/view/development/chapter04/creatingtoolsdir.html|/tools directory]]. Then, we will extract it to /opt/culfs/chroot/tools and prepare the chroot environment.
+===== Building the Tools Tarball =====
+CD back into the lfscript directory:
+  cd lfscript
+Run the following command to build the tools tarball:
+  ./lfscript -B
+Note that lfscript will exit with a reported failure. This is because lfscript will attempt to continue on after chapter 5 and begin building chapter 6. It will fail at building chapter 6 (because we don't use it for that part on Cucumber Linux), but will still build the tools tarball successfully. Therefore, this error can be safely ignored. When you reach the error message, press ''^C''.
+===== Installing /tools to the Chroot =====
+CD to the chroot environment and extract the tools tarball:
+  cd ../chroot
+  tar -xvJf ../lfscript/packages-*/lfs-*/toolchain.bak.txz
+===== Preparing the Chroot =====
+There are a few additional steps that must be performed before entering the chroot environment.
+==== Set the LFS Variable ====
+Set the LFS variable to the chroot directory:
+  export LFS=/opt/culfs/chroot
+==== Set up the Kernel File Systems ====
+For details about this step see [[http://www.linuxfromscratch.org/lfs/view/8.0/chapter06/kernfs.html]].
+Various file systems exported by the kernel are used to communicate to and from the kernel itself. Create the mount points for them:
+  mkdir -pv $LFS/{dev,proc,sys,run}
+Perform a bind mount for /dev:
+  mount --bind /dev $LFS/dev
+Mount the remaining virtual file systems:
+  mount -vt devpts devpts $LFS/dev/pts -o gid=5,mode=620
+  mount -vt proc proc $LFS/proc
+  mount -vt sysfs sysfs $LFS/sys
+  mount -vt tmpfs tmpfs $LFS/run
+==== Important Note about the Chroot Environment ====
+If at any point you reboot your build system or close out of the shell you are performing the build in, it will be necessary to repeat the steps in the "Preparing the Chroot" section before reentering the chroot environment.
+====== Chapter 6 - Building Phase 2 ======
+===== Introduction =====
+This phase is akin to Chapter 6 of Linux from Scratch: Installing Basic System Software. The majority of this chapter will be automated using the official Cucumber Linux ports tree and the portmake utility; however, a few of the early packages will need to be built by hand.
+At the end of this phase, the system will be capable of running itself (without the /tools directory), but will not be capable of compiling itself yet.
+===== Enter the Chroot =====
+After completing the "Preparing the Chroot" section of Chapter 5, enter the chroot environment:
+  chroot "$LFS" /tools/bin/env -i \
+    HOME=/root                  \
+    TERM="$TERM"                \
+    PS1='\u:\w\$ '              \
+    PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin/:/tools/sbin \
+    /tools/bin/bash --login +h
+===== Final Preparations =====
+==== Create Essential Directories ====
+It is time to create some structure in the LFS file system. Create a standard directory tree by issuing the following commands:
+  mkdir -pv /{bin,boot,etc/{opt,sysconfig},home,lib/firmware,mnt,opt}
+  mkdir -pv /{media/{floppy,cdrom},sbin,srv,var}
+  install -dv -m 0750 /root
+  install -dv -m 1777 /tmp /var/tmp
+  mkdir -pv /usr/{,local/}{bin,include,lib,sbin,src}
+  mkdir -pv /usr/{,local/}share/{color,dict,doc,info,locale,man}
+  mkdir -v  /usr/{,local/}share/{misc,terminfo,zoneinfo}
+  mkdir -v  /usr/libexec
+  mkdir -pv /usr/{,local/}share/man/man{1..8}
+  case $(uname -m) in
+   x86_64) mkdir -v /lib64 ;;
+  esac
+  mkdir -v /var/{log,mail,spool}
+  ln -sv /run /var/run
+  ln -sv /run/lock /var/lock
+  mkdir -pv /var/{opt,cache,lib/{color,misc,locate},local}
+==== Create Essential Symlinks ====
+Some programs use hardwired paths to other programs that do not exist yet. Create some temporary symlinks to satisfy them:
+  install -m 755 -d /bin /usr/{bin,lib} /etc /sbin /var/log
+  ln -sv /tools/bin/{bash,cat,echo,pwd,stty} /bin
+  ln -sv /tools/bin/perl /usr/bin
+  ln -sv /tools/lib/libgcc_s.so{,.1} /usr/lib
+  ln -sv /tools/lib/libstdc++.so{,.6} /usr/lib
+  sed 's/tools/usr/' /tools/lib/libstdc++.la > /usr/lib/libstdc++.la
+  ln -sv bash /bin/sh
+  ln -sv /proc/self/mounts /etc/mtab
+  ln -sv /tools/sbin/installpkg /sbin/
+  ln -sv /tools/sbin/makepkg /sbin/
+  ln -sv /tools/sbin/upgradepkg /sbin/
+  ln -sv /tools/bin/m4 /usr/bin/
+==== Create Essential Files ====
+Create a basic /etc/passwd:
+  cat > /etc/passwd << "EOF"
+  root::0:0:root:/root:/bin/bash
+  bin:x:1:1:bin:/dev/null:/bin/false
+  daemon:x:6:6:Daemon User:/dev/null:/bin/false
+  messagebus:x:18:18:D-Bus Message Daemon User:/var/run/dbus:/bin/false
+  nobody:x:99:99:Unprivileged User:/dev/null:/bin/false
+  EOF
+Create a basic /etc/group:
+  cat > /etc/group << "EOF"
+  root:x:0:
+  bin:x:1:daemon
+  sys:x:2:
+  kmem:x:3:
+  tape:x:4:
+  tty:x:5:
+  daemon:x:6:
+  floppy:x:7:
+  disk:x:8:
+  lp:x:9:
+  dialout:x:10:
+  audio:x:11:
+  video:x:12:
+  utmp:x:13:
+  usb:x:14:
+  cdrom:x:15:
+  adm:x:16:
+  messagebus:x:18:
+  systemd-journal:x:23:
+  input:x:24:
+  mail:x:34:
+  nogroup:x:99:
+  users:x:999:
+  EOF
+==== Initialize Log Files ====
+The login, agetty, and init programs (and others) use a number of log files to record information such as who was logged into the system and when. However, these programs will not write to the log files if they do not already exist. Initialize the log files and give them proper permissions:
+  touch /var/log/{btmp,lastlog,wtmp}
+  chgrp utmp /var/log/lastlog
+  chmod 664  /var/log/lastlog
+  chmod 600  /var/log/btmp
+===== Build a few Packages Manually =====
+The following packages must be built manually: pkgtools, which and tar 1.13.
+First, cd to the directory the sources are in:
+  cd /sources
+==== Install pkgtools ====
+Install pkgtools to /tools. Note: substitute the version of pkgtools you downloaded in chapter 3 for "c2.0.0"
+  tar -xzf c2.0.0.tar.gz
+  cd pkgtools-*
+  make DESTDIR=/tools install
+==== Install Tar 1.13 ====
+Install Tar 1.13 to /tools:
+  cd ..
+  tar -xzf tar-1.13.tar.gz
+  cd tar-1.13
+  ./configure --prefix=/tools --program-suffix=-1.13
+  make
+  make install
+==== "Install" Which ====
+The full fledged version of Which is not necessary at this point, so we will install a minimal script to emulate the essential functionality:
+  cat > /tools/bin/which << "EOF"
+  #!/bin/bash
+  type -pa "$@" | head -n 1 ; exit ${PIPESTATUS[0]}
+  EOF
+  chmod -v 755 /tools/bin/which
+===== Building the Phase 2 Bootstrap =====
+It is necessary to build a small set of bootstrap packages manually before building the rest of phase 2.
+First, create a dummy gpg to use as a workaround. We will not install gpg until later, but certain packages will not build without it installed. This is safe to do since we already verified the integrity of the downloaded sources when we initially downloaded them in chapter 3. It will be placed in /tools to ensure that it is deleted at the end of the build and will not leak into the final system.
+  cat > /tools/bin/gpg << EOF
+  #!/bin/bash
+  exit 0
+  EOF
+  chmod 755 /tools/bin/gpg
+Build and install them:
+  /usr/ports/utilities/tools/portmake install linux-headers
+  /usr/ports/utilities/tools/portmake install man-pages
+  /usr/ports/utilities/tools/portmake install glibc
+Run the test suites for glibc. This step is considered critical and should not be skipped. A few test failures is nothing to be alarmed about; it is usually cause for alarm only if a large percentage of the test fail.
+  cd /tmp/glibc-*/src/glibc-*/build
+  make -j $(nproc) check
+===== Adjusting the Toolchain =====
+This section is based off of [[http://www.linuxfromscratch.org/lfs/view/8.0/chapter06/adjusting.html]].
+==== Perform the Adjustment ====
+First, backup the /tools linker, and replace it with the adjusted linker we made in chapter 5. We'll also create a link to its counterpart in /tools/$(uname -m)-pc-linux-gnu/bin:
+  mv -v /tools/bin/{ld,ld-old}
+  mv -v /tools/$(uname -m)-pc-linux-gnu/bin/{ld,ld-old}
+  mv -v /tools/bin/{ld-new,ld}
+  ln -sv /tools/bin/ld /tools/$(uname -m)-pc-linux-gnu/bin/ld
+Next, amend the GCC specs file so that it points to the new dynamic linker. Simply deleting all instances of “/tools” should leave us with the correct path to the dynamic linker. Also adjust the specs file so that GCC knows where to find the correct headers and Glibc start files. A sed command accomplishes this:
+  case $(uname -m) in
+    x86_64) export LIBDIRSUFFIX=64 ;;
+  esac
+  gcc -dumpspecs | sed -e 's@/tools@@g'                   \
+    -e '/\*startfile_prefix_spec:/{n;s@.*@/usr/lib${LIBDIRSUFFIX}/ @}' \
+    -e '/\*cpp:/{n;s@$@ -isystem /usr/include@}' >      \
+    `dirname $(gcc --print-libgcc-file-name)`/specs
+It is a good idea to visually inspect the specs file to verify the intended change was actually made.
+==== Sanity Check the Toolchain ====
+It is imperative at this point to ensure that the basic functions (compiling and linking) of the adjusted toolchain are working as expected. To do this, perform the following sanity checks:
+  cd /tmp
+  echo 'int main(){}' > dummy.c
+  cc dummy.c -v -Wl,--verbose &> dummy.log
+  readelf -l a.out | grep ': /lib'
+There should be no errors, and the output of the last command will be (allowing for platform-specific differences in dynamic linker name):
+  [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
+Now make sure that we're setup to use the correct start files:
+  grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log
+The output of the last command should be (allowing for platform specific variations):
+  /usr/lib/../lib/crt1.o succeeded
+  /usr/lib/../lib/crti.o succeeded
+  /usr/lib/../lib/crtn.o succeeded
+Verify that the compiler is searching for the correct header files:
+  grep -B1 '^ /usr/include' dummy.log
+This command should return the following output:
+  #include <...> search starts here:
+   /usr/include
+Next, verify that the new linker is being used with the correct search paths:
+  grep 'SEARCH.*/usr/lib' dummy.log |sed 's|; |\n|g'
+References to paths that have components with '-linux-gnu' should be ignored, but otherwise the output of the last command should be:
+  SEARCH_DIR("/usr/lib")
+  SEARCH_DIR("/lib")
+Next make sure that we're using the correct libc:
+  grep "/lib.*/libc.so.6 " dummy.log
+The output of the last command should be:
+  attempt to open /lib/libc.so.6 succeeded
+Lastly, make sure GCC is using the correct dynamic linker:
+  grep found dummy.log
+The output of the last command should be (allowing for platform-specific differences in dynamic linker name):
+  found ld-linux-x86-64.so.2 at /lib/ld-linux-x86-64.so.2
+===== Building the Remainder of Phase 2 =====
+Now it is time to build the rest of phase 2. Fortunately, the remainder of this process can be automated with portmake. Run the following (note that this command will take a //long// time):
+  /usr/ports/utilities/tools/portmake install phase2
+====== Chapter 7 - Building Phase 3 ======
+This phase is akin to building a few select packages from the Beyond Linux from Scratch Book.
+At the end of this phase the system will be capable of both running *and* compiling itself; however, it will not be complete yet.
+====== Chapter 8 - Building Phase 4 ======
+This phase is akin to building additional packages from the Beyond Linux from Scratch Book.
+At the end of this phase the system will be complete.

Cucumber Linux Wiki

User Tools

Site Tools

Differences

Page Tools