TianoCore on Fredora Core 8

My attempts building anything of the TianoCore EDK2 codebase on FreeBSD/amd64 have been extremely frustrating so far. I guess trying to build on an unsupported operating system and on an unsupported architecture may have been too much of a hurdle for the beginning. So I figured I'd try building the EDK Unix Simulator (Trunk Revision 4679) on Fredora Core 8. Here's what I've done.

I pretty much followed this tutorial. It's for Gentoo, so a little tweaking was needed.

First, I downloaded and installed the JDK 6 from SUN via the RPMs they provide. The JDK ends up in /usr/java/jdk1.6.0_04, so the JAVA_HOME environment variable needs to be set to that.

Second, I didn't bother installing the needed (Java) tools through the Fedora Package Manager but instead downloaded the files manually and placed them under /opt. This is similar to what I did for my FreeBSD attempts. I needed two symlinks like the tutorial says:

$ cd /opt/apache-ant-1.7.0/lib
$ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
$ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar

Third, I needed to install the e2fsprogs-devel package. The e2fsprogs package (without the "-devel" suffix) isn't enough. Also, I had to install the X development packages. I don't know what exact package was needed, but the Fedora Core 8 package manager has this option that lets you install some pre-selected packages related to X development.

Forth, I had to apply the following patch (the tutorial mentions this):

Index: EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c
===================================================================
--- EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Revision 4679)
+++ EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Arbeitskopie)
@@ -21,7 +21,7 @@

 --*/

-#include "Pcibus.h"
+#include "pcibus.h"
 #include "PciHotPlugSupport.h"

 EFI_PCI_HOT_PLUG_INIT_PROTOCOL  *gPciHotPlugInit;

Finally, contrary to what the tutorial says, I used the following script to set up the environment. Note that I didn't include the TOOL_CHAIN line and that I didn't build the PE/COFF capable GCC.

export JAVA_HOME=/usr/java/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0 
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/edk2/edk2
export PATH="$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin" 

The build went smoothly after that and I was able to use the EDK Unix environment. Note that the thing should not be called "Unix" Package, since it heavily assumes that it runs on Linux in some areas.

TianoCore on FreeBSD/amd64, Take 2

Now that I have build a cross compiler, I'm trying to build the TianoCore EDK again.

In this post I list a few environment variables that need to be set in order to build the EDK. Because the build process needs to use the cross compiler, another environment variable is needed:

$ export CC=/opt/i386-tiano-pe/bin/gcc

Since I'm lazy and all, I put everything in a little script called env.sh:

export WORKSPACE=/home/phs/edk2
export JAVA_HOME=/usr/local/diablo-jdk1.5.0
export ANT_HOME=/opt/apache-ant-1.6.5
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
export CC=/opt/i386-tiano-pe/bin/gcc

Also, the EDK build notes mention that the default build target is the Windows NT Emulation environment. However, that target cannot be built using GCC, so I needed to edit the file Tools/Conf/target.txt and change the ACTIVE_PLATFORM to:

ACTIVE_PLATFORM       = EdkUnixPkg/Unix.fpd

Now, I can tip of the build process as follows:

$ cd ~/edk2
$ . env.sh
$ . edksetup.sh newbuild

Note that the build script must be "sourced", not executed. Unfortunately, the build process still fails, but now with a different error:

       [cc] 1 total files to be compiled.
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:27:20: error: jni_md.h: No such file or directory
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:45: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jsize'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:104: error: expected specifier-qualifier-list before 'jbyte'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:193: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1834: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1842: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1851: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1888: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1927: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1930: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1933: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1937: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1940: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:29: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

The root cause of this is that the compiler can't find the jni_md.h header which is located in $JAVA_HOME/include/freebsd (at least on my system). I worked around this problem by editing $JAVA_HOME/include/jni.h as follows:

--- jni.h.orig  2008-02-06 11:50:05.000000000 +0100
+++ jni.h       2008-02-06 11:50:16.000000000 +0100
@@ -24,7 +24,7 @@
 /* jni_md.h contains the machine-dependent typedefs for jbyte, jint
    and jlong */
 
-#include "jni_md.h"
+#include "freebsd/jni_md.h"
 
 #ifdef __cplusplus
 extern "C" {

Now I'm stuck with yet another compilation error:

init:
     [echo] Building the EDK Tool Library: CompressDll

Lib:
       [cc] 1 total files to be compiled.
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c: In function 'Java_org_tianocore_framework_tasks_Compress_CallCompress':
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:57: warning: overflow in implicit constant conversion
       [cc] Starting link
       [cc] /usr/bin/ld: /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a(EfiCompress.o): relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC
       [cc] /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a: could not read symbols: Bad value

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

Well, I guess we'll see where this ends...

Building a Cross Compiler on FreeBSD

I'm currently trying to build a cross compiler (and other required tools) on FreeBSD. The compiler will run on FreeBSD/amd64 and should produce i386 binaries. This wouldn't be too hard since that task can easily be accomplished by using the FreeBSD source tree. However, I need the toolchain to produce binaries in the PE/COFF format instead of the default ELF format.

Building the toolchain is somewhat tricky, at least I found it to be poorly documented. But maybe I just looked in the wrong places. Building the toolchain requires:

• Building binutils.
• Locating some header files.
• Building the compiler.
• Building a C Library.
• Building the compiler again so it uses the C library that was just built.

For my needs, I found that the last two steps weren't needed. I wrote a script that downloads the sources, extracts the archives and builds the toolchain. Here's the script in full quote (I really wish there was a way to upload files to this thing):

#/bin/sh

#
# Copyright (c) 2008 Philip Schulz 
#

#
# This script downloads, builds and installs GCC and GNU Binutils that can
# produce x86 binaries in PE/COFF Format. The cross toolchain needs some headers
# that aren't usually present on the host system. However, those headers can be
# obtained from the cygwin sources, that's why a snapshot of the cygwin sources
# is downloaded.
#
# After the script finishes, the tools will be located at
# ${PREFIX}/${TARGET_ARCH}/bin. Some other binaries will be installed in
# ${PREFIX}/bin with their own prefix of ${TARGET_ARCH} but I don't know that
# they are for.
#

# Prefix where the Cross-Tools will live
PREFIX="${PREFIX:-/opt}"

# Target architecture.
TARGET_CPU="${TARGET_CPU:-i386}"
TARGET_ARCH=${TARGET_CPU}-tiano-pe

# Program that can fetch the files.
FETCH_COMMAND="/usr/bin/ftp -V"

# GNU Make
GNU_MAKE=`which gmake`

################################################################################
#
# GCC settings.
#
################################################################################
# What version of GCC will be fetched, built and installed 
GCC_VERSION=gcc-4.2.3
# What mirror to use.
GCC_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the GCC sources. Should probably not be changed.
GCC_ARCHIVE=$GCC_VERSION.tar.bz2
# Where the GCC Sources can be fetched from. Should probably not be changed.
GCC_URL=$GCC_MIRROR/gcc/$GCC_VERSION/$GCC_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
GCC_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
GCC_CONFIGURE_ARGS+="--with-gnu-as --with-gnu-ld --with-newlib "
GCC_CONFIGURE_ARGS+="--disable-libssp --disable-nls --enable-languages=c "
GCC_CONFIGURE_ARGS+="--program-prefix=${TARGET_ARCH}- "
GCC_CONFIGURE_ARGS+="--program-suffix=-4.2.3 "


################################################################################
#
# Binutils settings.
#
################################################################################
# What version of the GNU binutils will be fetched, build and installed
BINUTILS_VERSION=binutils-2.18
# What mirror to use.
BINUTILS_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the binutils sources. Should probably not be changed.
BINUTILS_ARCHIVE=$BINUTILS_VERSION.tar.gz
# Where the GCC Sources can be fetched from. Should probably not be changed.
BINUTILS_URL=$BINUTILS_MIRROR/binutils/$BINUTILS_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
BINUTILS_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
BINUTILS_CONFIGURE_ARGS+="--disable-nls "

################################################################################
#
# Cygwin settings.
#
################################################################################
CYGWIN_SNAPSHOT=20080129
CYGWIN_ARCHIVE=cygwin-src-${CYGWIN_SNAPSHOT}.tar.bz2
CYGWIN_MIRROR=http://cygwin.com/
CYGWIN_URL=${CYGWIN_MIRROR}snapshots/${CYGWIN_ARCHIVE}
CYGWIN_DIR=cygwin-snapshot-${CYGWIN_SNAPSHOT}-1

################################################################################
#
# Batch code.
#
################################################################################
#
# Fetches the files.
#
do_fetch() {
        if [ \! \( -f $GCC_ARCHIVE \) ] ; then
                echo "Fetching ${GCC_URL}"
                ${FETCH_COMMAND} ${GCC_URL}
        else
                echo $GCC_ARCHIVE already locally present.
        fi

        if [ \! \( -f $CYGWIN_ARCHIVE \) ] ; then
                echo "Fetching ${CYGWIN_URL}"
                ${FETCH_COMMAND} ${CYGWIN_URL}
        else
                echo $CYGWIN_ARCHIVE already locally present.
        fi

        if [ \! \( -f $BINUTILS_ARCHIVE \) ] ; then
                echo "Fetching ${BINUTILS_URL}"
                ${FETCH_COMMAND} ${BINUTILS_URL}
        else
                echo $BINUTILS_ARCHIVE already locally present.
        fi
}

#
# Extracts the archives.
#
do_extract() {
        # Remove already extracted files first.
        rm -rf ${GCC_VERSION}
        rm -rf ${CYGWIN_DIR}
        rm -rf ${BINUTILS_VERSION}

        # Extract the archives
        if [ -f $GCC_ARCHIVE ] ; then
                echo "Extracting ${GCC_ARCHIVE}"
                tar -jxf ${GCC_ARCHIVE}
        fi

        if [ -f $CYGWIN_ARCHIVE ] ; then
                echo "Extracting ${CYGWIN_ARCHIVE}"
                tar -jxf ${CYGWIN_ARCHIVE}
        fi

        if [ -f $BINUTILS_ARCHIVE ] ; then
                echo "Extracting ${BINUTILS_ARCHIVE}"
                tar -xzf ${BINUTILS_ARCHIVE}
        fi
}


BUILD_DIR_PREFIX=build-

#
# Builds Binutils.
#
do_binutils_build() {
        BUILD_DIR_BINUTILS=${BUILD_DIR_PREFIX}binutils-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_BINUTILS ] ; then
                rm -rf $BUILD_DIR_BINUTILS
        fi

        echo "Building binutils..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_BINUTILS} && cd ${BUILD_DIR_BINUTILS};
                # Configure, build and install binutils
                ../${BINUTILS_VERSION}/configure ${BINUTILS_CONFIGURE_ARGS} &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )

        # Remove build dir
        rm -rf $BUILD_DIR_BINUTILS

        echo "Binutils Build done."
}

#
# "Builds" cygwin. Actually, it only copies some headers around.
#
do_cygwin_build() {
        HEADERS=${PREFIX}/${TARGET_ARCH}/sys-include

        mkdir -p $HEADERS  &&
        cp -rf ${CYGWIN_DIR}/newlib/libc/include/* $HEADERS &&
        cp -rf ${CYGWIN_DIR}/winsup/cygwin/include/* $HEADERS
}

#
# Builds GCC
#
do_gcc_build() {
        BUILD_DIR_GCC=${BUILD_DIR_PREFIX}gcc-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_GCC ] ; then
                rm -rf $BUILD_DIR_GCC
        fi

        echo "Building GCC..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_GCC} && cd ${BUILD_DIR_GCC};
                # Configure, build and install GCC.
                ../${GCC_VERSION}/configure $GCC_CONFIGURE_ARGS &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )
        rm -rf $BUILD_DIR_BINUTILS

        echo "GCC Build done."
}

do_fetch
do_extract
do_binutils_build
do_cygwin_build
do_gcc_build

Unfortunately, the gcc binary built by the script, located in /opt/i386-tiano-pe/bin, can't produce binaries. Invoking the compiler on a source file ("Hello, World!" program) dies with:

$ /opt/i386-tiano-pe/bin/gcc main.c -o main
/opt/lib/gcc/i386-tiano-pe/4.2.3/../../../../i386-tiano-pe/bin/ld: crt0.o: No such file: No such file or directory
collect2: ld returned 1 exit status

I assume this is because I skipped the last two steps in the list at the beginning of this post. However, using the compiler to generate an assembler file (parameter -S) and then running the assembler on that file to produce an object file does indeed produce a PE/COFF object file.

$ cd /opt/i386-tiano-pe/bin
$ ./gcc -S ~/main.c
$ ./as -o main.o main.s
$ file ./main.o
./main.o: MS Windows COFF Intel 80386 object file

TianoCore on FreeBSD/amd64

I'm attempting to build the TianoCore code base on FreeBSD/amd64. Here's what I did so far.

• In order to be able to check out the EDK2 sources, I installed the devel/subversion port. To check out the source tree, I did this in my home directory:

  $ svn co https://edk2.tianocore.org/svn/edk2/trunk/edk2 edk2
  



• I installed the java/diablo-jdk15 port.


• Downloaded Apache Ant 1.6.5. I didn't install it through the port but instead downloaded the binary distribution and extracted the archive under /opt since the EDK2 build framework requires very specific versions of the tools.


• Did the same thing with Ant-Contrib 1.0b3, XMLBeans 2.1.0 and Saxon 8.1.1.


• I created a symbolic link at /opt/xmlbeans-2.1.0/lib to /opt/saxon-8.1.1/saxon8.jar. The build notes for the EDK said a copy was needed, but a symbolic link works just as good. I guess they were running Windows or didn't know about links. Whatever.


• Then I set up the environment for the build process as described in the build notes. This is what I did (note that my shell is bash):

  $ export WORKSPACE=/home/phs/edk2
  $ export JAVA_HOME=/usr/local/diablo-jdk1.5.0
  $ export ANT_HOME=/opt/apache-ant-1.6.5
  $ export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
  $ export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
  



• I kicked off the build process with this command:

  $ bash edksetup.sh newbuild
  

  Unfortunately, the build fails with an error:

  BUILD FAILED
  /usr/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
  /usr/home/phs/edk2/Tools/CCode/Source/build.xml:247: The following error occurred while executing this line:
  /usr/home/phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
  /phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
  

  This error can be solved by using a GCC that produces PE/COFF binaries instead of the default ELF images.

Qemu and Coreboot on FreeBSD/amd64

Today I tried to boot FreeBSD inside Qemu, using coreboot. I already wrote a short guide on how to do this, but I only tested it on a FreeBSD/i386 host. Now I'm trying to do the same thing, except this time the host is a FreeBSD/amd64 system.

I encountered several problems and I'm going to describe them along with the workarounds here.

First, the devel/dev86 port I created was marked as running on i386 only. I initially did this because I wasn't sure if the software would run on amd64, but after testing it I can say that it does work. So the port needs a little tweaking, namely this line

ONLY_FOR_ARCHS= i386

needs to be extended to

ONLY_FOR_ARCHS= i386 amd64

or maybe even removed completely.

Second, the RomCC utility used in the coreboot build process crashes, apparently due to a change in GCC 4 as opposed to GCC 3.4 used in older versions on FreeBSD. It crashes in a function called "free_basic_block" and the workaround is this:

Index: util/romcc/romcc.c
===================================================================
--- util/romcc/romcc.c  (revision 3088)
+++ util/romcc/romcc.c  (working copy)
@@ -15083,6 +15083,8 @@

static void free_basic_block(struct compile_state *state, struct block *block)
{
+       return;
+
      struct block_set *edge, *entry;
      struct block *child;
      if (!block) {

Third, the linker on FreeBSD/amd64 does not understand the -m32 flag. So I needed a cross compiler for i386. Warner Losh has an entry in his blog that describes how to cross build FreeBSD. I don't need all of FreeBSD but only the compiler, so I use this:

$ cd /usr/src
$ export TARGET=i386
$ export TARGET_ARCH=i386
$ make toolchain

This gives me an i386 cross compiler in /usr/obj/i386/usr/src/tmp/usr/bin that I need to use to compile coreboot. I did this to tell the GNU make utility to use the cross compiler:

gmake CC=/usr/obj/i386/usr/src/tmp/usr/bin/gcc

There is one outstanding problem: Qemu crashes when I try to use coreboot as a BIOS replacement. It may have to do with the version of Qemu I'm using right now. I'll try to use an older version later this afternoon to see if it solves the problem.

Using Qemu 0.9.0 with the required patches (see the Qemu Build Tutorial) made the problem go away and I'm now able to boot FreeBSD/i386 8-CURRENT inside Qemu, using coreboot and ADLO. I've uploaded a complete archive of the port files to the coreboot wiki, see this link.