TianoCore and coreboot, again

It has been over two years since the last time I worked on TianoCore for coreboot. These days I had some free time to spend and I used it to continue the project. I updated my code to the latest versions of coreboot and TianoCore and got it to work again in QEMU.

Here's a "screenshot" and a few words on what it's all about.

The new GenFw Tool

I've re-written the GenFw tool part of the TianoCore BaseTools project. The source code can be found here. In order to use the tool, the file Source/C/GenFw/GenFw.c must be replaced with the re-written one. Then, the base tools must be re-built. After that, the EDK2 build process can be started. It will automatically pick up the new tool which will brand an ELF file with an UEFI file type.

Currently, the re-written tool will not compile on Linux. The reason is that Linux lacks implementations of err(3), errx(3), warn(3), etc. library functions which the BSDs have. It should be easy to add some compatibility macros using a combination of fprintf(3), strerror(3) and exit(3). I might add those should the need arise.

Update (Dec 3rd, 2008): I've added the compatibility macros for Linux. An updated version of the source code can be downloaded here.

More on TianoCore for coreboot

It's been a while since I last worked on combining TianoCore and coreboot. Tonight I had some spare time and tried to pursue the project.

The previously mentioned build failure does indeed stem from the fact that the build tools cannot cope with ELF binaries. Especially problematic is the GenFw tool which is supposed to convert the binary file into an UEFI firmware volume file. In order to do that, it parses the header information of the input binary executable file and encodes the type of file (in UEFI terms) in a spare header field. The tool expects to work on PE32 files but the TianoCore developers have added code which converts an ELF image into a PE32 image internally if the tool is pointed at an ELF file. However, this facility is only compiled in if #defined(Linux) is true. Of course, that won't work on FreeBSD but changing the relevant pre-processor condition allowed me to produce an UEFI firmware volume without any further changes to the code.

However, this shortcut will only work on x86 and only if the target platform is x86, too. The real solution is to avoid the conversion and instead encode the UEFI file type directly into the ELF header. I've already done this for my thesis project (*) and back then it seemed that re-writing the GenFw tool was easier than fixing the existing implementation. Well, here's the next item on the ToDo list...

(*) I used the Java-based tools for the thesis project which means that a different tool with essentially the same functionality was the culprit.

The beginnings of coreboot and TianoCore

In order to create a UEFI payload for coreboot, I've started a coreboot platform as part of the TianoCore EDK II. The sources for the platform can be obtained here. Note that the CorebootPkg directory must be placed in the TianoCore $WORKSPACE directory.

To build the package on FreeBSD, a GNU toolchain from vendor sources must be used. This is because the TianoCore tools use some compiler/linker flags unknown to the toolchain included in the FreeBSD base system. The path as well as the names of the toolchain binaries must be adjusted in Conf/tools_def.txt. Because I built the toolchain according to these instructions, the preprocessor will not look in /usr/include for headers which causes errors in the ProcessorBind.h header when it attempts to include stdint.h. This patch can be applied to fix this.

Note that the build process still cannot complete as the tools producing the final Firmware Volume (FV) cannot cope with the ELF binaries produced by the GNU toolchain.

TianoCore and the Python-based Build Process, Part 3

This is part III of my attempts to build the TianoCore EDK II with the Python-based tools. In order to circumvent the error that stopped me in part II, the build process needs to be taught to use GNU make, i.e. gmake, on FreeBSD instead of make, which is BSD make. This can be done by editing the *_ELFGCC_*_MAKE_PATH variable in Conf/tools_def.txt.

The tools_def.txt file is automatically copied from a template part of the BaseTools sources. This patch fixes the template so the changes described above do not have to be applied manually.

At this point, the build process starts and does actually build some modules. However, the UnixPkg cannot be built completely on FreeBSD. This is because the code makes some assumptions only true on Linux, e.g. the presence of the sys/vfs.h header.

TianoCore and the Python-based Build Process, Part 2

So here's the "sequel" to Part One. This time I'm trying to actually build a Firmware Volume with the Python-based tools.

Prerequisites for build.py

The core of the tools is build.py, a Python script which invokes the tools in order to build a Firmware Volume (FV). On FreeBSD, build.py cannot be run until the following requirements are met:

• SQLite3 for Python, which can be installed through the databases/py-sqlite3 port.
• The Python module for ANTLR, a parser generator.
• Installing the module mentioned above requires EasyInstall, or rather: I don't know how it can be done otherwise.

Because I could not find a port for EasyInstall, I did this to install the script on FreeBSD:

$ fetch http://peak.telecommunity.com/dist/ez_setup.py
$ chmod +x ez_setup.py
$ ./ez_setup.py

Note that this isn't the whole truth as the path to the interpreter in the script, i.e. the first line aka "shebang", must be adjusted to /usr/local/bin/python before the script can be executed.
After that, the easy_install command is available and the ANTLR module can be installed by running this:

$ eazy_install \
  http://www.antlr.org/download/Python/antlr_python_runtime-3.0.1-py2.5.egg

Running build.py

In theory, running build.py and thus building a Firmware Volume should be as easy as this:

$ cd path/to/edk2
$ export PYTHONPATH=/path/to/basetools/Source/Python
$ . edksetup.sh BaseTools
$ python $PYTHONPATH/build/build.py

Unfortunately, the last step initially aborted with this error:

build...
 : error 5000: Please execute /home/phs/sandbox/basetools/Bin/FreeBSD-i386:/sbin: \
/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin: \
/home/phs/bin/edksetup.bat to set /home/phs/sandbox/basetools/Bin/Freebsd7 in \
environment variable: PATH!



- Failed -

After some try'n'error time, I think that the above error is caused by user error: I had previously copied the compiled C programs from Source/C/bin to bin/FreeBSD-i386 (paths relative to /path/to/basetools). After removing bin/FreeBSD-i386, I created a link to BinWrappers/PosixLike at the same location:

$ cd /path/to/basetools
$ ln -s BinWrappers/PosixLike Bin/FreeBSD-i386

I then re-ran build.py (see above) and it produced some output that didn't look like errors:

00:44:09, Sep.21 2008 [FreeBSD-7.1-PRERELEASE-i386-32bit-ELF]

WORKSPACE                = /usr/home/phs/sandbox/edk2
EDK_SOURCE               = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg
EFI_SOURCE               = /usr/home/phs/sandbox/edk2/EdkCompatibilityPkg
EDK_TOOLS_PATH           = /home/phs/sandbox/basetools

TARGET_ARCH              = IA32
TARGET                   = DEBUG
TOOL_CHAIN_TAG           = ELFGCC

Active Platform          = UnixPkg/UnixPkg.dsc
Flash Image Definition   = UnixPkg/UnixPkg.fdf

Processing meta-data . . . . . . .

Unfortunately, though, right after the dots, an error occured:

build...
UnixPkg/UnixPkg.dsc(...): error 4000: Instance of library class [NetLib] is not found
        in [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf] [IA32]
        consumed by module [MdeModulePkg/Universal/Network/ArpDxe/ArpDxe.inf]
 

- Failed -
00:44:17, Sep.21 2008 [00:08]

Fixing the UnixPkg

The UnixPkg part of the EDK II seems to be broken as the error above indicates a dependency error between modules which is caused by an incorrect platform definition file (*.dsc). Applying this patch fixes the problem.

The patch ensures that all dependencies are met, but the build process still fails with this error:

Processing meta-data . . . . . . . . done!
make: don't know how to make pbuild. Stop


build...
 : error 7000: Failed to execute command
        make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate]

Waiting for all build threads exit...
make: don't know how to make pbuild. Stop


build...
 : error 7000: Failed to execute command
        make pbuild [/usr/home/phs/sandbox/edk2/Build/Unix/DEBUG_ELFGCC/IA32/MdePkg/Library/BaseLib/BaseLib]


build...
 : error F002: Failed to build module
        MdePkg/Library/BaseTimerLibNullTemplate/BaseTimerLibNullTemplate.inf [IA32, ELFGCC, DEBUG]

- Failed -
01:01:43, Sep.21 2008 [00:09]

Oh, well, to be continued...

TianoCore and the Python-based Build Process

Now that I can use coreboot and libpayload on FreeBSD, it's time to try the new Python-based build process for the TianoCore EDK II on FreeBSD.

Prerequisites are:

• A copy of the EDK II sources, available at https://edk2.tianocore.org/svn/edk2/trunk/edk2.
• The sources of the Python tools which can be obtained from https://buildtools.tianocore.org/svn/buildtools/trunk/BaseTools.
• The misc/e2fs-libuuid port.
• GNU Make, i.e. the devel/gmake port.

Note that Subversion access requires a user account at the TianoCore project.

Installing the e2fs-libuuid port

This is trivially easy, just do:

$ cd /usr/ports/misc/e2fs-libuuid
$ sudo make install

That's all. The headers and libraries are installed under /usr/local.

Building the Base Tools

Compiling the Base Tools, i.e. the Python-based TianoCore build tools, isn't complicated but doesn't work out of the box, either. First, these two patches (patch 1, patch 2) must be applied:

$ cd /path/to/basetools
$ patch -p0 < basetools_include.diff
$ patch -p0 < basetools_make.diff

The first patch adjusts some include paths so that /usr/local/include is searched, too, which is required in order to find the uuid/uuid.h header. The second patch replaces invocations of make to use the $(MAKE) variable which holds the name of invoked the make binary. This is required as in FreeBSD (and other BSDs), make is not GNU make, however the latter is required to build the Base Tools. Consequently, when building the project, make sure that gmake is used:

$ gmake

Compiling the EDK II

To be continued...

The TianoCore Contributor's Agreement

So, I finally found some time to crawl through the TianoCore project's Contributor's Agreement. Here's what I think it means.

• Preample: So Intel has decided to release some code under what it calls the "BSD license". Personally, I have think the BSD license is something else or maybe even something like this. I don't think a link to an incomplete license stub is enough, though. But enough of the ranting.
  Just to be clear here, I think it is safe to assume that Intel released their code under the following license (note that it's just the stub they provide a link to, filled in with some meaningful values):

  Copyright (c) 2006-2008, Intel Corp.
  All rights reserved.
  
  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are
  met:
  
  * Redistributions of source code must retain the above copyright notice, this
    list of conditions and the following disclaimer.
  
  * Redistributions in binary form must reproduce the above copyright notice,
    this list of conditions and the following disclaimer in the documentation
    and/or other materials provided with the distribution.
  
  * Neither the name of Intel Corp. nor the names of its contributors may be
    used to endorse or promote products derived from this software without
    specific prior written permission.
  
  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
  LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
  CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
  SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
  CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
  POSSIBILITY OF SUCH DAMAGE.
  

  In addition to their own code which they release under the "BSD license", there is some code in the TianoCore tree that is released under other licenses. Specifically the FAT32 code, which is apparently covered by some patents. If other licenses apply, and that's the key point here, the license is either in the source files themselves or packaged with the source files.
• Definitions: I'm a "contributor" if I hold the copyright on some code that I contribute to the TianoCore project. If I represent a company, all my employees are considered part of my company and not separate contributors. A "contribution" is anything I sent to the TianoCore project, e.g. via mail, snail mail, telephone, etc. as long as I don't explicitly mark it as "not a contribution".
• License for Contributions: So when I decide to contribute something to the TianoCore project, I automatically agree to provide it under a license. That can be found in the contributor's agreement. The bullet points a) and b) are pretty clear: The permisssion to use and redistribute my contributions, provided that the three conditions laid out in the "BSD license" quoted above are met.
  The next bullet point, c), is somewhat harder to understand. I interpret is as: If I hold a patent on something, and my contribution infringes that patent, I automatically grant a patent license. I grant it to everybody, who wants to exercise his rights I granted him with the copyrigth license mentioned above. However, here's the catch: That patent license applies only to my unmodified contribution.
  I'm not sure what to think about that. I think, Intel is trying to protect their own patents. So if they release some code to the TianoCore project which is covered by a patent they own, the only grant a minimum patent license. What remains unclear is whether the patent license is still valid, even if I modify their code as permitted by the copyright license they granted.
  The last bullet point, d), is an easy one again. It's simply the "provided 'as is'" part in the copyright license cited above.
• Representations: By signing the agreement, I promise that I created the code myself. If my employer does have any rights, I promise that it explicitly permitted me to contribute my code.
• Third Party Contributions: If I chose to contribute third party code, I need to explicitly mark it as such. It also must be separate from my own contributions.
• Miscellaneous. The agreement is in English and translation are not authorative. If I want to take the whole thing to court, I need to to it in Delaware (US).

So what's the conclusion here? I think Intel is pretty open about releasing their code. However, they are not so open about creating an open source project around their code. What I mean is that there are quite some legal hurdles one has to pass when contributing code to the TianoCore project. In effect, they force the BSD license on any code I contribute and I think that's OK. On the other hand, however, they prevent me from forking the project by introducing that stupid patent clause since I have no easy way of checking whether a specific piece of code infringes one of their patents.
I really wonder if they only want to protect themselves from getting sued over some code contributed the project from non-Intel employees. Or are they really trying to create an impression of an Open Source project when it's really not?

Porting TianoCore to a new platform, part 2

So it took me exactly one day to start the TianoCore DXE Core on a new platform. Of course, this doesn't count the roughly 10 weeks it took me to understand how the TianoCore Codebase works ;-) Also, it took me a fair amount of work to fix one thing or the other.

Anyways, I wanted to take a note that the generic memory test module included in the TianoCore Codebase is nowhere near "quick", despite the fact that it has a mode called QuickMode, when you attempt to throw it at a 8 GByte memory range.

Reminder about porting TianoCore to a new Platform

Just a quick note to remind myself that when porting the TianoCore stack to a new platform, the PEI needs an implementation of the PEI_LOAD_FILE_PPI before it can load any PEIMs from the FV.

For the platforms I have worked with so far, the PPI was implemented in the SEC.

TianoCore on QEMU

There is a bios.bin binary file for use with QEMU available at http://fabrice.bellard.free.fr/qemu/efi-bios.tar.bz2. It is meant to be used as a BIOS replacement for QEMU and provides an EFI interface. It is compiled from the TianoCore sources, at least that's what the QEMU homepage suggests.

The problem with this file is that it can only be used with very few versions of QEMU, that's why I'm writing this.

I've had success with version 0.9.0 when the patches linked from the coreboot wiki were applied. I've also had success with a CVS snapshot from July 3rd, 2007. Version 0.9.1 or the stock 0.9.0 do not work.

TianoCore on IBM's OpenClient

I've started work at IBM yesterday. Today I found a room and got all my user accounts and passwords set up so I can actually start to work.

I'm trying to build the TianoCore EDKII (SVN Revision 4792) on IBM's own Linux distribution, called OpenClient. The distribution sucks but it's the only kind of Linux we're allowed to use around here. It's based on Red Hat (I think), but it feels "different".

To bootstrap the TianoCore Build Environment, I sort of followed my FreeBSD and Fedora Core notes. Here's what I did:

• Installed Java JDK 6 Update 4 (JDK 1.6.0_04) for Linux through the self-extracting binary file (not the RPM) that Sun provides. I placed the JDK in /opt.
• Installed the binary distribution of Apache Ant 1.7.0, Saxon 8.1.1, XMLBeans 2.1.0 as well as ant-contrib 1.0b3 and placed all of them in /opt.
• Created the symlinks:

  $ cd /opt/apache-ant-1.7.0/lib
  $ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
  $ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar
  

I then created a small script that automates the build tool bootstrapping process:

export JAVA_HOME=/opt/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/Sources/edk2
export PATH=$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin
. edksetup.sh ForceRebuild

Sourcing the script successfully builds the build tools. Then, in the file $WORKSPACE/Tools/Conf/target.txt, two settings need to be adjusted:

ACTIVE_PLATFORM=EdkUnixPkg/Unix.fpd
TOOL_CHAIN_TAG=ELFGCC

Of course, the previously mentioned patch needs to be applied. After that, the EDKII Unix Emulation Package can be built and run as described in the tutorial:

$ cd $WORKSPACE/
$ build
$ cd Build/Unix
$ . run.cmd

I found that the IBM OpenClient distribution already includes the e2fsprogs-devel package as well as the relevant X11 development packages. Please also note that it is not neccessary to build an PE32+ cross compiler on Linux.

TianoCore on Fredora Core 8

My attempts building anything of the TianoCore EDK2 codebase on FreeBSD/amd64 have been extremely frustrating so far. I guess trying to build on an unsupported operating system and on an unsupported architecture may have been too much of a hurdle for the beginning. So I figured I'd try building the EDK Unix Simulator (Trunk Revision 4679) on Fredora Core 8. Here's what I've done.

I pretty much followed this tutorial. It's for Gentoo, so a little tweaking was needed.

First, I downloaded and installed the JDK 6 from SUN via the RPMs they provide. The JDK ends up in /usr/java/jdk1.6.0_04, so the JAVA_HOME environment variable needs to be set to that.

Second, I didn't bother installing the needed (Java) tools through the Fedora Package Manager but instead downloaded the files manually and placed them under /opt. This is similar to what I did for my FreeBSD attempts. I needed two symlinks like the tutorial says:

$ cd /opt/apache-ant-1.7.0/lib
$ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
$ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar

Third, I needed to install the e2fsprogs-devel package. The e2fsprogs package (without the "-devel" suffix) isn't enough. Also, I had to install the X development packages. I don't know what exact package was needed, but the Fedora Core 8 package manager has this option that lets you install some pre-selected packages related to X development.

Forth, I had to apply the following patch (the tutorial mentions this):

Index: EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c
===================================================================
--- EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Revision 4679)
+++ EdkModulePkg/Bus/Pci/PciBus/Dxe/PciHotPlugSupport.c (Arbeitskopie)
@@ -21,7 +21,7 @@

 --*/

-#include "Pcibus.h"
+#include "pcibus.h"
 #include "PciHotPlugSupport.h"

 EFI_PCI_HOT_PLUG_INIT_PROTOCOL  *gPciHotPlugInit;

Finally, contrary to what the tutorial says, I used the following script to set up the environment. Note that I didn't include the TOOL_CHAIN line and that I didn't build the PE/COFF capable GCC.

export JAVA_HOME=/usr/java/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0 
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/edk2/edk2
export PATH="$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin" 

The build went smoothly after that and I was able to use the EDK Unix environment. Note that the thing should not be called "Unix" Package, since it heavily assumes that it runs on Linux in some areas.

TianoCore on FreeBSD/amd64, Take 2

Now that I have build a cross compiler, I'm trying to build the TianoCore EDK again.

In this post I list a few environment variables that need to be set in order to build the EDK. Because the build process needs to use the cross compiler, another environment variable is needed:

$ export CC=/opt/i386-tiano-pe/bin/gcc

Since I'm lazy and all, I put everything in a little script called env.sh:

export WORKSPACE=/home/phs/edk2
export JAVA_HOME=/usr/local/diablo-jdk1.5.0
export ANT_HOME=/opt/apache-ant-1.6.5
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
export CC=/opt/i386-tiano-pe/bin/gcc

Also, the EDK build notes mention that the default build target is the Windows NT Emulation environment. However, that target cannot be built using GCC, so I needed to edit the file Tools/Conf/target.txt and change the ACTIVE_PLATFORM to:

ACTIVE_PLATFORM       = EdkUnixPkg/Unix.fpd

Now, I can tip of the build process as follows:

$ cd ~/edk2
$ . env.sh
$ . edksetup.sh newbuild

Note that the build script must be "sourced", not executed. Unfortunately, the build process still fails, but now with a different error:

       [cc] 1 total files to be compiled.
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:27:20: error: jni_md.h: No such file or directory
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:3,
       [cc]                  from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:45: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jsize'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:104: error: expected specifier-qualifier-list before 'jbyte'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:193: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1834: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1842: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1851: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1888: error: expected specifier-qualifier-list before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1927: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1930: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1933: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1937: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jint'
       [cc] /usr/local/diablo-jdk1.5.0/include/jni.h:1940: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'
       [cc] In file included from /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:17:
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:29: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'jbyteArray'

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

The root cause of this is that the compiler can't find the jni_md.h header which is located in $JAVA_HOME/include/freebsd (at least on my system). I worked around this problem by editing $JAVA_HOME/include/jni.h as follows:

--- jni.h.orig  2008-02-06 11:50:05.000000000 +0100
+++ jni.h       2008-02-06 11:50:16.000000000 +0100
@@ -24,7 +24,7 @@
 /* jni_md.h contains the machine-dependent typedefs for jbyte, jint
    and jlong */
 
-#include "jni_md.h"
+#include "freebsd/jni_md.h"
 
 #ifdef __cplusplus
 extern "C" {

Now I'm stuck with yet another compilation error:

init:
     [echo] Building the EDK Tool Library: CompressDll

Lib:
       [cc] 1 total files to be compiled.
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c: In function 'Java_org_tianocore_framework_tasks_Compress_CallCompress':
       [cc] /home/phs/edk2/Tools/CCode/Source/CompressDll/CompressDll.c:57: warning: overflow in implicit constant conversion
       [cc] Starting link
       [cc] /usr/bin/ld: /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a(EfiCompress.o): relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC
       [cc] /home/phs/edk2/Tools/CCode/Source/Library/libCommonTools.a: could not read symbols: Bad value

BUILD FAILED
/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/build.xml:254: The following error occurred while executing this line:
/home/phs/edk2/Tools/CCode/Source/CompressDll/build.xml:45: gcc failed with return code 1

Well, I guess we'll see where this ends...

Building a Cross Compiler on FreeBSD

I'm currently trying to build a cross compiler (and other required tools) on FreeBSD. The compiler will run on FreeBSD/amd64 and should produce i386 binaries. This wouldn't be too hard since that task can easily be accomplished by using the FreeBSD source tree. However, I need the toolchain to produce binaries in the PE/COFF format instead of the default ELF format.

Building the toolchain is somewhat tricky, at least I found it to be poorly documented. But maybe I just looked in the wrong places. Building the toolchain requires:

• Building binutils.
• Locating some header files.
• Building the compiler.
• Building a C Library.
• Building the compiler again so it uses the C library that was just built.

For my needs, I found that the last two steps weren't needed. I wrote a script that downloads the sources, extracts the archives and builds the toolchain. Here's the script in full quote (I really wish there was a way to upload files to this thing):

#/bin/sh

#
# Copyright (c) 2008 Philip Schulz 
#

#
# This script downloads, builds and installs GCC and GNU Binutils that can
# produce x86 binaries in PE/COFF Format. The cross toolchain needs some headers
# that aren't usually present on the host system. However, those headers can be
# obtained from the cygwin sources, that's why a snapshot of the cygwin sources
# is downloaded.
#
# After the script finishes, the tools will be located at
# ${PREFIX}/${TARGET_ARCH}/bin. Some other binaries will be installed in
# ${PREFIX}/bin with their own prefix of ${TARGET_ARCH} but I don't know that
# they are for.
#

# Prefix where the Cross-Tools will live
PREFIX="${PREFIX:-/opt}"

# Target architecture.
TARGET_CPU="${TARGET_CPU:-i386}"
TARGET_ARCH=${TARGET_CPU}-tiano-pe

# Program that can fetch the files.
FETCH_COMMAND="/usr/bin/ftp -V"

# GNU Make
GNU_MAKE=`which gmake`

################################################################################
#
# GCC settings.
#
################################################################################
# What version of GCC will be fetched, built and installed 
GCC_VERSION=gcc-4.2.3
# What mirror to use.
GCC_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the GCC sources. Should probably not be changed.
GCC_ARCHIVE=$GCC_VERSION.tar.bz2
# Where the GCC Sources can be fetched from. Should probably not be changed.
GCC_URL=$GCC_MIRROR/gcc/$GCC_VERSION/$GCC_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
GCC_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
GCC_CONFIGURE_ARGS+="--with-gnu-as --with-gnu-ld --with-newlib "
GCC_CONFIGURE_ARGS+="--disable-libssp --disable-nls --enable-languages=c "
GCC_CONFIGURE_ARGS+="--program-prefix=${TARGET_ARCH}- "
GCC_CONFIGURE_ARGS+="--program-suffix=-4.2.3 "


################################################################################
#
# Binutils settings.
#
################################################################################
# What version of the GNU binutils will be fetched, build and installed
BINUTILS_VERSION=binutils-2.18
# What mirror to use.
BINUTILS_MIRROR=ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/ftp.gnu.org
# File name of the binutils sources. Should probably not be changed.
BINUTILS_ARCHIVE=$BINUTILS_VERSION.tar.gz
# Where the GCC Sources can be fetched from. Should probably not be changed.
BINUTILS_URL=$BINUTILS_MIRROR/binutils/$BINUTILS_ARCHIVE
# Arguments for the GCC configure script. Should probably not be changed.
BINUTILS_CONFIGURE_ARGS="--prefix=${PREFIX} --target=${TARGET_ARCH} "
BINUTILS_CONFIGURE_ARGS+="--disable-nls "

################################################################################
#
# Cygwin settings.
#
################################################################################
CYGWIN_SNAPSHOT=20080129
CYGWIN_ARCHIVE=cygwin-src-${CYGWIN_SNAPSHOT}.tar.bz2
CYGWIN_MIRROR=http://cygwin.com/
CYGWIN_URL=${CYGWIN_MIRROR}snapshots/${CYGWIN_ARCHIVE}
CYGWIN_DIR=cygwin-snapshot-${CYGWIN_SNAPSHOT}-1

################################################################################
#
# Batch code.
#
################################################################################
#
# Fetches the files.
#
do_fetch() {
        if [ \! \( -f $GCC_ARCHIVE \) ] ; then
                echo "Fetching ${GCC_URL}"
                ${FETCH_COMMAND} ${GCC_URL}
        else
                echo $GCC_ARCHIVE already locally present.
        fi

        if [ \! \( -f $CYGWIN_ARCHIVE \) ] ; then
                echo "Fetching ${CYGWIN_URL}"
                ${FETCH_COMMAND} ${CYGWIN_URL}
        else
                echo $CYGWIN_ARCHIVE already locally present.
        fi

        if [ \! \( -f $BINUTILS_ARCHIVE \) ] ; then
                echo "Fetching ${BINUTILS_URL}"
                ${FETCH_COMMAND} ${BINUTILS_URL}
        else
                echo $BINUTILS_ARCHIVE already locally present.
        fi
}

#
# Extracts the archives.
#
do_extract() {
        # Remove already extracted files first.
        rm -rf ${GCC_VERSION}
        rm -rf ${CYGWIN_DIR}
        rm -rf ${BINUTILS_VERSION}

        # Extract the archives
        if [ -f $GCC_ARCHIVE ] ; then
                echo "Extracting ${GCC_ARCHIVE}"
                tar -jxf ${GCC_ARCHIVE}
        fi

        if [ -f $CYGWIN_ARCHIVE ] ; then
                echo "Extracting ${CYGWIN_ARCHIVE}"
                tar -jxf ${CYGWIN_ARCHIVE}
        fi

        if [ -f $BINUTILS_ARCHIVE ] ; then
                echo "Extracting ${BINUTILS_ARCHIVE}"
                tar -xzf ${BINUTILS_ARCHIVE}
        fi
}


BUILD_DIR_PREFIX=build-

#
# Builds Binutils.
#
do_binutils_build() {
        BUILD_DIR_BINUTILS=${BUILD_DIR_PREFIX}binutils-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_BINUTILS ] ; then
                rm -rf $BUILD_DIR_BINUTILS
        fi

        echo "Building binutils..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_BINUTILS} && cd ${BUILD_DIR_BINUTILS};
                # Configure, build and install binutils
                ../${BINUTILS_VERSION}/configure ${BINUTILS_CONFIGURE_ARGS} &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )

        # Remove build dir
        rm -rf $BUILD_DIR_BINUTILS

        echo "Binutils Build done."
}

#
# "Builds" cygwin. Actually, it only copies some headers around.
#
do_cygwin_build() {
        HEADERS=${PREFIX}/${TARGET_ARCH}/sys-include

        mkdir -p $HEADERS  &&
        cp -rf ${CYGWIN_DIR}/newlib/libc/include/* $HEADERS &&
        cp -rf ${CYGWIN_DIR}/winsup/cygwin/include/* $HEADERS
}

#
# Builds GCC
#
do_gcc_build() {
        BUILD_DIR_GCC=${BUILD_DIR_PREFIX}gcc-${TARGET_ARCH}

        # Remove dir if it exists.
        if [ -d $BUILD_DIR_GCC ] ; then
                rm -rf $BUILD_DIR_GCC
        fi

        echo "Building GCC..."

        # Changing directory, so use a sub-shell (?)
        (
                # Create a the build directory.
                mkdir ${BUILD_DIR_GCC} && cd ${BUILD_DIR_GCC};
                # Configure, build and install GCC.
                ../${GCC_VERSION}/configure $GCC_CONFIGURE_ARGS &&
                ${GNU_MAKE} -j 12 -w all && ${GNU_MAKE} -w install
        )
        rm -rf $BUILD_DIR_BINUTILS

        echo "GCC Build done."
}

do_fetch
do_extract
do_binutils_build
do_cygwin_build
do_gcc_build

Unfortunately, the gcc binary built by the script, located in /opt/i386-tiano-pe/bin, can't produce binaries. Invoking the compiler on a source file ("Hello, World!" program) dies with:

$ /opt/i386-tiano-pe/bin/gcc main.c -o main
/opt/lib/gcc/i386-tiano-pe/4.2.3/../../../../i386-tiano-pe/bin/ld: crt0.o: No such file: No such file or directory
collect2: ld returned 1 exit status

I assume this is because I skipped the last two steps in the list at the beginning of this post. However, using the compiler to generate an assembler file (parameter -S) and then running the assembler on that file to produce an object file does indeed produce a PE/COFF object file.

$ cd /opt/i386-tiano-pe/bin
$ ./gcc -S ~/main.c
$ ./as -o main.o main.s
$ file ./main.o
./main.o: MS Windows COFF Intel 80386 object file

TianoCore on FreeBSD/amd64

I'm attempting to build the TianoCore code base on FreeBSD/amd64. Here's what I did so far.

• In order to be able to check out the EDK2 sources, I installed the devel/subversion port. To check out the source tree, I did this in my home directory:

  $ svn co https://edk2.tianocore.org/svn/edk2/trunk/edk2 edk2
  



• I installed the java/diablo-jdk15 port.


• Downloaded Apache Ant 1.6.5. I didn't install it through the port but instead downloaded the binary distribution and extracted the archive under /opt since the EDK2 build framework requires very specific versions of the tools.


• Did the same thing with Ant-Contrib 1.0b3, XMLBeans 2.1.0 and Saxon 8.1.1.


• I created a symbolic link at /opt/xmlbeans-2.1.0/lib to /opt/saxon-8.1.1/saxon8.jar. The build notes for the EDK said a copy was needed, but a symbolic link works just as good. I guess they were running Windows or didn't know about links. Whatever.


• Then I set up the environment for the build process as described in the build notes. This is what I did (note that my shell is bash):

  $ export WORKSPACE=/home/phs/edk2
  $ export JAVA_HOME=/usr/local/diablo-jdk1.5.0
  $ export ANT_HOME=/opt/apache-ant-1.6.5
  $ export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
  $ export PATH=$PATH:$ANT_HOME/bin:$XMLBEANS_HOME/bin
  



• I kicked off the build process with this command:

  $ bash edksetup.sh newbuild
  

  Unfortunately, the build fails with an error:

  BUILD FAILED
  /usr/home/phs/edk2/Tools/build.xml:22: The following error occurred while executing this line:
  /usr/home/phs/edk2/Tools/CCode/Source/build.xml:247: The following error occurred while executing this line:
  /usr/home/phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
  /phs/edk2/Tools/CCode/Source/PeCoffLoader/build.xml:68: ar failed with return code 139
  

  This error can be solved by using a GCC that produces PE/COFF binaries instead of the default ELF images.