Wednesday, March 12, 2008

Embedding binary BLOBs into an ELF file

I needed this yesterday, found a link describing it - and forgot it by today :-(

For a project I'm working on, I need to embed a file into an ELF executable. The executable then needs to do things with the embedded file, i.e. it has to know where in memory the file resides and how large it is.

So here it goes, largely copied from the link mentioned above.
  • Create an object file from the binary blob:
    $ ld -r -b binary -o example.o example.bin
  • In the sources, declare the symbols:
    extern char _binary_example_bin_start[];
    extern char _binary_example_bin_end[];
    
  • Make sure the object file is linked with the other sources:
    $ gcc -o example example.c example.o
    

Tuesday, March 11, 2008

TianoCore on QEMU

There is a bios.bin binary file for use with QEMU available at http://fabrice.bellard.free.fr/qemu/efi-bios.tar.bz2. It is meant to be used as a BIOS replacement for QEMU and provides an EFI interface. It is compiled from the TianoCore sources, at least that's what the QEMU homepage suggests.

The problem with this file is that it can only be used with very few versions of QEMU, that's why I'm writing this.

I've had success with version 0.9.0 when the patches linked from the coreboot wiki were applied. I've also had success with a CVS snapshot from July 3rd, 2007. Version 0.9.1 or the stock 0.9.0 do not work.

QEMU and kqemu on IBM's OpenClient

Today I wanted to try QEMU on IBM's OpenClient Linux distribution. Unfortunately, I was unable to install a binary package through yum because the default package repositories don't provide one. So I ended up installing QEMU from the sources.

Getting the sources is easy. To fetch the latest sources (i.e. CVS HEAD), just run:
$ cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/qemu \
 co qemu
In my case, I wanted version QEMU version 0.9.1, so I did this:
$ cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/qemu \
 co -rrelease_0_9_1 qemu
Building the sources is trivial as well. The usual three step process (configure, make, make install) works like a charm. If PREFIX isn't set, QEMU installs in /usr/local, but I want it in /opt. So here's what I did:
$ ./configure --prefix=/opt
$ make
$ sudo make install
Now I had a bunch of QEMU executables in /opt/bin, each one for a different architecture. But I wanted kqemu, the kernel accellerator for QEMU, as well. Through the QEMU home page, I found this site which provides kqemu RPMs for RHEL and Fedora.

For the IBM OpenClient distribution, I had to do this:
$ wget http://dl.atrpms.net/all/kqemu-1.3.0-2.el5.i386.rpm
$ wget http://dl.atrpms.net/all/kqemu-kmdl-2.6.18-53.1.13.el5-1.3.0-2.el5.i686.rpm
$ sudo rpm -iv kqemu-1.3.0-2.el5.i386.rpm kqemu-kmdl-2.6.18-53.1.13.el5-1.3.0-2.el5.i686.rpm
In case the links to the RPMs are truncated, there is a kqemu RPM and a kqemu-kmdl RPM.

Finally, in order to actually load the kernel module, I did this:
$ sudo modprobe kqemu
Everything described here is pretty straight forward, but I wanted to make sure I document the installation of the kqemu module somwhere, hence this post.

Friday, March 7, 2008

Cell SDK 3.0 on IBM's OpenClient Distribution

Today I was finally able to build a simple HelloWorld-Application with and within the TianoCore build environment. This is good news, as it leads me to the next task: Building the same HelloWorld-Application, but this time for Linux on 64-Bit PowerPC.

Since I do not yet have access to a 64-Bit PowerPC machine running Linux, I'm going to use the Cell SDK 3.0 for now. It can be used from an i386 machine and includes a toolchain as well as the full system simulator. The toolchain includes a cross-compiler that is cabable of producing binaries for the Cell BE's PPU, which is essentially a 64-Bit PowerPC processor. The system simulator simulates a Linux-installation running on Cell.

I'm still on IBM's OpenClient Linux distribution, which is apparently based on RHEL 5.1, at least according to /etc/redhat-release
Red Hat Enterprise Linux Client release 5.1 (Tikanga)
This is good on one hand, but made things slightly more complicated on the other hand. But first things first. Here's what I did to prepare the Cell SDK installation:
  • I went to the developerWorks download page for the Cell SDK 3.0 and downloaded the RHEL 5.1 packages.
  • I had to download the "basic libraries and headers for cross-compiling to Cell Broadband Engine's PPU", both the 32-Bit version and the 64-Bit version, from the Barcelona Supercomputer Center (BSC). Note that I could have built those RPMs myself, but only if I had a few other required RPMs like e.g. a glibc for PowerPC. Apparently those required RPMs are provided on the RHEL installation CDs, however, I'm on IBM's OpenClient and thus do not have access to the installation CDs. The good thing is, the Fedora RPMs provided by the BSC turned out to work just fine.
  • For the full system simulator, I had to download the sysroot image from the BSC website.
So that's it for the preparation part, now to actually installing the SDK.
  • I installed the installer RPM like this:
    # rpm -ivh cell-install-3.0.0-1.0.noarch.rpm
    This installs the installer to /opt/cell.
  • Now I needed to install the cross-compilation libraries and headers: # rpm -ivh ppu-sysroot-f7-2.noarch.rpm # rpm -ivh ppu-sysroot64-f7-2.noarch.rpm
  • Next I ran the installer as instructed by the installation manual: # cd /opt/cell # ./cellskd --iso /home/phs/Downloads install
After successfully running the installer, I found a functioning cross-compiler in /opt/cell/toolchain/bin.

For the system simulator, I had to install the sysroot imae RPM like this:
# rpm -ivh sysroot_image-3.0-7.noarch.rpm
Unfortunately, I wasn't able to make the system simulator work because of a missing dependency on a simulation library.

By the way, there's also official documentation available here.

Thursday, March 6, 2008

Setting up a Subversion Server

Ok, there's probably a million Blog posts, tutorials and HowTos on how to do this already. Yet I still find it hard to find short instructions on how to set up a subversion server quickly. I just had to do it again, and it took me longer than expected, so here it goes.

My requirements are pretty basic:
  • WebDAV so I can access the repository over HTTP.
  • I don't need too much security, so SSL won't be needed.
  • I want SVN:Web, a web front end to the repositories.
Here's what I did for the Subversion+WebDAV part. Note that I'm currently on IBM's OpenClient Linux Distribution which is based on RedHat.
  • I made sure that the httpd package is installed. I didn't have to install it, so I guess it's installed by default.
  • I had to install the mod_dav_svn package with yum:
    $ sudo yum install mod_dav_svn
  • The package installs a config file at /etc/httpd/conf.d/subversion.conf. By default, the <Location> tag is commented out. I just copied it, removed the comment signs and adjusted the values to my needs. This is what it now looks like:
    LoadModule dav_svn_module     modules/mod_dav_svn.so
    LoadModule authz_svn_module   modules/mod_authz_svn.so
    
    <Location /repos>
       DAV svn
       SVNParentPath /var/www/svn
    
       <LimitExcept GET PROPFIND OPTIONS REPORT>
          AuthType Basic
          AuthName "Authorization Realm"
          AuthUserFile /var/www/passwd
          Require valid-user
       </LimitExcept>
    </Location>
    
  • Now the Apache HTTP Server will serve the contents of /var/www/svn via WebDAV (I think), but it will query for a valid user entry. The entry must be created as follows:
    # cd /var/www
    # htpasswd -c passwd <username>
    
  • Next, I created the direcories for the repositories. As root, I ran these commands:
    # cd /var/www
    # mkdir svn
    # cd svn
    # svnadmin create <name of repository>
    # chown -R apache:apache <name of repository>
    
  • I then activated the HTTP service so the Apache Web Server would be started at boot time. I did this using a GUI tool called system-config-services. I had to check the box next to the httpd entry. I didn't want to reboot right aways, so I clicked Start in the toolbar. The tool told me that the service would now be running and I could verify this by going to http://localhost/. Testing the Subversion part was easy, too. Navigating to http://localhost/repos/example did the trick. Note that the actual repository name must be used instead of "example".
  • Oh, and I want at least minimal security. That is, I want the HTTP Server to serve pages only to the local machine. Therefore, I changed the Listen directive in the server config uration file to this:
    Listen 127.0.0.1:80
    
The second part was installing the SVN::Web CGI Script. Here's what I did to do it.
  • First, I had to install the subversion-perl package with yum. As root, I ran
    # yum install subversion-perl
  • Second, I installed the actuall SVN::Web script through CPAN. Again as root, I did
    # cpan install SVN::Web
  • I then created a directory that would hold all the SVN::Web files:
    #cd /var/www
    #mkdir svnweb
    
  • In that directory, I let the Perl script set itself up using default values:
    # cd /var/www/svnweb
    # svnweb-install
    
  • The last step created a file called config.yaml. It must be edited so the CGI script finds the repositories. Near the end, I edited the reposparent value:
    reposparent: '/var/www/svn'
  • Now, as the final step, the script needs to be introduced to the Apache Server. I created a file svnweb.conf in /etc/httpd/conf.d with the following contents:
    Alias /svnweb /var/www/svnweb
    
    AddHandler cgi-script .cgi
    
    <Directory /var/www/svnweb>
            Options All ExecCGI
            DirectoryIndex index.cgi
    </Directory>
    
After restarting the Apache HTTP Server, I could access http://localhost/svnweb and see the repositories.

Tuesday, March 4, 2008

TianoCore on IBM's OpenClient

I've started work at IBM yesterday. Today I found a room and got all my user accounts and passwords set up so I can actually start to work.

I'm trying to build the TianoCore EDKII (SVN Revision 4792) on IBM's own Linux distribution, called OpenClient. The distribution sucks but it's the only kind of Linux we're allowed to use around here. It's based on Red Hat (I think), but it feels "different".

To bootstrap the TianoCore Build Environment, I sort of followed my FreeBSD and Fedora Core notes. Here's what I did:

  • Installed Java JDK 6 Update 4 (JDK 1.6.0_04) for Linux through the self-extracting binary file (not the RPM) that Sun provides. I placed the JDK in /opt.
  • Installed the binary distribution of Apache Ant 1.7.0, Saxon 8.1.1, XMLBeans 2.1.0 as well as ant-contrib 1.0b3 and placed all of them in /opt.
  • Created the symlinks:
    $ cd /opt/apache-ant-1.7.0/lib
    $ sudo ln -s /opt/ant-contrib/ant-contrib-1.0b3.jar ant-contrib.jar
    $ sudo ln -sf /opt/saxonb8.1.1/saxon8.jar /opt/xmlbeans-2.1.0/lib/saxon8.jar
    
I then created a small script that automates the build tool bootstrapping process:
export JAVA_HOME=/opt/jdk1.6.0_04
export XMLBEANS_HOME=/opt/xmlbeans-2.1.0
export ANT_HOME=/opt/apache-ant-1.7.0
export WORKSPACE=/home/phs/Sources/edk2
export PATH=$PATH:$XMLBEANS_HOME/bin:$ANT_HOME/bin
. edksetup.sh ForceRebuild
Sourcing the script successfully builds the build tools. Then, in the file $WORKSPACE/Tools/Conf/target.txt, two settings need to be adjusted:
ACTIVE_PLATFORM=EdkUnixPkg/Unix.fpd
TOOL_CHAIN_TAG=ELFGCC
Of course, the previously mentioned patch needs to be applied. After that, the EDKII Unix Emulation Package can be built and run as described in the tutorial:
$ cd $WORKSPACE/
$ build
$ cd Build/Unix
$ . run.cmd
I found that the IBM OpenClient distribution already includes the e2fsprogs-devel package as well as the relevant X11 development packages. Please also note that it is not neccessary to build an PE32+ cross compiler on Linux.