Thursday, May 20, 2010

OpenEmbedded on Ubuntu 10.04

NOTE: In light of the instructions received from Koen, the Angström head developer, you should avoid following the instructions below, since the deb-based OE is appearantly faulty by design and bitbake 1.10.x should be used instead of Ubuntu's one. Use these setup scripts to set-up your dev branch instead. I'm not deleting the post in case the same errors occur in some other way as well.

Now's the time to talk about...setting up OpenEmbedded on Ubuntu 10.04...and...building the Ångström distribution for the BeagleBoard! (I do realize I've created some suspense surrounding this particular subject with my previous posts, but despite the problems one may run into it's not that terrifying really :))


An advantage of setting up OE on Ubuntu (or any other .deb-based system, for that matter) is that one is able to use the apt-get'able OpenEmbedded. Following the instructions on the blog post (that is, get the repository in your sources list - sudo aptitude install openembedded-minimal - oe-setup-minimal org.openembedded.dev) the set-up part becomes a breeze - this will take care of all the necessary dependencies.


You may read from several sources that you absolutely shouldn't be using the bitbake that is in Ubuntu's own repositories (ie, things like DON'T sudo apt-get install bitbake) but from what I've seen, they've updated the repository and version 1.8.18 was directly available at the time of my writing this. You may want to check for yourself by executing bitbake --version once you have obtained the packages, anything over 1.8.16 should be fine.


Once you've set up the environment by executing oe-setup-minimal org.openembedded.dev (yes, using the dev branch is appearantly recommended, instead of the stable/2009 branch) you can make an attempt at trying out your new oven...oh, I mean, development environment, by trying out something smaller via giving this command:


MACHINE=beagleboard bitbake nano

While we're at it, we can take a short look at parts of this particular command. Not that it's that complicated but here goes anyway:
  • OpenEmbedded supports a lot of different target platforms, so we should specify the platform we'll be targeting. This is what the MACHINE=beagleboard part is for.
  • bitbake is the central make tool, used extensively in OpenEmbedded, and nano is the name of the "recipe" for the miniscule (but useful!) text editor. For a nice introduction to bitbake and its recipes system, you may want to read this.
Although nano itself isn't all that complicated, the build can take some time since the initial setting up of the toolchain and base libraries will also be done.

During the build process, you may run into several errors. I've documented the ones I've personally ran into and how I got over them.

Error #1: Extraction error while building expat
OE Message: ERROR: '/home/maltanar/oe/openembedded.org/recipes/expat/expat_2.0.1.bb' failed
Digging down a bit further reveals 




NOTE: Unpacking ../../sources/expat-2.0.1.tar.gz to ../../tmp/minimal/org.openembedded.dev/work/armv7a-oe-linux-gnueabi/expat-2.0.1-r3/

gzip: stdin: invalid compressed data--crc error

This is a known bug in the current gzip which appearantly affects several distributions on i386 computers. The workaround I did is as follows:
  1. Install the gzip recovery tools: sudo apt-get install gzrt
  2. cd into the OE sources folder ($OE_ROOT/sources, should be ~/oe/sources if you used the deb setup)
  3. Run gzrecover expat-2.0.1.tar.gz; mv expat-2.0.1.tar.recovered expat-2.0.1.tar; gzip expat-2.0.1.tar and say yes to the overwrite
  4. Generate the md5 and sha256 sums of the new file by running md5sum expat-2.0.1.tar.gz; sha256sum expat-2.0.1.tar.gz and note them somewhere. Edit the expat-2.0.1.tar.gz.md5 file to reflect the new MD5 checksum.
  5. cd into oe/openembedded.org/recipes/expat then edit the expat_2.0.1.bb file in your favorite text editor - change the MD5 and SHA256 checksums there to reflect the new ones, and save.
  6. Done! You can run MACHINE=beagleboard bitbake nano once again.
I didn't run into any more problems while building nano, and it was eventually successfully built. And now it's the time to try the bit-baking oven on something bigger...an image of the actual Ångström distribution. Sounds fancy? Well, it starts off as simple as

MACHINE=beagleboard bitbake -v console-image

provided that you want to start out with something simple, that is, the barebones console image. Notice the extra -v passed on the command line - it can be helpful to have a verbose bitbake while building something major.

I've also kept track of what happened during the console-image build process:

Error #2: Trouble with glibc version while building gtk+
OE Message: NOTE: Task failed: /home/maltanar/oe/tmp/minimal/org.openembedded.dev/work/i686-linux/gtk+-native-2.20.0-r8.0/temp/log.do_configure.11847

In more detail: 

| checking for BASE_DEPENDENCIES... configure: error: Package requirements (glib-2.0 >= 2.23.6    atk >= 1.29.2    pango >= 1.20    cairo >= 1.6) were not met:
| Requested 'glib-2.0 >= 2.23.6' but version of GLib is 2.22.1

For some reason, although many newer glib-2.0 recipes are available, OE chooses the older ones. To solve this, I went into the oe/openembedded.org/recipes/glib-2.0 and deleted all the recipe (*.bb) files with the -native suffix (for example, glib-2.0-native_2.2.3.bb), and re-started the build process.

I don't understand it completely but I think the old -native suffixed recipes are sort of left-overs, since OE seems to be using the BBCLASSEXTEND = "native" flag for this purpose now (the non-suffixed recipes all have this).

Error #3: Trouble with mtd-utils build
OE error message: NOTE: Task failed: /home/maltanar/oe/tmp/minimal/org.openembedded.dev/work/i686-linux/mtd-utils-native-1.3.1-r0/temp/log.do_compile.6960

crc32.o: could not read symbols: File in wrong format

Not sure why this happened either. I cd'd into the mtd-utils source directory (oe/tmp/minimal/org.openembedded.dev/work/i686-linux/mtd-utils-native-1.3.1-r0/git) and ran make clean followed by MACHINE=beagleboard bitbake mtd-utils which did complete successfully this time.

After these (and several hours later) the process was complete, and I ended up with the rootfs, uImage and u-boot images in the oe/tmp/minimal/org.openembedded.dev/deploy/images directory! The uImage was still causing a kernel panic for me upon boot with a proper USB cable, so one may want to use the uImage from the /boot folder from the last BeagleBoard demo image as I mentioned in my previous post. Or maybe use an USB cable which isn't working :)

19 comments:

  1. Hi Yaman,
    Do you have the steps to setup the development environment using Angstrom setup scripts.I am trying to figure out how to enable DSP on my beagleboard using Angstrom.
    My development environment is Ubuntu 9.10.
    Any help would be really appreciated.
    Thanks in advance.

    Ashwin Raut

    ReplyDelete
  2. Hi there!

    The setup scripts are quite self-documenting, so I didn't want to do one more post on how to set them up, but here's how I did it:

    cd ~
    mkdir oe
    git pull git://gitorious.org/angstrom/angstrom-setup-scripts.git angstrom-setup-scripts
    cd angstrom-setup-scripts
    ./oebb.sh config beagleboard
    ./oebb.sh update

    and then you're good to go! You can call bitbake in this manner:
    ./oebb.sh bitbake nano

    there's also an alternative way without using the script as a wrapper, but the script will tell you that itself anyway :)

    ReplyDelete
  3. Thanks for posting the additional info on getting the scripts from the git repository as I found it helpful. Although, I did find it necessary to use git clone rather than git pull.

    I ended up with BitBake Build Tool Core version 1.10.0, bitbake version 1.10.0

    Running "./oebb.sh bitbake nano" resulted in the same issue (error 1) with expat_2.0.1.bb that you had although the files were in different locations the fix was the same.

    Building the console-image "./oebb.sh bitbake -v console-image" ran to completion without error. The result looks like a rootfs in sysroots/armv7a-angstrom-linux-gnueabi and a uImage in beagleboard-angstrom-linux-gnueabi. I have not yet have a chance to play with trying to get my beagleboard to boot using them.

    ReplyDelete
  4. That was definitely meant to be git clone and not git pull, I must have drifted off :)

    The problem with expat (and possibly other random packages) will keep coming up since it's a gzip issue - you may find my post entitled "getting a working gzip" helpful, it saves some frustration.

    ReplyDelete
  5. Hi, I used the oebb.sh to successfully build the Angstorm image, but I can't find where the script put it.

    Can you plz help ?

    Cheers :)

    ReplyDelete
  6. Hi,

    it should be under something like ~/oe/setup-scripts/build/tmp-angstrom_2008_1/deploy/glibc/images/beagleboard

    a little adventure to find it the first time, yes :) usually, everything that bitbake produces ends up under deploy/glibc

    ReplyDelete
  7. hi maltanar...
    during build procedure, i got the following error

    | /home/adnan/oe/tmp/sysroots/i686-linux/usr/bin/install: writing `/home/adnan/oe/tmp/work/armv7a-angstrom-linux-gnueabi/gettext-0.18-r1/image/usr/share/doc
    NOTE: Task failed: /home/adnan/oe/tmp/work/armv7a-angstrom-linux-gnueabi/gettext-0.18-r1/temp/log.do_install.10649
    ERROR: TaskFailed event exception, aborting
    ERROR: Build of /home/adnan/oe/org.openembedded.dev/recipes/gettext/gettext_0.18.bb do_install failed
    ERROR: Task 607 (/home/adnan/oe/org.openembedded.dev/recipes/gettext/gettext_0.18.bb, do_install) failed
    NOTE: Tasks Summary: Attempted 396 tasks of which 396 didn't need to be rerun and 1 failed.
    ERROR: '/home/adnan/oe/org.openembedded.dev/recipes/gettext/gettext_0.18.bb' failed


    Can you please help me ?

    Regards,
    Adnan

    ReplyDelete
  8. Does this occur while you're using the setup script, or are you using the apt-get'able OE? If it's the latter you should really prefer the setup scripts instead. I recall running into similar problems while using the apt-get'able OE.

    ReplyDelete
  9. no, i am using the setup script.
    i tried bitbake x-load before it, it did not give error but it keep giving

    Note : missing inherit gettext

    Do you have any idea how could it be overcome ?
    I am just stucked into middle of work :(

    ReplyDelete
  10. Okay. You may want to add the -v (verbose) option while running bitbake to obtain some more information since the log is a bit cryptic now. You should ask in #beagle on irc.freenode.net, or perhaps in the angstrom/beagleboard mailing lists.

    ReplyDelete
  11. thanks alot maltanar....
    finally atleast this problem is solved........

    ReplyDelete
  12. Hi Maltanar,

    Having followed your blog about open embedded, I got over the problem of failing at expat when building angstrom.

    But, now I always get following message at the end when bitbake nano completes.

    NOTE: Tasks Summary: Attempted 813 tasks of which 794 didn't need to be rerun and 0 failed.

    I just started using beagleboard. could you please help me here? Thanks.
    Louie

    ReplyDelete
  13. Hmm, that's not very clear, is it? Can you try passing -v (verbose) while running bitbake to get a bit more information as to what might be causing the error?

    ReplyDelete
  14. Hi Maltanar,

    having tried running "bitbake nano -v", I copied following suspicious part from the entire log.

    NOTE: selecting gcc-cross-initial to satisfy virtual/arm-angstrom-linux-gnueabi-gcc-initial due to PREFERRED_PROVIDERS
    staging-nano-armv7a-angstrom-linux-gnueabi - 2.2.5-r1.0
    Removing package staging-nano-armv7a-angstrom-linux-gnueabi from root...
    Installing staging-nano-armv7a-angstrom-linux-gnueabi (2.2.5-r1.0) to root...
    Configuring staging-nano-armv7a-angstrom-linux-gnueabi.
    Installing staging-nano-armv7a-angstrom-linux-gnueabi (2.2.5-r1.0) to root...
    Collected errors:
    * check_data_file_clashes: Package staging-nano-armv7a-angstrom-linux-gnueabi wants to install file /home/louie/stuff/angstrom-setup-scripts/build/tmp-angstrom_2008_1/sysroots/armv7a-angstrom-linux-gnueabi/usr/share/info/dir
    But that file is already provided by package * staging-gnutls-armv7a-angstrom-linux-gnueabi
    * opkg_install_cmd: Cannot install package staging-nano-armv7a-angstrom-linux-gnueabi.
    NOTE: Preparing runqueue
    NOTE: Marking Active Tasks
    NOTE: Pruned 1395 inactive tasks, 813 left
    NOTE: Assign Weightings
    NOTE: Compute totals (have 1 endpoint(s))
    NOTE: Executing runqueue
    NOTE: Removing stamps: /home/louie/stuff/angstrom-setup-scripts/build/tmp-angstrom_2008_1/stamps/armv7a-angstrom-linux-gnueabi/nano-2.2.5-r1.0.*
    NOTE: Running task 795 of 813 (ID: 7, /home/louie/stuff/angstrom-setup-scripts/sources/openembedded/recipes/nano/nano_2.2.5.bb, do_setscene)
    NOTE: package nano-2.2.5-r1.0: task do_setscene: Started
    NOTE: Staging package found, using it for /home/louie/stuff/angstrom-setup-scripts/sources/openembedded/recipes/nano/nano_2.2.5.bb.
    NOTE: Failure installing prestage package
    NOTE: package nano-2.2.5-r1.0: task do_setscene: Succeeded.

    Can we see any thing wrong from there?

    ReplyDelete
  15. I'm by no means an OE expert but I think this might be caused by an untimely interruption. Have you tried doing doing this with a clean workspace (ie, do bitbake clean first) ? I can't really help otherwise, if the problem persists you may want to try the mailing list instead.

    ReplyDelete
  16. Okay I can now build, that was a bit of a challenge. Now I am trying to figure out how to deploy the results. Is there a decent reference for the current layout of things? I used to do unix systems, but it has been 30 years. Everything is moved, and I don't quite see where I should unpack what.

    ReplyDelete
  17. thank you so much for creating this blog.

    ReplyDelete
  18. Hello, Im trying to build Oe environment on ubuntu 10.04 for beagleborad-xM, following steps from the Oe website http://docs.openembedded.org/usermanual/html/gettingoe_getting_oe.html

    However I get the following error when executing bitbake x11-image.

    xeon@xeon-laptop:~/oe/build$ bitbake x11-image
    NOTE: Handling BitBake files: | (0001/0001) [100 %]
    Parsing of 1 .bb files complete (0 cached, 1 parsed). 1 targets, 0 skipped, 0 masked, 0 errors.
    ERROR: Nothing PROVIDES 'x11-image'

    Build Configuration:
    BB_VERSION = "1.10.2"
    METADATA_BRANCH = "master"
    METADATA_REVISION = "6148e6f"
    TARGET_ARCH = "arm"
    TARGET_OS = "linux-gnueabi"
    MACHINE = "beagleboard"
    DISTRO = "angstrom"
    DISTRO_VERSION = "v20110419"
    TARGET_FPU = "hard"

    Unknown Event:
    ERROR: Nothing PROVIDES 'x11-image'
    Command execution failed: Traceback (most recent call last):
    File "/home/xeon/oe/bitbake/lib/bb/command.py", line 88, in runAsyncCommand
    commandmethod(self.cmds_async, self, options)
    File "/home/xeon/oe/bitbake/lib/bb/command.py", line 174, in buildTargets
    command.cooker.buildTargets(pkgs_to_build, task)
    File "/home/xeon/oe/bitbake/lib/bb/cooker.py", line 782, in buildTargets
    taskdata.add_provider(localdata, self.status, k)
    File "/home/xeon/oe/bitbake/lib/bb/taskdata.py", line 354, in add_provider
    self.add_provider_internal(cfgData, dataCache, item)
    File "/home/xeon/oe/bitbake/lib/bb/taskdata.py", line 383, in add_provider_internal
    raise bb.providers.NoProvider(item)
    NoProvider: x11-image

    xeon@xeon-laptop:~/oe/build$


    please help because im new to beagleboard and linux

    ReplyDelete
  19. Hi, I am setting up OE env on Ubuntu 11.10. I am getting following error after doing "MACHINE=beagleboard ./oebb.sh bitbake virtual/kernel" command :

    DEBUG: BB /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/package_ipk.bbclass: handle(data, include)
    DEBUG: LOAD /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/package_ipk.bbclass
    DEBUG: BB configuration INHERITs:0: inheriting classes/image-prelink.bbclass
    DEBUG: BB /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/image-prelink.bbclass: handle(data, include)
    DEBUG: LOAD /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/image-prelink.bbclass
    DEBUG: BB configuration INHERITs:0: inheriting classes/license.bbclass
    DEBUG: BB /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/license.bbclass: handle(data, include)
    DEBUG: LOAD /home/vinay/BB/Aungstrom/setup-scripts/sources/openembedded-core/meta/classes/license.bbclass
    DEBUG: Clearing SRCREV cache due to cache policy of: clear
    ERROR: Error parsing configuration files
    Traceback (most recent call last):
    File "/home/vinay/BB/Aungstrom/setup-scripts/sources/bitbake/lib/bb/cooker.py", line 197, in BBCooker.loadConfigurationData():
    self.parseConfigurationFiles(self.configuration.prefile,
    > self.configuration.postfile)
    except SyntaxError:
    File "/home/vinay/BB/Aungstrom/setup-scripts/sources/bitbake/lib/bb/cooker.py", line 870, in BBCooker.parseConfigurationFiles(prefiles=[], postfiles=[]):
    if data.getVar("BB_WORKERCONTEXT", False) is None:
    > bb.fetch.fetcher_init(data)
    bb.codeparser.parser_cache_init(data)
    File "/home/vinay/BB/Aungstrom/setup-scripts/sources/bitbake/lib/bb/fetch2/__init__.py", line 228, in fetcher_init(d=):
    pass
    > revs.clear()
    else:
    File "/home/vinay/BB/Aungstrom/setup-scripts/sources/bitbake/lib/bb/persist_data.py", line 142, in SQLTable.clear():
    def clear(self):
    > self._execute("DELETE FROM %s;" % self.table)

    File "/home/vinay/BB/Aungstrom/setup-scripts/sources/bitbake/lib/bb/persist_data.py", line 66, in SQLTable._execute(*query=('DELETE FROM BB_URI_HEADREVS;',)):
    try:
    > return self.cursor.execute(*query)
    except sqlite3.OperationalError as exc:
    OperationalError: attempt to write a readonly database




    ANY HELP PLZ,

    ASHISH

    ReplyDelete