10 Replies Latest reply on Nov 21, 2011 2:15 PM by Marek Goldmann

    building ec2 ebs of fedora 16 base system fails on meta appliance

    bradrubenstein Newbie

      I'm brand new to boxgrinder, and I tried to build a Fedora 16 base system using the ec2 meta-appliance after reading the tutorials.  But it seems to fail, and I'm not sure what I'm missing.

       

      I apologize if I missed some step, or some piece of documentation.  I'm attaching the complete log file

       

      It seems like it fails when it tries to set up grub, whereas Fedora 16 uses grub2.  (Glancing through the ruby code, it does seem to be grub2 aware, though...)

       

      From log:

       

      D, [2011-11-18T16:52:15.145308 #18057] DEBUG -- : Writing grub /home/ec2-user/appliance/build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp/imgcreate-8leBaN/install_root/boot/grub/device.map

      D, [2011-11-18T16:52:15.314807 #18057] DEBUG -- : Writing grub config /home/ec2-user/appliance/build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp/imgcreate-8leBaN/install_root/boot/grub/grub.conf

      D, [2011-11-18T16:52:15.315963 #18057] DEBUG -- : Unable to create appliance : grub not installed : /home/ec2-user/appliance/build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp/imgcreate-8leBaN/install_root/usr/share/grub/i386-redhat/e2fs_stage1_5 not found

      D, [2011-11-18T16:52:15.746367 #18057] DEBUG -- : Unmounting directory /home/ec2-user/appliance/build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp/imgcreate-8leBaN/install_root/

       

      ...

      E, [2011-11-18T16:52:16.943202 #18057] ERROR -- : /usr/lib/ruby/gems/1.8/gems/boxgrinder-core-0.3.8/lib/boxgrinder-core/helpers/exec-helper.rb:85:in `execute'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb:165:in `execute_appliance_creator'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb:129:in `build_with_appliance_creator'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/fedora/fedora-plugin.rb:45:in `execute'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/base-plugin.rb:194:in `run'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:141:in `execute_plugin'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `each'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:120:in `create'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/bin/boxgrinder-build:177

      /usr/bin/boxgrinder-build:19:in `load'

      /usr/bin/boxgrinder-build:19

      F, [2011-11-18T16:52:16.944608 #18057] FATAL -- : RuntimeError: An error occurred while executing command: 'appliance-creator -d -v -t 'build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp' --cache=/var/cache/boxgrinder/rpms-cache/x86_64/fedora/16 --config 'build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp/f16-vpn.ks' -o 'build/appliances/x86_64/fedora/16/f16-vpn/1.0/fedora-plugin/tmp' --name 'f16-vpn' --vmem 256 --vcpu 1 --format raw', process exited with wrong exit status: 1

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-core-0.3.8/lib/boxgrinder-core/helpers/exec-helper.rb:92:in `execute'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb:165:in `execute_appliance_creator'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb:129:in `build_with_appliance_creator'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/os/fedora/fedora-plugin.rb:45:in `execute'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/plugins/base-plugin.rb:194:in `run'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:141:in `execute_plugin'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `each'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/lib/boxgrinder-build/appliance.rb:120:in `create'

      /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.7/bin/boxgrinder-build:177

      /usr/bin/boxgrinder-build:19:in `load'

      /usr/bin/boxgrinder-build:19

        • 1. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
          Marek Goldmann Master

          First of all you need to have latest version - 0.9.8. It's not yet in Fedora stable repos, you need to grab relevant bits from here: https://admin.fedoraproject.org/updates/FEDORA-2011-15920 You'll need also updated appliance-tools from here: https://admin.fedoraproject.org/updates/appliance-tools-006.1-1.fc15

           

          After these builds get pushed to stable - you'll have it automatically installed at meta-appliance boot time.

           

          --Marek

          • 2. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
            bradrubenstein Newbie

            Thank you for the response, Marek! 

             

            I also got a failure building the Fedora 14 base system using the meta-appliance (using LIBGUESTFS_MEMSIZE=300 on t1.micro instance of ami-98ab50f1).

             

            I'll look at the links, but I expect I'll need to wait until the meta-appliance is working.  (Although it occurs to me I could patch the meta-appliance instance... I have to stop thinking of them as "other people's machines" :-)).

             

            -- Brad

            • 3. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
              bradrubenstein Newbie

              OK - I added

               

              Nov 19 13:41:54 Updated: rubygem-boxgrinder-core-0.3.9-1.fc17.noarch

              Nov 19 13:41:54 Updated: appliance-tools-006.1-1.fc17.noarch

              Nov 19 13:41:55 Updated: rubygem-boxgrinder-build-0.9.8-2.fc17.noarch

               

              to the ami-98ab50f1 appliance, and did the base Fedora 16 build.  It got much farther, then failed at:

               

              T, [2011-11-19T14:10:21.122058 #944] TRACE -- : Mounting partitions...

              D, [2011-11-19T14:10:21.122226 #944] DEBUG -- : GFS: list_partitions

              T, [2011-11-19T14:10:21.123744 #944] TRACE -- : GFS: guestfsd: main_loop: proc 8 (list_partitions) took 0.00 seconds^M

              guestfsd: main_loop: new request, len 0x28

              D, [2011-11-19T14:10:21.124730 #944] DEBUG -- : GFS: list_partitions = ["/dev/vda1", "/dev/vda2"]

              D, [2011-11-19T14:10:21.126431 #944] DEBUG -- : GFS: vfs_type "/dev/vda1"

              T, [2011-11-19T14:10:21.127690 #944] TRACE -- : GFS: guestfsd: main_loop: proc 8 (list_partitions) took 0.00 seconds

              T, [2011-11-19T14:10:21.129351 #944] TRACE -- : GFS: guestfsd: main_loop: new request, len 0x38

              T, [2011-11-19T14:10:21.134558 #944] TRACE -- : GFS: blkid -o value -s TYPE /dev/vda1

              T, [2011-11-19T14:10:21.205137 #944] TRACE -- : GFS: guestfsd: error: /dev/vda1:

              D, [2011-11-19T14:10:21.205490 #944] DEBUG -- : GFS: vfs_type = NULL (error)

              F, [2011-11-19T14:10:21.206395 #944] FATAL -- : Guestfs::Error: vfs_type: /dev/vda1:

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:272:in `vfs_type'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:272:in `mountable_partitions'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:272:in `reject'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:272:in `mountable_partitions'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:255:in `mount_partitions'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:199:in `execute'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:174:in `customize'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:169:in `prepare_guestfs'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:77:in `log_callback'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:173:in `customize'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/image-helper.rb:129:in `customize'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb:134:in `build_with_appliance_creator'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/os/fedora/fedora-plugin.rb:45:in `execute'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/base-plugin.rb:192:in `run'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:139:in `execute_plugin'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `each'

              /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

              • 5. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                Marek Goldmann Master

                I'm pretty sure libguestfs needs to be updated to latest available version. Please run yum update *libguestfs* to perform the udate and re-run the build then.

                 

                LIBGUESTFS_MEMSIZE=300 is no longer required, as we use this setting by default now, see https://issues.jboss.org/browse/BGBUILD-246.

                 

                P.S. Fedora 14 will be EOLed in 2 weeks, but I can see that you're building F16.

                 

                --Marek

                • 6. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                  bradrubenstein Newbie

                  No joy yet.  Per your suggestion, on ami-98ab50f1, I updated libguestfs from the default repos, and rubygem-boxgrinder-build (and dependents) from rawhide.

                   

                  Nov 21 12:42:56 Updated: 1:libguestfs-1.10.9-2.fc15.x86_64

                  Nov 21 12:42:56 Updated: 1:ruby-libguestfs-1.10.9-2.fc15.x86_64

                  Nov 21 12:42:57 Updated: 1:guestfish-1.10.9-2.fc15.x86_64

                  Nov 21 12:43:53 Updated: rubygem-boxgrinder-core-0.3.9-1.fc17.noarch

                  Nov 21 12:43:54 Updated: appliance-tools-006.1-1.fc17.noarch

                  Nov 21 12:43:54 Updated: rubygem-boxgrinder-build-0.9.8-2.fc17.noarch

                   

                  The result of building a base F16 system continues to fail, while "synchronizing filesystems".  It looks like it fails at the same place.

                   

                  D, [2011-11-21T13:42:03.757196 #1021] DEBUG -- : GFS: set_e2label "/dev/vda2" "79d3d2d4"

                  D, [2011-11-21T13:42:03.870065 #1021] DEBUG -- : GFS: set_e2label = 0

                  D, [2011-11-21T13:42:03.870445 #1021] DEBUG -- : Copying files...

                  D, [2011-11-21T13:42:03.871761 #1021] DEBUG -- : GFS: cp_a "/in/" "/out"

                  D, [2011-11-21T13:49:57.858107 #1021] DEBUG -- : GFS: cp_a = -1 (error)

                  F, [2011-11-21T13:49:57.949241 #1021] FATAL -- : Guestfs::Error: unexpected end of file when reading from daemon.

                  See earlier debug messages.

                  Or you can run 'libguestfs-test-tool' and post the complete output into

                  a bug report or message to the libguestfs mailing list.

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/image-helper.rb:98:in `cp_a'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/image-helper.rb:98:in `sync_filesystem'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/platform/ec2/ec2-plugin.rb:42:in `execute'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/image-helper.rb:130:in `customize'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:176:in `customize'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:169:in `prepare_guestfs'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:77:in `log_callback'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:123:in `initialize_guestfs'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/guestfs-helper.rb:173:in `customize'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/helpers/image-helper.rb:129:in `customize'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/platform/ec2/ec2-plugin.rb:41:in `execute'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/plugins/base-plugin.rb:192:in `run'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:139:in `execute_plugin'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `each'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:105:in `execute_plugin_chain'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/lib/boxgrinder-build/appliance.rb:120:in `create'

                  /usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.9.8/bin/boxgrinder-build:189

                  /usr/bin/boxgrinder-build:19:in `load'

                  /usr/bin/boxgrinder-build:19

                   

                  I'm going to let this go for a while, and try a "fry" rather than "bake" approach.  I'll continue to follow the boxgrinder discussion, to see how it is progressing.  Thank you for your help, Marek!

                  • 7. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                    bradrubenstein Newbie

                    Here's the most recent build's log, indicating the failure.

                    • 8. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                      Marek Goldmann Master

                      It fails in completely different place. Could yu pleae run it with --trace to get more details about the failure?

                       

                      --Marek

                      • 9. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                        Marek Goldmann Master

                        It seems I was too quick, thanks for the log

                         

                        --Marek

                        • 10. Re: building ec2 ebs of fedora 16 base system fails on meta appliance
                          Marek Goldmann Master

                          In general - things like this doesn't happen. We need respin our meta-appliances to have more updated systems and most probably switch to F16. I'll try to reproduce your issue though on the AMI you mentioned. Maybe the 300 MB limit for guestfs is too low in your case.

                           

                          --Marek