Difference between revisions of "User:MarcusC/TalosOpBuild"

From RCS Wiki
Jump to navigation Jump to search
(Updated gentoo package list)
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
Some notes on building talos-op-build
+
Some notes on building talos-op-build.
 +
 
 +
'''NB:'''  A complete build takes quite a lot of time and space.  On my AmigaOne X5000 (dual core e5500 @ 2GHz) the build took about 15 hours.  The final size of the build tree was 18 GB.
  
 
== Required packages on gentoo ==
 
== Required packages on gentoo ==
Line 8: Line 10:
 
  dev-perl/XML-LibXML
 
  dev-perl/XML-LibXML
 
  dev-util/cscope
 
  dev-util/cscope
 +
dev-util/ctags
 +
dev-libs/expat
 +
dev-libs/libxslt
 
  sys-libs/zlib[static-libs]
 
  sys-libs/zlib[static-libs]
 +
sys-devel/bc
  
 
== host-libopenssl-1.0.2m builds in 32-bit mode ==
 
== host-libopenssl-1.0.2m builds in 32-bit mode ==
Line 20: Line 26:
 
  op-build
 
  op-build
  
−
== Broken hcode reference ==
+
== FTBFS in hcode ==
 +
 
 +
Compiling the current master of hcode (e45e1293f43a534fc180133ec588733035cd2dc8), the following fatal error is generated:
 +
 
 +
../../import/chips/p9/procedures/ppe/tools/ppetracepp//ppetracepp.C: In function
 +
  ‘int main(int, char**)’:
 +
../../import/chips/p9/procedures/ppe/tools/ppetracepp//ppetracepp.C:1097:36: err
 +
or: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with att
 +
ribute warn_unused_result [-Werror=unused-result]
 +
          fgets(buf, MAX_BUFFER, PWD);
 +
                                    ^
 +
cc1plus: all warnings being treated as errors
 +
 
 +
To fix, place the following patch in openpower/package/hcode/fgets.patch:
 +
 
 +
--- a/import/chips/p9/procedures/ppe/tools/ppetracepp/ppetracepp.C 2018-02-15 11:37:48.995064250 +0100
 +
+++ b/import/chips/p9/procedures/ppe/tools/ppetracepp/ppetracepp.C 2018-02-15 11:47:24.463767244 +0100
 +
@@ -1094,7 +1094,7 @@
 +
 
 +
          string pwd;
 +
          FILE* PWD = popen("pwd", "r");
 +
-        fgets(buf, MAX_BUFFER, PWD);
 +
+        if(!fgets(buf, MAX_BUFFER, PWD)) fprintf(stderr, "Failed to run pwd\n");
 +
          pwd = buf;
 +
          pclose(PWD);
 +
          time_t tt = time(NULL);
 +
 
 +
== FTBFS in host-squashfs ==
 +
 
 +
There is a missing #include directive in the squashfs sources used, which leads to compilation errors such as these:
 +
mksquashfs.c: In function ‘create_inode’:
 +
mksquashfs.c:991:24: error: called object ‘major’ is not a function or function
 +
pointer
 +
    unsigned int major = major(buf->st_rdev);
 +
                        <nowiki>^~~~~</nowiki>
 +
 
 +
To fix, place the following patch in buildroot/package/squashfs/sysmacros.patch:
 +
 
 +
--- a/squashfs-tools/mksquashfs.c      2018-02-15 14:11:24.661930637 +0100
 +
+++ b/squashfs-tools/mksquashfs.c      2018-02-15 14:12:28.218412025 +0100
 +
@@ -44,6 +44,7 @@
 +
  #include <setjmp.h>
 +
  #include <sys/types.h>
 +
  #include <sys/mman.h>
 +
+#include <sys/sysmacros.h>
 +
  #include <pthread.h>
 +
  #include <regex.h>
 +
  #include <fnmatch.h>
 +
--- a/squashfs-tools/unsquashfs.c      2018-02-15 14:25:34.608263881 +0100
 +
+++ b/squashfs-tools/unsquashfs.c      2018-02-15 14:26:01.376410327 +0100
 +
@@ -35,6 +35,7 @@
 +
  #include <sys/types.h>
 +
  #include <sys/time.h>
 +
  #include <sys/resource.h>
 +
+#include <sys/sysmacros.h>
 +
  #include <limits.h>
 +
  #include <ctype.h>
 +
 +
 
 +
== sbe fails to build with python 3 ==
 +
 
 +
This package uses the command "python" to launch scripts containing legacy constructs which are only valid Python 2, not Python 3.  Thus, if "python" invokes a Python 3 interpreter they will fail.  To fix, place the following patch in openpower/package/sbe/python.patch:
 +
 
 +
--- a/src/build/Makefile 2018-01-30 11:34:43.000000000 +0100
 +
+++ b/src/build/Makefile 2018-02-16 09:56:05.406489933 +0100
 +
@@ -165,7 +165,7 @@
 +
  # ---- SEEPROM Image  ------
 +
  # This removes all unecessary headers from the ELF executable
 +
  $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).dis: $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out
 +
- ./parsAndCutElf.py $(IMAGE_SUFFIX) $(IMG_DIR)
 +
+ python2 ./parsAndCutElf.py $(IMAGE_SUFFIX) $(IMG_DIR)
 +
  $(OBJDUMP) -S $< > $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).dis
 +
 
 +
  #create a linked ELF executable
 +
@@ -231,7 +231,7 @@
 +
  tar install
 +
 
 +
  buildinfo: $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin
 +
- ./updateBuildTag.py $(P9_XIP_TOOL) $(IMG_DIR) $(IMAGE_SEEPROM_NAME)
 +
+ python2 ./updateBuildTag.py $(P9_XIP_TOOL) $(IMG_DIR) $(IMAGE_SEEPROM_NAME)
 +
 
 +
  add_LoaderAddr: $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out
 +
  $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin set L1_LoaderAddr 0x`nm $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out | grep __l1Loader | cut -f 1 -d " "`
 +
@@ -253,7 +253,7 @@
 +
 
 +
  # Create build Info file
 +
  buildInfo:
 +
- python buildInfo.py $(GENFILES_DIR)
 +
+ python2 buildInfo.py $(GENFILES_DIR)
 +
 
 +
 
 +
  #Create an obj directory if needed
 +
@@ -316,7 +316,7 @@
 +
  $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin normalize
 +
 
 +
  compress:
 +
- $(BOOT_SRCDIR)/sbeCompression.py -l $(IMG_DIR) -i $(IMAGE_SEEPROM_NAME).bin
 +
+ python2 $(BOOT_SRCDIR)/sbeCompression.py -l $(IMG_DIR) -i $(IMAGE_SEEPROM_NAME).bin
 +
 
 +
  defaultset:$(SBE_TOOLS) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin normalize
 +
  $(TOOLS_ATTR_DIR)/ppeSetFixed.pl $(IMG_DIR) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin
 +
$(IMPORT_XML_DIR)/attribute_info/p9_sbe_attributes.xml $(ATTRFILES)
 +
@@ -327,7 +327,7 @@
 +
 
 +
  # generate whitelist and blacklist security algorithm
 +
  security:
 +
- python $(SECURITY_SRC_DIR)/securityRegListGen.py -f $(SECURITY_LIST) -o $(GENFILES_DIR)
 +
+ python2 $(SECURITY_SRC_DIR)/securityRegListGen.py -f $(SECURITY_LIST) -o $(GENFILES_DIR)
 +
 
 +
  # Build hwp_error_info.H.  If the script fails then print the contents of
 +
  # the header and then delete whatever garbage the script left to force it to
 +
 
 +
Then, make the following change in openpower/package/sbe/sbe.mk:
 +
 
 +
--- a/openpower/package/sbe/sbe.mk
 +
+++ b/openpower/package/sbe/sbe.mk
 +
@@ -23,7 +23,7 @@ endef
 +
 
 +
  define SBE_INSTALL_IMAGES_CMDS
 +
        $(INSTALL) -D $(@D)/images/p9_ipl_build  $(HOST_DIR)/usr/bin/
 +
-      python $(@D)/src/build/sbeOpDistribute.py --sbe_binary_dir=$(STAGING_DIR)/sbe_binaries --img_dir=$(@D)/images
 +
+      python2 $(@D)/src/build/sbeOpDistribute.py --sbe_binary_dir=$(STAGING_DIR)/sbe_binaries --img_dir=$(@D)/images
 +
        cp $(@D)/src/build/sbeOpDistribute.py $(STAGING_DIR)/sbe_binaries/
 +
  endef
 +
 
 +
 
 +
== openpower-pnor fails to install with python 3 ==
 +
 
 +
This package uses one of the python scripts from sbe, again using "python" to run it.  To fix, place the following patch in openpower/package/openpower-pnor/python.patch:
  
−
The Makefile for the hcode package (openpower/package/hcode/hcode.mk) specifies 37069e1756e2b8e70e8e185d28609414033dbc02 as the version to checkout.  However, no such commit exists in the git repo (https://scm.raptorcs.com/scm/git/talos-hcode).  Current master is at e45e1293f43a534fc180133ec588733035cd2dc8.  Maybe Raptor forgot to push something?
+
--- a/update_image.pl  2018-02-15 16:27:30.407412998 +0100
 +
+++ b/update_image.pl  2018-02-15 16:27:39.982815072 +0100
 +
  @@ -213,7 +213,7 @@
 +
 
 +
  # SBE image prep
 +
  if ($release eq "p9") {
 +
-    run_command("python $sbe_binary_dir/sbeOpDistribute.py --install --buildSbePart $hb_image_dir/buildSbePart.pl --hw_ref_image $hb_binary_dir/p9n.ref_image.bin --sbe_binary_filename $sbe_binary_filename --scratch_dir $scratch_dir --sbe_binary_dir $sbe_binary_dir");
 +
+    run_command("python2 $sbe_binary_dir/sbeOpDistribute.py --install --buildSbePart $hb_image_dir/buildSbePart.pl --hw_ref_image $hb_binary_dir/p9n.ref_image.bin --sbe_binary_filename $sbe_binary_filename --scratch_dir $scratch_dir --sbe_binary_dir $sbe_binary_dir");
 +
  }
 +
  else {
 +
      run_command("cp $hb_binary_dir/$sbe_binary_filename $scratch_dir/");
  
−
As a workaround, change the value of HCODE_VERSION in the Makefile from 37069e1756e2b8e70e8e185d28609414033dbc02 to e45e1293f43a534fc180133ec588733035cd2dc8.
+
----

Latest revision as of 03:56, 16 February 2018

Some notes on building talos-op-build.

NB: A complete build takes quite a lot of time and space. On my AmigaOne X5000 (dual core e5500 @ 2GHz) the build took about 15 hours. The final size of the build tree was 18 GB.

Required packages on gentoo

When building on gentoo, the following packages need to be installed: 

dev-perl/XML-Simple
dev-perl/XML-LibXML
dev-util/cscope
dev-util/ctags
dev-libs/expat
dev-libs/libxslt
sys-libs/zlib[static-libs]
sys-devel/bc

host-libopenssl-1.0.2m builds in 32-bit mode

If a compilation error complaining about the lack of <gnu/stubs-32.h> appears, this is because the libopenssl package is trying to compile in 32-bit mode and no 32-bit CRT is installed.

To fix, set the environment variable KERNEL_BITS to force a 64-bit build: 

export KERNEL_BITS=64
rm -rf output/build/host-libopenssl-1.0.2m
op-build

FTBFS in hcode

Compiling the current master of hcode (e45e1293f43a534fc180133ec588733035cd2dc8), the following fatal error is generated: 

../../import/chips/p9/procedures/ppe/tools/ppetracepp//ppetracepp.C: In function
 ‘int main(int, char**)’:
../../import/chips/p9/procedures/ppe/tools/ppetracepp//ppetracepp.C:1097:36: err
or: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with att
ribute warn_unused_result [-Werror=unused-result]
         fgets(buf, MAX_BUFFER, PWD);
                                    ^
cc1plus: all warnings being treated as errors

To fix, place the following patch in openpower/package/hcode/fgets.patch: 

--- a/import/chips/p9/procedures/ppe/tools/ppetracepp/ppetracepp.C	2018-02-15 11:37:48.995064250 +0100
+++ b/import/chips/p9/procedures/ppe/tools/ppetracepp/ppetracepp.C	2018-02-15 11:47:24.463767244 +0100
@@ -1094,7 +1094,7 @@
 
         string pwd;
         FILE* PWD = popen("pwd", "r");
-        fgets(buf, MAX_BUFFER, PWD);
+        if(!fgets(buf, MAX_BUFFER, PWD)) fprintf(stderr, "Failed to run pwd\n");
         pwd = buf;
         pclose(PWD);
         time_t tt = time(NULL);

FTBFS in host-squashfs

There is a missing #include directive in the squashfs sources used, which leads to compilation errors such as these: 

mksquashfs.c: In function ‘create_inode’:
mksquashfs.c:991:24: error: called object ‘major’ is not a function or function 
pointer
   unsigned int major = major(buf->st_rdev); 
                        ^~~~~

To fix, place the following patch in buildroot/package/squashfs/sysmacros.patch: 

--- a/squashfs-tools/mksquashfs.c       2018-02-15 14:11:24.661930637 +0100
+++ b/squashfs-tools/mksquashfs.c       2018-02-15 14:12:28.218412025 +0100
@@ -44,6 +44,7 @@
 #include <setjmp.h>
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <sys/sysmacros.h>
 #include <pthread.h>
 #include <regex.h>
 #include <fnmatch.h>
--- a/squashfs-tools/unsquashfs.c       2018-02-15 14:25:34.608263881 +0100
+++ b/squashfs-tools/unsquashfs.c       2018-02-15 14:26:01.376410327 +0100
@@ -35,6 +35,7 @@
 #include <sys/types.h>
 #include <sys/time.h>
 #include <sys/resource.h>
+#include <sys/sysmacros.h>
 #include <limits.h>
 #include <ctype.h>

sbe fails to build with python 3

This package uses the command "python" to launch scripts containing legacy constructs which are only valid Python 2, not Python 3. Thus, if "python" invokes a Python 3 interpreter they will fail. To fix, place the following patch in openpower/package/sbe/python.patch: 

--- a/src/build/Makefile	2018-01-30 11:34:43.000000000 +0100
+++ b/src/build/Makefile	2018-02-16 09:56:05.406489933 +0100
@@ -165,7 +165,7 @@
 # ---- SEEPROM Image  ------
 # This removes all unecessary headers from the ELF executable
 $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).dis: $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out
-	./parsAndCutElf.py $(IMAGE_SUFFIX) $(IMG_DIR)
+	python2 ./parsAndCutElf.py $(IMAGE_SUFFIX) $(IMG_DIR)
 	$(OBJDUMP) -S $< > $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).dis
 
 #create a linked ELF executable
@@ -231,7 +231,7 @@
 		tar install
 
 buildinfo: $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin
-	./updateBuildTag.py $(P9_XIP_TOOL) $(IMG_DIR) $(IMAGE_SEEPROM_NAME)
+	python2 ./updateBuildTag.py $(P9_XIP_TOOL) $(IMG_DIR) $(IMAGE_SEEPROM_NAME)
 
 add_LoaderAddr: $(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out
 	$(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin set L1_LoaderAddr 0x`nm $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).out | grep __l1Loader | cut -f 1 -d " "`
@@ -253,7 +253,7 @@
 
 # Create build Info file
 buildInfo:
-	python buildInfo.py $(GENFILES_DIR)
+	python2 buildInfo.py $(GENFILES_DIR)
 
 
 #Create an obj directory if needed
@@ -316,7 +316,7 @@
 	$(P9_XIP_TOOL) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin normalize
 
 compress:
-	$(BOOT_SRCDIR)/sbeCompression.py -l $(IMG_DIR) -i $(IMAGE_SEEPROM_NAME).bin
+	python2 $(BOOT_SRCDIR)/sbeCompression.py -l $(IMG_DIR) -i $(IMAGE_SEEPROM_NAME).bin
 
 defaultset:$(SBE_TOOLS) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin normalize
 	$(TOOLS_ATTR_DIR)/ppeSetFixed.pl $(IMG_DIR) $(IMG_DIR)/$(IMAGE_SEEPROM_NAME).bin 
$(IMPORT_XML_DIR)/attribute_info/p9_sbe_attributes.xml $(ATTRFILES)
@@ -327,7 +327,7 @@
 
 # generate whitelist and blacklist security algorithm
 security:
-	python $(SECURITY_SRC_DIR)/securityRegListGen.py -f $(SECURITY_LIST) -o $(GENFILES_DIR)
+	python2 $(SECURITY_SRC_DIR)/securityRegListGen.py -f $(SECURITY_LIST) -o $(GENFILES_DIR)
 
 # Build hwp_error_info.H.  If the script fails then print the contents of
 # the header and then delete whatever garbage the script left to force it to

Then, make the following change in openpower/package/sbe/sbe.mk: 

--- a/openpower/package/sbe/sbe.mk
+++ b/openpower/package/sbe/sbe.mk
@@ -23,7 +23,7 @@ endef
 
 define SBE_INSTALL_IMAGES_CMDS
        $(INSTALL) -D $(@D)/images/p9_ipl_build  $(HOST_DIR)/usr/bin/
-       python $(@D)/src/build/sbeOpDistribute.py --sbe_binary_dir=$(STAGING_DIR)/sbe_binaries --img_dir=$(@D)/images
+       python2 $(@D)/src/build/sbeOpDistribute.py --sbe_binary_dir=$(STAGING_DIR)/sbe_binaries --img_dir=$(@D)/images
        cp $(@D)/src/build/sbeOpDistribute.py $(STAGING_DIR)/sbe_binaries/
 endef

openpower-pnor fails to install with python 3

This package uses one of the python scripts from sbe, again using "python" to run it. To fix, place the following patch in openpower/package/openpower-pnor/python.patch: 

--- a/update_image.pl   2018-02-15 16:27:30.407412998 +0100
+++ b/update_image.pl   2018-02-15 16:27:39.982815072 +0100
@@ -213,7 +213,7 @@
 
 # SBE image prep
 if ($release eq "p9") {
-    run_command("python $sbe_binary_dir/sbeOpDistribute.py --install --buildSbePart $hb_image_dir/buildSbePart.pl --hw_ref_image $hb_binary_dir/p9n.ref_image.bin --sbe_binary_filename $sbe_binary_filename --scratch_dir $scratch_dir --sbe_binary_dir $sbe_binary_dir");
+    run_command("python2 $sbe_binary_dir/sbeOpDistribute.py --install --buildSbePart $hb_image_dir/buildSbePart.pl --hw_ref_image $hb_binary_dir/p9n.ref_image.bin --sbe_binary_filename $sbe_binary_filename --scratch_dir $scratch_dir --sbe_binary_dir $sbe_binary_dir");
 }
 else {
     run_command("cp $hb_binary_dir/$sbe_binary_filename $scratch_dir/");
Retrieved from "https://wiki.raptorcs.com/w/index.php?title=User:MarcusC/TalosOpBuild&oldid=713"