[gijoe77]

ok I think I might have figured something out. I just noticed that those bzlib.h unresolved symbols are only happeneing with mipspro and not gcc, so I re-ran everything using gcc, it appears not to see/use the nekoware/include/bzlib.h at all:

Code:

export CC=/usr/nekoware/gcc-4.7/bin/gcc
export CXX=/usr/nekoware/gcc-4.7/bin/g++
export CFLAGS="-std=gnu99 -g0 -O2 -mips4"
export CXXFLAGS="-g0 -O2 -mips4"

./configure --prefix=/usr/local --includedir=/usr/local,/usr/nekoware

<snip>
checking bzlib.h usability... no
checking bzlib.h presence... no
checking for bzlib.h... no
checking for BZ2_bzDecompressInit in -lbz2... no
<snip>

gmake
  CCLD     bsdtar
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_new" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_free" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "EVP_CIPHER_CTX_reset" -- 1st referenced by ./.libs/libarchive.a(archive_cryptor.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: INFO    152: Output file removed because of error.
gmake[1]: *** [bsdtar] Error 2

so gcc doesn't try to use bzlib.h, mipspro does - so I modified the configure script to disable bzlib and it's now behaving like gcc:

Code:

export CC=cc
export CXX=CC
export CFLAGS="-c99 -g0 -O2 -mips4 -I/usr/nekoware/include -I/usr/local/include -I/usr/local/include/openssl"
export CXXFLAGS="-g0 -O2 -mips4"

./configure --prefix=/usr/local --includedir=/usr/local,/usr/nekoware --without-bz2lib
/usr/nekoware/bin/sed -i "s/-Wall -Wformat/-woff all/g" Makefile
gmake

<snip>
  CCLD     bsdtar
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_new" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_free" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "EVP_CIPHER_CTX_reset" -- 1st referenced by ./.libs/libarchive.a(archive_cryptor.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: INFO    152: Output file removed because of error.
gmake[1]: *** [bsdtar] Error 2

I suppose it would be nice to add bz2lib support. For giggles I copied /usr/nekoware/include/bzlib.h to /usr/include/ and the configure script had no issues this time with bzlib.h being usable, but the linker still bombed out with unresolved BZ2* symbols.

So it would seem to me that somewhere, somehow the configure script needs to be pointed to the correct include dir and libraries dir, even though I passed "--includedir=/usr/local,/usr/nekoware " to the configure script and manually put in " -I/usr/nekoware/include -I/usr/local/include -I/usr/local/include/openssl" in the $CFLAGS. I'll see if I can dig around some more, but I am just completely confused with this configure/automake stuff. I am betting that once I can figure that out the rest of the openssl problems will also get resolved.

I got a good lead to look at CPPFLAGS from Tru in some other thread so I'll be investigating that

[Raion]

So yeah Gijoe, here's the reason I'm interested in getting this working.

We don't need BSD cpio, but I want BSD tar for irisware so we can distribute packages as tardist.xz and GNU tar doesnt work with inst/swmgr.

If you can get a patch out and dependency lists published, then it resolves most of our issues.

[jpstewart]

<QUOTE author="gijoe77" post_id="1826" time="1532988641" user_id="243">

ok I think I might have figured something out. I just noticed that those bzlib.h unresolved symbols are only happeneing with mipspro and not gcc, so I re-ran everything using gcc, it appears not to see/use the nekoware/include/bzlib.h at all:

Code:

export CC=/usr/nekoware/gcc-4.7/bin/gcc
export CXX=/usr/nekoware/gcc-4.7/bin/g++
export CFLAGS="-std=gnu99 -g0 -O2 -mips4"
export CXXFLAGS="-g0 -O2 -mips4"

... snip ...
<QUOTE author="gijoe77" post_id="1826" time="1532988641" user_id="243">

so gcc doesn't try to use bzlib.h, mipspro does - so I modified the configure script to disable bzlib and it's now behaving like gcc:

Code:

export CC=cc
export CXX=CC
export CFLAGS="-c99 -g0 -O2 -mips4 -I/usr/nekoware/include -I/usr/local/include -I/usr/local/include/openssl"
export CXXFLAGS="-g0 -O2 -mips4"

./configure --prefix=/usr/local --includedir=/usr/local,/usr/nekoware --without-bz2lib
/usr/nekoware/bin/sed -i "s/-Wall -Wformat/-woff all/g" Makefile

I notice a couple of things there. In the MIPSpro version, you've got "-I/usr/nekoware/include" but it's missing from the GCC compilation. That's why it's not finding nekoware/include/bzlib.h. You need the -I option regardless of which compiler you use. You'll also need a corresponding "-L/usr/nekoware/lib" option (either in LDFLAGS <I>or</I> in <I>both</I> CFLAGS and CXXFLAGS). Probably also "-L/usr/local/lib" to correspond to the other -I options. That will tell the linker where to find all the necessary libraries to get rid of the linker errors.

Also, your "--includedir=/usr/local,/usr/nekoware" option doesn't make sense. The --include option to configure tells it where to <I>install</I> headers belonging to the package currently being built. It looks to me like you're trying to use --include to tell it the path to search, but that's what the -I option in CFLAGS is for. If you install a package built with that --include option it will likely take that to be a single, literal path name where it will put the header files!

Oh, and all those -I options need to be in both CFLAGS and CXXFLAGS (for the C++ compiler to see them too, if you're building C++ sources) or in CPPFLAGS (because technically they're for the pre-processor to both compilers).

[gijoe77]

<QUOTE author="jpstewart" post_id="1828" time="1532994237" user_id="129">

I notice a couple of things there. In the MIPSpro version, you've got "-I/usr/nekoware/include" but it's missing from the GCC compilation. That's why it's not finding nekoware/include/bzlib.h. You need the -I option regardless of which compiler you use. You'll also need a corresponding "-L/usr/nekoware/lib" option (either in LDFLAGS <I>or</I> in <I>both</I> CFLAGS and CXXFLAGS). Probably also "-L/usr/local/lib" to correspond to the other -I options. That will tell the linker where to find all the necessary libraries to get rid of the linker errors.

Also, your "--includedir=/usr/local,/usr/nekoware" option doesn't make sense. The --include option to configure tells it where to <I>install</I> headers belonging to the package currently being built. It looks to me like you're trying to use --include to tell it the path to search, but that's what the -I option in CFLAGS is for. If you install a package built with that --include option it will likely take that to be a single, literal path name where it will put the header files!

Oh, and all those -I options need to be in both CFLAGS and CXXFLAGS (for the C++ compiler to see them too, if you're building C++ sources) or in CPPFLAGS (because technically they're for the pre-processor to both compilers).

ok thank you for clearing things up for me - I've been messing with the configure script for a few programs and either I guess they seem to take different options or I'm just confusing myself further - but I bet its both . I usually just add the CXX and CXXFLAGS just in case, not sure if that's a good habit to develop...

so I cleaned it all up and here is basically what I have:

added below to line 112 of "libarchive/archive_read_disk_posix.c" (really could put it anywhere I guess)

Code:

#define dirfd(dp) (((DIR *) (dp))->__dd_fd)

Code:

export CC=cc
export CFLAGS="-c99 -g0 -O2 -mips4"
export CPPFLAGS="-I/usr/nekoware/include -I/usr/local/include -L/usr/nekoware/lib -L/usr/local/lib"

./configure --prefix=/usr/local
/usr/nekoware/bin/sed -i "s/-Wall -Wformat/-woff all/g" Makefile
#or
#/usr/nekoware/bin/sed -i "s/-Wall -Wformat/-woff 1009,1014,1110,1116,1185,1188,1204,1230,1233 -Wl,-woff,85,-woff,84/g" Makefile
gmake

<snip>

  CCLD     bsdtar
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_new" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "HMAC_CTX_free" -- 1st referenced by ./.libs/libarchive.a(archive_hmac.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: ERROR   33 : Unresolved text symbol "EVP_CIPHER_CTX_reset" -- 1st referenced by ./.libs/libarchive.a(archive_cryptor.o).
        Use linker option -v to see when and which objects, archives and dsos are loaded.  
ld32: INFO    152: Output file removed because of error.
gmake[1]: *** [bsdtar] Error 2

I also tried setting the -I and -L dir's in the CFLAGS, same result

Code:

export CC=cc
export CFLAGS="-c99 -g0 -O2 -mips4 -I/usr/nekoware/include -I/usr/local/include -L/usr/nekoware/lib -L/usr/local/lib"
export CPPFLAGS="-I/usr/nekoware/include -I/usr/local/include -L/usr/nekoware/lib -L/usr/local/lib"

I'm still digging around why these openssl symbols are unresolved, I'm not really getting anywhere yet. /usr/local/openssl/hmac.h and /usr/local/openssl/evp.h are there...

I verified the openssl I compiled works Here is a speed test of openssl, this was on a maxed out 4x1ghz Tezro if anyone wants to compare:

Code:

-bash-4.3$ /usr/local/bin/openssl speed
<snip>
OpenSSL 1.1.0h  27 Mar 2018
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: cc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\""
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md2                  0.00         0.00         0.00         0.00         0.00         0.00
mdc2              1679.24k     2144.30k     2305.28k     2349.38k     2361.71k     2361.71k
md4               5485.22k    18879.36k    53822.26k   100255.76k   134069.34k   137324.22k
md5              14853.98k    37399.52k    70471.58k    90895.93k    99257.45k    99936.92k
hmac(md5)         4957.30k    16473.75k    43942.19k    75905.11k    96199.83k    98128.65k
sha1             13578.18k    33305.02k    61799.26k    78516.48k    85377.02k    86210.53k
rmd160            3912.26k    11385.43k    24840.79k    35287.04k    40283.27k    40566.78k
rc4              93837.20k   104095.42k   106840.49k   107541.50k   108112.48k   107702.95k
des cbc          13698.03k    14193.62k    14312.28k    14341.80k    14341.46k    14346.92k
des ede3          4944.55k     4994.43k     5011.71k     5016.23k     5016.23k     5018.97k
idea cbc         13859.51k    14442.03k    14595.75k    14635.35k    14644.57k    14647.30k
seed cbc         20394.14k    21708.89k    22018.65k    22080.51k    22145.71k    22151.17k
rc2 cbc          12599.09k    13049.71k    13213.28k    13199.02k    13208.23k    13205.50k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00         0.00
blowfish cbc     23596.45k    25537.69k    26027.35k    26240.77k    26178.90k    26187.09k
cast cbc         19278.85k    20492.35k    20813.40k    20895.06k    20964.94k    20911.45k
aes-128 cbc      28373.99k    30690.20k    31554.10k    31636.14k    31694.85k    31686.66k
aes-192 cbc      24464.65k    26226.88k    26773.33k    26997.30k    26943.49k    26951.68k
aes-256 cbc      21505.01k    22854.40k    23282.18k    23389.18k    23423.66k    23507.48k
camellia-128 cbc    23301.61k    25018.05k    25614.56k    25630.72k    25690.11k    25695.57k
camellia-192 cbc    18439.47k    19490.24k    19802.20k    19862.53k    19967.66k    19906.56k
camellia-256 cbc    18425.55k    19488.70k    19801.77k    19862.87k    19901.10k    19906.56k
sha256            7491.75k    16702.04k    28744.52k    35118.39k    37420.18k    37754.67k
sha512            6255.31k    24216.60k    37256.56k    51836.66k    58491.98k    59031.72k
whirlpool         4343.17k     9351.83k    15760.78k    19030.99k    20306.97k    20370.77k
aes-128 ige      27734.15k    30056.17k    30753.54k    30920.36k    30968.49k    31041.93k
aes-192 ige      24031.93k    25772.18k    26285.74k    26398.04k    26432.85k    26427.39k
aes-256 ige      21261.07k    22520.43k    22909.95k    22999.04k    23027.71k    23019.52k
ghash            48929.67k    56663.93k    59056.81k    59735.04k    59902.63k    59927.21k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000631s 0.000057s   1585.4  17550.1
rsa 1024 bits 0.002959s 0.000146s    337.9   6843.6
rsa 2048 bits 0.018469s 0.000412s     54.1   2428.7
rsa 3072 bits 0.053404s 0.000899s     18.7   1112.1
rsa 4096 bits 0.105474s 0.001374s      9.5    727.8
rsa 7680 bits 0.640000s 0.004973s      1.6    201.1
rsa 15360 bits 4.673333s 0.019065s      0.2     52.5
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000814s 0.000614s   1229.0   1629.7
dsa 1024 bits 0.001981s 0.001709s    504.8    585.3
dsa 2048 bits 0.005517s 0.004995s    181.3    200.2
                              sign    verify    sign/s verify/s
160 bit ecdsa (secp160r1)   0.0011s   0.0036s    946.0    277.7
192 bit ecdsa (nistp192)   0.0011s   0.0043s    870.4    234.8
224 bit ecdsa (nistp224)   0.0010s   0.0038s    967.4    265.4
256 bit ecdsa (nistp256)   0.0011s   0.0042s    920.6    236.1
384 bit ecdsa (nistp384)   0.0022s   0.0088s    453.9    113.7
521 bit ecdsa (nistp521)   0.0050s   0.0187s    200.7     53.5
163 bit ecdsa (nistk163)   0.0016s   0.0058s    634.4    173.6
233 bit ecdsa (nistk233)   0.0024s   0.0081s    413.1    123.3
283 bit ecdsa (nistk283)   0.0039s   0.0161s    253.7     61.9
409 bit ecdsa (nistk409)   0.0089s   0.0333s    111.9     30.0
571 bit ecdsa (nistk571)   0.0192s   0.0699s     52.2     14.3
163 bit ecdsa (nistb163)   0.0016s   0.0061s    635.3    163.3
233 bit ecdsa (nistb233)   0.0024s   0.0086s    417.2    116.1
283 bit ecdsa (nistb283)   0.0040s   0.0173s    252.2     57.9
409 bit ecdsa (nistb409)   0.0090s   0.0367s    111.6     27.2
571 bit ecdsa (nistb571)   0.0191s   0.0782s     52.3     12.8
                              op      op/s
160 bit ecdh (secp160r1)   0.0031s    323.1
192 bit ecdh (nistp192)   0.0035s    284.7
224 bit ecdh (nistp224)   0.0031s    321.0
256 bit ecdh (nistp256)   0.0036s    276.6
384 bit ecdh (nistp384)   0.0072s    138.2
521 bit ecdh (nistp521)   0.0156s     63.9
163 bit ecdh (nistk163)   0.0029s    350.7
233 bit ecdh (nistk233)   0.0040s    249.6
283 bit ecdh (nistk283)   0.0079s    127.1
409 bit ecdh (nistk409)   0.0166s     60.3
571 bit ecdh (nistk571)   0.0346s     28.9
163 bit ecdh (nistb163)   0.0030s    328.7
233 bit ecdh (nistb233)   0.0042s    238.3
283 bit ecdh (nistb283)   0.0086s    116.3
409 bit ecdh (nistb409)   0.0183s     54.7
571 bit ecdh (nistb571)   0.0392s     25.5
253 bit ecdh (X25519)   0.0000s      inf
-bash-4.3$

[gijoe77]

Ok I got it figured out - It just dawned on me as I was re-reading my post. I know for a fact "HMAC_CTX_new", "HMAC_CTX_free" and "EVP_CIPHER_CTX_reset" are in openssl1.1.0 so those unresolved symbols should not be happening, and I read somewhere those are actually new to 1.1.0, so I figured that the configure script was using an older version of openssl, which it turns out is installed in my nekoware install.

As a few of you have helpfully pointed out to me, order matters - and I had the nekoware include and libs first in the CPPFLAGS order, therefore now it makes sense why the compiler was using an older/wrong version of openssl! So this is what I did:

Code:

export CC=cc
export CFLAGS="-c99 -g0 -O2 -mips4"
export CPPFLAGS="-I/usr/local/include -I/usr/nekoware/include -L/usr/local/lib -L/usr/nekoware/lib"

./configure --prefix=/usr/local
/usr/nekoware/bin/sed -i "s/-Wall -Wformat/-woff all/g" Makefile
gmake

Unfortunately I don't yet know how to test it - my dgen compile that was going to use it is failing, I will post that failure in the dgen thread (here: <URL url="https://forums.irix.cc/viewtopic.php?f=10&t=220&p=1865#p1865"><LINK_TEXT text="viewtopic.php?f=10&t=220&p=1865#p1865">https://forums.irix.cc/viewtopic.php?f=10&t=220&p=1865#p1865</LINK_TEXT>)

[gijoe77]

<QUOTE author="Raion-Fox" post_id="1827" time="1532989335" user_id="49">

So yeah Gijoe, here's the reason I'm interested in getting this working.

We don't need BSD cpio, but I want BSD tar for irisware so we can distribute packages as tardist.xz and GNU tar doesnt work with inst/swmgr.

If you can get a patch out and dependency lists published, then it resolves most of our issues.

I can work on getting all that sorted, but I'm really green behind the ears so I might be asking for some help until I fully understand what I'm doing

btw, I'm not sure if I'm doing this right, but perhaps there is still a problem with this compile, I would assume the below should create a .xz archive:

Code:

-bash-4.3$ ls
-bash-4.3$ echo test > test.txt
-bash-4.3$ /usr/local/bin/bsdtar -cJf test.xz test.txt
bsdtar: Unsupported compression option --xz
-bash-4.3$ /usr/local/bin/bsdtar -cJ -f  test.xz test.txt
bsdtar: Unsupported compression option --xz

I'll have to take a closer look and see what is missing when I get some more free time

[gijoe77]

<QUOTE author="gijoe77" post_id="1866" time="1533083493" user_id="243">

<QUOTE author="Raion-Fox" post_id="1827" time="1532989335" user_id="49">

So yeah Gijoe, here's the reason I'm interested in getting this working.

We don't need BSD cpio, but I want BSD tar for irisware so we can distribute packages as tardist.xz and GNU tar doesnt work with inst/swmgr.

If you can get a patch out and dependency lists published, then it resolves most of our issues.

I can work on getting all that sorted, but I'm really green behind the ears so I might be asking for some help until I fully understand what I'm doing

btw, I'm not sure if I'm doing this right, but perhaps there is still a problem with this compile, I would assume the below should create a .xz archive:

Code:

-bash-4.3$ ls
-bash-4.3$ echo test > test.txt
-bash-4.3$ /usr/local/bin/bsdtar -cJf test.xz test.txt
bsdtar: Unsupported compression option --xz
-bash-4.3$ /usr/local/bin/bsdtar -cJ -f  test.xz test.txt
bsdtar: Unsupported compression option --xz

I'll have to take a closer look and see what is missing when I get some more free time

I compiled dgen with the libarchive library and it does work with the xz format, so I verified xz support does work, not sure why bsdtar doesn't seem to support it. I'll do some more digging

[gijoe77]

so I've been wondering about this

Code:

#if defined(__sgi__) && !defined(dirfd)
#define dirfd(dp) (((DIR *) (dp))->__dd_fd)
#endif

I had to remove the below lines because it wasn't working:

Code:

#if defined(__sgi__) && !defined
#endif

what are these things with all the underscores anyway? Extensions? I just can't seem to figure out why there is so many underscores everywhere. how would I know if "__sgi__" is defined or not? Is there some sort of standard of what gets defined? I'm not even sure what to google to look this up - can someone point me in a good direction?

[dexter1]

These are preprocessor symbols or 'define's defined by including header files and performing a compile which includes a preprocessor stage, like most regular compile stages.

The symbol '__sgi' (not '__sgi__') is defined when compiling c code under IRIX. I'm not sure if gcc will set that define but I use it mostly in MIPSPro ports to keep the sgi specific code separate from the main code branch.

[gijoe77]

<QUOTE author="dexter1" post_id="2052" time="1533555984" user_id="141">

These are preprocessor symbols or 'define's defined by including header files and performing a compile which includes a preprocessor stage, like most regular compile stages.

The symbol '__sgi' (not '__sgi__') is defined when compiling c code under IRIX. I'm not sure if gcc will set that define but I use it mostly in MIPSPro ports to keep the sgi specific code separate from the main code branch.

so lets say I have "__sig__" in my code, can i "trigger" this code by doing a "-D__sgi__" in CPPFLAGS? I'm just trying to get my head around all this stuff. Are there certain "-D" strings you use for porting stuff to IRIX? my notes from long ago have "-DIP32 -DIRIX" but I figure those are not very helpful in most software packages