my o2 is running waaay too slow -
gijoe77 - 11-11-2018
so I've been installing everything on the ftp, and I must of done something that is really screwing up my o2. Basically its not very responsive, it seems to have long periods of time between processing certain commands, for example if I try to uncompress and untar a file, it takes way too long - for example a 200mb .tgz file is taking anywhere from 30-50 minutes to uncompress/untar.
here is a video showing the issue:
https://youtu.be/05Z4iH2Y3bY
the thing that is interesting is that not all commands seem to be having an issue. here is a diskperf:
Code:
-bash-4.2# diskperf -W -D -n "o2 500GB SSD" /tmp/testfile
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name : o2 500GB SSD
# Test date : Fri Nov 9 11:42:19 2018
# Test machine : IRIX o2 6.5 10070056 IP32
# Test type : XFS data subvolume
# Test path : /tmp/testfile
# Request sizes : min=4096 max=33553920
# Parameters : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 524288000 bytes
#---------------------------------------------------------
# req_size fwd_wt fwd_rd bwd_wt bwd_rd rnd_wt rnd_rd
# (bytes) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s)
#---------------------------------------------------------
4096 9.72 8.72 9.78 7.49 9.74 7.50
8192 14.82 13.21 14.87 11.53 14.81 11.48
16384 19.94 17.52 20.02 15.34 19.85 15.41
32768 24.15 20.98 24.22 19.20 24.18 19.22
65536 27.02 24.22 27.08 22.75 27.05 22.82
131072 28.60 25.26 28.66 23.70 28.65 23.78
262144 29.48 25.18 29.56 23.66 29.53 23.72
524288 29.94 25.88 30.00 25.47 29.98 25.48
1048576 30.18 26.19 30.23 25.09 30.22 25.09
2097152 30.55 26.53 30.63 25.70 30.61 25.66
4194304 31.15 27.22 31.24 26.81 31.22 26.79
8388608 31.58 27.63 31.79 27.39 31.81 27.49
16777216 31.62 27.91 32.15 27.70 32.14 27.83
-bash-4.2#
here is a mkfile command:
Code:
-bash-4.3# time mkfile 10g /tmp/tempfile
real 6m6.163s
user 0m0.160s
sys 1m0.216s
Currently the o2 has a 500GB SSD, I cloned the disk to a 15K 300GB brand new SCSI disk and it's having the same issue, so I proved out my HDD. I've been chkconfig'ing off most things, I've been disabling stuff in /etc/rc2.d, I just can't seem to figure out what the issue is. I figured I'd post and perhaps someone might have some ideas where to look next.
here is my current chkconfig:
Code:
-bash-4.3# chkconfig -s
Flag State
==== =====
autofs on
desktop on
ipaliases on
jaleolic on
network on
nsd on
routed on
rtmond on
savecore on
sdpd on
snetd on
sysevent on
tfxd on
timed on
verbose on
visuallogin on
vswap on
windowsystem on
xdm on
xlv on
ad_elmd off
appletalk off
array off
audit off
autoconfig_ipaddress off
automount off
aw_flexlm off
cesag off
cpumeter off
dl_lucid off
dl_stone off
dl_sws off
dl_vic off
dl_wire off
dl_wiretap off
esp off
fcagent off
flexlm off
fontserver off
fsr_xfs off
gated off
ldap off
lockd off
lp off
maple_license off
mediad off
mrouted off
named off
nds off
neko_ldap off
netwr_client off
nfs off
noiconlogin off
nostickytmp off
nss_fasttrack off
ntp off
pmcd off
pmie off
privileges off
proclaim_relayagent off
proclaim_server off
proxymngr off
quickpage off
rarpd off
rsvpd off
rwhod off
sar off
sendmail off
sendmail_cf off
sesdaemon off
sesinetd off
sgi_apache off
Shout_Server off
soundscheme off
Thomson_Licenses off
timeslave off
ts off
webface off
webface_apache off
yp off
ypmaster off
ypserv off
my /etc/rc* files:
Code:
-bash-4.3# ls -l /etc/rc*
-rwxr-xr-x 1 root sys 1037 Mar 13 2018 /etc/rc0
-rwxr-xr-x 1 root sys 1441 Mar 13 2018 /etc/rc2
-rwxr-xr-x 1 root sys 439 Mar 13 2018 /etc/rc3
/etc/rc0.d:
total 4
-rwxr-xr-x 1 root sys 1443 Aug 17 22:45 K00ad_elmd
lrwxr-xr-x 1 root sys 14 Aug 30 08:43 K00qx -> /etc/init.d/qx
lrwxr-xr-x 1 root sys 17 Jun 8 17:28 K01atalk -> /etc/init.d/atalk
lrwxr-xr-x 1 root sys 21 Jun 8 17:28 K01qpage -> ../init.d/qpageserver
lrwxr-xr-x 1 root sys 19 Jun 8 17:28 K01teleffect -> ../init.d/teleffect
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K02midi -> ../init.d/midi
lrwxr-xr-x 1 root sys 13 Jun 8 17:28 K02pcp -> ../init.d/pcp
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K02pmie -> ../init.d/pmie
lrwxr-xr-x 1 root sys 18 Jun 8 17:28 K02sdpd -> ../init.d/run-sdpd
lrwxr-xr-x 1 root sys 20 Aug 29 21:09 K02wdl_lucid -> /etc/init.d/dl_lucid
lrwxr-xr-x 1 root sys 13 Jun 8 17:28 K02xdm -> ../init.d/xdm
lrwxr-xr-x 1 root sys 18 Jun 8 17:28 K03announce -> ../init.d/announce
lrwxr-xr-x 1 root sys 18 Jun 8 17:28 K05availmon -> ../init.d/availmon
lrwxr-xr-x 1 root sys 22 Aug 29 21:09 K06s+w_stop -> /etc/init.d/stone+wire
lrwxr-xr-x 1 root sys 16 Jun 8 17:28 K09mediad -> ../init.d/mediad
lrwxr-xr-x 1 root sys 19 Jun 8 17:28 K09proxymngr -> ../init.d/proxymngr
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K15cron -> ../init.d/cron
lrwxr-xr-x 1 root sys 13 Jun 8 17:28 K17nds -> ../init.d/nds
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K20mail -> ../init.d/mail
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K21ldap -> ../init.d/ldap
lrwxr-xr-x 1 root sys 13 Jun 8 17:28 K24esp -> ../init.d/esp
-rwxr-xr-x 1 root sys 653 Sep 28 17:23 K24neko_ldap
lrwxr-xr-x 1 root sys 19 Jun 8 17:28 K24neko_ldap.O -> ../init.d/neko_ldap
lrwxr-xr-x 1 root sys 12 Jun 8 17:28 K25lp -> ../init.d/lp
lrwxr-xr-x 1 root sys 18 Jun 8 17:28 K25sysevent -> ../init.d/Sysevent
lrwxr-xr-x 1 root sys 15 Jun 8 17:28 K27audit -> ../init.d/audit
lrwxr-xr-x 1 root sys 25 Jun 8 17:28 K27nss_fasttrack -> /etc/init.d/nss_fasttrack
lrwxr-xr-x 1 root sys 22 Jun 8 17:28 K27sgi_apache -> /etc/init.d/sgi_apache
lrwxr-xr-x 1 root sys 16 Jun 8 17:28 K30flexlm -> ../init.d/flexlm
lrwxr-xr-x 1 root sys 17 Jun 8 17:28 K32fcagent -> ../init.d/fcagent
lrwxr-xr-x 1 root sys 19 Jun 8 17:28 K32sesdaemon -> ../init.d/sesdaemon
lrwxr-xr-x 1 root sys 26 Jun 8 17:28 K34webface_apache -> /etc/init.d/webface_apache
lrwxr-xr-x 1 root sys 17 Jun 8 17:28 K35webface -> ../init.d/webface
lrwxr-xr-x 1 root sys 20 Jun 8 17:28 K38aliases-ip -> ../init.d/aliases-ip
lrwxr-xr-x 1 root sys 17 Jun 8 17:28 K40network -> ../init.d/network
-rwxr-xr-x 1 root sys 657 Jun 10 14:00 K50eclipse
lrwxr-xr-x 1 root sys 15 Jun 8 17:28 K75array -> ../init.d/array
lrwxr-xr-x 1 root sys 20 Jun 8 17:28 K75pppstartup -> ../init.d/pppstartup
lrwxr-xr-x 1 root sys 18 Jun 8 17:28 K80postinst -> ../init.d/postinst
lrwxr-xr-x 1 root sys 12 Jun 8 17:28 K80ts -> ../init.d/ts
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K81dlif -> ../init.d/dlif
lrwxr-xr-x 1 root sys 17 Jun 8 17:28 K82sysetup -> ../init.d/sysetup
lrwxr-xr-x 1 root sys 14 Jun 8 17:28 K83swap -> ../init.d/swap
lrwxr-xr-x 1 root sys 21 Jun 8 17:28 K84filesystems -> ../init.d/filesystems
lrwxr-xr-x 1 root sys 24 Jun 8 17:26 K90nlskeyserver -> /etc/init.d/nlskeyserver
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 K98jaleolic -> /etc/init.d/jaleolic
lrwxr-xr-x 1 root sys 13 Jun 8 17:28 K98usr -> ../init.d/usr
lrwxr-xr-x 1 root sys 24 Jun 8 17:26 K99Shout_Server -> /etc/init.d/Shout_Server
lrwxr-xr-x 1 root sys 28 Jun 8 17:26 K99Thomson_Licenses -> /etc/init.d/Thomson_Licenses
lrwxr-xr-x 1 root sys 20 Jun 8 17:28 K99disk_patch -> ../init.d/disk_patch
/etc/rc2.d:
total 5
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S00announce -> ../init.d/announce
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S00disk_patch -> ../init.d/disk_patch
lrwxr-xr-x 1 root sys 13 Jun 8 17:26 S04usr -> ../init.d/usr
lrwxr-xr-x 1 root sys 21 Jun 8 17:26 S12filesystems -> ../init.d/filesystems
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S16postinst -> ../init.d/postinst
lrwxr-xr-x 1 root sys 17 Jun 8 17:26 S20sysetup -> ../init.d/sysetup
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S21perf -> ../init.d/perf
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S23autoconfig -> ../init.d/autoconfig
lrwxr-xr-x 1 root sys 17 Jun 8 17:26 S30network -> ../init.d/network
lrwxr-xr-x 1 root sys 22 Jun 8 17:26 S31proclaim -> ../init.d/run-proclaim
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S32aliases-ip -> ../init.d/aliases-ip
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S34snmp -> ../init.d/snmp
lrwxr-xr-x 1 root sys 26 Jun 8 17:26 S34webface_apache -> /etc/init.d/webface_apache
lrwxr-xr-x 1 root sys 15 Jun 8 17:26 S35audit -> ../init.d/audit
lrwxr-xr-x 1 root sys 17 Jun 8 17:26 S35webface -> ../init.d/webface
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S41ldap -> ../init.d/ldap
lrwxr-xr-x 1 root sys 16 Jun 8 17:26 S45flexlm -> ../init.d/flexlm
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S48savecore -> ../init.d/savecore
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S49swap -> ../init.d/swap
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S50mail -> ../init.d/mail
lrwxr-xr-x 1 root sys 21 Jun 8 17:26 S51aw_flexlm -> /etc/init.d/aw_flexlm
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S58rmtmpfiles -> ../init.d/rmtmpfiles
lrwxr-xr-x 1 root sys 12 Jun 8 17:26 S60lp -> ../init.d/lp
lrwxr-xr-x 1 root sys 15 Jun 8 17:26 S62array -> ../init.d/array
lrwxr-xr-x 1 root sys 25 Jun 8 17:26 S63nss_fasttrack -> /etc/init.d/nss_fasttrack
lrwxr-xr-x 1 root sys 22 Jun 8 17:26 S63sgi_apache -> /etc/init.d/sgi_apache
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S75cron -> ../init.d/cron
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S75pppstartup -> ../init.d/pppstartup
lrwxr-xr-x 1 root sys 13 Jun 8 17:26 S76esp -> ../init.d/esp
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S77sysevent -> ../init.d/Sysevent
lrwxr-xr-x 1 root sys 13 Jun 8 17:26 S83nds -> ../init.d/nds
lrwxr-xr-x 1 root sys 12 Jun 8 17:26 S85ts -> ../init.d/ts
lrwxr-xr-x 1 root sys 19 Jun 8 17:26 S88configmsg -> ../init.d/configmsg
lrwxr-xr-x 1 root sys 16 Jun 8 17:26 S90chkdev -> ../init.d/chkdev
lrwxr-xr-x 1 root sys 24 Jun 8 17:26 S90nlskeyserver -> /etc/init.d/nlskeyserver
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S90roboinst -> ../init.d/roboinst
lrwxr-xr-x 1 root sys 16 Jun 8 17:26 S91ts -> ../init.d/ts_old
lrwxr-xr-x 1 root sys 17 Jun 8 17:26 S92fcagent -> ../init.d/fcagent
lrwxr-xr-x 1 root sys 19 Jun 8 17:26 S92sesdaemon -> ../init.d/sesdaemon
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S95availmon -> ../init.d/availmon
lrwxr-xr-x 1 root sys 20 Jun 8 17:26 S96fontserver -> ../init.d/fontserver
lrwxr-xr-x 1 root sys 16 Jun 8 17:26 S97mediad -> ../init.d/mediad
lrwxr-xr-x 1 root sys 19 Jun 8 17:26 S97proxymngr -> ../init.d/proxymngr
lrwxr-xr-x 1 root sys 20 Jun 8 17:25 S98jaleolic -> /etc/init.d/jaleolic
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S98midi -> ../init.d/midi
lrwxr-xr-x 1 root sys 13 Jun 8 17:26 S98pcp -> ../init.d/pcp
lrwxr-xr-x 1 root sys 14 Jun 8 17:26 S98pmie -> ../init.d/pmie
lrwxr-xr-x 1 root sys 16 Jun 8 17:26 S98rtmond -> ../init.d/rtmond
lrwxr-xr-x 1 root sys 18 Jun 8 17:26 S98sdpd -> ../init.d/run-sdpd
lrwxr-xr-x 1 root sys 13 Jun 8 17:26 S98xdm -> ../init.d/xdm
lrwxr-xr-x 1 root sys 24 Jun 8 17:26 S99Shout_Server -> /etc/init.d/Shout_Server
lrwxr-xr-x 1 root sys 28 Jun 8 17:26 S99Thomson_Licenses -> /etc/init.d/Thomson_Licenses
lrwxr-xr-x 1 root sys 17 Jun 8 17:26 S99atalk -> /etc/init.d/atalk
lrwxr-xr-x 1 root user 15 Jun 8 17:26 S99avidd -> ../init.d/avidd
lrwxr-xr-x 1 root user 14 Jun 8 17:26 S99errs -> ../init.d/errs
lrwxr-xr-x 1 root sys 22 Jun 8 17:26 S99netwr_client -> ../init.d/netwr_client
lrwxr-xr-x 1 root sys 21 Jun 8 17:26 S99qpage -> ../init.d/qpageserver
lrwxr-xr-x 1 root sys 19 Jun 8 17:26 S99teleffect -> ../init.d/teleffect
lrwxr-xr-x 1 root sys 14 Aug 30 08:43 _S47qx -> /etc/init.d/qx
-rwxr-xr-x 1 root sys 657 Jun 10 14:00 _S50eclipse
-rwxr-xr-x 1 root sys 653 Sep 28 17:23 _S60neko_ldap
lrwxr-xr-x 1 root sys 20 Aug 31 21:42 _S89sesinetd -> /etc/init.d/sesinetd
-rw-r--r-- 1 root sys 130 Aug 31 00:58 _S98wp
lrwxr-xr-x 1 root sys 20 Aug 29 21:09 _S98zdl_lucid -> /etc/init.d/dl_lucid
-rwxr-xr-x 1 root sys 1443 Aug 17 22:45 _S99ad_elmd
lrwxr-xr-x 1 root sys 23 Aug 30 01:24 _S99bg_mod_link -> /etc/init.d/bg_mod_link
lrwxr-xr-x 1 root sys 22 Aug 29 21:09 _S99stone+wire -> /etc/init.d/stone+wire
lrwxr-xr-x 1 60000 user 23 Sep 2 02:01 _maple_license -> ../init.d/maple_license
/etc/rc3.d:
total 0
-bash-4.3#
any ideas?
RE: my o2 is running waaay too slow -
mamorim01 - 11-11-2018
No sensible ideas, sorry. It seems to be idling out (0%) when the decompression freezes, my gut feeling is that it could be memory or controller related, although an issue with the disk itself would have been my first thought as well.
An entirely wild guess since you seem to have disk_patch in your rc.d folder.
http://nixdoc.net/man-pages/irix/man1/diskpatch.1.html
Ignore this, disk_patch seems to be scheduled by cron and should not be active otherwise.
Quite a few software license servers in your rc.d folder, as well. These give me the willies.
RE: my o2 is running waaay too slow -
gijoe77 - 11-12-2018
I cloned my disk before I started installing everything on the FTP (sometimes I'm not an idiot I guess...), I put the older-cloned disk into the o2 and it was running normal, here is a video of how the o2 should be running:
https://youtu.be/zUj8CbuPacw
From what I can tell I have ruled out a bad hdd (since I cloned my SSD to a brand new 15k SCSI disk, the problem was also seen on that SCSI disk). I don't know if I can rule out some sort of OS corruption, but I think I would be seeing some serious issues if it were an OS issue.
What I really want to do is go item by item/line by line of the processes or services that get started as the OS loads and try to find the offending process. I was under the impression that would be the /etc/rc2.d/S* scripts, is there some other place that gets loaded at startup that a program might have installed some startup script?
RE: my o2 is running waaay too slow -
mamorim01 - 11-12-2018
(11-12-2018, 12:16 AM)gijoe77 Wrote: What I really want to do is go item by item/line by line of the processes or services that get started as the OS loads and try to find the offending process. I was under the impression that would be the /etc/rc2.d/S* scripts, is there some other place that gets loaded at startup that a program might have installed some startup script?
If you suspect a software process then that is a good place to start.
Scripts are stored in the /etc/init.d folder with symbolic links to start (sXXfile) and stop/kill (kXXfile) placed at the relevant runlevel folders (typically rc0.d at shutdown and rc3.d or rc2.d at startup for multiuser with or without networking). As you know, you can later on run
top or
ps to confim that the relevant startup daemon has not been started and to get a picture of which process are still active as you go through the list. If you take batches of startup script links out the rc3.d and rc.2 folders you will eventually narrow it down to the offending one. Comparing the contents of these folders with the contents of your original drive to determine which new scripts have been installed since can further help narrow down the problem.
I guess you could also try to start in single user mode (run level 1), I do not believe that startup daemons are run under that mode. The hard drive would be working just fine in that mode if your theory is correct.
RE: my o2 is running waaay too slow -
gijoe77 - 11-13-2018
so I basically disabled all the startup scripts in /etc/rc2.d (except the ones that at least need to be started to boot, like S30network!! ) and still it runs very slow.
I ran the system in single user mode, in system administrator mode (init 1), and it was still working slow.
I mounted the disk externally on a different system and ran "xfs_repair" on it, still nothing.
I made sure it was actually the gtar in the nekoware dir and ldd'd it to make sure it wasn't using anything I've been compiling and playing with in my /usr/local path, everything looks correct.
I'm at a loss, I think I'm just gonna have to restore from backup, reinstall everything again like I was doing all summer but do more testing after I install each package. It's a bit of a bummer, but oh well

I'll take this inconvenience over unrecoverable data loss anytime
RE: my o2 is running waaay too slow -
mamorim01 - 11-13-2018
That is quite interesting. I too am at a loss, starting the system in administrator mode should work just fine if a rogue non-system process was originally at fault. Since the problem has replicated on the cloned disk, could it possibly be due to a damaged system file or swap file maybe?
I do agree that a complete restore could be a feasible workaround.
RE: my o2 is running waaay too slow -
Jacques - 11-13-2018
I noticed on the 'slow' version you're running bash and on the 'normal' one xwsh, a coincidence?
RE: my o2 is running waaay too slow -
dexter1 - 11-13-2018
Those diskperf number are absolutely fine.
An IDE diagnostic run may shake something loose. These should run within 50 minutes.
If nothing comes out of that i would bite the bullet and do a normal irix 6.5 install on a clean disk. The only way to be sure. Do keep your old disk or back it up, it looks like there is a lot of unusual software on it.
RE: my o2 is running waaay too slow -
gijoe77 - 11-16-2018
(11-13-2018, 03:13 PM)gijoe77 Wrote: I'll take this inconvenience over unrecoverable data loss anytime
I had to say that.... ^^^^^ but anyway....
I backed up my SSD to a brand new 15K 300GB SCSI disk, tested it, it worked fine.
I started cloning an old backup disk to the SSD, but then I thought of something I wanted to try, so I canceled that (it had already destroyed all the original SSD data). I wanted to restore my SSD drive back to the original state, so I put the SSD-cloned 15k SCSI disk into the o2 as the master and I was going to clone it back to the SSD.
wouldnt you believe it, something went wrong with the SSD-backup - xfs errors everywhere, system wouldn't even boot, it would just start dumping memory to /var/adm/crash and everything got all messed up. I ran xfs_repair on it, and it did a ton of stuff. I'm not sure if its bootable or not yet, and there was a *LOT* of stuff in the lost+found directory. Oh well, at least its not all gone, I have most of the things I need to have backed up in other spots and my older cloned drives will be of help. I'm probably going to have to spend a week or two trying to rebuild the system from all the salvageable bits. bummer. I was going to do n+1 backups of the SSD, but didnt think it was needed. Well I guess that was a silly mistake.
I did make some interesting observations. the SSD blows away the results of diskperf, but in real applications it seems to not be really blowing away spinning disks on an o2. so I used the with the same compressed archive for my testing on the SSD and a 10K and 15k (time gtar zxvf <filename>) :
http://www.mediafire.com/file/1izsdjfsq3t83wc/opencascade-7.3.0.tgz/file
I also used a totally different o2 (200mhz R5k), and the tests were sort of surprising. for this particular test the SSD wasn't necessarily faster than the disks . I'm currently copying whats left of the SSD filesystem and am hoping I could salvage all my custom scripts and all the notes I had on porting software, should take 20 hours lol Once I get everything back in order I'll have to see where I was keeping notes on the completion times for SSD vs 15k vs 10k and I'll post a follow up and my thoughts maybe tomorrow
RE: my o2 is running waaay too slow -
Trippynet - 11-16-2018
To be honest, I'm not too surprised overall. The limiting factor on the O2 is the SCSI speed, which is 40MB/s. Most later HDDs can easily saturate this, so it's only the seek time that you improve on. Hence if you do something with lots and lots of very small files, I would expect a performance improvement (albeit not a massive one), but for a lot of normal tasks, I wouldn't expect much improvement overall. It's mainly the lack of noise where the SSD will win against most HDDs.