The start of a LONG Fuel repair thread...
#11
RE: The start of a LONG Fuel repair thread...
The Odyssey Graphics is the V6-V12 boards used for Chimera and Speedracer designs, so the designation here isn't a problem.

I would not rule out the V10 actually being okay. You can for a short time turn the env monitoring off just to test if you can get a picture -- that's what I'd do. Just keep an IR thermometer on that heatsink. That will at least tell you if other components may be at fault here.

If you don't get a picture after turning of env monitoring and getting to a prompt, then yeah, she dead.

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
10-29-2021, 10:36 PM
#12
RE: The start of a LONG Fuel repair thread...
Sufficiently scared aside, these same messages appear in other ENV disease posts from Neko-archive. So I agree that nothing here distinctively rules the V10 as toast.

I'd actually rather NOT disable env monitoring because I honestly need it to tell if my fans are broken as well as over temp scenarios during testing:-) Since I HAVE the ICs already, it's probably better for me to fix it and move on, I'm not looking to poke the bear and I'd need to run the system for a good while (like 15-20 mins at the least) to get my sh*t together to test boot and then run IDE diags from CDROM anyway...

Since I already have the supplies, I'll plan to just do the IC changes and see where that gets me. I'd rather have a working monitoring system during test or inspection then not (for obvious safety).

But no one has chimed in yet, does this look like main/core ENV disease? I've never seen one in person so, I'm asking all of you. Does this look like it (so far)?
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
10-29-2021, 10:46 PM
#13
RE: The start of a LONG Fuel repair thread...
Yo All,
Okay, a few steps are done...few more to go.

I've successfully removed all the DS1780 chips from the mainboard and cleaned the mainboard areas where the chips used to be. I've cleaned the old chips and will be checking them BEFORE I start on the mainboard to see if I can find out which one failed for my symptoms and I will be probing the mainboard BEFORE I plant new chips on it...so I can then probe the new mounted chips for whisker connections/bridges. I need to know if any unwanted connections occur and unless I know none of the IC legs internally are connected and none of the nearby pads on the mainboard are connected, then I can reasonably trust I've not bridged anything when I do the final assessment.

I have decided to approach this soldering job using soldering paste and a hoof drag tip. After I cleaned the old ICs I realized using a micro-tip iron on each leg with a line of solder paste will still result in issues, getting a good, fluxed, drag on very minimal solder paste line will be the best approach.

I posted pictures for all to see.

I have a Fuel disassembly Album:
https://siliconimage.irixnet.org/index.p...isassembly

and a Fuel mainboard desoldering/masking Album:
https://siliconimage.irixnet.org/index.p...780-Repair

I'm happy with how the planning on the masking has worked out so far, I think the coverage will be sufficient for my planned installation. I'm of course worried about glancing the iron tip into other components and of course exact registration of the ICs. My hope is that with the aid of the paste (slight adhesive power) and a microscope I can align and then tack-down the ICs, then drag solder them and examine, re-dragging if needed. I'd say if I was used to doing this, I'd likely only need about 40-50 mins to do and inspect all three ICs, but of course I'll be taking my time here to hopefully guarantee success!

More to come, enjoy the pics!
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
10-31-2021, 05:28 AM
#14
RE: The start of a LONG Fuel repair thread...
This gives me hope for the origin 300 I obtained from BlueandGold

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
10-31-2021, 05:36 AM
#15
RE: The start of a LONG Fuel repair thread...
Me too, if this goes well, I may no longer be so afraid of it..success has that effect :-). Fingers crossed, wish me luck!
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
10-31-2021, 05:55 AM
#16
RE: The start of a LONG Fuel repair thread...
Okay...a lot happened today and I still haven't yet placed a DVI monitor on the Fuel and done a Full online pwr up to PROM yet...

I went through 4 items today.

I finished repairing/replacing the three DS1780 ICs on the mainboard.  That went VERY SMOOTH, I had the right equipment and my only issues were getting good alignment and tacking of the IC accurately.  Otherwise...a breeze!  However, all the old/removed ICs tested good on the curve tracer!  That's not good!!!

OK well I replaced them anyway, reinstalled the board in the Fuel...NO CHANGE...SAME PROBLEM...ARRRGHHH


Fuel_log_mainboard_DS1780_replaced:
Code:
ALERT: Error initializing the ODYSSEY monitor, no acknowledge
ALERT: Error setting fan speed on ODYSSEY monitor, no acknowledge




SGI SN1 L1 Controller

Firmware Image B: Rev. 1.28.3, Built 03/20/2004 00:01:57





001a01-L1>ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge

INFO: 001a01 will power up system in  5 seconds...

INFO: 001a01 powering up the system.

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error initializing the ODYSSEY monitor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 12V bias) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 2.5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 3.3V aux) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY temperature monitoring: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error initializing the ODYSSEY monitor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 12V bias) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 2.5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 3.3V aux) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY temperature monitoring: no acknowledge

ALERT: Error initializing the ODYSSEY monitor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 12V bias) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 2.5V) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY power (XIO 3.3V aux) monitoring: no acknowledge

ALERT: Error configuring ODYSSEY temperature monitoring: no acknowledge

ERROR: 001a01 auto power up error.

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge

eALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

nv

Environmental monitoring is enabled, but not running (configuration error).



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V   Wait Pwr  10%  10.80/ 13.20  20%   9.60/ 14.40    0.06

        12V IO   Wait Pwr  10%  10.80/ 13.20  20%   9.60/ 14.40    0.06

            5V   Wait Pwr  10%   4.50/  5.50  20%   4.00/  6.00    0.10

          3.3V   Wait Pwr  10%   2.97/  3.63  20%   2.64/  3.96    0.64

          2.5V   Wait Pwr  10%   2.25/  2.75  20%   2.00/  3.00    0.00

          1.5V   Wait Pwr  10%   1.35/  1.65  20%   1.20/  1.80    0.00

        5V aux   Wait Pwr  10%   4.50/  5.50  20%   4.00/  6.00    5.04

      3.3V aux   Wait Pwr  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias   Wait Pwr  10%  10.80/ 13.20  20%   9.60/ 14.40    0.06

     Fuel SRAM   Wait Pwr  10%   2.25/  2.75  20%   2.00/  3.00    0.06

      Fuel CPU   Wait Pwr  10%   1.13/  1.38  20%   1.00/  1.50    0.01

    PIMM0 1.5V   Wait Pwr  10%   1.35/  1.65  20%   1.20/  1.80    0.04

PIMM0 3.3V aux   Wait Pwr  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux   Wait Pwr  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias   Wait Pwr  10%  10.80/ 13.20  20%   9.60/ 14.40    0.00

        XIO 5V   Wait Pwr  10%   4.50/  5.50  20%   4.00/  6.00    0.00

      XIO 2.5V   Wait Pwr  10%   2.25/  2.75  20%   2.00/  3.00    0.00

  XIO 3.3V aux   Wait Pwr  10%   2.97/  3.63  20%   2.64/  3.96    0.00



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST   Wait Pwr          920            0

FAN 1       HD   Wait Pwr         1560            0

FAN 2      PCI   Wait Pwr         1120            0

FAN 3    XIO 1   Wait Pwr         1600            0

FAN 4    XIO 2   Wait Pwr         1600            0

FAN 5       PS   Wait Pwr         1349            0



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0           Wait Pwr    [Autofan Control]    75C/167F   15C/ 59F

1 NODE 1           Wait Pwr    [Autofan Control]    75C/167F   15C/ 59F

2 NODE 2           Wait Pwr    [Autofan Control]    75C/167F   15C/ 59F

3 PIMM             Wait Pwr    [Autofan Control]    75C/167F   15C/ 59F

4 ODYSSEY          Wait Pwr    [Autofan Control]    75C/167F    0C/ 32F

5 BEDROCK          Wait Pwr  Not currently available



001a01-L1>ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading monitor ODYSSEY interrupt status 1: no acknowledge

ALERT: Error reading the ODYSSEY temperature sensor, no acknowledge


Well...many swear words later....the errors say ODYSSEY, and so I'm going to replace the DS1780 IC on the V10...why not...right?  OK, Done..replaced and the old IC did measured shorted in circuit, so we're on to something!


Reinstalled V10 into Fuel...WOW there is a difference:
Code:
SGI SN1 L1 Controller

Firmware Image B: Rev. 1.28.3, Built 03/20/2004 00:01:57





001a01-L1>eINFO: 001a01 will power up system in  5 seconds...

INFO: 001a01 powering up the system.

nv

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.06

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.25

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.48

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2149

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2191

FAN 4    XIO 2    Enabled         1600         2070

FAN 5       PS    Enabled         1349         2083



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   14C/ 57F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   14C/ 57F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   14C/ 57F

3 PIMM              Enabled    [Autofan Control]    75C/167F   17C/ 62F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   17C/ 62F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   14C/ 57F



001a01-L1>

001a01 ATTN: XIO 12V bias low fault limit reached  2.688V.



001a01 ATTN: brick auto power down in 30 seconds

Wow, it worked I'm good....Wait...what this ATTN?  Darn the 12V XIO is being dragged down, I cannot see that on the ENV but I believe it's real...because I just replaced all the ICs!  Okay, so the V10 has a shorted rail...hmm...how should I deal with this?  Well the DS1780 that was installed on the V10 measure shorted between VCC and GRD on it's pins WHILE installed...remeasured...it's still shorted (only in DIODE test mode, not shorted in Ohms..~40 Ohms)...but DS1780 IC works now at the low voltage!

Okay, well I cannot find another VCC line that I KNOW is a VCC and shorted, because the V10 PCB layout does NOT help me see the power rails/track easily...weird.

Okay so I went ahead and injected 1v @ 1A between PCB ground and the VCC pin 9 on the V10's DS1780 (thank god for SGI test points!, they surround nearly all chips and allow easier testing!), I see a direct short dragging down my bench DC PSU, Okay let's follow that.

I tried injection three times with the same values and my thermal imager COULD NOT see any real heat bloom...it might be under the heatsink but it's not near the edge of any heatsink, I decided to bump the voltage injection to 1V @ 1.5A.  Nothing, short is gone....again...nothing short is gone...darn!!!!

This is the second time in recent memory that I did a very conservative LOW VOLTAGE injection with what I feel was a LOW current limit...and the short burned itself out!!!  There like an 80% chance, in my head, that the shorted component is just a passive (cap or diode) used for protection/decouping or whatever.  So let's try it anyway...new diode mode test reading is high ~0.98v instead of the old 0.004v I was getting at the short.

Threw it back into the Fuel:
Code:
SGI SN1 L1 Controller

Firmware Image B: Rev. 1.28.3, Built 03/20/2004 00:01:57





001a01-L1>INFO: 001a01 will power up system in  5 seconds...

INFO: 001a01 powering up the system.

env

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.06

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.25

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.48

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2149

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2163

FAN 4    XIO 2    Enabled         1600         2033

FAN 5       PS    Enabled         1349         2083



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   14C/ 57F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   14C/ 57F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   13C/ 55F

3 PIMM              Enabled    [Autofan Control]    75C/167F   16C/ 60F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   17C/ 62F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   13C/ 55F



001a01-L1> env

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.10

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.25

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.88

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2149

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2177

FAN 4    XIO 2    Enabled         1600         2045

FAN 5       PS    Enabled         1349         2083



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   14C/ 57F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   14C/ 57F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   13C/ 55F

3 PIMM              Enabled    [Autofan Control]    75C/167F   16C/ 60F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   17C/ 62F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   13C/ 55F



001a01-L1>env

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.10

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.24

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.88

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2191

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2177

FAN 4    XIO 2    Enabled         1600         2045

FAN 5       PS    Enabled         1349         2096



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   16C/ 60F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   15C/ 59F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   13C/ 55F

3 PIMM              Enabled    [Autofan Control]    75C/167F   19C/ 66F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   17C/ 62F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   16C/ 60F



001a01-L1>env

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.10

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.24

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.88

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2191

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2191

FAN 4    XIO 2    Enabled         1600         2045

FAN 5       PS    Enabled         1349         2096



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp      

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   17C/ 62F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   15C/ 59F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   14C/ 57F

3 PIMM              Enabled    [Autofan Control]    75C/167F   20C/ 68F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   16C/ 60F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   17C/ 62F



001a01-L1>pwr down

001a01-L1>


12 XIO Bias is sticking at 11.88v...it's not going lower and the system is still happy...fans are running...it wants to pwr up. I'm not seeing any ATTN Alerts anymore, voltages are stable.

So I turned it off at this point, I have stuff I have to do to prepare for a contractor tomorrow (shouldn't take long) so I'm stopping for now.

Still no idea if the V10 even produces a picture at this point but what has been learned?  The mainboard DS1780s were fine, V10 was draggin down the XIO 12V line from an ACTIVE SHORT, this is was in tandem with the DS1780 on the V10 shorting out or maybe because of it.  The new DS1780 IC wasn't shorting out and could work at the lower rail voltage as it take like ~2v-5V variable VCC input to run.  Again, I have no cause and effect order and I couldn't find WHAT was shorted out, but it burnt out during testing...which means it was likely something the size of a period on a book page to burn out immediately at 1.5A @ 1v.

I give about a 75% chance it was just a decoupling cap or something harmless.  The system now thinks everything's good (comm-wise and voltage-wise).

I've reassembled the Fuel and will then get to setting it up with peripherals and an Irix 6.5.30 disc 1 to try to boot to PROM screen, then try to run IDE diags from CD (I think I can do that manually from boot -f stand?), anyone want to give me a good hint and how to run the EXTENSIVE diags from PROM using the install media so I get this testing underway (assuming any picture shows up at PROM).  Also, yeah I may have to use a terminal for the extended test on graphics...I just don't have a good way of sneaking the serial cable out of the case with the side panel on...any suggestions?

I think things are looking up!  Comments?!?!
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
11-01-2021, 03:23 AM
#17
RE: The start of a LONG Fuel repair thread...
Good job Weblacky! Nice going figuring out the Odyssey's DS1780 was the issue.

I can probably tell you why it failed:

As I've stated in another thread, the Fuel's cooling system is simply... godawful. Air traveling at right angles, tiny fans, etc. That would suggest that inadequate ventilation of the V10 may have contributed to its failure -- either that or it in particular is likely vulnerable to env monitoring damage.

My thoughts on fixing the fuel's fan system: Reverse the intake fan, add another shroud and fan at the other end that helps push air out? Then maybe change the small fans pushing air into the VPro area with a large fan with a HIGH static pressure (none of the Noctuas, they're great, but they have low static pressure so they're not suitable here).

I'm not nearly as intelligent as you are in the electrical department, though.

If you want to prove my theory, it's easy: Attach with some temporary siliicone caulk a sheet of plexiglass to the side of the Fuel to allow transparency.

Get a "Smoke machine" used by mechanics to find vacuum leaks. Blow smoke into a running system and watch the path the air takes. You'll see it sort of splits into a "T" shape from the intake fan that then exhausts from the (stock) PSU and VPro cage

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
11-01-2021, 03:50 AM
#18
RE: The start of a LONG Fuel repair thread...
I know for a concentrated air movement (direct flow) you have to use a low-pressure method (slow intake, fast exhaust...so if that was the case than we'd need to somehow pull air out the back of the case faster than the fans in front bring it in, that will create central air flow directly in the passage. I believe that's what you're expressing.

The lack of rear room makes that a little hard...we COULD use a squirrel cage blower mounted UNDER the metal shield (between the metal shield and the rear of the heatsink, inline, pointed towards the rear video ports to accelerate air as exhaust? That would likely be the only way to do what you want without compromising the metal shield by cutting a hole in it.

We'd have to take some measurements and tap the metal shield to hold the blower upside down inside the card's empty space between the rear video output ports and the main heatsink. But the V10 has a lot of room towards the back (behind the video puts)...so having a blower there, aimed to the rear...sucking from the front...would be the simplest way to make that mod.

If this things runs, I'll see about looking into options....
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
11-01-2021, 04:05 AM
#19
RE: The start of a LONG Fuel repair thread...
In any case, I just know that currently the way the Fuel is designed is not smart -- and IR cameras can only show surface temps so when people talk about that I wanna facepalm.

You can verify before investing time engineering a solution, of course. The fogging stuff should work.

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
11-01-2021, 04:52 AM
#20
RE: The start of a LONG Fuel repair thread...
Okay, so things were going good...but I've hit a standstill, the V10 is broken in a way that prevents startup.

With the V10 in I get a: INFO: System failed pre-power check

And that error is a google hack...no results.  So fixing the health monitoring, while having more damage, created a scenario never seen/published before:

Code:
SGI SN1 L1 Controller

Firmware Image B: Rev. 1.28.3, Built 03/20/2004 00:01:57


001a01-L1>INFO: 001a01 will power up system in  5 seconds...

INFO: 001a01 powering up the system.



ERROR: command not found.

001a01-L1>env

Environmental monitoring is enabled and running.



Description    State       Warning Limits     Fault Limits       Current

-------------- ----------  -----------------  -----------------  -------

           12V    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.94

        12V IO    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

            5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

          3.3V    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.35

          2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

          1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.47

        5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

      3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

PIMM0 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   12.00

     Fuel SRAM    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.51

      Fuel CPU    Enabled  10%   1.13/  1.38  20%   1.00/  1.50    1.24

    PIMM0 1.5V    Enabled  10%   1.35/  1.65  20%   1.20/  1.80    1.49

PIMM0 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.29

  PIMM0 5V aux    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.02

  XIO 12V bias    Enabled  10%  10.80/ 13.20  20%   9.60/ 14.40   11.88

        XIO 5V    Enabled  10%   4.50/  5.50  20%   4.00/  6.00    5.07

      XIO 2.5V    Enabled  10%   2.25/  2.75  20%   2.00/  3.00    2.47

  XIO 3.3V aux    Enabled  10%   2.97/  3.63  20%   2.64/  3.96    3.30



Description    State       Warning RPM  Current RPM

-------------- ----------  -----------  -----------

FAN 0  EXHAUST    Enabled          920         1180

FAN 1       HD    Enabled         1560         2220

FAN 2      PCI    Enabled         1120         1520

FAN 3    XIO 1    Enabled         1600         2205

FAN 4    XIO 2    Enabled         1600         2070

FAN 5       PS    Enabled         1349         2096



                              Advisory   Critical   Fault      Current

Description       State       Temp       Temp       Temp       Temp       

----------------- ----------  ---------  ---------  ---------  --------- 

0 NODE 0            Enabled    [Autofan Control]    75C/167F   16C/ 60F

1 NODE 1            Enabled    [Autofan Control]    75C/167F   15C/ 59F

2 NODE 2            Enabled    [Autofan Control]    75C/167F   13C/ 55F

3 PIMM              Enabled    [Autofan Control]    75C/167F   19C/ 66F

4 ODYSSEY           Enabled    [Autofan Control]    75C/167F   13C/ 55F

5 BEDROCK           Enabled    [Autofan Control]    85C/185F   16C/ 60F



001a01-L1>pwr up

INFO: Power is already on



INFO: System failed pre-power check


The front lights go from Red to a blinking white (two white LEDs) and stay blinking all the time (never quite).  If I remove the V10 the card misses the I2c bus interface (error is shown on L1) and cannot communicate with BEDROCK and claims ODYSSEY is missing (no duh).

Then the L1 claims it thought it was already powered up, cannot understand why it's not.  The kickstart button doesn't do anything on the ATX adapter other than the FAN starts (which they are running).  Turning off env does NOTHING, pre-power check is still fails with nothing.  Case front power button does nothing.

So something on the V10 is broken...I'm at a loss for how to proceed.  I've tried to look around for power system shorts...there are none anymore.  Whatever shorted out that I couldn't find...seems critical to the card working.  Fixing the DS1780 IC allowed the I2c monitoring to come back up, this made the env bus whole again...but more damage is there...likely on the V10...since it's DS1780 was the only one damaged among those ICs, so far.

So I'm at an impasse without a Fuel V10 graphics card that works.

I'm going to place a wanted ad for a V10, if that V10 fixes the rest of my issues...I MAY be able to use to fix mine if the problem can be detected by comparison easily enough...but so far...I have serious doubts about fixing the card I have.  Since you cannot complete the I2c bus without the Graphics card I obviously cannot boot without a graphics card.

So off to find a replacement graphics card!  Horrible, I know...I don't have a choice to get further...this really is disappointing but it's not a simple straight short on the bus line anymore...something burnt out and I don't know what that something on the card was (no marks or odor).

I just need to start the system to tell if that's all!

Arrghhh...so close..but mainboard and PIMM seem happy so...fingers crossed.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
11-03-2021, 01:27 AM


Forum Jump:


Users browsing this thread: 2 Guest(s)