Origin 350 - PCI 5V missing
#1
Origin 350 - PCI 5V missing
Hello all,

I'm posting on your forum, regarding an issue i'm having on one of my Origin 350 Brick.

As soon as i power it on, the red fault led turn on and the message "PCI 5V low fault reach @ 0.104 V" is displayed, then the brick turn itself off.
To diagnose it i removed every PCI cards including the IO9 board and the cpu node, but the fault is still present.
I also tried swapping the 2U Pci riser with the one from another brick (that has no issue), but the issue remains.

At the moment i'm trying to do component level board repair on what i would call the motherboard (2U Interface 030-1809-006).
And on it, i measured a capacitor (220µF 6V) with 38mV on its +Pin.
This cap is near an LTC1629CG (synchronous step-down switching regulator) and we can spot some components shown in the typical application (attached) from the datasheet.
The 4 Mosfets (D-pack), 2 Inductors, 2 3mOhms resistors, etc

I measured the obvious continuity/voltages.
- I have 12V at Vin (through a 10 Ohm res)
- The gate of each Mosfet is connected to its TG1/TG2/BG1/BG2 respectiv pin
- The 2 0.003Ohm Res are connected to their respectiv sense pin (through a 100Ohms Res)
- There is no obvious shorts on the mosfets, diodes nor ceramic caps
- I measure a brief 5V at startup, and quickly it falls down to near 0V
- There is a 300 Ohms resistance when measuring between the + and GND of the output Cap (the 220µF 6V one) which is about the same compared to the other LTC1629 circuit on that board ( 3.3V output IIRC)

What else could i miss ?
Could a zener diode be dead ?
A reset i'm not aware of ?
Or maybe a DS1780 not properly measuring and/or shorted ?

I would greatly appreciate your input/ideas if you are a seasoned component level repair expert as i am not necessarily one.
I'm pretty comfortable with electronics and desoldering, i also had great success repairing numerous switching PSUs (easy when there is an obvious short... Biggrin ), but on this one i'm a bit dry.

So far i managed to gather informations on the internet (including your forum) every-time i had an issue, but this time i think i might need help.
I'll keep looking for answer on my side (reading datasheets, measuring, etc) and post it if i find anything valuable.

Thank you all for reading, and maybe your future posts.


Attached Files Image(s)
           
.txt Log Test short.txt Size: 2.36 KB  Downloads: 204
aSpartan111
O2

Trade Count: (0)
Posts: 6
Threads: 1
Joined: Aug 2023
Location: Fr
Find Reply
09-25-2023, 01:07 PM
#2
RE: Origin 350 - PCI 5V missing
(09-25-2023, 01:07 PM)aSpartan111 Wrote:  Or maybe a DS1780 not properly measuring and/or shorted ?
That would be my bet based on the measurements you shared. We have seen a lot of failed monitoring chips.

Personaliris O2 Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT Indigo2 Indy   (past: 4D70GT)
robespierre
refector peritus

Trade Count: (0)
Posts: 640
Threads: 3
Joined: Nov 2020
Location: Massholium
Find Reply
09-25-2023, 05:08 PM
#3
RE: Origin 350 - PCI 5V missing
Hi aSpartan111,

as per robspierre's feedback, the DS1780 environment chips are known to fail (there are 3 of them on O350 board I believe).

Hence Weblacky offers replacement service: https://forums.irixnet.org/thread-3207.html

If you want to do yourself then you will find that there are quite a few posts providing details of location of chips and replacement tips.

I am pretty sure I posted a picture in one of the post of O350 board and env chip locations (but they are pretty easy to find).

I think one is up near the power connectors (top right in your picture) and a pair between the blue positioning post and the DALLAS NVRAM.

This thread has pics of locations: https://forums.irixnet.org/thread-3424.html and links to a number of discussions on O350 voltage and temp monitoring.

Cheers from Oz,


jwhat/John.
(This post was last modified: 09-26-2023, 05:29 AM by jwhat.)
jwhat
Octane/O350/Fuel User

Trade Count: (0)
Posts: 513
Threads: 29
Joined: Jul 2018
Location: Australia
Find Reply
09-26-2023, 04:47 AM
#4
RE: Origin 350 - PCI 5V missing
(09-26-2023, 04:47 AM)jwhat Wrote:  Hi aSpartan111,

as per robspierre's feedback, the DS1780 environment chips are known to fail (there are 3 of them on O350 board I believe).

Hence Weblacky offers replacement service: https://forums.irixnet.org/thread-3207.html

If you want to do yourself then you will find that there are quite a few posts providing details of location of chips and replacement tips.

I am pretty sure I posted a picture in one of the post of O350 board and env chip locations (but they are pretty easy to find).

I think one is up near the power connectors (top right in your picture) and a pair between the blue positioning post and the DALLAS NVRAM.

This thread has pics of locations: https://forums.irixnet.org/thread-3424.html and links to a number of discussions on O350 voltage and temp monitoring.

Cheers from Oz,


jwhat/John.


Thanks Jwhat,
I wouldn't have a problem doing this board if you'd like me to but since your location says France I'm assuming it wouldn't be very economical. These actually have better spacing than a Fuel does. So if you're at all handy with soldering, these would probably be easier for someone who is has less experience and less accurate tools actually do the work correctly.

Fuel motherboards are incredibly tight and that's where specialized tools can make or break the entire procedure. This is not the case with your board.

In the case of the board images posted I would say you have a good shot to simply surround the chips with yellow polyimide tape and use something like a chipquik or low melt solder to simply heat up and drench both sides of the chip and gently pluck off the chip with just way too much low melt solder on it.  Then use either solder wick or a desoldering gun to clean up all the excess low melt solder from the pads. Clean the area with isopropyl alcohol and an appropriate swab. Then the next challenge is putting a new chip on and getting it aligned correctly while also not using so much solder that you end up creating bridges between the legs which will short things.

I am normally not a fan of using a hot air tool and so I don't recommend it be used by someone without extensive experience in this case. What I would recommend you do though is go ahead and get a very small iron tip that'll help prevent you from putting too much solder each leg and just put flux on the pads and try to afix the chip as best you can and then touch each leg and try to tack it down with the minimal amount of solder for each pin. Repeat as needed until you have some of it looks good. You can reflow just watch those bridges. I normally use a multimeter and a microscope inspection tool after I solder to help insure no bridges on these chips. They can be pretty deceptive but it's not that difficult. Take your time and keep inspecting before first power on. In this case I think someone has a better shot because you have so much room to work with compared to a Fuel.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
09-26-2023, 07:38 AM
#5
RE: Origin 350 - PCI 5V missing
Hello All,

Thank you for all your inputs !

Yeah, i saw Weblacky's post about the replacement service.
And like @Weblacky said it wouldn't be very economical to me.

Noneless i've read that thread and found some precious info, but i still have some questions.
Do DS1780 always tends to short the 2-wire bus when they fail ?
And did you have any case where a chip falsely reported only one of it's input voltage and without shorting the bus (the datasheet lists 5 different voltage inputs possible) ?
I know considering electronic chips malfunctions everything is possible, but the last possibility seems unintuitive to me.

I can easily order (and will, eventually) the part here but it's still 6.5€/pcs (with 5pcs min. order).

Also thank you @jwhat for the link you provided, i flew over the thread (i will read it carefully when i'm finished with this answer).
And apparently there is 3 locations for the DS1780 on the 2U_Interface. ( I already had the two obvious in G7G5 & H5G5 but to the sneaky in N4K4).
Now i will have to find which chip is responsible for the PCI 5V voltage, and i think it will be easier to me to do the measurements on my working brick (as it dosn't power itself off).
Once I have all 5V measured on the working one it will be quicker to measure on the "dead" one.

And considering my soldering skills, i have no fear.
I did some tsop28 de/re-soldering (for BIOS flashing with a TL866) on my OG xbox without low-melt solder, flux or hot-air station.
And since then I've treated myself with some equipments and consumables (all sort of flux, kapton tape, thin solder wire, solder paste, etc).
So far i have no experience with BGA and very few with QFN :/

Thank you all for your precious answers, reading me and your future answers.
I will keep you informed of my findings.
aSpartan111
O2

Trade Count: (0)
Posts: 6
Threads: 1
Joined: Aug 2023
Location: Fr
Find Reply
09-26-2023, 08:40 AM
#6
RE: Origin 350 - PCI 5V missing
A little advice, don't bother trying to detect which of the DS1780 chips is malfunctioning and replace all three of them. I have personally seen completely shorted DS1780 dragging down the i2c bus, I have personally seen DS1780 with slightly damaged inputs giving slightly incorrect results only for a specific voltage and/or a fan RPM (while showing no other signs of malfunction), I have seen totally absent DS1780 chips shorted open where the bus cannot find them as if they had been removed. Replacement is faster than troubleshooting and preferred as slight damage can go unnoticed as you've seen.

DS1780 where the inputs are slightly off is the rarest I have seen but I have seen it twice in my life. Once here: http://forums.irixnet.org/thread-3238-po...l#pid23284

I have an unpopular theory as to what's damaging the chips but suffice it to say that the chip inputs have little to no protection from overvoltage or momentary overvoltage due to ripple this is also true of its power input. The data sheet specifically says that it should not be given voltage higher than 5.75v. These later SGI designs runs this 5V for these sensors directly from the power supply 5 V rail. Usually part of it standby 5 V ability as well. There is no VRM's and very little filtering in the way (on Fuel this is true, likely on O350, it's true...may not be true on Tezro. I've not looked but likely not due to low DS1780 Tezro failures). The only filtering I've ever encountered so far is a resistor series network that provides some amount of current limiting around the inputs of the DS1780 on fuel main board as well as fuel PIMM and fuel graphics cards.

Yes I'm aware that DS1780 ICs are $6-$7 USD a piece unless you buy a large quantity. You're just going have to bite that bullet and consider yourself lucky you don't have like a ton of them in a rack where you have to do the chip replacement on all of them.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
09-26-2023, 07:14 PM
#7
RE: Origin 350 - PCI 5V missing
(09-26-2023, 07:14 PM)weblacky Wrote:  I have an unpopular theory as to what's damaging the chips

I'm not saying you're wrong, but DS1780 problems were already plaguing O3K bricks when these systems were new and used by their original owners. Over-voltage glitches are probably more of a power on/power down problem and the typical O3K owner would run their systems 24/7.
jan-jaap
SGI Collector

Trade Count: (0)
Posts: 1,049
Threads: 37
Joined: Jun 2018
Location: Netherlands
Website Find Reply
09-27-2023, 07:19 AM
#8
RE: Origin 350 - PCI 5V missing
(09-27-2023, 07:19 AM)jan-jaap Wrote:  
(09-26-2023, 07:14 PM)weblacky Wrote:  I have an unpopular theory as to what's damaging the chips

I'm not saying you're wrong, but DS1780 problems were already plaguing O3K bricks when these systems were new and used by their original owners. Over-voltage glitches are probably more of a power on/power down problem and the typical O3K owner would run their systems 24/7.

Yes....however...not all the "final" SGI systems have this issue.  So I blame individual design decisions (on a bad general design of 5V straight from the PSU to run these things) and the varying quality of each system's PSU model.  Why does Tezro not have this issue?  Why is Fuel the worst?  So on and so forth.  I think some PSUs were better designed and likely those system model that were better matched to their PSU designs did better.  Hence the differing rate of failure of a component they all share in common with the design of their architecture.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
09-27-2023, 08:09 AM
#9
RE: Origin 350 - PCI 5V missing
Hello,

I'm back to you with some new informations.

I did some measurements on my working O350 brick.
And i can safely assume that the DS1780 G7G5 is the one not working.
I confirmed by doing some continuity test on the dead one (G7G5 DS1780, Pin 16).
It also goes through the 1K resistor network array H1G5 (haven't check the others DS1780, but there is 2 others resistors arrays near H5G5).

I soon will pull the trigger (to continue the weblacky's bullet metaphor) on a 5 or 10 pcs order (just in case and it's a bit less expensive starting 10 Pcs), and i will keep you update on the repair process and the result.
I will preventively change all of them (maybe after changing G7G5 to confirm) anyway to be ready for the next 15 years (hopefully Smile).

On the subject on the different tensions i have some questions that haven't found any answer.
I have 6-7 threads to read about some similar subjects including weblacky's rabbit hole, so maybe some of those answers will be inside (sorry in advance if that's the case).

- So first the PSU output a 15V tension (15V BIAS) but i don't know what's its purpose and where is it measured ?
(There is no info in the Env. variables and the DS1780 has no 15V input, so maybe it's not monitored)
- *** There is some DC-DC conversion on the CPU node, so I assume there is also a DS1780 on it to monitor the voltage. Have you already disassembled the CPU node heat sink/metal duck ? Is there any DS1780 (haven't found any on the underside ***read NB at bottom) ?
- The CPU heatsinks seems to be riveted to the metal ducts and I am wandering if it's worth tearing down to check and maybe change the thermal paste ?
- I'm also asking myself why, over the years, no schematics have leaked. Also at some point most electronics products (i know it was really a thing in consumer Hi-Fi) had their schematics laid out in the manual or at least in a service manual. It was a great source of informations for diagnostics.

I also noticed a DS1780 on the 2U_PCI_Riser E9B0 and this one measure 1.8V on Pin 19 and 2.5V on pin 18.
So this one should/could/would be changed also preventively.
There is also an LTC3728 doing the DC to Dual DC (1.8V + 2.5V) conversion.

So far i have found (maybe for future reference):
G7G5 :
-Pin 17 : PCI 3.3V or 3.3V
-Pin 16 : PCI 5V
-Pin 15 : 12V #2 or 12V IO or IP59 12V
-Other ADC input pins 0V
H5G5 :
-Pin 17 : IP59 3.3V Aux or 3.3V Aux
-Pin 16 : PCI 5V Aux or 5V Aux
-Pin 15 : 12V #2 or 12V IO or IP59 12V
-Other ADC input pins 0V
N4K4 :
-Pin 15 : 12V #2 or 12V IO or IP59 12V
-Other ADC input pins 0V

E9B0 (2U_PCI_Riser):
-Pin 19 : 1.8V
-Pin 18 : PCI 2.5V
-Pin 17 : PCI 3.3V or 3.3V
-Pin 16 : PCI 5V Aux or 5V Aux
-Pin 15 : 12V #2 or 12V IO or IP59 12V
-Pin 14 : IDK, measured at 3.145V

Thank you all for your precious inputs and for reading my long and hopefully useful post.
I'll be waiting for the reception of the ICs and i hope I'll have some good news.

***NB :- I have found the DS1780 on the CPU node, it is located between the two 2.5V DC/DC conversion boards (VMEM & Bedrock) name : B3D3.
-Also a DS1780 can measure a sixth tension on pin 14
aSpartan111
O2

Trade Count: (0)
Posts: 6
Threads: 1
Joined: Aug 2023
Location: Fr
Find Reply
10-02-2023, 01:35 PM
#10
RE: Origin 350 - PCI 5V missing
Hello All,

I'm finally back with some news.
I received the chips and some others components.

This week, i replaced the G7G5 DS1780 chip which is in charge for measuring the PCI 5V.
And i have the regret to inform you that the issue remained.

So after an extensive reading of the LTC1629 datasheet and measuring some key tensions ( RUN/SS, EAIN, Ith, etc).
I decided to exchange this chip with the one in charge of the 3.3V.
And to my surprise the PCI 5V is now operational and so does the 3.3V next door.

Well at least the 3.3V was operational for a few minutes, and is now faulty with 0.98V.
So i guess that the LTC1629 is faulty and i'm gonna need to bite another bullet (16€/piece).

Thank you for reading me and I will keep you informed ASAP.


Attached Files
.txt LTC1629 Exchange log short.txt Size: 4.64 KB  Downloads: 200
aSpartan111
O2

Trade Count: (0)
Posts: 6
Threads: 1
Joined: Aug 2023
Location: Fr
Find Reply
11-10-2023, 10:42 AM


Forum Jump:


Users browsing this thread: 1 Guest(s)