Red Fever Trouble
#11
RE: Medicine for your red fever
You best be careful probing around in that charred section, I spot a CNYxx optocoupler there so likely half that circuit is not galvanically separated from the 230VAC.

I know exactly that smell you mention, and it's been > 10 years I last smelled it Expressionless 

Don't despair though. I could very well have some of the things you need to get that thing going:
1. I'm pretty sure I have one last set of bus bar adapters for a PowerOne PSU.
2. I have several of the PowerOne PSUs, but no idea what works and what doesn't. There's also my own Crimson which is that last of my PowerSeries that doesn't have a rebuilt PSU and currently craps out before it finishes booting. That's the least of your worries though, you can still buy PowerOne PSUs.
3. The extra circuit in the SGI version of the PSU is pretty trivial:

[Image: DSC_5007.JPG]

It's a one-layer PCB and I have the circuit diagram somewhere. Replicating this will be straightforward, and the PowerOne PSU can be made to work (temporarily) without if you don't mind that the power switch at the front doesn't function and the system is switched on/off with the breaker at the back.

So there's one possible way forward. Repair of your existing PSU is another. Either way it will probably take a while to find all the relevant parts and that's too bad because you must be anxious to run the system.
jan-jaap
SGI Collector

Trade Count: (0)
Posts: 1,048
Threads: 37
Joined: Jun 2018
Location: Netherlands
Website Find Reply
03-29-2021, 07:43 AM
#12
RE: Medicine for your red fever
(03-29-2021, 12:44 AM)weblacky Wrote:  While I've never been inside these PSUs, I've been inside a few SGI workstation PSUs.  Yes many are customized, but take heart in the that they aren't designed by SGI, they were all designed by power supply companies of the day. That often means, off the shelf parts.  So I'd be very hard pressed to believe those parts are wholly-proprietary to SGI (weren't not talking Apple Computer here).  Yes, the ICs will be discontinued and may not have modern-day formfactor replacements, yeah the connectors are also the same (but you'd often reuse those), but the power circuitry, I'd bet fair money you can find modern replacements for those parts.

You just have to be very meticulous in your desoldering, recording, documenting.  But unless you find a custom management IC that has burnt out, I'd guess you'll do fine given time and care.

There is this IC (custom management IC?) near the charred area of which I hope does still work. There are also lots of small components which are not that easy to identify, of which I also hope to reuse...

Then I plan to redesign and order a pcb - the best would be copper based with a lot of thermal conductance. The culprit resistors have to be changed with a heat-sinked version of some sorts (what a strange design!?)

(03-29-2021, 07:43 AM)jan-jaap Wrote:  You best be careful probing around in that charred section, I spot a CNYxx optocoupler there so likely half that circuit is not galvanically separated from the 230VAC.

I know exactly that smell you mention, and it's been > 10 years I last smelled it Expressionless 

Don't despair though. I could very well have some of the things you need to get that thing going:
1. I'm pretty sure I have one last set of bus bar adapters for a PowerOne PSU.
2. I have several of the PowerOne PSUs, but no idea what works and what doesn't. There's also my own Crimson which is that last of my PowerSeries that doesn't have a rebuilt PSU and currently craps out before it finishes booting. That's the least of your worries though, you can still buy PowerOne PSUs.
3. The extra circuit in the SGI version of the PSU is pretty trivial:

It's a one-layer PCB and I have the circuit diagram somewhere. Replicating this will be straightforward, and the PowerOne PSU can be made to work (temporarily) without if you don't mind that the power switch at the front doesn't function and the system is switched on/off with the breaker at the back.

So there's one possible way forward. Repair of your existing PSU is another. Either way it will probably take a while to find all the relevant parts and that's too bad because you must be anxious to run the system.

This PCB on the other hand would be much easier to rebuild, especially with a diagram. But at the moment there are no SPM5-AD-AEs offered on eBay but lots of SPM5s with other voltage configurations.

So at the moment I try to rebuild the Cherokee I think. But I'm quite laid back concerning getting the Crimson to run, it is an interesting project and the machine is in an overall excellent shape.

And of course thanks to you guys, especially Jan-Jaap - I do not take your help for granted. Be it for the Tezro or now with the Crimson.

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
03-29-2021, 04:19 PM
#13
RE: Medicine for your red fever
Tomorrow is reassembly-day...


Attached Files Image(s)
       

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
03-30-2021, 09:31 PM
#14
RE: Medicine for your red fever
I checked the repaired circuit several times. Then I removed it from its original position to the airflow of the fan, which has been assembled now outside in front the PSU to create space for it.

Well what did I get: A short screeching noise from the PSU and then nothing. A visual inspection yielded only this charred board, but there may be several possibilities for futher failure: by adding cables (0,75 cm² diameter because of the high current inside that thing) I changed the characteritics of that circuit, or there were simply more failed components inside this PSU, or I did something else wrong.

Well now I could ask a company to repair this PSU for me or try to get a PowerOne.


Attached Files Image(s)
       

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
03-31-2021, 10:18 PM
#15
RE: Medicine for your red fever
(03-31-2021, 10:18 PM)Geoman Wrote:  I checked the repaired circuit several times. Then I removed it from its original position to the airflow of the fan, which has been assembled now outside in front the PSU to create space for it.

Well what did I get: A short screeching noise from the PSU and then nothing. A visual inspection yielded only this charred board, but there may be several possibilities for futher failure: by adding cables (0,75 cm² diameter because of the high current inside that thing) I changed the characteritics of that circuit, or there were simply more failed components inside this PSU, or I did something else wrong.

Well now I could ask a company to repair this PSU for me or try to get a PowerOne.


I didn't see a picture of the charred board after this run.  Can you post it, I'm interested to see.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
03-31-2021, 11:36 PM
#16
RE: Medicine for your red fever
These are other before-pictures. I'm going to photograph the recent aftermath somwhen later...


Attached Files Image(s)
       

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
04-01-2021, 04:18 AM
#17
RE: Medicine for your red fever
(04-01-2021, 04:18 AM)Geoman Wrote:  These are other before-pictures. I'm going to photograph the recent aftermath somwhen later...

You're probably aware of how a traditional PSU works: galvanic isolation using a transformer, rectify, and buffer capacitors to smooth the output. The problem with that is that for high power PSUs you need really bulky transformers. Much smaller transformers are possible if you use higher frequencies than 50 or 60Hz.

SM-PSUs all more or less follow this mode of operation: rectify the AC input to 300+ VDC, 'chop' this into a square wave which is at least several dozen KHz, run this through a transformer, then stabilize and filter to required voltage(s). There is a chicken-egg problem though: the chopper and control circuitry needs power too, so you need some aux power supply. The chopper circuit is *before* the transformer so not galvanically separated from mains input. So to create a feedback loop from the output circuit to the chopper, you need galvanic isolation once again. This is usually done with an optocoupler: basically an LED and a photosensitive transistor in one package. Now look at this photo:

[Image: attachment.php?aid=1937]

This looks like such a feedback circuit.

The CNY17 is the optocoupler The LM339 is a differential comparator.

The functionality of the circuit is likely along the lines of: current flows through big resistors, causing voltage drop, amount of drop compared by differential comparator to ref value after being galvanically isolated by optocoupler.

You'd have to trace out the circuit to figure out the fault mode: obviously a lot of current flowed through the big resistors causing the burn, but is it because the feedback loop is broken (optocouplers wear out), the resistors shorted, or the comparator is faulty?
jan-jaap
SGI Collector

Trade Count: (0)
Posts: 1,048
Threads: 37
Joined: Jun 2018
Location: Netherlands
Website Find Reply
04-01-2021, 07:43 AM
#18
RE: Medicine for your red fever
@weblacky: Now I took apart the power supply coming home from work with a fresh mind (i e. fresh concerning computer related stuff). My repaired board looks 'perfect' under the magnifying glass: no blown caps, burns or exploded ICs. The fuse is still intact and I cannot see any problems with the rest of the PSU in a visual and olfactory (not a single source of magic smoke) inspection. So pictures before and after are literally the same.

But I found my error: a very small part of my soldered cables came in contact with the heatsink, on which it was attached to, despite my isolation. A corrected that, and the PSU worked again for a brief moment until the circiut braker tipped again, just after the relais inside switched on and off in a fast succession for 10x or so.

@jan-jaap: Thanks for the excellent explanation of this board and PSUs in general! Yes, I tested the optocoupler in my "transistor tester" and the other components, except for the comparator-IC. For that I would have to scope it, and I must admit that I lack the education and tools to diagnose a live and possibly deadly PSU.
________________________
At first I saw a broken guarantee seal, during this second thorough inspection I spotted a well crafted repair of another charred area in the past.

So my decision is to wait for a buying opportunity of a known-working PowerOne for Power/Crimson.

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
(This post was last modified: 04-01-2021, 08:19 PM by Geoman.)
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
04-01-2021, 08:18 PM
#19
RE: Medicine for your red fever
Good to hear you took another look at it! From my limited experience relays are normally the activated protection for voltage out of range issues. As high-current shorts normally happen so fast that it needs to blow a fuse (relay's aren't fast enough). That does rule them out as overcurrent protectors, it's just not for short conditions.

A curve tracer is better for testing an optocoupler, but you need to actually check that the LED inside actually works (do this with a multimeter in diode mode if that's what you have). A burnt out LED (inside optocoupler) will mean no correction signal takes place, assuming you just said you checked the transistor side. You SHOULD find if you"check" the LED with a multimeter in diode mode that the transistor side should conduct! That is the optocoupler actually operating.

As it concerns the card with the big resistors. I'd suggest two basic things.

1. Resistors can be odd, while you can measure them in circuit and if you find them HIGHER then stated, that's bad. But you could find them lower (due to parallel/series arrangements). So I'd advise you take the easy route (assuming you have a desoldering gun or sucker) and desolder one leg of the diodes and resistors and lift the part a little bit out of the PCB on that desoldered side and measure it "out of circuit". Check for Resistor drift. The colored bands even tell you the margin of error (1%,5%, 10%), check the the numbers you get. If you find something out spec, that's important and I'd look into replacing that.

The green-tinted resistors MIGHT be inductors and not resistors. If all of those all start measuring weird that likely that's what they are.

2. My next basic suggestion is about the ICs in general (again while there are better tools for this), most ICs have protection diodes in most or all pins. Sometimes they don't on output pins but VCC/GRD and IO normally do (at least for ESD protection). I would desolder the ICs and test the leg of each pin again the Ground and possibly VCC pins to check for the presence of a DIODE, if you get a resistance instead...that could indicate an internal short! If you can find a datasheet, "like legs" normally have a similar structure. For example. If you have 3 "output pins", often the 3 output pins have identical protection, resistance, etc.. so you can use that do compare one pin against its sibling (same function, if possible) when it's out of circuit. That will often show a damaged pin among a family of pins on an IC.

To that end while you have them out, just check the tracks going to nearby components from the hole's left in the IC footprint. Just to see if you have track damage at all.


I kind of think that since one "pair" of the resistor banks is so much worse than the other. That you have a problem with that bank (or the rail it monitors). Also a manufacturing trick if you're going to desolder stuff, increase the height of the big resistors above the board surface (don't touch PCB), the higher in they are in air the more airflow and distance from the PCB they have...the less heat damage they can do.

Also note that PERHAPS, the reason for the increase in heat isn't necessarily current as we'd label it, maybe it's ripple. Ripple causes heating as well and since this is obviously second stage from the chopper it's possible the MAIN filter caps, combined with any other secondary filters are so bad, that the ripple on the 300V backbone, was bad...really bad.

I know you're going to wait for another PSU, You can KIND OF see this affect with a Multimeter set on AC (mV). Ripple will often register as AC if it's big enough on the DC outputs. With PSUs, rule of thumb in the industry is measured AC V/DC V (both Volts units, not mV /V) needs to be 1% or LESS to be acceptable. Anything higher is unacceptable. So that might be a way of proving that. But if it were me and you now have a PSU that is successfully protecting itself. I'd see about good, high quality Caps (yeah the big ones too).

On it's face (without knowing more) this smells like a lack of filtering causing poor power regulation (hence bad feedback?). Yeah could be a drifted passive part (I'm not saying it's not). But this smells like the filtering function of your large caps is junk and nothing else can work now.

If you're relay is going on that means the basic feedback circuit works...however you have no idea if what it's reading is real, only that the feedback does reach a decision maker. So I'd still do a very careful (part by part) examination on that board you showed (burnt one) as it may be getting wild signals as well.

Also, I'd suggest a few more ideas (not destructive) once you get your replacement PSU. With PSU off and discharged, I'd measure the resistance of the voltage rails by ramming a multimeter lead up the voltage connector output pin and the PSU ground (Not chasiss). Those resistance values should be similar between PSUs for each voltage rail. Then Swap the leads (red on PSU ground, black on voltage rail). You should get a different readings (two readings by swapping leads), one reading is smaller than the other (Black -> ground is smaller) If you get the same reading for both directions that proves you have a failed secondary diode in that rail and you're probing the secondary winding of your chopper in both directions (a shorted voltage rail).

You can tell a lot about a PSU from the outside. You cannot really hurt it when it's off. Just use your meter and probe around on the DC outputs and the AC connector. Note any real difference from the other PSU you get, that will point you where to look! If you get good resistance values you could try DC injection next. But capacitors don't register on DC (they do on AC, that's how ESR meters work). IF your outward measurements all basically look the same then I'd say it's a filtering + logic issue. Just my hunch with what I see now.

Let us know how you progress.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
04-01-2021, 10:39 PM
#20
RE: Medicine for your red fever
@robespierre, GeekLucanis, weblacky, and jan-jaap: thank you for all the great input, I learned a lot! Especially from jan-jaap and weblacky.

But I also have to admit that I work in a profession with very few free time, so I decided to "kill" this problem with money.

SGI - the legend will never die!!

Indy Indigo Crimson Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT O2 O2 Octane Octane2 Octane2 Tezro
Geoman
Crimson to Tezro

Trade Count: (0)
Posts: 162
Threads: 13
Joined: May 2018
Location: Germany
Find Reply
04-02-2021, 10:31 AM


Forum Jump:


Users browsing this thread: 1 Guest(s)