[weblacky]

Did you place the 800 MHz PIMM in the system before flashing or after?

[jwhat]

Hi Weblacky,

I first swapped out CPU module and machine booted up reporting as 600 MHz.

Then did flash and reboot.

After reboot it reported as 800 MHz machine.

Cheers from Oz,

jwhat/John

[jan-jaap]

I think what we're all interested to know is what happens if you now put the 600MHz CPU back in.

The chicken-egg problem here is that if you need a working system to flash the parameters but the wrong parameters prevent your system from booting you're stuck. If that 800MHz CPU dies, will you be able to back to 600MHz?

Flashing using the L1 would sidestep the 'working system' requirement. A mainboard booting in some failsafe mode if the parameters are wrong would too.

[jwhat]

Hi Jan-Jaap,

now I have finally managed to get machine running at 800 MHz, not keen to take it back to 600 MHz….

Seriously though this CPU module was originally shipped from China (Shanghai) in Feb 2021. Is has taken 8 months to get to me. The package was badly handled and a lot of stuff was smashed / bent. The CPU module survived, but with lots of squashed pins on CPU connector, as well as cracked connector plastic. I very carefully unbent the pins with very fine tweezers and gave it a go, not expecting it would boot. But I got lucky and it was ok, so did flash to get it back running at 800 MHz. As the connector is now in poor condition I am not keen to take it back out at the moment.

Based on prior very old posting from Nekochan (and Hamei at start of this thread), the only way to recover a Fuel board that has been flashed to higher clock speed is to put a module of right speed into the machine and then use that to down flash it to lower speed. Then swap in the slower CPU module.

As part of shipment from China, I also got the Fuel system board that went with CPU, but that has got at least one capacitor ripped off board and a lot of very badly bent fan and cable pins. I might see if I can set this up as bench test machine.. then I could play with that and slower CPU.

Likely a task beyond my repair skills.

EDIT #1: looking at log and seeing this running, the flash is doing more than just putting a couple of values in an eeprom. It first wiped the “Bedrock PROM” (which had already been written to 14 time according to log) and then rewrote to this (Writing 1476168 bytes of data). So if people are looking for the undocumented POD way of doing this then it would likely be knowing offset and value to stuff data into “Bedrock PROM”.

EDIT #2: thinking about this a bit more…. On O350 Chimera, the Bedrock ASIC (had hence the referenced PROM) is on the removable CPU board, not the system interface board. That is why the PROM and board go together as one and there is no need to flash PROM cpu speed when you swap an O350 CPU board over. On Fuel the Bedrock is on system board and not the PIMM. So separation of these creates need for flashing of Bedrock PROM. Me thinks, the PROM is not on the IO9 board, so looking for one there is not going to find the common Bedrock related components. So to find common components you need to look at Fuel system board and O350 CPU boards….

Cheers from Oz,

jwhat/John.

[weblacky]

Hi Jan-Jaap,

now I have finally managed to get machine running at 800 MHz, not keen to take it back to 600 MHz….

Seriously though this CPU module was originally shipped from China (Shanghai) in Feb 2021.  Is has taken 8 months to get to me. The package was badly handled and a lot of stuff was smashed / bent.  The CPU module survived, but with lots of squashed pins on CPU connector, as well as cracked connector plastic. I very carefully unbent the pins with very fine tweezers and gave it a go, not expecting it would boot. But I got lucky and it was ok, so did flash to get it back running at 800 MHz.  As the connector is now in poor condition I am not keen to take it back out at the moment.

Based on prior very old posting from Nekochan (and Hamei at start of this thread), the only way to recover a Fuel board that has been flashed to higher clock speed is to put a module of right speed into the machine and then use that to down flash it to lower speed. Then swap in the slower CPU module.

As part of shipment from China, I also got the Fuel system board that went with CPU, but that has got at least one capacitor ripped off board and a lot of very badly bent fan and cable pins. I might see if I can set this up as bench test machine.. then I could play with that and slower CPU.

Likely a task beyond my repair skills.

EDIT #1: looking at log and seeing this running, the flash is doing more than just putting a couple of values in an eeprom. It first wiped the “Bedrock PROM” (which had already been written to 14 time according to log) and then rewrote to this (Writing 1476168 bytes of data). So if people are looking for the undocumented POD way of doing this then it would likely be knowing offset and value to stuff data into “Bedrock PROM”.

EDIT #2: thinking about this a bit more…. On O350 Chimera, the Bedrock ASIC (had hence the referenced PROM) is on the removable CPU board, not the system interface board. That is why the PROM and board go together as one and there is no need to flash PROM cpu speed when you swap an O350 CPU board over. On Fuel the Bedrock is on system board and not the PIMM. So separation of these creates need for flashing of Bedrock PROM. Me thinks, the PROM is not on the IO9 board, so looking for one there is not going to find the common Bedrock related components. So to find common components you need to look at Fuel system board and O350 CPU boards….

Cheers from Oz,

jwhat/John.

jwhat,
Please provide photos of the damaged areas of that board.  When I do my Fuel board I can make notes and pics of what components you need.

[jwhat]

Hi Weblacky,

here you go (lots of pins squashed down onto the array plastic and plastic locator damaged:



Cheers from Oz,

jwhat/John

[weblacky]

Yeah, the only thing stopping you is that it's controlled BGA rework. They still make these connectors and they seem to be about ~$25 dollars a piece if you can find a distributor of quantity = 1.

https://www.amphenol-icc.com/product-ser...array.html

https://www.digikey.com/short/z544ncnj


I don't know the board mating height or seat height, but I count 240 pins and row of 8, so that gets only 3 options for seating heights. The support document details a BGA machine ramp-up/down sequence (though fails to disclose the peak temp and it refers to a table in the doc that doesn't exist :-(.)

If I was more confident in my BGA soldering, I'd try it...but BGA soldering is an experience I don't have enough of, especially with LARGE items. The support document claims you need to preheat the PCB to about ~400F then quickly ramp up and into the critical heat region for soldering and then down under it in like <40 secs! I don't think a human has a chance on these. Man I wish I had the tools/toys to do this kind of work!

[jwhat]

Hi Weblacky,

I think as long I don't move the board out of machine it will be ok, with my pin unbending job and to keep things simple maybe consider putting a bit of glue on array plastic to stop it breaking further.

On another item:

I keep my Fuel plugged into power and have noticed that it emits a high pitched sound when powered down (I got the kids to confirm as their hearing is much better than mine).

This only started recently, as I have not noticed it before.

Is this a potential warning of pending power supply failure ?

What is your advise on keeping machines power up ?

I kept the Fuel powered on (though not running) to stop it losing time clock, but since I replaced the Snaphat battery this is not really needed anymore.

Is it better to switch off power completely or leave it still “on” with L1 going ?

On all my O350 boxes I shut off the power completely.

Cheers from Oz,

jwhat/John.

[weblacky]

Yo Jwhat,
What you describe sounds like “capacitor whine” and maybe the physical oscillation of one or more capacitors (normally it’s the large filters, but not exclusively). I’ve heard it before on ATX PSUs, it’s not a “stage” in the life cycle you always get to before failure but it is a very good indication of age (Or if PSU is new, substandard caps) related failure.

I would 100% approach that symptom as a gift, giving you advanced warning a PSU going out of ripple/filtering tolerance as it’s now physically moving inside due to the AC Cycle, because it can no longer absorb/tolerate the demand placed on it.

Short answer, failure is imminent. If you do tear down and cap replacement right now, you have a great chance of having a working PSU. You maybe able able to buy time by just replacing the filter caps, then plugging it in and seeing if the whine goes away. I suggest a full cap replacement but the main, large, filter(s) do a huge amount of work so start with that. I noticed on my new, broken, fuel the original PSU is extremely sparse and light, so it may have only a single filtering cap (like Tezro). This cheaper design only accelerated failure as there was no second cap to help share the filtering load.

My advice is to replace all the caps with exactly the manufacturer they are now, same family line if you can get them. The filtering caps don’t have to be the same family/manufacturer just make sure they are high quality. A good filter cap is ~$7USD here in the states.

If whine is happening, your filtering is compromised, semiconductors failure won’t be far behind, unless a cap shorts, but likely you’ll blow a diode before then.

In terms of the “left on” question. Here’s what I believe but it’s not gospel: some of these modern PSUs have an inrush bypass to help but most of these use a NTC inrush limiter that heats up (kind of like a light bulb affect) to slowly engage power on the high voltage AC-side of the PSU. The PSU stops itself from running the main DC by simply shutting down its main switch power transistor(s) just before the main Transformer. There is normally a standby voltage that it keeps making usable voltage on an aux path to either a smaller transformer or a regulator for that small standby power for soft-power-on. Obviously Fuel uses the standby to run the L1 and cooling (at times?), so work and power draw.

So when you’re not using a PC (or the Fuel to a greater extent) your high voltage end of your PSU is still filtering the AC coming into it, still heating its inrush limiter to allow passing of current (unless it has a relay bypass) most old PSUs don’t, still sending aux voltage, still running L1.

I think you see where I’m going. You’re working the PSU (to a lesser extent than running the full machine) even when it’s off…and you’re in bed sleeping/not using the machine. It’s working, 24/7, filtering power on the large cap(s), oscillating with the AC frequency, creating a more crude power for the aux voltage (not as smooth, not as clean, simpler design for smaller implementation of the AC to DC conversion).

So you’re asking it to preform that much work, just plugged in and not running.

I think from my long description you read my point, unplug it or hit the OFF switch on a power-strip. Turn off PSUs when not in use. Saves power, heat, wear. However if this were a data center and you told me this was happening and you NEEDED this machine up, I’d tell you to turn it in and NEVER SHUT IT DOWN, while I ordered you a new PSU. On a healthy PSU, shutdown and startup are no problem. Startup however uses a few components that can fail with age and heat as well. Many network gear PSUs will fail to comeback online after running 4+ years straight in a data center because running a PSU at idle (like car motor) is different from the startup process (different process). The startup process can fail yet the PSU (once started) can keep running.

That’s the only case I would tell you to risk it and keep it running as part of a production system.

In this case, I’m being thoughtful of the equipment’s age and value and the rarity and telling you to unplug and risk more startups/restarts than to leave it running fully time. But I would still warn about startups being heavy so be aware that startup/shutdown is the prime time of when components actually fail!

Most people with a broken PSU think it failed during startup, most semiconductors seem to fail on last shutdown, it’s just silent. It can happen whenever though, nothing is exclusive once you fall out of tolerance.

[jwhat]

Hi Weblacky,

thanks for very thorough answer.

Your feedback on recapping vs replacing Fuel power supply is worth a separate thread…

Back to PROM flashing, I just did re-read of SGI O350 doco (via “Jurassic techpubs” ;-) ) to confirm that Bedrock PROM is on O350 CPU board. My recollection was correct and SGI doco clearly has the Bedrock ASIC as being under the metal cover on the O350 board. So likely the Bedrock PROM is under the cover as well. This is why people cannot see it.

I would be willing to bet a broken / bent SGI system board ;-) , that the little pluggable chip that is common to the Fuel and O350 IO9 board is not the Bedrock PROM. Could this be sitting under one of the big heat sinks on Fuel?

The Bedrock provides the CPU / CPU and memory comes: “Bedrock ASIC (or hub ASIC), which enables communication between the processors, memory, and I/O devices.”

So it would need cooling as much as CPU.

If I had a spare O350 CPU, I would open it up and have a look, but I sold the last spare I had to help fund replacement quad 800.

Another thing to note: Octane also has flashable PROM and does not need to be “re-flashed” like Fuel to accommodate CPU swaps. So I think it’s PROM is also on the CPU module. Unfortunately my Octane is dead due to V12 failure at the moment, but another useful test would be to see if PROM version changes as a result of Octane CPU swap (Octane XBOW switch has similar architecture to O3000 according to old SGI marketing/technical docs). On O350 I have seen PROM change happen a few time when adding new boards / chassis into my numalink’ed O350. It reports different PROM version across chassis on boot and I have had to reflash to bring new board/chassis up to alignment with the rest of them. The flash utility only flashes the chassis that has the older PROM version and leaves the rest untouched.

Anyhow, enough “chewing the fat” on this one.

Cheers from Oz,

jwhat/John