Fuel needs to be power cycled to start
#1
Fuel needs to be power cycled to start
Hello,
I have another issue with my Fuels I’m trying to restore from their deadness.
I experienced this behaviour with already two machines so there’s a chance it’s a common issue. When I connect the PSU to power and try to press the power button, nothing happens, not even L1 outputs anything to its console. What helps is to power cycle the 5V AUX power branch by reconnecting the 24pin power connector from motherboard. Sometimes it’s enough to do this once, sometimes twice. After this procedure I got the L1 console output and I’m able to start the machine that functions normally. The problem occurs again when I disconnect the power cord for at least few seconds - enough time to discharge PSU caps.

The PSU is ok, works fine with another machine.

Any ideas? Many thanks.
(This post was last modified: 04-07-2021, 06:40 PM by cz7asm.)
cz7asm
O2

Trade Count: (0)
Posts: 25
Threads: 6
Joined: Feb 2021
Location: Czech Republic
Find Reply
04-07-2021, 05:55 PM
#2
RE: Fuel needs to be power cycled to start
What is the hardware config (and graphics) of this machine, what is the configuration of the "working" system?
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
04-07-2021, 11:32 PM
#3
RE: Fuel needs to be power cycled to start
From what I tried it really doesn't matter what PIMM module or GPU is in - it happens the same way even if I disconnect them. However, GPU is V10 and PIMM with 600MHz CPU.

Another observation is a difference in power consumption. When I test it using bench power supply for 5V AUX rail, it behave the same but at least I can see the difference in current - the state when it hangs eats about ~270mA whereas the working state jumps to ~280mA.

I thought of a couple of tests I must try out, one of which is using different NVRAM module. Yesterday I found out that the machine doesn't start L1 when the NVRAM Dallas chip is not inserted so there might be some connection. Also some bad capacitors might cause this but I have a feeling this is not the case.
cz7asm
O2

Trade Count: (0)
Posts: 25
Threads: 6
Joined: Feb 2021
Location: Czech Republic
Find Reply
04-08-2021, 07:36 AM
#4
RE: Fuel needs to be power cycled to start
Based on your response, I'd suggest monitoring the PWR_OK pin of the main PSU connector. https://wiki.preterhuman.net/Fuel_PSU_repair

My guess would be it's not being signalled fast enough due to PSU age, restarting a "charged" PSU is working...but not a cold start-up.. You may need a basic oscilloscope to see this signal. Most modern PCs need the signal within about 400ms (300ms is perfect), I've seen "zombie" PCs (fans start but board never posts) where the signal was at 800ms (but all voltages were GOOD!). Many boards need the signal within a specific power-on time period (they don't just wait forever then start at the signal). This indicates that the PSU is taking too long to stabilize its outputs for clean start-up

That's my basic guess, it's the PSU signalling. If you want to try faking this (dangerous as PSU really hasn't stabilized, which is what PWR-OK is for, you could remove the PWR_OK pin from your connector and use a bench PSU with a resistor on the connector trace or solder leg to fake a good signal immediately from power-on...likely you want to try VERY low current on this signal pin until it works...likely nothing greater the a few hundred MilliAmps to make the "signal" (pull "high" be present from the start).

However there is a reason for the signal. I'd start by doing a basic measurement and measure the time from powering on until the signal goes fully high. That time will be less than 1 second...and likely needs to be less than 500ms for the board to actually turn on correctly. You'll also get the max voltage you need to be at to tune a test-signal hookup.

Let us know what you find.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
04-08-2021, 08:08 AM
#5
RE: Fuel needs to be power cycled to start
The PWR_OK signal is used to signalize stabilized voltage rails *after* PSU powers up (PS_ON pin is asserted). The scenario I'm describing happens *before* PSU powers up - only the 5V AUX (standby) line is on. From this line the L1 controller is sourced plus all its peripherals like the L1 console, NVRAM + EEPROM or monitoring chips.
As I mentioned, I tried different PSU's that normally work with my other Fuels. Also to get L1 going one doesn't need to use a PSU, basically any 5V voltage source is enough.

However! Some moments ago I managed to try the NVRAM theory I mentioned in my previous post and it seems to help. It looks like this behavior is caused (or contributed) by NVRAM chip that has a dead battery. What I cannot explain why the power cycling helps though.

It's not easy to buy these NVRAM DS1742W-120 chips that still have healthy battery. I tried several pieces from eBay but they all seem to have dead battery or even none inside. I'm playing with an idea to try to build a circuit that would emulate it as there seems to be no replacement from Dallas available.

Anyways, now it needs some time to verify it really helped and it's not just a temporary glitch caused by today's snow calamity. Also many thanks for elaborating on the PSU PWR_OK issue - I'm sure I will encounter this problem sooner or later.
(This post was last modified: 04-08-2021, 12:56 PM by cz7asm.)
cz7asm
O2

Trade Count: (0)
Posts: 25
Threads: 6
Joined: Feb 2021
Location: Czech Republic
Find Reply
04-08-2021, 11:32 AM
#6
RE: Fuel needs to be power cycled to start
Perhaps it’s your description of events but something doesn’t seem properly described.

Do you have L1 console output that suggests you have a dead battery? On that RTC? For that matter have you checked your yellow snaphat battery as well (much more likely to be dead). Does your L1 output during start suggest it’s defaulting and filling values after checksum errors or encountering bad values?

Otherwise, the reason I have trouble with this theory is, you claimed in your description that if you leave the Fuel off and then try to start it you have to cycle 5v power to get it to start.

Now I assume you meant this literally, like you have a running Fuel after your fiddling, you say shutdown in the PROM or Irix and it does so. Then you leave it alone and do not actually cut PSU AC power, so 5v standby has been feeding the RTCs all this time. Yet you still have power on issues on your next attempt?

The L1 RTC (for sure, not sure about snaphat, but likely the same) battery is for when the board doesn’t receive 5v. Which would only happen if you actually pulled AC wall power between system startups.

Please clarify startup testing procedures. Because while I could understand cycling to fill the NVRAM of a blanked RTC because of dead battery after loss of wall power. After successful boot, your RTC values are now filled out and running. Unless you’re pulling PSU standby power before every boot up attempt, your PSU should be feeding standby 5v to the RTCs to retain the newly booted NVRAM data which is valid after last successful startup and it should simply boot normally as long as you’ve not interrupted PSU wall power between power-on attempts.

You suggest in your description that this isn’t the case and that you need to cycle the connector on every boot-up attempt.

Please clarify if, after a successful boot, you can continue to restart and shutdown and start normally as long as you’ve not unplugged the PSU. Otherwise I think you have something else going on. As the RTC docs are clear about when battery is used. You can check incoming mainboard power at the RTCs to confirm RTCs are being externally feed retention/operating power between boot attempts.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
04-08-2021, 01:21 PM
#7
RE: Fuel needs to be power cycled to start
Quote:Do you have L1 console output that suggests you have a dead battery? On that RTC?  For that matter have you checked your yellow snaphat battery as well (much more likely to be dead). Does your L1 output during start suggest it’s defaulting and filling values after checksum errors or encountering bad values?


Yes, in case of NVRAM chip with dead battery the L1 is complaining about NVRAM checksum so here I assume in this case it's reinitialized from the Atmel EEPROM. The snaphat battery is fine (I suppose it's not even used in L1).

Quote:Otherwise, the reason I have trouble with this theory is, you claimed in your description that if you leave the Fuel off and then try to start it you have to cycle 5v power to get it to start.

This is when I leave the Fuel off with PSU disconnected from mains. Then I have to power cycle the 5V standby rail in order to start L1 and the machine. If I leave the Fuel off with PSU connected to main then the 5V standby line is still on and I can power on the Fuel without any problems - this is because the L1 is initialized and running.

Quote:Now I assume you meant this literally, like you have a running Fuel after your fiddling, you say shutdown in the PROM or Irix and it does so.  Then you leave it alone and do not actually cut PSU AC power, so 5v standby has been feeding the RTCs all this time.  Yet you still have power on issues on your next attempt?

Already answered above - no, there are no power on issues in this case.


Quote:The L1 RTC (for sure, not sure about snaphat, but likely the same) battery is for when the board doesn’t receive 5v.  Which would only happen if you actually pulled AC wall power between system startups.


Agreed.

Quote:Please clarify startup testing procedures. 

I'll do so verbatim:
  • the Fuel is disconnected from AC, I plug in the L1 console cable
  • the Fuel is connected to AC, the power on button doesn't work, no L1 console output
  • reconnecting AC cable doesn't help - this is very likely caused by PSU capacitors not resetting L1
  • this is the point where I power cycle the 5V AUX rail by unplugging the 24pin ATX-like power supply connector from the motherboard
  • if the above doesn't help to get L1 console output, I repeat
  • now I have L1 output and can power on the machine
  • until I unplug the AC cord, all consequent power on attempts are without any issues
cz7asm
O2

Trade Count: (0)
Posts: 25
Threads: 6
Joined: Feb 2021
Location: Czech Republic
Find Reply
04-08-2021, 01:55 PM
#8
RE: Fuel needs to be power cycled to start
I have this issue on a fuel and by power cycling it works around it in the same way, its a new PSU (corsair rm650x ATX PSU) did replacing the NVRAM DS1742W-120 resolve the issue in the end? I will need to track one down.
teng
O2

Trade Count: (0)
Posts: 3
Threads: 0
Joined: Nov 2024
Location: UK
Find Reply
11-26-2024, 09:10 PM
#9
RE: Fuel needs to be power cycled to start
Cz7asm,
A couple things that kind of don't mesh with this logic that I'll bring up.

Firstly the fuel is not based on the ATX standard, it's based on the WTX specification that started up and failed to take hold.

https://phantom.sannata.org/download/file.php?id=30455

So your assumption of a 5 V only standby line is incorrect. While I didn't find the main PSU document specification the above file states clearly in the last section that the WTX standard provides both 5V and 3.3V standby for the system.

I know you claimed that the power supply you're using works fine another system. I would still feel better if you had another power supply, even if it was an ATX power supply using the established adapter board to confirm that versus one power supply that might be suspect.

To me this sounds like a power problem. Now I think you should attempt to use a multimeter that has a mini and max feature and see if you can probe the back of ATX connector while it's on the board and cycled the power to the power supply using a power strip or the power cord on the wall and see what Your meter says. My thought is that you're just barely getting the correct voltage for things to work.


3.3 V requirements are actually much more common than a 5 V requirement for surface mount components that are also semiconductors. So I'd like to believe that the trick of constantly unplugging and re-plugging the motherboard load using the 24 pin connector from the power supply is in fact resetting the power supply Over and over again until it produces the correct voltage.


Now there is the possibility that you have a failing semiconductor in the power path on the mainboard that is sometimes sticking in the on or off position that is causing what you might consider to be a short circuit on the motherboards power system that appears to be a load that isn't normally there at that stage and potentially that's pulling the voltage lines a little too far down.

If you want to compare the boards with the problem to the board that seems to work with the same power supply what I would advise you to do then is to get a good quality multimeter and use both ohms and diode mode tests on every single pin on that 24 pin connector to main ground through the connector. The multi-pin connector that's much smaller is the 12 V VRM power system. It's important but it's not seemingly something that you're involved with so I can easily say that 12 V power is not your issue here nor is that VRM as it only starts up through auto power on second stage. So you're squarely looking at 5 V and 3.3 V in that main 24 pin connector. Go ahead and make a simple listing of the pins and the diode mode reading you get snd the ohms mode readings you get. To be thorough you need to reverse the polarity for another round of testing.


That is to say you need to first do multimeter black common probe to ground and red lead to the component under test on each of the 24 pins that isn't ground. Then start another list and reverse the leads with the red probe being on ground and your black probe being used for probing. This reverse polarity should often come up with a different measurement. Sometimes it'll come up with appears to be a direct short and those are normally protection diodes to ground That are usually fine. You're not totally caring about what the values are only what they are across all three samples.


The point of doing both these tests and the reverse polarity of the same tests on all the boards is looking for differences. Your supposition is there something going on with just the two boards. Making the assumption that it's not the RTC module which is modular and you've already plugged in and out then it must be something not modular on the board. Since the electronics won't come up they're not meeting some form of voltage threshold or current threshold that the system requires to start those components.


Create a listing for all three boards involved. The two that don't work and the one that does. If you find one or more pins that do not measure anywhere near the same. Then you have your culprit for investigation. I would say anything that measures outside of 10% is something you should at least note but I would say it needs to measure 20% outside or more for you to really put your attention on it.

The vast majority of repairs I do are based on a comparative method. I'm not an electrical engineer and I don't have secret schematics that I hide from everyone. I go by comparison most people do this as well. You have a working board and you have a non-working board and as long as you use a multimeter With the board with unpowered there's very little chance you can do any damage except for scratching something with the probe tip. I do not recommend doing much live testing as that's not what I do and one slip up could cause horrific damage. So I recommend boards off and disconnected from the power Supply and removal of both their wiring harnesses. You do not want the power supplies wiring harness to be part of the circuit while you're measuring it. So you want to act like the boards have been removed from the case in that there's no power supply connected.


I think this technique will give you a much greater insight if there is a variance in the board as perceived by the power supplies 24 pin connector. Remember the 24 pin connector is what you're fiddling with so something on that is where the problem is connected to.

My view is still but the power supply is weak but That isn't 100% reality unless you can get another working power supply and prove the same issue. If that's not available then performing both diode forward voltage base measurements and ohm resistance measurements in both polarities on every one of those pins is the only way to tell whether there is a difference or not from the power supply's perspective at that point.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
11-26-2024, 10:29 PM
#10
RE: Fuel needs to be power cycled to start
Hi Weblacky,
Apologies if some confusion (my fault I revived a several years old thread here) I saw cz7asm thread from some years ago as he outlined exactly the same symptoms and workaround i’m observing entirely independently on a newly acquired Fuel with a different PSU. Pleased to meet you, I’m teng Smile

cz7asm outlined in his earlier post that he swapped the NVRAM DS1742W-120 and the issue seemed to go away. I can say that my PSU is basically new in box, manufacture circa 2018 corsair ATX supply and ATX adaptor board. I replied as my issue is uncanny and exactly the same issue.

Internal L1 will not come up unless the power switch is toggled. on, off, on It does this very reliably

My speculation on the sequence
toggle power on NVRAM DS1742W-120 is flat, L1 hangs on start and stays crashed circuit gets energised with some volts
toggle power off L1 terminates, NVRAM DS1742W-120 circuit will retain some volts until the PSUs/mainboard filter caps discharge
toggle power on, NVRAM DS1742W-120 circuit is already energised, L1 will start up as normal

toggle off, on allways works 100% of the time, this is what we both observe

I could perhaps try some tests on the NVRAM DS1742W-120 I’m not quite sure what the relationship is with the top hat, am i correct that fuels have 2 separate batteries in?
(This post was last modified: 11-26-2024, 11:19 PM by teng.)
teng
O2

Trade Count: (0)
Posts: 3
Threads: 0
Joined: Nov 2024
Location: UK
Find Reply
11-26-2024, 11:12 PM


Forum Jump:


Users browsing this thread: 1 Guest(s)