Soft Updates being removed from OpenBSD - Printable Version
+- IRIX Network Forums (//forums.irixnet.org)
+-- Forum: Other Platforms (//forums.irixnet.org/forum-5.html)
+--- Forum: Other UNIX (//forums.irixnet.org/forum-30.html)
+--- Thread: Soft Updates being removed from OpenBSD (/thread-3997.html)
+- IRIX Network Forums (//forums.irixnet.org)
+-- Forum: Other Platforms (//forums.irixnet.org/forum-5.html)
+--- Forum: Other UNIX (//forums.irixnet.org/forum-30.html)
+--- Thread: Soft Updates being removed from OpenBSD (/thread-3997.html)
Soft Updates being removed from OpenBSD - Raion - 07-17-2023
https://undeadly.org/cgi?action=article;sid=20230706044554
It is cited as an impediment to properly implementing a VFS layer for OpenBSD.
Once upon a time maybe about 10 to 12 years ago I was a huge proponent of soft updates. However considering it never had wide use (NetBSD only briefly supported it and FreeBSD uses journaling primarily) it's not much of a loss seeing it go because it was never really able to be competitive with a journaling file system.
For those confused why journals or soft updates are necessary, it's a combination of performance and crash safety.
Old file systems like FAT32, EFS, Ext2 etc. Are synchronous file systems. Every operation is completed in the order it is specified and thus during an unclean shutdown it must be walked through with a file system checking utility such as fsck or chkdsk.
The problem with synchronous file systems is that it basically grinds performance to a halt if you have a very big file that needs to be written. Thus the first way of dealing with this was journaling or log file systems. Most journaling file systems only preserve the metadata of the files which basically is all of the referential information needed to retrieve that file. This might include size, sector locations etc depending upon the data structures in the file system. Journaling file systems work pretty well because they allow for decoupling operations in an asynchronous manner.
Soft Updates/ Soft Dependencies by comparison does not use a journaling structure. Instead it builds a dependency graph and using a small amount of memory as a writeback buffer, performs operations in a very specific order specified by the dependency graph. The benefit of this is that you can use asynchronous writes. The drawback is that it doesn't necessarily guarantee that your file system is always clean. On reboot instead of simply checking the index for a file system check you instead have to walk through the entire file system but that can be done in the background _because_ the file system can only have a storage leak or so the original paper involved claimed. A storage link is when a file is deleted but the space is marked used or that an incomplete file write was in progress but not not completed and thus the file is removed.
The main issues with soft updates were that it didn't provide the performance benefits promised and didn't make maintenance necessarily easier. Instead I would say the ultimate issue with UFS is that it is an obsolete file system. It has never been very fast, it's not well optimized for modern use cases and when you really pull away the layer of fanboyism overlaying all of it, it's an ancient file system that's not capable of keeping up with today's performance demands.
At the end of the day, many BSD users use ZFS instead but that has never been available for OpenBSD. ZFS is not a horrible file system but it is not a general purpose file system. Using it for all general purpose situations is a lot like using a hydraulic ram to crack open peanuts.
Ultimately I hope that one or more of the BSDs decides to ditch the file system but I'm not holding my breath.
RE: Soft Updates being removed from OpenBSD - vishnu - 07-17-2023
I have to respectfully disagree, I use ZFS on my Linux computers, EXT4 doesn't have checksumming, when I was working at Northern Telecom (decades ago) I was one of the programmers who wrote their checksumming code, so, obviously, I'm a big fan of the concept.
RE: Soft Updates being removed from OpenBSD - johnnym - 07-17-2023
(07-17-2023, 08:37 PM)Raion Wrote: https://undeadly.org/cgi?action=article;sid=20230706044554As long as they don't ditch the sgi platform, I'm good!
It is cited as an impediment to properly implementing a VFS layer for OpenBSD.
Damn it, they already did...
RE: Soft Updates being removed from OpenBSD - vishnu - 07-17-2023
(07-17-2023, 11:13 PM)johnnym Wrote:(07-17-2023, 08:37 PM)Raion Wrote: https://undeadly.org/cgi?action=article;sid=20230706044554As long as they don't ditch the sgi platform, I'm good!
It is cited as an impediment to properly implementing a VFS layer for OpenBSD.
Damn it, they already did...
If I was to pick a close second to ZFS as a filesystem, XFS would be it...
RE: Soft Updates being removed from OpenBSD - Raion - 07-17-2023
My issues with checksumming is that it's only practical in COW filesystems which are far harder on memory and performance than a traditional file system. I'm also refusing to use ZFS anymore on new installations because ZFS-On-Linux is now the upstream and that was a stupid move. Congregating around the worst implementation of ZFS.
RE: Soft Updates being removed from OpenBSD - vishnu - 07-18-2023
(07-17-2023, 11:42 PM)Raion Wrote: My issues with checksumming is that it's only practical in COW filesystems which are far harder on memory and performance than a traditional file system. I'm also refusing to use ZFS anymore on new installations because ZFS-On-Linux is now the upstream and that was a stupid move. Congregating around the worst implementation of ZFS.
Linux has been COW for golly, at least since 1995! I've been using the latest open source release of ZFS for about a month now, no issues. But by the same token, I've never had any issues using ZFS....
RE: Soft Updates being removed from OpenBSD - Raion - 07-18-2023
Copy-on-Write is a very specific filesystem setup not suited for low memory infrastructure or plain, just really ugly and complex. It doesn't have some of the raw single disk performance of a traditional FS, and while checksums are fine in COW (because snapshots) if none of that is set up, e.g. for a user that just wants something that "works" or that lacks the memory and background resources, checksumming on a normal filesystem is just a plain annoyance.
COW systems are not my personal thing anymore. It's a meme, a pseudo-killer app but we didn't need it for nearly 40 years prior, so we probably don't need it now.
RE: Soft Updates being removed from OpenBSD - vishnu - 07-18-2023
(07-18-2023, 01:11 AM)Raion Wrote: Copy-on-Write is a very specific filesystem setup not suited for low memory infrastructure or plain, just really ugly and complex. It doesn't have some of the raw single disk performance of a traditional FS, and while checksums are fine in COW (because snapshots) if none of that is set up, e.g. for a user that just wants something that "works" or that lacks the memory and background resources, checksumming on a normal filesystem is just a plain annoyance.
COW systems are not my personal thing anymore. It's a meme, a pseudo-killer app but we didn't need it for nearly 40 years prior, so we probably don't need it now.
Again, I respectfully have to disagree. When I was at Northern Telecom, and we were marketing massively expensive private branch exchanges to, more or less, huge corporates, where the dropping of a call was absolutely not, well, we just couldn't allow it. We wrote test suites up that we ran for weeks, including ROM checksums (back in those days the systems didn't contain any RAM), and prior to that, when I worked at AT&T they were doing the same thing (though I wasn't programming the checksum code). But seriously, I don't know anything about BSD, are you saying they DON'T use copy on write? I mean, in my opinion, and other opinions will no doubt vary, what is the point in NOT using copy on write? It conserves memory it doesn't abuse it! Well, whatever, maybe I don't know what I'm talking about anymore, I haven't written any copy on write code since the 80s.
RE: Soft Updates being removed from OpenBSD - jwhat - 07-18-2023
Hi Raion & Vishnu,
I found this thread in FreeBSD forum: https://forums.freebsd.org/threads/gjournal-vs-soft-updates-journaling.54689/
Interesting as the NetBSD announcement kind had a dig at Kirk McKusick for not contributing to NetBSD, so problem of not having detailed knowledge of Soft Updates was as much a contributor to its "removal" as technical merit...
All this is well beyond my area of interest but at least now after all these years of using FreeBSD seeing the install question to enable or disable Soft Updates I understand at least what it was trying to do.
Cheers from Oz.
jwhat/John.
RE: Soft Updates being removed from OpenBSD - vishnu - 07-18-2023
So okay, first of all, I credit Kirk McKusick, and Eric Allman, and Bill Joy, and all those guys for everything they've all done for free software, but really, none of those guys are anywhere near the tip of the spear anymore. ZFS (and EXT4) are actively maintained by a whole bunch of guys who know far more about filesystems than I could ever hope to conceive. Perhaps as some members here know, I'm not a systems programmer, I'm an end user programmer, I've haven't written a single line of code that accesses hardware in the last 30 years. Code in Motif? Yes definitely. Xt? Yes that too. X11? Painful but sort of. Anything beyond that? I'm a basket case. So I give a great deal of thanks to the people who understand that kind of shit, because just thinking about it makes my brain melt...
