Success getting files off SGI XFS-formatted disks using "modern" hardware
#1
Success getting files off SGI XFS-formatted disks using "modern" hardware
I've been frustrated for years that there seemed to be no easy way to get data directly off of XFS formatted SGI disks using "modern" hardware and operating systems. Recently, I decided to revisit this problem and, as it turns out, it's not that hard.

Because I don't have infinite space, I wanted something compact -- specifically a laptop solution. What you need:
  • An Adaptec 1460 PCMCIA Fast SCSI adapter (~$40 on ebay) with the proprietary cable, or
  • An Adaptec 1480 Cardbus UltraSCSI adapter (~$125 on ebay) in order to use newer Ultra-SCSI-only drives (e.g. Fujitsu MA_ drives)
  • A laptop with a PCMCIA/cardbus slot (I use a Thinkpad T500, which has a great mix of "modern" power/virtualization support and useful IO)
  • I tested this with Linux Mint 20.04 on the T500. I assume many other host OS's would work.
  • An external disk housing to supply molex power and provide a Centronics -> IDC50 cable (or a HDPB50 -> IDC50 if you're using the 1480)
  • A 50 pin to SCA adapter if you're using SCA drives
  • Virtualbox (or your hypervisor of choice, unfortunately new versions of VMWare Player don't support the Core2Duo in my T500 anymore)
  • An old linux distro, I used Ubuntu 4.10 because 1) it's the oldest Ubuntu distro so I assumed it has the best backwards compatibility, 2) it's readily available with a working package manager (with small tweaks to the /etc/apt/sources.list file, replacing "archive" with "old-releases" in each url), and 3) I'm most familiar with Ubuntu / Debian. 
  • Edit: Ubuntu 8.04 is also confirmed working as well, so long as you mount the device to the IDE bus. 

You can still download Ubuntu install media from the original sources. The x86 install CD includes everything you need for basic xfs volume repairing/mounting. If you want to create/manipulate xfsdump files, it's best to use 8.04.
  • 8.04 http://old-releases.ubuntu.com/releases/hardy/ (This is the first version to have xfsdump available through apt-get.) This is the first Ubuntu LTS release, so it has better driver support and much more recent package updates. It also supports SMP out of the box, if that's important. It was released in Oct 2008.
  • 4.10 https://old-releases.ubuntu.com/releases/warty/ (This is the oldest Ubuntu release, from Oct 2004, so it may have slightly better compatibility)

Once you have this setup, it's actually fairly straightforward. First install Virtualbox and get your old Linux distro installed. Make sure you use an IDE controller type for your hard disk images. 4.10 doesn't seem to support SATA at all, and I had better luck with the loopback device mounted to IDE in 8.04 as well. xfsprogs should be installed by default, but might as well run a sudo apt-get update and sudo apt-get upgrade (you will need to update the sources.list file, per below).

Hardy Herron (8.04) is the first Ubuntu release to include xfsdump through apt-get. Warty Warthog also has xfsdump available, so suggest just using it. The top answer here (https://superuser.com/questions/339537/w...u-versions) walks through updating your apt sources.list file to get apt-get working.
  • The tl;dr is to edit the /etc/apt/sources.list file (will need to use sudo) and find/replace "archive" and "security" with "old-releases" 
  • For 4.10 you'll also need to uncomment the lines with the urls. That's it, super easy.

Then image your drive:
  1. Connect your SCSI drive and power it on,
  2. Insert the Adaptec PCMCIA card,
  3. On the host OS, type "dmesg | tail -20" to see the hardware ID of the SCSI drive (e.g. /dev/sda1 or whatever ... if there is none troubleshoot)
  4. "sudo fdisk -l" should also give you the disks and partitions -- although your new linux distro won't actually be able to mount the disk properly

Armed with that info, use dd to create an image file, loopback device pointing at that image file, and virtualdisk VMDK file pointing at that loopback device (you could also skip the image file/loopback device and just create a VMDK pointing at the disk itself, which allows direct manipulation of the disk in the guest OS, but that seems risky):
  1. Enter the command "sudo dd if=/dev/yourscsidevice of=filename.dd bs=512 status=progress" (Note this maxes out at 1MB/s for me on a 1460 adapter, so be patient)
  2. Mount this file as a loopback device using "sudo losetup /dev/loop0 filename.dd"
  3. Create a vmdk pointing to this loopback device using "sudo VBoxManage internalcommands createrawvmdk -filename ./loop0.vmdk -rawdisk /dev/loop0"

Now add this VMDK to your virtual machine and boot it up. Note that in Linux Mint I found I had to run virtualbox using sudo, even adding my user to the "disk" group didn't allow access otherwise.

For some reason, I wasn't able to mount the disks immediately in the guest OS, I had to repair the images first (note these are all tested images that are known to work in actual SGI hardware)
  1. Use "sudo fdisk -l" to view the disks/partitions. In my case, the XFS partition was something like /dev/hdb1 (4.10) and /dev/sdb1 (8.04)
  2. use "sudo xfs_repair -L /dev/hdb1" to dump the pending log files
  3. use "sudo xfs_check /dev/hdb1" to verify the image, this should be very quick for an image/loopback device
  4. mount the image using "sudo mount -t xfs /dev/hdb1 ./yourmountpoint"
  5. ????
  6. Profit

Note: I have not/not tested whether these repaired images are backwards compatible with IRIX. The primary intent with this approach is to be able to pull the files themselves without relying on vintage hardware. If you want to duplicate a disk, however, I assume the dd images would work if written back to an appropriate disk.

I've confirmed this to work perfectly with a host OS of Linux Mint 20.04 and a guest OS of Ubuntu Warty Warthog (4.10) and Hardy Herron (8.04), using images both from an Indy (6.5.22) and Octane (6.5.30). I would like to test with 6.2 (I anticipate there may be problems as it uses an earlier xfs version -- which is one reason I started with 4.10), but I don't have any 6.2 installs handy. If anyone can host an installed 6.2 disk image, let me know.



Note this also works perfectly with SCSI2SD images (tested), which you can also dd directly from the SD card (using the seek and count options to copy just the appropriate sectors). This is way faster than using the PCMCIA SCSI card.

There are many ways to get the files back off of the host OS. I just used a large USB drive formatted via fat32, which I passed through to the guest OS to drag/drop files from the mounted SGI volume, then I detached the USB device through virtualbox and was able to access all of the files back on the host OS. Ext3 probably would be better (to retain more file metadata and allow larger files sizes), but I stuck with fat32 for my testing because I was sure it would be supported. 

Edit: Got everything working on 8.04 Hardy Herron as well! Moved the loopback device over to the IDE bus and was able to manipulate it as /dev/sdb1 without issue. Note that after further experimentation, it does seem that 4.10 more reliably accesses the loopback drive without issue, so that would be my go-to OS if you don't need to use xfsdump/xfsrestore.

Winter solstice edit: After frustrating attempts to compile xfsdump (and dependencies, most notably uuid-dev) from source in 4.10, I realized complied versions *are* in fact still available from ubuntu, I just hadn't fully updated my sources.list file properly (doh). With that sorted, I've installed and tested xfsdump/xfsrestore in 4.10 against a 6.5.22 image, which seems to work well enough. So I see no reason to go with 8.04, which seems to have more trouble consistently mounting the drive/loopback device passed through from virtualbox. Tweaked language above to correct my mistakes/avoid confusion.
(This post was last modified: 12-21-2021, 04:22 PM by callahan.)
callahan
Octane

Trade Count: (0)
Posts: 147
Threads: 20
Joined: Dec 2018
Location: East Coast, USA
Find Reply
12-20-2021, 04:49 AM
#2
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
Oh yeah, super old linux distros had XFS v2 at least, the problem I've found is that many distros have PURGED their online archives of old distros. I still have some old Redhat CDs from like 1998 so if you found the right distro that supported it out of the box, great! I'll remember that.

But I found that most of the time XFS support wasn't installed by default and wasn't on the original media and then you go to apt-get or yum it and....file not found.

Yes, XFS v1 was used until (or was it after) Irix 6.5.5.

Also same problem with like SSH, after you got the files, unless you're saving them onto another drive that's formatted with something you can read, you cannot even SSH or SCP the files over because there have been too many changes between the old version and modern version so, connection refused.

I've got both the Adaptec PCMCIA & CardBus adapters (both versions) and all that...software is the biggest problem and I still have zero idea why they removed XFS v1 & v2..after all Linux is about data recovery and digital archeology now...why remove them? Once I have a fully-working SGI up and can use FTP, I can get the files.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
12-20-2021, 07:25 AM
#3
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
Just one tip: when you use dd with the cooked device and bs=512, it's going to be very slow.
The correct way to do this is with the raw device and a larger blocking size, such as bs=16k, depending on how much buffer capacity your adapters can manage. SCSI disks usually have large buffers and can do transfers of 64k and larger without any issue, but the same cannot necessarily be said for dodgy adapters.

I'm surprised that fdisk recognized the MIPS partition map.

Personaliris O2 Indigo2 R10000/IMPACT Indigo2 R10000/IMPACT Indigo2 Indy   (past: 4D70GT)
robespierre
refector peritus

Trade Count: (0)
Posts: 640
Threads: 3
Joined: Nov 2020
Location: Massholium
Find Reply
12-20-2021, 02:16 PM
#4
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
Modern Linux fdisk definitely recognizes (as well as Linux kernel) SGI disk labels! I tried this very recently when experimenting with HDD boot media creation. However it does NOT HONOR partition numbering correctly. I will display different functional numbers for your partitions than FX, which is irritating, but you can tell which you want but looking at that partition type (swap, volhdr, cos, etc).

There is also no real equivalent to the FX expert menu for manipulating the SGI disk label as there is in FX (fdisk runs very short on this). Also default fdisk SGI disk label creation isn’t the default we know on Irix. It’s something else.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
12-20-2021, 02:54 PM
#5
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
Glad to hear about your success.

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
12-20-2021, 03:29 PM
#6
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
@weblacky -- Realizing folks often have strong opinions about Linux / BSD distros, one really nice thing about Ubuntu is that every release is still available from the original source. Also, xfsprogs is installed by default. You do have to add xfsdump, but since the apt repository is also still supported, that is super easy. To transfer data I just passed through a fat32-formatted USB drive to the guest and dropped files there. This seemed to work.

@robespierre -- The PCMCIA card seems to be the hard limit on bandwidth. I tried block sizes up to 4MB with no increase in performance. Larger blocks probably would work fine, but I like to stick with 512 byte block sizes 1) because I'm never fully sure how some of the archaic hardware will handle big blocks and 2) I regularly use SCIS2SD devices, which have a default block size of 512 bytes, so I don't have to do mental math to target the right sector.

Also -- I was able to get Hardy Herron working just fine, including testing xfsrestore on a xfsdump produced on a 6.5.22 Indy. Will update the post above with links and a few more details for posterity.
(This post was last modified: 12-20-2021, 04:26 PM by callahan.)
callahan
Octane

Trade Count: (0)
Posts: 147
Threads: 20
Joined: Dec 2018
Location: East Coast, USA
Find Reply
12-20-2021, 03:53 PM
#7
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
There's no BSD OS with functional XFS drivers, so you won't hear a peep from me. This is fine and I appreciate you using what is possible and available to educate others. That's the spirit.

I'm the system admin of this site. Private security technician, licensed locksmith, hack of a c developer and vintage computer enthusiast. 

https://contrib.irixnet.org/raion/ -- contributions and pieces that I'm working on currently. 

https://codeberg.org/SolusRaion -- Code repos I control

Technical problems should be sent my way.
Raion
Chief IRIX Officer

Trade Count: (9)
Posts: 4,240
Threads: 533
Joined: Nov 2017
Location: Eastern Virginia
Website Find Reply
12-20-2021, 04:25 PM
#8
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
I started as a Redhat user in 1996, had to abandon it for Debian in 2004, not the biggest Ubuntu fan due to gui choices, use mint now when I really need a Linux desktop.

However I didn’t know Debian has kept stuff that old on the official repositories. So that is real news to me! I install straight Debian distro for servers due to the base being less hassle and works.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
12-20-2021, 09:42 PM
#9
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
(12-20-2021, 09:42 PM)weblacky Wrote:  I started as a Redhat user in 1996, had to abandon it for Debian in 2004, not the biggest Ubuntu fan due to gui choices, use mint now when I really need a Linux desktop. 

However I didn’t know Debian has kept stuff that old on the official repositories.  So that is real news to me!  I install straight Debian distro for servers due to the base being less hassle and works.

Yes, the "modern" ubuntu UI drives me crazy. I also use Mint when I'm using Linux. It mostly just works and has a very decent UI/window manager.
callahan
Octane

Trade Count: (0)
Posts: 147
Threads: 20
Joined: Dec 2018
Location: East Coast, USA
Find Reply
12-21-2021, 04:32 PM
#10
RE: Success getting files off SGI XFS-formatted disks using "modern" hardware
Callahan, along with your SCSI2SD config archive maybe you’d consider creating a console-only, minimalist, 4.10 Linux install with stable XFS image mounting support but the newest SSH you can find for virtual box or whichever VM system, BZIP2 compressing it, and publishing it on your archive for easy download/use by others?

While it’s not terribly hard to recreate the steps you’ve published, why not smooth it a little bit and just share a like 200-300MiB image for others to just download and use.

That way (just like your SCSI2SD config files) you can dehassle others. If you have the apt-get sources pointed correctly for the time being, people can always add the X server and window manager themselves if they need it on their own copy.

That way even if a user wants to do this, they can just download your prior work instead of fighting for 45 mins to an hour to try to put it together.
weblacky
I play an SGI Doctor, on daytime TV.

Trade Count: (10)
Posts: 1,716
Threads: 88
Joined: Jan 2019
Location: Seattle, WA
Find Reply
12-21-2021, 04:47 PM


Forum Jump:


Users browsing this thread: 1 Guest(s)