New to server operation

So this post is going to be part rant, part journal, part update.

First, ive decided with certainty that FreeBSD is the way to go at this point. Lots of hardening options, better and more precise control over what the OS and hardware are doing at any given moment, and better compatibility with more modern software. A number of challenges have been beating me in the head the last several days ive been researching this.

What ive learned may very well be useful to any other folks looking to throw an old Mac Pro tower or maybe even an OG Motorola mac up as a relay, as the hardware despite its age, is still very well suited to what a relay needs. Well, maybe the PPC and OG macs wont line up so well here but thats irrelevant. Anyway, firstly, Apple and most of the fanbois were full of shit when they said that the Firmware upgrade from 1.1 to 2.1 would fix the 32 bit EFI. Nope. It does not. It applies a sort of hacky patch that STILL loads a 32 bit kernel on boot, then straps into a 64 bit OS. Apple isnt very open about what and how they do what they do, but heres what ive learned so far. In short, the bootrom on those old macs are basically hardcoded to shit the bed if an x64 kernel tries to load. It wont even show up on the boot menu. So the obvious solution (at least to me) was to just replace the apple boot manager with grub, or something similar.

Nope.
Dont do that. During the boot process, the mac loads bootrom first, then looks for a 32bit EFI file bootstrap from there. But where is this firmware and bootloader stored? Dunno, couldnt figure it out. Still cant. Some sources say its loaded in Mac OS, others claim its loaded into a separate chip somewhere else on the board, and flashing it to include EFI64 bit, or UEFI is unrealistic, dangerous, and is probably impossible, and im reasonably certain attempting to do so will brick the board completely. Both answers cant be correct, yet both have merit and ground to stand on. Personally, (opinion this time, not fact) im about 85% sure the bootrom is stored on a chip somewhere on the board that isnt accessible to plebs like us, only to apple engineers, and can only be flashed or modified by an Apple signed Firmware update program, and im about 90% sure the bootloader just loads and points to an EFI file loaded into Mac OS, no Mac OS, no EFI file. Not completely sure either way, but i am sure that attempting to mod the bootrom will destroy the machine.

There was one solution from a very enterprising gentlemen from around 2011 or so that went the extra mile but writing a 32bit EFI for booting, but it isnt persistent. Upgrading to a more recent version of the OS, or doing updates has a high potential of rolling back the mods to the kernel, and on reboot will attempt to load a 64bit EFI or UEFI image, then promptly shits the bed. No good.

Ive stated before, and ill state again, im not a fan of hacky half assed work arounds. A native solution is always the best answer when possible. And its possible here. A hacky work around to boot FreeBSD (at least to me) defeats the purpose of using BSD in the first place. Its rock stable, and if its not, then its operator Error.

So i started looking hard into side loading a bootloader that will just run over top of the default apple bootloader without erasing or replacing it. One solution is to use a USB stick loaded with rEFIt or rEFInd. Hold option/alt on boot, select bootloader USB, from USB loaded bootloader, load OS of choice. Once OS is installed, 2 options to make it persistent. Install a copy of that USB sticks boot loader onto a boot only partition of the primary harddrive. Mac boots rEFIt partition first, then rEFIt bootstraps BSD. Maybe, ill give this option a solid 5/7.

The next, more stable option, is wipe all disks on the array. Start fresh. Install fresh copy of OS X of your choice, SL for me because i have a SL stick lying around for the bad days. Yes, you can use a usb stick to boot a DvD iso of OS X. Use unetbootin, and you can get a legitimate copy of OS X SL on MR (https://www.macintoshrepository.org/)
If your a Mac guy/gal, that resource is fucking amazing for vintage macs that predate App Store being the default for software distribution. They even have a copy of GTA vice city that was written for Mac OS (i fail to recall version atm, look it up if your curious, really cool)

Anyway. SL freshly installed, only installed on 1 drive, no hard or soft raid. Then try to install rEFInd, which has an mpkg GUI installer, theres also an option for CLI install if thats your preference. More info on this boot manager can be found at http://refit.sourceforge.net/
Technically, this version is deprecated, but it was the go to for the 1.1 and 2.1 MP, as well as many of the early EFI MBPs and Mac Mini. The newest version is rEFInd and can be found at The rEFInd Boot Manager
According to what ive learned so far, there are 2 ways to make this work. Install rEFInd/t onto a USB stick, and just leave it plugged in while booting. Non optimal.
The second, more stable option i eluded to above is to load this bootloader in to OS X SL, and install BSD onto a second disk in the array. But dual booting is also non optimal, but its as close a native solution ive cooked up yet.

My primary goal, was and still is ultimately to run BSD on bare metal. Virtualization may very be more than practical for most people in most situations. But im broke, and most of my shit is a bundled clusterfuck of jury rigged shit my neighbors and work throw out. I very rarely have the funds to purchase brand new hardware, and even when i do its never more than 75 per module. So that gives you an idea for what ive been working with over the years, native and optimal solutions to non native and non optimal hardware most of the time.

Something to keep in mind, regardless of what method is chosen, NVRAM doesnt clear on every reboot, so sometimes even after a successful install, it wont show, leading the user to believe the install is busted. Sometimes not, reboot your Mac at least twice, i suggest 3 times to clear any old bootstrap instructions from bootrom and NVRAM. You can also clear VRAM and multireboot by holding the keys Option-Command-P-R on a cold boot, and continue holding them until you hear the Mac boot and reboot 3 times, and make the chime. On the third reboot, everything possible left over is definitely cleared from memory.

Im going to attempt another bare metal install tonight and will update tomorrow or the day after with what ive got so far if anything. Since the Tor community is primarily full of non mac bros, i know most of you will find this post and its predecessors irrelevant and even hilarious. But for you other 6 Darwin bros running relays, i salute you, and hope that some of these discussions and posts will spark some curiosity in using Macintosh computers for relays in the future. Or barring that, if your in the same boat where the best machine to do this on is a 17 year old Cheesegrader in the closet, its possible. Might not be easy all the time, but its definitely possible, and certainly cheaper to get an eBay 5,1 Mac Pro (which mercifully has a 64 bit bootloader thank fuck) in good condition for a few hundred bucks.

Why would you want to? Server grade Xeon hardware and registered memory makes these things absolutely fantastic for server operations if your just getting started, and building a Xeon work station from scratch is not only difficult, but expensive. These towers come packed with workstation grade hardware designed to last, and if you strip Mac OS off of it, and run BSD or linux on a 5,1, it will serve you and the tor community well for many years to come as a relay or bridge.

It just now occurred to me I didn’t respond to part of your message, my bad yo.
I considered a VPS, but cloud hosting a service isn’t my thing, and its expensive anyway.
Yeah, running the relay from a VM would be fine, but im against the idea of running it in a VM either way. Im kind of against virtualization in general honestly. Im one of those native purist types I guess. I like having direct control over the hardware and such through the OS. Running it in VM wouldn’t solve the problem anyway I dont think, as the machine and hardware are definitely more than capable of handing 30MB/s data streams both ways with little to no issue. Even with natd, dhcpd, and Tor all running at the same time barely took more than 20% hardware resource, and didn’t even break 2gb of active memory.
I even ran a 200GB file transfer over nat and wire to test stability, 35% CPU usage is where it topped out at while transferring 125MB/s one direction, this is even with Tshark running a ring buffer 512MB capture in the background, DHCPd, and Tor eating 4MB up and down of bandwidth.

The problem was the kernel panic during the DDoS, running it in virtualization would solve part of the problem, but im reasonably confident the host OS (SL in this case) is the problem. Running BSD in a Box would just exasperate the issue, BSD could handle the DOS, but SL running under it couldn’t.
Native Bare Metal is the best solution here.
And yes, Snort does have some IPS capabilities hooked in, Im putting together a modified version of the community rulesets for it and will try to build in some DOS mitigation and such too, after I get BSD on bare metal. I dont give a fuck what that Cheesegrader says or wants, its going to get BSD whether it likes it or not. SL is just too old, and not intended to be used in the capacity im using it for.
Linux is the obvious choice, but in recent years, OS flavors like ubuntu have moved away from configurability to more baked in bloat. And most other newer Distros are unstable and wonky anyway. Fedora is a great example of this, its a great OS, probably a great server OS too, but its too modern and bloated, and prone to getting smashed all to shit after an update. That’s No bueno per nada (slang for no worky for shit)

Im leaning BSD because its rock fucking stable, and small. For reference, a modern Ubuntu OS image is around 7-9GB for a more or less base install.
FreeBSD’s basic memory stick installer (usb image) is just shy of 1BG, and the full DVD disc image is just over 4GB. Much much much less unstable and useless code. Less attack surface for a bad agent.

As for the Tor dev team git-lab? Not a bad idea, I probably should hit them up and see if they have a suggestion. But for the moment, im still working on getting BSD to install bare metal. Must overcome the 32bit EFI thing first however.

Hey there. How’d the BSD install go? I have a bit of possibly relevant info about boot rom and EFI which might be irrelevant but also might be good food for thought. Check out this article by Howard Oakley at The Eclectic Light Company:

He’s a wealth of information on Macs. Check his tags on firmware etc. Long story short, your intuition about modding the bootrom is probably right—especially if your machine’s bootrom version has been upgraded to 144.000.000. Of course, your mileage may vary (as they say).
Anyhow, good luck with BSD. Sounds like the optimal solution. :blush:

1 Like

Well shit, nice to have confirmation I suppose as that does answer a few questions I’ve been trying to answer.

No I havent managed to get FreeBSD onto the Mac yet unfortunately. Though im still actively working the issue daily with little to no progress at the moment. im in the middle of moving apartments and ISPs on top of everything, and so that’s slowed down things quite a bit.

Also, I decided to erase and install FreeBSD 13.0 onto my fairly newish MacBook Pro, just so I would have a working and running copy to do development on. Im going to attempt to modify an ISO or .img to be 32bit EFI compliant with an x86_64 Kernel on boot, and the info in this article will help a lot with that. I did manage however to get the damn thing to at least recognize an EFI bootable stick in the machine with rEFInd on it, granted booting to it didn’t actually do anything as the only available bootable image was itself and the OS X on the raid array but progress is progress.

Im thinking I might relocate one of the external USB ports to internal, and just tape it to the side of the power supply or something with a 1g thumb drive loaded with rEFInd on it, machine boots and loads apples bootrom, then loads rEFInd, and it loads the x86 FreeBSD kernel and just specify a boot order and shorter timer to keep bootstrapping time down. But I think its worth trying anyway.

If that doesn’t pan, I’ve got a spare PowerMac G4 from 2002 and another DDR2 machine with an old processor in it, I was thinking of installing bare metal, moding the boot image on it, remove and relocate the drive to the Mac Pro and see if it’ll boot. More to come.