jl_678 wrote: ↑Sat Apr 13, 2019 3:58 am
This Moosefs thing looks very cool! I am considering doing the same thing to replace an aging NAS. Can you answer a couple of simple questions about it?
Sure, I'd love to!
jl_678 wrote: ↑Sat Apr 13, 2019 3:58 am
1. Give the setting of 2, how much usable capacity do you have?
I've got 12TB of disks "in service" (as technically my fourth node does not yet have a hard drive. I'll be purchasing one this weekend, bringing it up to 16TB of raw capacity).
Of that, I have about 11TiB of storage after overheads and conversion from TB to TiB etc. So 11TiB represents the "real" storage capacity, if that makes sense.
With a goal setting of 2, I could store roughly half of that, so 5.5TiB.
One of the really cool things about MooseFS, however, is that you can set goals on a folder-by-folder or file-by-file basis. This means you can do things like have the default copies setting be 2 (as in my cluster) but set special folders to have more or less copies. My ISO images folder is set to 1 copy as I don't care if I lose it, but my partner's music collection is set to 3 copies for added durability at the cost of a bit more disk usage.
jl_678 wrote: ↑Sat Apr 13, 2019 3:58 am
2. Do you need a full 4TB in the master server? It looks like it only holds Metadata and so might need significantly less.
The only reason the master server has a 4TB disk is because it also runs a chunkserver. The moosefs-master service and a moosefs-chunkserver service both live on it.
In a similar vein, the remaining servers run a moosefs-chunkserver service and *also* run a moosefs-metalogger service.
jl_678 wrote: ↑Sat Apr 13, 2019 3:58 am
3. I like the self healing idea; however, the master server seems to be a single point of failure. How hard would it be to recover that if it died?
Yes, the master is a single point of failure. However, the metaloggers running on the other nodes are designed to "follow" the master and each keeps a copy of the metadata set and tries to keep it up to date.
In the event of the master drastically failing, you can "promote" a metalogger to a master using its copy of the metadata (either by changing which IP the clients point to, or by moving the IP to the metalogger you are promoting).
In practice, recovering from a failed master is actually very easy using that method.
For what it's worth, maintenance in general is a breeze - when I add the fourth and final hard drive I'll be able to put it into service without taking down or interrupting MooseFS in any way. It's completely transparent most of the time.
jl_678 wrote: ↑Sat Apr 13, 2019 3:58 am
4. I am looking forward to your technical blog and so definitely share it
Thanks! I'm hoping to publish it in the next week or so but have been very busy.
Definitely let me know how you get on if you try it before my posts hit. If you want to talk more about Moose you can PM me on here, hit me on Twitter (@gnomethrower) or email me.