New compute cluster
As my drone ambitions grew, so did the
need to have a compute cluster that could handle the processing of
the images and data sets. Not to mention, it really comes in handy
for running bioinformatics at home. The genomes I work with range
in size from 750mb to 1.2gb so working with sequence data can be
fairly computationally intensive at times. The R710 handles the file system and runs a series of docker containers. This server directly interfaces with the storage array which currently has 10 3TB drives configured in Raid6 with a hot spare giving me 21TB of usable space. The built in Raid5 array is primarily used for bioinformatics and has 10TB of usable space.
The R910 affectionately named "Spoolbus" handles the more computationally intensive workloads and in its current config has 496GB of ram, 40 cores - 80 threads, and 500GB of swap configured in a Raid0 for the lowest possible latency. I know its flashing an error, I had a DIMM fail and I haven't gotten around to swapping it out.
All in all, configuring everything has been quite the learning experience. It was very interesting to get the containers set up and playing nicely on the network, and I've come to appreciate the value of containerizing certain workloads. Not to mention, it's very convenient to spin up a container to process a bioinformatics workload when that container already has all the dependencies satisfied.