About

After I had completed my Linux, Networking, and A+ courses I wanted to use what I learned to help with my partners work. I worked with her to find out what her work flow was like and how technology could save time and expand her capabilities of analysis. The key issues found:

• Waiting days to download data from the external server
• Constantly having to manage data to make room on the company provided external hard drive
• One study would take up around 10 terabytes of data
• One component of analysis would take over 10 days of constant compute time. In practice this part of the workflow would take a month to complete. The analysis software would lock her computer up, so she would have to do it in chunks to get other work done.
• Other components of analysis would take hours to complete

With all this in mind I set out to provide a tech solution. I gave myself a month to procure the parts and configure the setup.

Below is a list of features I created to solve all these issues:

• A server that runs Ubuntu Server for low overhead resource usage and compabitble with her anaylsis software.
• She can run the CPU analysis software on the server freeing up her laptop for other work.
• 48 terabyte network share with a 512 gigabyte cache drive and a 2.5 gbps network speed.
• I originally used the SMB protocol for the network drive but was limited to around 32 mbps speeds. Since permissions in SMB are client side there is a lot of CPU overhead which became the bottleneck. I switched to the NFS service which would get up to 1600 mbps of tranfer speeds
• I use a cron job to rsync the network share with the external server every day. So when a new study drops into the external server it immediatley downloads to our local server.
• With all this space she can create multiple copies of projects and download new projects without having to worry or spend time on managing data.
• The component of anaylsis that would take 10 days of compute time was a program that would take splices of images and render them in a 3D format with data bits like flow rates attached to each pixel. I researched the program online and found it could be run on a GPU instead of a CPU. I got a Tesla K80 which is a 6 year old server grade graphics card. Using this card for processing took the time down to analyze each subject from 2 hours to 3 minutes. Running a full study would take 6 hours instead of 10 days. This not only saved her time but gave her the capability to try different options and re-run the analysis multiple times; expanding her capabilities of anaylsis.
• There are around 100 subjects per study. To run the subjects all together in sequence requries a script with around 100 commands. Each subject file name only differs with a different identyfying number. I created a For Sed loop in Linux to autogenerate the script to run all these subjects in sequence
• The router hosts its own VPN service so she can access our home network resources while at work.
• I use the raspberry pi 4 for local DNS resolution. So we can refer to the network resources with friendly names. The Pi is also what is being used to host this website.

Below are images of each component with a list of parts, configuration, and functions.

Server

The server has 3 16 terabyte hard drives in RAID 0 with a 512GB cache drive attached to it. The drives are IronWolf Pro with a Mean Time Between Failure of 6 years given they are always running. I am not worried about drive failure in the near future, but a future upgrade will be to add two more drives and run the hard drives as a RaidZ2 for redundancy. The cache drive is a Western Digital 500GB WD Red SN700 NVMe. As a cache drive the most important factor is write endurance, this drive is rated for 5,100 Terabytes Written. I used the Linux tool Bcache to attach this drive as a cache drive to the Raid0 Hard Drives. This improved write speeds from 80Mb/s up to 200 Mb/s.

I used a 12th gen i5 for the processor. It is a little slower than her laptop which is a 13th gen i7. I was limited to the 12th gen i5 because I used a mATX case and the CPU fan could only handle the TDP of the 12th gen i5. The benefit to using this CPU is that she can run the analysis on it and still use her laptop for other work.

Tesla K80

I got a Tesla K80 which is a 6 year old server grade graphics card. It has the cuda cores equivalent to a NVIDIA 3070ti which is a $500 graphics card. Since the Tesla K80 has no video output and Bitcoin Mining crashed the price for this card fell drastically to $60. It has no internal cooling mechanism, consumes 300 watts of power, and does not fit inside the mATX case. I got a 3d printed fan insert and GPU mount off eBay and found a cheap power supply to use for the card. When the card needs to be used I plug in a riser cable to attach it to the server. We do not keep it like this permanently for safety as well as the fans being very loud.

About

Router

I used this Micro Firwall Mini Pc as the router. It uses pfSense as the OS which has a lot of documentation,features, and is often used in small to medium sized companies.

It hosts the VPN with the OpenVPN service. OpenVPN was used for ease of use and great compatibility. It can run on all OSs and phones.

The router provides DynamicDNS updates to CloudFlare so my kehoe.pro domain points to my changing Dynamic IP. It also provides Network Address Translation to route website queries to the device hosting this website.

• I use CloudFlare's Proxy service to protect my own actual Public IP. The IP that you are seeing for this website is a CloudFlare Proxy server

About

Raspberry Pi 4

The Raspberry Pi provides local DNS resolution so we can refer to the network resources with friendly url names. It is also what is being used to host this website.