About

After I had completed my Linux, Networking, and A+ courses I wanted to use what I learned to help with my partners work. I worked with her to find out what her work flow was like and how technology could save time and expand her capabilities of analysis. They key issues found:

• Waiting days to download data from the external server
• Constantly having to manage data to make room on the company provided external hard drive
• One study would take up around 10 terabytes of data
• One component of analysis would take over 10 days of constant compute time. In practice this part of the workflow would take a month to complete. The analysis software would lock her computer up, so she would have to do it in chunks to get other work done.
• Other components of analysis would take hours to complete

With all this in mind I set out to provide a tech solution. I gave myself a month to procure the parts and configure the setup.

Below is a list of features I created to solve all these issues:

• A server that runs Ubuntu Server for low overhead resource usage and compabitble with her anaylsis software.
• She can run the CPU analysis software on the server freeing up her laptop for other work.
• 48 terabyte network share with a 512 gigabyte cache drive and a 2.5 gbps network speed.
• I originally used the SMB protocol for the network drive but was limited to around 32 mbps speeds. Since permissions in SMB are client side there is a lot of CPU overhead which became the bottleneck. I switched to the NFS service which would get up to 1800 mbps of tranfer speeds. Once I upgraded the CPU the SMB transfer speeds went up to 2500mbps
• I use a cron job to rsync the network share with the external server every day. So when a new study drops into the external server it immediatley downloads to our local server.
• With all this space she can create multiple copies of projects and download new projects without having to worry or spend time on managing data.
• The component of anaylsis that would take 10 days of compute time was a program that would take splices of images and render them in a 3D format with data bits like flow rates attached to each pixel. I researched the program online and found it could be run on a GPU instead of a CPU. I got a Tesla K80 which is a 6 year old server grade graphics card. Using this card for processing took the time down to analyze each subject from 2 hours to 3 minutes. Running a full study would take 6 hours instead of 10 days. This not only saved her time but gave her the capability to try different options and re-run the analysis multiple times with different options; expanding her capabilities of anaylsis.
• There are around 100 subjects per study. To run the subjects all together in sequence requries a script with around 100 commands. Each subject file name only differs with a different identyfying number. I created a For Sed loop in Linux to autogenerate the script to run all these subjects in sequence
• The router hosts its own VPN service so she can access our home network resources while at work.
• I use the raspberry pi 4 for local DNS resolution. So we can refer to the network resources with friendly names. The Pi is also what is being used to host this website.

Below are images of each component with a list of parts, configuration, and functions.

Server

The server has 3 16 terabyte hard drives in RAID 0 with a 512GB cache drive attached to it. The drives are IronWolf Pro with a Mean Time Between Failure of 6 years given they are always running. I am not worried about drive failure in the near future, but a future upgrade will be to add two more drives and run the hard drives as a RaidZ2 for redundancy. The cache drive is a Western Digital 500GB WD Red SN700 NVMe. As a cache drive the most important factor is write endurance, this drive is rated for 5,100 Terabytes Written. I used the Linux tool Bcache to attach this drive as a cache drive to the Raid0 Hard Drives. This improved write speeds from 80Mb/s up to 250 Mb/s.

Services that run on this server:

• I use a cron script to update my dynamic IP to cloudflare to update DNS resolution for my website and VPN
• I also use a cron script to rsync this home server with my partner's work server, that way she always has up to date information
• I run OpenVPN for my VPN service. OpenVPN is great for client compatibility, it works on phones, computers, and many other devices. I use certificates for authentication.
• I use SMB and NFS protocols for the network share. My partner uses Linux and NFS works better with permissions on Linux. SMB is for my windows machine and our phones.
• I uploaded all our family videos and share them with my family via the OpenVPN and a streaming service called Jellyfin. Now my family can stream home movies like its Netflix
• I installed a Remote Desktop Protocol. My partner's work VPN is HTML GUI based. So to connect to it I tried running her VPN through the Remote Desktop Protocol. This did not work, I think for security reasons they don't want their vpn to be installed and always running.

I used a 12th gen i5 for the processor. I orignally had an I3 with no video output. When I installed the Tesla K80 graphics card the server would not run its services. With the i5 I could read the output and diagnose the issues (it just needed me to select the kernel) It is a little slower than her laptop which is a 13th gen i7. I was limited to the 12th gen i5 because I used a mATX case and the CPU fan could only handle the TDP of the 12th gen i5. The benefit to using this CPU is that she can run the analysis on it and still use her laptop for other work.

Tesla K80

I got a Tesla K80 which is a 6 year old server grade graphics card. It has the cuda cores equivalent to a NVIDIA 3070TI which is a $500 graphics card. Since the Tesla K80 has no video output and Bitcoin Mining crashed the price for this card fell drastically to $60. It has no internal cooling mechanism, consumes 300 watts of power, and does not fit inside the mATX case. I got a 3d printed fan insert and GPU mount off eBay and found a cheap power supply to use for the card. When the card needs to be used I plug in a riser cable to attach it to the server. We do not keep it like this permanently for safety as well as the fans being very loud.

About

Router

I previously used this Micro Firwall Mini Pc as the router. It uses pfSense as the OS which has a lot of documentation,features, and is often used in small to medium sized companies.

It had hosted the VPN with the OpenVPN service. OpenVPN was used for ease of use and great compatibility. It can run on all OSs and phones.

The router provides DynamicDNS updates to CloudFlare so my kehoe.pro domain points to my changing Dynamic IP. It also provides Network Address Translation to route website queries to the Raspberry Pi.

The SSD on this router had died. It took weeks for the vendor to send a replacement. In that time I got the dynamic DNS updater and OpenVPN service to run on the server. Now I no longer need this device. I am going install linux and set up services for it and give it to a friend.

About

Raspberry Pi 4

The Raspberry Pi provides local DNS resolution so we can refer to the network resources with friendly url names. It is also what is being used to host this website.