Home avatar

Just a tech guy that likes playing on the internet

Optimizing Your CDN Cache With Cloudflare and Nginx

Optimizing your CDN cache with Cloudflare and Nginx

Recently I have been working on a personal project to help support the Linux distro I use as a daily driver, Manjaro. I decided to set up a package mirror and allow users from the community to connect to my server and pull updates or new packages for install. Doing some research I found that when updates get pushed this can result in large spikes in traffic and I would have to be prepared to serve around 1TB of traffic a week. Because I am doing this on a budget I was trying to work out the most cost-efficient way to support all this traffic when I don’t run a home lab. So I was looking for a cloud server and a way to help reduce or at least manage traffic. Due to the way the list of mirror servers are published, it’s not something I can opt-out of quickly if I hit a data limit or ran into some other cost barrier.

Linux Mirrors


I wanted to put out a post explaining what a Linux mirror is and how I set one up. A Linux mirror is a web server that holds a "mirror" copy of the distro's packages. So if a Linux machine needs to pull updates or new packages they can contact a mirror server or the main server to pull these binaries. This helps distribute the load so the distro can support a larger member base without been the sole source of all their packages. Mirrors are normally community-run or sponsored, a lot of educational institutions offer free mirror hosting to help out the Linux communities. Also, some ISP's will offer to host a mirror as they also benefit by keeping the data usage inside their network.

The distro I choose to use, Manjaro was set up in this very classic way. But unfortunately out of the two mirrors set up inside Australia, both were having issues. One was completely non-responsive and the other was very patchy. I had been looking for an opportunity to give back to the Linux community as I enjoy all the free products they offer and use them every day. The usual way to give back to the Linux community is to work on the tools and programs every one makes use of. You can offer development skills, problem-solving or even just writing documentation it all helps the community grow and support its self.