The Operations Team at Delphix has recently published an Ansible role called delphix.package-caching-proxy. We are using this role internally to host binary packages (ex. RPMs, Python “pip” packages), as well as to locally cache external packages that our infrastructure and build process depends upon.
This role provides:
- A generic HTTP caching proxy
- An internal Docker registry server
- A private PyPI registry server
- An APT (Debian) package caching proxy
- And a YUM repo server
Why Is this Useful?
This sort of infrastructure can be useful in a variety of situations, for example:
- When your organization has remote offices/employees whose productivity would benefit from having fast, local access to large binaries like ISOs, OVAs, or OS packages.
- When your dev process depends on external dependencies from services that are susceptible to outages, ex. NPM or PyPI.
- When your dev process depends on third-party artifacts that are pinned to certain versions and you want a local copy of those pinned dependencies in case those specific versions become unavailable in the future.
--- - hosts: all roles: - delphix.package-caching-proxy
Underneath the Covers
This role works by deploying a front-end Nginx webserver to do HTTP caching, and also configures several Nginx server blocks (analogous to Apache vhosts) which delegate to Docker containers for the apps that run the Docker Registry, the PyPI server, etc.
Downloading, Source Code, and Additional Documentation/Examples
This role is hosted in Ansible Galaxy at https://galaxy.ansible.com/list#/roles/3008, and the source code is available on GitHub at: https://github.com/delphix/ansible-package-caching-proxy.
Additional documentation and examples are available in the README.md file in the GitHub repo, at: https://github.com/delphix/ansible-package-caching-proxy/blob/master/README.md
Shoutouts to some deserving folks:
- My former Development Infrastructure Engineering Team at VMware, who proved this idea out by implementing a similar set of caching proxy servers for our global remote offices in order to improve developer productivity.
- The folks who conceived the Snakes on a Plane Docker Global Hack Day project.