Automating My Development Environment

A recent conversation with a co-worker reminded me about my intention to share how and why I’ve automated the setup and configuration of my development environment. This is a quick write up on just that.

The Why

To set the scene of when I first had this idea, my work machine was a MacBook and my PC was dual booting Ubuntu and Windows (where I’d use Ubuntu via WSL). My goal was to have the same tools and config across all three without having to copy changes between them.

Resiliency was another key factor. I quickly realised that this approach would mean that I could switch machines with minimal effort. In fact, my compute can be replaced and I can be back up and running in less than an hour.

Finding a Solution

You may have noticed that I actually had two problems that I wanted to solve. The first was having the same tools available across my machines, and the second was having the same config for those tools across my machines. Let’s go over how I solved both of these problems (in reverse).

Problem 1 - Sharing Config Between Machines

When I was looking for a solution to the config problem, I discovered the concept of a dotfiles Git repository. Dotfiles refer to the config files found on *nix operating systems - the name is derived from the fact that these files and directories often start with a dot (e.g. ~/.bashrc or ~/.config/). This felt like a good starting point, so I created aadam-ali/dotfiles with a basic set of config files.

Despite coming across both chezmoi and GNU Stow, I opted to use a bash script to handle the symlinking of the config files to their canonical locations. Both tools handle symlinking, with the former offering more features including templating. However, I didn’t want to add an extra dependency with no tangible benefit.

The initial structure was simple: a few config files and a setup script to symlink the files to the correct locations. Whilst the structure has since changed, the idea remains the same (source code).

1
2
3
4
5
6

dotfiles/
├── .bashrc
├── .gitconfig
├── .profile
├── README.md
└── setup

This was a quick and effective solution for sharing my config files and is still what I do to this day. Though, I have considered considered revisiting chezmoi to check whether I now find any benefit in its additional features.

Problem 2 - Ensuring that the Same Tools Are Available Between Machines

When considering how to solve this problem, there were two solutions that seemed quite obvious: a shell script or a configuration management tool.

Given my earlier aversion to introducing dependencies, you may have assumed that I opted for a shell script. In that case, you’d be wrong! I decided to use Ansible for no real reason other than as an excuse to learn it. And it’s stuck ever since. My playbook has gone through a couple refactors but has eventually ended up as a refined version of my initial approach.

The structure of my playbook is as follows (source code):

1
2
3
4
5
6
7
8

bootstrap/
├── bootstrap.yaml
├── common.yaml
├── fedora.yaml
├── macosx.yaml
├── README.md
├── ubuntu.yaml
└── variables.yaml

Without going into too much detail, bootstrap.yaml is the entrypoint. It does the following:

• Loads the variables from variables.yaml
• Runs the tasks from the appropriate <distribution>.yaml e.g. installing packages using the distro’s package manager
• Runs the tasks from common.yaml e.g. downloading fonts

A minimal example playbook only requires one file as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

- name: Setup Machine
  hosts: localhost
  connection: local

  tasks:
    - name: Update Apt Cache & Packages
      ansible.builtin.apt:
        name: "*"
        state: latest
        update_cache: true
      become: true

    - name: Install Packages
      ansible.builtin.apt:
        name:
            - fzf
            - ripgrep
        state: present
      become: true

At this point, I had the same tools available to me across my machines but how would I deal with project specific requirements? It’s not uncommon for the projects that I work on to require different versions of Terraform or Python etc.

This is where version managers come in. At the time I was using pyenv, tfenv etc. because this is what my team at the time was using. This was fine at the start but as I began introducing new tools e.g. Golang and kubectl, I eventually wanted a single tool to manage their versions. At first, I used asdf because I had just started a new job where it was widely adopted but I eventually settled on Mise.

Mise allows you to define which versions of tools to use on a global and per project basis using a ~/.config/mise/config.toml or mise.toml file, respectively (example). It’s also compatible with asdf’s .tool-versions file as well as the version files from various other version managers (source). This makes it a great choice because you can use it without needing the rest of your team to. And that’s without taking into account its other features like tasks.

Tying It All Together

Taking the setup script from earlier and extending it’s logic to handle a few preparation tasks as well as running the Ansible playbook has resulted in me having a single command install.

DO NOT RUN THESE COMMANDS WITHOUT UNDERSTANDING WHAT THEY DO!

1
2
3

bash <(curl -s https://raw.githubusercontent.com/aadam-ali/dotfiles/HEAD/scripts/bootstrap)
# OR
bash <(wget -qO- https://raw.githubusercontent.com/aadam-ali/dotfiles/HEAD/scripts/bootstrap)

The boostrap script (source code):

1. Installs any dependencies (the main one being Git)
2. Creates an SSH key if one doesn’t exist and prompts me to add it to my GitHub account
3. Clones my dotfiles repo or pulls the latest changes if it has already been cloned
4. Symlinks the config files to their correct locations
5. Installs Ansible
6. Runs the Ansible playbook

The bootstrap script is then available on my path where I can run it to pull the latest changes to my dotfiles and run the Ansible playbook.

My Takeaways

The Good

• My environment is consistent across all machines, whether it’s my work laptop, my PC, or even a remote server.
• My environment is portable - as long as I have access to a Linux or MacOS install.
• My environment is version controlled, so if I ever need to restore an old behaviour it’s as simple as checking out a specific revision of a config file.

The Bad

There isn’t a ‘bad’ thing that I can think of given that the alternative is not having an automated or version controlled development environment. If I had to be nitpicky, Ansible requiring Python is a little annoying. I would prefer if there was a tool that was compiled into a single binary - an arugment for using good old shell scripts.

What’s Next?

I’m happy with what I have at the moment and probably won’t be making significant changes any time soon. However, there are a few things that I’d like to experiment with when I get some time:

1. Moving to a container based workflow (with devcontainers) where my dotfiles are built into a base image.
2. Exploring reproducible environments (with nix) over repeatable ones.

Thank you for taking some time out of your day to read this post! Please feel free to share this with anyone who you think might benefit from it.

If you would like to stay up to date with future posts, subscribe to the RSS feed.