Woodpecker CI is very straight forward and seems to basically have what I want from a CI tool without much beyond that.

• In-repo pipeline definition
• Docker job execution
• Docker image for deploying server and agents
• Parallel execution on a single agent (looking at you GitHub)

But of course I still had to fight with it for a couple hours to make it do something super simple. Because everything is awful and I'm an idiot.

I just wanted to have it build (zola build) this site and deploy it (rsync).

Building was easy enough - I needed a docker image with Zola on it. I couldn't use the "official" zola one, though, because it lacks even /bin/sh and Woodpecker needs some way to execute your scripts on the container you want a job to run in.

So I made a small alpine based image to do that - no problem.

Then I needed to rsync. And SSH auth and secrets.

There are a million ways for things to go wrong here that are painful and just take time to figure out.

Here are the issues I ran into - many were my fault and others were simply not-great UX from Woodpecker.

• Woodpecker supports per-repo secrets - great - but they have to be specified by name in the YAML of each pipeline step that wants to use them. This is clearly stated in the docs and I followed the instructions clearly but at one point I removed them from the YAML and didn't realize it.

  Unfortunately Woodpecker doesn't have a great way of letting you know you're referencing secrets which you haven't asked for because it exposes secrets as just plain environment variables.

  I don't really know what I would want from Woodpecker here - this was my fault and it was hard to notice once I'd mistakenly removed the secrets.

• Putting "-" in your secret names breaks your YAML and Woodpecker fails without much information when you give it broken pipeline YAML.

• Copying environment variables into files and using them as ssh keys is super annoying and error prone. Between key file permissions and the whole process failing because the file contents were empty because the secret being echo'ed into the file was actually not present. Ugh.

• SSH TOFU is great for users but annoying for automation. You have to pick from one of several annoying options.

  • use ssh-keyscan during the automation to trust the host key
  • Muck with ssh command line arguments to turn StrictHostKeyChecking off
  • Pre-emptively load an authorized_hosts file onto the machine you're deploying from.

  The last option is probably the best because it's the most secure but it's also annoying because it means you can't use a generic image for that deployment step.

Ultimately it all worked out - I ended up using some Alpine docker images, sshpass and rsync.

But it's always a fight to get these things working. Even when I'm using software that seems like such a nice fit for my personal approach like Woodpecker.