It's funny, this is almost exactly the argument we used to switch to non-BYO systems – moving from Jenkins to CircleCI.
We had Jenkins, but it required so much engineering time to manage, secure, upgrade, etc. Then when we hit scaling limits as our team grew, scaling it out to multiple machines took more time and cost a significant amount as we had to have capacity for peak time, which when all engineers are in one timezone is a significant peak compared to, say, the weekend.
We moved to CircleCI and while I have many frustrations with it, parallelism and ability to speed up the pipeline with minimal development overhead are not really frustrations I have. The cost is also minimal compared to the developer time, and while we're getting "less for our money", because we only pay for the active time, it's actually cheaper for us than Jenkins was just for hardware rental, let alone developer time managing Jenkins.
I can completely see how a different org with different constraints, different deployments, clouds, strategies, provisioning, distribution of engineers, etc, could come to the conclusion that you did – that BYO is better, but I think it does depend on so much.
I think that's more of a comparison of different pieces of software rather than BYO or not. Jenkins is quite the beast no matter what size your projects. There are other fully self-hosted CI solutions, but Jenkins is the biggest one... the hardest one... usually the most fragile one... and for some reason the most popular one...
Yeah that's definitely a factor. There's part of it that's not related though – scaling out build capacity. Setting up a Jenkins build node is actually quite straightforward and reliable on the Jenkins side, similar to a BuildKite node for example, the issue is where is that node, how does it get provisioned, how is it managed, removed, etc.
For us, it was a bare metal machine where we had to email a sales rep to get a machine added, then spend ~2 hours setting up firewall stuff with semi-manual Ansible scripts. Add to that minimum contract terms and difficulty cleaning machines, and it was a pain to manage.
Conversely, if you've got a reliable autoscaling solution of some sort, and your build manager is capable of poking that as necessary to scale up and down (possible with Jenkins, but hard), then this could be really easy to do and BYO may be feasible.
Having a CI provider give us ~unlimited pay-as-you-go capacity that needed no management on our end and was always a clean environment, that was worth a lot to us in engineering time.
> For us, it was a bare metal machine where we had to email a sales rep to get a machine added, then spend ~2 hours setting up firewall stuff with semi-manual Ansible scripts. Add to that minimum contract terms and difficulty cleaning machines, and it was a pain to manage.
That'll do it. I'm using the Buildkite elastic stack, which took me about 20 minutes to start using and 4-5 hours to dial in to ideal settings (eg adding IAM to allow deploys from agents, getting the right size spot instances etc).
We had Jenkins, but it required so much engineering time to manage, secure, upgrade, etc. Then when we hit scaling limits as our team grew, scaling it out to multiple machines took more time and cost a significant amount as we had to have capacity for peak time, which when all engineers are in one timezone is a significant peak compared to, say, the weekend.
We moved to CircleCI and while I have many frustrations with it, parallelism and ability to speed up the pipeline with minimal development overhead are not really frustrations I have. The cost is also minimal compared to the developer time, and while we're getting "less for our money", because we only pay for the active time, it's actually cheaper for us than Jenkins was just for hardware rental, let alone developer time managing Jenkins.
I can completely see how a different org with different constraints, different deployments, clouds, strategies, provisioning, distribution of engineers, etc, could come to the conclusion that you did – that BYO is better, but I think it does depend on so much.