With seemingly everyone who’s ever written an app or booted a VM jumping on the cargo ship at the moment, it’s hardly surprising to see the launch this week of Rocket; a credible challenger to Docker in the container space.
Reading through the launch announcement, I found two of the motivations given by the CoreOS team especially interesting. They set me wondering: perhaps looking for “simple, lightweight enterprise solutions” is actually a pipe dream, and we’d be better served by looking for “enterprise features” from Day 1.
But back to the Rocket announcement. First, the following:
“From a security and composability perspective, the Docker process model – where everything runs through a central daemon – is fundamentally flawed. To ‘fix’ Docker would essentially mean a rewrite of the project, while inheriting all the baggage of the existing implementation.”
This flaw has been known and discussed pretty openly for a while, but hasn’t quite so bluntly been labelled a “fatal flaw” up until now. To me, the main takeaway here is a reaffirmation of a fact every experienced operations engineer already knows: building a reliable, secure runtime is hard! So hard, in fact, that even the very smart team at Docker have yet to get it right.
So why are so many companies with a fraction of the experience of the Docker engineers trying to build their own container-based PaaS platform right now? With the general state of being dazzled by the shiny new toy starting to wear off, we can hopefully dare to accept the fact that trying to build your own runtime environments, by stringing together a combination of brand new technologies, is Just Not A Good Idea. Not unless you have:
- People in your organization who understand the complexities of building and maintaining these systems
- A lot of cycles on your hands
- Are working in a timeframe that has room for a couple of painful iterations until the technologies mature, anyway.
And then this one:
“Docker now is building tools for launching cloud servers, systems for clustering, and a wide range of functions: building images, running images, uploading, downloading, and eventually even overlay networking, all compiled into one monolithic binary running primarily as root on your server. The standard container manifesto was removed. We should stop talking about Docker containers, and start talking about the Docker Platform. It is not becoming the simple composable building block we had envisioned.”
My first thought was: so, in a simply, composable, non-platformy Rocket world, will all this complexity
- …be handled by CoreOS (and how is that not simply a different choice of platform)?
- …be handled by the App Container Runtime, but then somehow in a “less heavyweight” manner?
- …simply go away?
To be clear: I’m not trying to be critical of Rocket or the Rocket team in any way here. I think that added diversity in the container space comes not a day too soon, and am really looking forward to see what Rocket will bring to the table. What does interest me, however, is the notion of simplicity as an achievable trait. Appealing as it may be, is it realistic in an operational context?
As a developer, I totally appreciate the desire to have “simple” and “lightweight” solutions for everything. I get that “urgh” feeling when working with clunky, slow, antiquated tools and processes just as everyone else does. But working day-in, day-out with teams introducing enterprise tools for Continuous Delivery and Devops, I’ve come to accept that some of the “weight” of enterprise tools is actually necessary. It was a painful truth to me: those “enterprise requirements,” the ones that cause much of that “heavyweight” feeling, the ones that we all want to banish…they’re real. They’re never going away. Delivering a business service at a certain scale (not just running the underlying tech, which is hard enough in itself!) just comes with someintrinsic complexity that doesn’t simply evaporate “if only you use the right tools”, much as we would like to believe that.
A colleague, a highly experienced architect and consultant, put it this way:
“I do understand the path that Docker is taking: just having a Docker container is almost as exciting as having a Java EAR file. The major difference that most operations departments know what to do with an EAR file (sort of). To run Docker-based applications 24×7 in production requires a platform, and any container is just a building block.“
Sure, if you’re experimenting with in a lab environment, or if you’re a startup with a handful of people and a couple of VMs in the cloud, the “simple”, “lightweight” and “latest and greatest” tech will work for you. But once you try to move out of the lab and into production, or once your startup grows from 5 to 50 and then 500 people, all those boring, enterprise problems – scalability, auditability, reporting, managing multiple versions across many release pipelines, ease-of-use etc. – will be right there, just when you have absolutely no time to get to grips with them.
Of course, there’s always the option of staying in the lab and throwing your app over the wall, or of leaving for a new startup once you hit 50 or 500 people, but that’s just shifting the problem to someone else. Let’s call a spade a spade: the “Simple, Lightweight Enterprise Platform” for consistently delivering your applications doesn’t exist.
Don’t get me wrong: I’m tremendously excited by the potential of microservices as an application paradigm and containers as an implementation technology for them. I firmly believe we need new ideas and, yes, new tooling to make proper use of the potential of microservices and containers and to address the new challenges (such as runtime dependency management) that they introduce. And of course we should avoid and eliminate unnecessary complexity and weight wherever possible.
But let’s stop this mad rush to operationalize a bunch of “we-called-it-1.0-because-0.2-sounds-scary” tooling. Let’s stop making believe that we can grow our projects and companies to any kind of meaningful size with a bag of operational tools of which many are “simple” because they are simplistic and “lightweight” because they lack essential features down the road. This time, let’s try to ensure that security, scalability, reporting, usability etc. is in place before we’re scrambling to hook up hundreds of new servers a day, or getting hammered with audit requirements, and need to divert precious time to get our shiny, “simple”, “lightweight” tooling to cope.
Enterprise features may be boring and “heavyweight”, but for your next round of Devops, Continuous Delivery, microservice and container tooling, look for them from Day 1. Because you will need them.