The Power Of Abstraction

One of the most useful skills in our developer work is the skill to make abstractions over the subject area. The abstraction is the simplest object-oriented principle and we use it widely to manage complexity. Engineers use it to decompose complex systems into smaller ones. Abstractions hide details under the hood and provide clarity of vision.

In DevOps world, we typically use one or another solution for deployment applications (or services) that is PaaS (or orchestration if you wish). The general way of deploying an application is applying manifest via CLI or REST API. So, look at the manifest for docker compose (yes, it is orchestration too):

version: '3'
services:
  coolapp:
    image: com.company/coolapp:1.0.0
    ports:
      - "8080:8080"
    deploy:
      replicas: 6
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M

This manifest consists of two things that matter for us. First thing is its structure (or in other words how we could describe our application). And the second is its content (or in other words what we want to deploy exactly).

Why do these two things matter? Because our needs in resources, replicas, constraints, etc are not dependent on a manifest structure or a chosen solution. Even if you meet the eye that Marathon and Kubernetes are two very different things. This fact is lead us to think that structure is just the derivative of content and concrete implementation. And we could generate it from some more abstract presentation.

But why we should do that instead of simply putting manifests into service repos like YAML-file for Travis, or Jenkinsfile?

Let’s look at the benefits and drawbacks of this approach. Pros, I think, are clear. We just write the manifest for the service and then apply it manually or automatically. There is no magic, no implicit, simple and clear. Also, all features that are supported by orchestrator could be applied easily.

But on the other hand, we have cons like copy-paste manifests from one repo to another. Сopy-paste manifests for test, stage, and production environments. Search and maintain all manifests if we would like to change something for all (or a significant part) applications. Security breaches if secrets are located inside of deployment file (don’t do that, guys). And we achieve vendor lock on the concrete solution such as Kubernetes or Swarm.

This approach may be useful if you have your own orchestrator (in-house developed private PaaS) and you take care of backward compatibility and security (secrets come outside of manifest). And even in that case (hand on heart, guys, the majority of us do not use in-house cloud) the problem of copy-pasting and maintaining manifest still exists. The question is does it make your life worse, or do you love doing a boring job?

In the case of using the third-party solution, you have all the drawbacks that we met early. Some of the problems you could eliminate with additional tools like Helm (package manager for Kubernetes) that allow you to template deployment with config files and Lua scripts. But Helm still focuses on one concrete solution, so what if we go further.

Imagine that we have own format of application manifest that we could process into parts of deployment:

application:
   name: coolapp
   image: coolapp
   port: 8080
test:
      cpu: 0.01
      memory: 128.0
      instances: 2
   
   prod:
      cpu: 0.1
      memory: 512.0
      instances: 6
use_mongo: True
   use_queue: True

What the difference between this description and the previous one?

Firstly, some things as docker image version are explicit. We do not describe them in manifest because they are frequently changing, but they come to us in any convenient way.

Secondly, convention over configuration (you could hate this approach for a lot of magic). Why is the image called “coolapp”? Does it come from official docker registry or from corporate one? We make an abstraction from this to our developers. Moreover, we could simply eliminate the property image from our manifest because it is the same as name. We need to find the balance between the size of the average manifest and the amount of magic that comes out nowhere.

In addition, standardized features like using mongo, or some queue in our application are connected via one line (like a checkbox in graphical interface). Yes, it could be some other options for customizing features, but for typical service (or type of service) developer could not think about how he should connect them.

How to implement it?

You have a wide range of options starting from tools like Ansible (yes, yes, yes) to your own implementation. You are bounded only by your time, skills and fantasy.

So, if you pick Ansible for this you could template manifest for any orchestrator and apply it via standard modules like uri and command or you could use more specialize modules or feel free to write your own. Implementation of features is depended on your architecture patterns. It could be sidecars or simple passing credentials of mongo cluster outside the cloud to environmental variables. Again you still have a multiple choice here.

For example, we could template our description into Kubernetes deployment manifest:

apiVersion: v1
kind: Service
metadata:
  name: "coolapp-service"
  labels:
    run: "coolapp"
spec:
  type: NodePort
  ports:
  - port: 8080
    protocol: TCP
    name: http
  selector:
    app: "coolapp"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: "coolapp-deployment"
spec:
  selector:
    matchLabels:
      app: "coolapp"
  replicas: 2
  template:
    metadata:
      labels:
        app: "coolapp"
    spec:
      containers:
      - name: "coolapp"
        image: "com.company/registry/coolapp:1.0.0"
        resources:
          requests:
            memory: "128.0Mi"
            cpu: "0.01"
          limits:
            memory: "128.0Mi"
            cpu: "0.01"
        ports:
        - containerPort: 8080
        env:
        - name: SOME_ENV
          value: "some value"
        command: [ "java", "-jar", "coolapp.jar" ]

Then deploy it via kubectl using the specialized module or just run a shell command and check the result after that.

And what we achieve?

To begin with, we have an agnostic description of applications. We have an abstraction over a deployment process. We could change deployment behavior at any time without changing manifests because our needs are not changed, but our environment is. Even more, we could migrate from one solution to another like from Marathon to Kubernetes or vice versa or just support both of them at the same time.

Furthermore, we could easily maintain common parts for any application that we have already deployed. We could add new behavior or fix some things on the go without searching, regexping, and copy-pasting.

Finally, we provide a high-level abstraction for developers who could not care (or just do not want to do that actually) about applied features. They could focus on business features instead.

And what the price, man?

I would not be totally honest if I do not tell about what we actually paid. Abstractions are very powerful but they сould leak. What if you need to make something special only for one application? Whether we add this for all with default turned-off behavior or we should add a possibility to paste raw definition (for example custom labels for Marathon)?

These questions are the same that all of us face when we code something. Nothing new. Just think all the time.

Another problem is testing. Changing one file that affects one application is not the same that changing one file that could affect to all of your applications. It leads to improve the whole delivery process or maybe you are never making a mistake (sarcasm here).

And the last one is limitations that we should admit. Some concepts in orchestrators could be completely different or be the same but have a completely different implementation. For example, you want to launch some job periodically after the application was deployed. Ok, if you use Mesos ecosystem you can pick Chronos for that. Let’s look at the spec for schedule definition:

{
  "schedule": "R/2018-09-25T17:22:00Z/PT2M",
  ...
}

As you see, Chronos use own format mixed with ISO 8601 format for date and time and duration. If you want to do the same in Kubernetes you can use CronJob. Let’s look at its spec:

...
spec:
  schedule: "*/1 * * * *"
...

This is just a cron definition. And the question is how should we define that in our own abstract manifest? Should we use cron and retranslate it to Chronos format? Whether we should use Chronos format and retranslate it into cron or use our own format and do the same action? We need to think again.

To sum up, we could say that the more standardized applications you have, the more benefits could be achieved with this approach. Limitations usually are not evil and introducing them the sooner the better. Own format and standardized process of deployment make you more adaptable to changes. The price that we pay is our time to think, develop and support it. There are no miracles.

If you at the starting point, think about the nearest future. But if you want to migrate from one solution to another with more than several dozens of applications without annoying teams, you really have no choice.