All articles

Python is booming and Flask is a pretty popular web-framework nowadays. Probably, quite some new projects are being started in Flask. But people should be aware, it's synchronous by design and ASGI is not a thing yet. So, if someday you realize that your project really needs asynchronous I/O but you already have a considerable codebase on top of Flask, this tutorial is for you. The charming gevent library will enable you to keep using Flask while start benefiting from all the I/O being asynchronous. In the tutorial we will see:

• How to monkey patch a Flask app to make it asynchronous w/o changing its code.
• How to run the patched application using gevent.pywsgi application server.
• How to run the patched application using Gunicorn application server.
• How to run the patched application using uWSGI application server.
• How to configure Nginx proxy in front of the application server.
• [Bonus] How to use psycopg2 with psycogreen to make PostgreSQL access non-blocking.

Read more

Regardless of whether it's the end of the calendar decade or not it's the end of a programming decade for me. I started early in 2010 and since then I've been programming almost every day, including weekends and vacations. This was a really exciting period in my life and I realized that it's been a while since 2010 only recently. So, I decided to put into words some of my learnings from that time. Warning: the content of this article is highly opinionated and extremely subjective.

Read more

Well, at this moment we already know what we need to deal with. In order to use runc from code we need to implement our shim as a daemon and this daemon has to be as long-lived as the underlying container process. In this article, we will try to develop a minimum viable runtime shim and integrate it with our experimental container manager.

The minimal shim implementation takes as its input a path to the container bundle (with the config.json) as well as the list of the predefined locations (for the container log file, container pidfile, container exit status file, etc). The shim needs to create a container by executing runc with the provided parameters and then serve the container process until its termination. The planned functionality of this shim version includes:

• Detaching the shim from the container manager process.
• Launching runc and handling container creation errors.
• Reporting the status of the container creation back to the manager.
• Streaming container's stdout and stderr to the log file.
• Tracking and reporting the exit code of the container.

Read more

A container runtime shim is a piece of software that resides in between a container manager (containerd, cri-o, podman) and a container runtime (runc, crun) solving the integration problem of these counterparts.

Layered Docker architecture

The easiest way to spot a shim is to inspect the process tree on a Linux host with a running docker container:

ps auxf output on a host running docker run -it ubuntu bash; notice containerd-shim process in between containerd and bash.

On the one hand, runtimes need shims to be able to survive managers restarts. On the other hand, shims are helping container managers to deal with the quirky behavior of runtimes. As a part of the container manager implementation series, we will try to create our own shim and then integrate it with conman, an experimental container manager. Hopefully, during the development, we will gain an in-depth understanding of the topic.

However, before jumping to the shim development, we need to familiarize ourselves with the container runtime component of the choice. Unsurprisingly, conman uses runc as a container runtime, so I will start the article by covering basic runc use cases alongside its design quirks. Then I'll show the naive way to use runc from code and explain some related pitfalls. The final part of the article will provide an overview of the shim's design.

Read more

When I'm diving into a new codebase, I always start from the project structure analysis. And my favorite tool is tree. However, not every project is perfectly balanced. Some files and folders tend to be more popular and contain much more code than others. Seems like yet another incarnation of the Pareto principle.

So, when the tree's capabilities aren't enough, I jump to cloc. This tool is much more powerful and can show nice textual statistics for the number of code lines and programming languages used per the whole project or per each file individually.

However, some projects are really huge and some lovely visualization would be truly helpful! And here the FlameGraph goes! What if we feed the cloc's output for the Kubernetes codebase to FlameGraph? Thanks to the author of this article for the original cloc-to-flamegraph one-liner:

git clone https://github.com/brendangregg/FlameGraph
go get -d github.com/kubernetes/kubernetes

cd $(go env GOPATH)/src/github.com/kubernetes/kubernetes

cloc --csv-delimiter="$(printf '\t')" --by-file --quiet --csv . | \
    sed '1,2d' | \
    cut -f 2,5 | \
    sed 's/\//;/g' | \
    ~/FlameGraph/flamegraph.pl \
        --width=3600 \
        --height=32 \
        --fontsize=8 \
        --countname=lines \
        --nametype=package \
    > kubernetes.html

open kubernetes.html

Read more