This article takes you to understand the past and present of Mimir, Grafana's latest open source project

Just a few days ago (2022/3/29) Grafana announced the official open source of its time series database Mimir. Once open sourced, it caused widespread discussion in the community. Not only did the discussion on hacknews continue to heat up, but it also quickly gained 1K attention on Github.

What is it that keeps it so popular, strength or controversy?

What is Mimir

Grafana Mimir is an open source, horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus.

Mimir is officially introduced

You read that right, itIt is another time series database, and it also solves the problems of long-term storage, horizontal expansion, and multi-tenancy of Prometheus.

Although it claims to be fully compatible with Prometheus, its goal is not just to be a better Prometheus, it positions itself as the ultimate solution for back-end storage of metrics in observability, capable ofCompatible with various metrics protocols, as shown in the figure:

Mimir's selling point

You may ask, Prometheus has so many long-term storage community solutions, Thanos, Cortex, M3, Victoriametrics, why another one, timingIs the database really that volume?

Let's take a look at some of Mimir's own selling points:

• Super-large-scale production verification (over 2 billion active indicators)

• Simple deployment, supports both single/microservice deployment methods, less dependencies (only OSS, Cortex dependencies you know)

• Support high cardinality metrics compression and query

• Support same tenant BlockFragment compression (over 64G limit)

• Support query and calculation lower layer

• Naturally supports multi-tenancy and supports global cross-tenant query

• Compatible with all metircs protocols (currently supporting Prometheus)

Are these promotional selling points really worth our migration?This is also a point that everyone discussed on HN. The general conclusion is as follows:

• Cortex is too difficult to use. If you are using Cortex now, you can try to upgrade to Mimir.

• Thanos is currently working very well, unless you encounter problems such as high cardinality/large block compression, there is no need to migrate.

• VictoriaMetrics is also working very well, and it is necessary to add a comparison test with VictoriaMetrics neutrally in the community to facilitate selection.

I generally agree with the above viewpoints.

Relationship with Cortex

After receiving the news that Grafana Mimir is open source, students who are familiar with Cortex may have the same instinct as me, Grafana may have to give up Cortex, after all, Mimir and CortexIt's too similar, and most of the previous core maintainers of Cortex (4/7) were from Grafana.

First of all Mimir is indeedBased on Cortex, you can understand that it is the 2.0 version of Cortex, which is not difficult to explain why the first version of Mimir open source is v2.0.

As for why this is done, we can see the reason from HN's discussion:

• Cortex has a large technical debt and is veryIt is difficult to meet some requirements in Mimir

• The change of open source protocol is more suitable for the development of Grafana's products. Cortex belongs to Apache-2.0 License, while MimirThe protocol is AGPLv3.

I personally think that the change of the protocol should be the decisive factor, because AGPLv3 is the open source protocol mainly used by Grafana at present, you can refer to [Grafana Open Source Protocol Change Q&A].

But it seems that CNCF requires projects to be incubated with Apache-2.0 License, so how Mimir enters CNCF for incubation in the future is worth paying attention to.

Is Grafana really giving up on Cortex?

The current official explanation is that Cortex will find new maintainers, and the original Grafana maintainers need to put more energy into Mimir.

So the answer is yes, I personally think that Grafana is really going to give up Cortex, but this does not mean that Cortex will stop maintenance, which will test subsequent maintainers.

Cortex's current value is the existing users and its Apache-2.0 protocol. In case Grafana comes with another protocol modification, after all, it is not very good for a single company to dominate.

Is it worth learning?

I personally think it is worthwhile from an architectural point of view, for the following reasons:

Fully distributed, good scalability and clusteringThe management ability is a good reference for us to manually build a multi-tenant time series database through Thanos.

For some selling point functions, such as single-tenant sharding compression, query operator push-down can be learned well (it is expected that Thanos will follow up in the future)

Summary