Schema design

[john.sanda]

I have been continuing to think more on design and starting writing up some notes. Take a look at my design doc[1], the schema section in particular I provided an example of how we might declare tenant-level pre-computed aggregation and retention settings

[1] https://docs.google.com/document/d/1Il9wHSKQD4QwVmgmZhrA7ghRejAt2Qka98_qoQFDfUU/edit?usp=sharing

[mithomps]

You have probably already seen this, but here is another schema from: pyr/cyanite · GitHub - project that is a C* based backend replacement for graphite.

It has tenants and rollup...

CREATE KEYSPACE metric WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '1' }; 
USE metric; 
CREATE TABLE metric ( period int,
 rollup int,
 tenant text,
 path text,
 time bigint,
 data list<double>, 
PRIMARY KEY ((tenant, period, rollup, path), time) ) 
WITH bloom_filter_fp_chance=0.010000 AND caching='KEYS_ONLY' AND comment='' AND dclocal_read_repair_chance=0.000000 AND gc_grace_seconds=864000 AND index_interval=128 AND read_repair_chance=0.100000 AND replicate_on_write='true' AND populate_io_cache_on_flush='false' AND default_time_to_live=0 AND speculative_retry='NONE' AND memtable_flush_period_in_ms=0 AND compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'LZ4Compressor'}; 

[john.sanda]

I have seen it. In fact, you may recall that the RHQ core dev team discussed it some a while back when we first started discussing RHQ Metrics. Gotta love that it is in Clojure The path column is the metric identifier. One thing that I find interesting is that in index of paths is maintained. From what I can see so far, that index is used in querying/search for metrics from the HTTP interface. By default the index is stored in memory, but it also supports using ElasticSearch. I am not sure about all of the search capabilities that are available to clients, but it definitely offers a nice, flexible way to find metrics.

[john.sanda]

My Clojure skills are admittedly rusty, but it looks like cyanite only supports a few aggregation functions - max, min, mean, and sum. It does support pre-computed aggregates in addition to downsampling.

[mithomps]

I was only thinking that the schema might provide multi-tenant and rollup  ideas that may be useful. That's all.

[mithomps]

What about units of measure (UoM) for the values. Different metrics will have various UoM.  I didn't see anything in the schema to capture that?

[john.sanda]

Mike Thompson wrote:

What about units of measure (UoM) for the values. Different metrics will have various UoM.  I didn't see anything in the schema to capture that?

My design doc is a work in progress. Take a look at the notes and examples for the numeric_data table. It includes an attributes column now.