Propagate alert definition templates between RHQ installations

[swiderski.maciej]

Hi,

would like to ask if there is a way to sort of export import alert definitions from one RHQ server to another. Imagine situation where there is one server that is considered test that usually acts as playground environment that allows to verify certain functionality and then another server is covering pre production and production systems. There are bunch of alert definitions that shall be moved from test to production RHQ server. Is there a way to achieve that?

Alternatively would it be possible to create scripts for setting up alert definitions instead of using GUI?

Thanks in advance

Maciej

[pathduck]

Hi Maciej,

This has been discussed before and there is an RFE on Bugzilla:

https://bugzilla.redhat.com/show_bug.cgi?id=535829

It is something I am very interested in as well.

If you look at the attachments to the BZ there is a plugin you can try, to export alert definitions:

https://bugzilla.redhat.com/show_bug.cgi?id=535829#c11

I have tried it but it needs more work to actally be really usable. Also I got some errors and not even sure if it works in newer versions of RHQ.

The developers would probably say to script it through the CLI - but not everyone can spend a long time learning the scripting API and language and would prefer just a simple export/import interface.

[lkrejci]

The problem with alert definition / templates export/import is that alert

notifications are a) pluggable and b) can reference resources and/or users,

roles, etc. which are in generic sense not transferable between RHQ

installations (id's won't match, even resource ancestry might be different if

stuff is for example deployed in a way that affects resource key generation).

So in order for this to work reliably, the users would have to provide a map

that would relate corresponding resources in the 2 environments. Not that that

is not doable but it is something that we haven't found a nice way of doing it

in the GUI, but even if we constrained ourselves to CLI, the map could be

quite tricky to produce because the entities in the RHQ data model are quite a

bit interrelated (users reference roles, which reference groups (and bundle

groups), which reference resources. So to properly map users including their

settings between the two environments, you'd basically have quite a big graph

to cover).

There's support for system settings and metric definitions sync using the

SynchronizationManager in the CLI. I'd imagine you could extend that to also

support alert definitions/templates if you would like to contribute a generic

purpose solution for this.

Or as Stian said, you will have better luck with a custom scripted solution to

this that would be tailored to your environment.

There's also https://github.com/rpelisse/rhq-sync-tool but that seems a little

bit dead and doesn't seem to support alert definitions either.

I think I've found the code for vlad's plugin (from BZ 535829) here:

https://github.com/1and1/rhq-infrro

Sorry to not have any better news, but that's how things are ATM.

As for the CLI - RHQ 4.13 finally contains the "rhqapi" module which has a

MUCH nicer API than the raw remote API exposed by the CLI by default (but RHQ

4.5.1 and up should be able to use that module, too).

It doesn't cover the whole remote API but you should definitely give it a try

(I did and was impressed http://lukas.krejci.pw/posts/2014/11/24/playing-with-rhqapi/).

Lukas

[swiderski.maciej]

thanks a lot guys for all the details. I'll be giving a try with cli to be able to sort of record and replay configuration for alerts across various installations. That should give us quite a lot of flexibility. Will post any questions that I might run into.

Maciej

[swiderski.maciej]

one more question, is there any reference for Remote API and CLI api that I could use for AlertDefinitionManager and ResourceTypeManager?

Maciej

[swiderski.maciej]

ok, managed to use CLI to fetch all alert definitions I need but it seems that CLI lacks operation to create alert definitions then, is that correct or am I missing something?

If so, would be possible to use remoteClient.remoteInvoke to create alert definitions somehow... or any other way to achieve that

[pilhuhn]

Am 12.12.2014 um 09:59 schrieb Maciej Swiderski <[e-mail]>:

JBoss Developer

Propagate alert definition templates between RHQ installations

reply from Maciej Swiderski in RHQ - View the full discussion

ok, managed to use CLI to fetch all alert definitions I need but it seems that CLI lacks operation to create alert definitions then, is that correct or am I missing something?

This is very possible.

If so, would be possible to use remoteClient.remoteInvoke to create alert definitions somehow... or any other way to achieve that

The rest-api allows to retrieve and create alert definitions.

See e.g. https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Operations_Network/3.3/html/Rest_API/index.html#N69932

[swiderski.maciej]

Thanks Heiko,

is this REST api available in JON 3.2 as well?

[pilhuhn]

is this REST api available in JON 3.2 as well?

Yes. And is not too different from what is in 3.3 (we fixed some issues in 3.3 mostly)

[swiderski.maciej]

alright, it does works well on 3.2 as well. Though few points that are sort of locking further usage:

• not possible to filter alert definitions to return templates only (still a minor limitation that can be workaround)
• AlertDefinitionRest does not specify what resource id is is defined for so resource type cannot be found for it and by that cannot be created on another server instance

so with these I won't be able to use REST api for both fetching available definitions and the creating them on second instance.

Any ideas if resource id can be fetched somehow for alert definition via REST?

If not then I think I'll attempt to create mixed utility:

• fetch alert definitions via scripting
• create alert definitions via REST api

hopefully both can be done from within script (java script and executed via rhq client cli). Any thoughts on this approach?

Thanks
Maciej

[jayshaughnessy]

There is AlertDefinitionCriteria.addFilterAlertTemplateOnly(boolean filterAlertTemplateOnly).  I guess the REST interface isn't offering this criteria.

[lzoubek]

Hi Maciej,

You're right, AlertDefinitionRest does not specify resourceId nor resourceGroupId nor template, but at least you can access those by reading "links" with rel="resource" then you should be able to parse resourceId. But you could also be able to pull all definitions by knowing all resource IDs and then GET /alert/definitions/?resourceId=123

REST API is certainly missing exposing alert definitions for resource types or groups.

Not sure if resourceId helps enough, as your target instance has different IDs anyway.

Some time ago I was working on setting up alert definitions for testing JON instance in QE. We needed to setup several alerts right after we deployed JON + several agents. I started a python client, which is capable doing that (it cannot setup alerts per resource type or groups). Main idea here is to keep definitions in the script (or some additional json/py file) and ignore resourceId at all - resource is identified as a path in resource tree. By using wildcards in path elements I was able achieve something similar to resource type based alert definitions, but with worse management capabilities (alert definitions are still bound to particular resources). Above python client is just a rough start and would need more work (become idempotent, be able to update existing definitions in case it differs from what it has, support more condition and notification types, and of course support resource-type and group based definitions).

If you decide to go this way I'd love to assist you and extract that stuff into standalone module/repo and help with enhancements.

Also, note  [BZ 617202] AlertConditionTemplate allows creation of AlertDefinitions e... by burmanm · Pull Request #143 · rhq-project… which adds support for creating or updating alert definitions to CLI.

[swiderski.maciej]

thanks guys for very valuable input.

I am following REST based approach so far and looks promising. One thing I don't know how to gather is MeasurementDefinition of alert condition as it seems to be an internal identifier. Any idea how to get hold of that value based on some criteria like name or so?

Thanks

Maciej

[lzoubek]

Yes, measurementDefinitionId is internal thing. You need to enrich your export data model for several more values, which will help you find the correct measurementDefinitionId when importing. Ideally, it could be a triple (pluginName, resourceTypeName, metricName), but at this time, neither RHQ 4.13 nor JON 3.3 has a capability of retrieving measurementDefinition by those 3. I've filed a BZ for it and hopefully push it to 3.3.1. A workaround could be "just" to remember scheduleName at export time (you can get it by GET /metric/definition/{id}) and then at import time, you get to it through resource, similar to this. I would work only for resource-related alerts, for group alerts or template alert definitions it will be difficult and it requires imported resources in inventory.

I am interested in your export/import tool. How are you going to deal with resourceIDs? Are you going to export to file or just memory and do immediate import? Do you have a plan/permission to share your tool at some point?

[swiderski.maciej]

Libor,

will do share my work as soon as it's usable I am somewhere in the middle of the work. My main focus is actually to copy alert definition templates and not individual alerts per resource. Anyway once I have that we can take a look and see how that can be improved.

The implementation is mainly utilizing REST api and slightly remote api to fetch measurement definitions so they can be used to map between installations. Export and import is done at the same time, meaning fetching items from one server and applying them directly on the other after doing the mapping of ids. So there is no saving stuff to a file or so, but I guess it could be done as well as I do operate on JSON format so it would be just a matter of dumping that.

I plan to have some working version by the end of the week and then will push that to github, and once it's there feedback is more than welcome.

Thanks for all the help

Maciej