Distributing Narayana JTS via Docker

[gytis]

Hello,

I'm posting here in order to get some feedback regarding the way to distribute Narayana JTS via Docker. I have a dockerfile which starts transaction service and recovery manager and then registers them with JacORB name server running in the same container. Currently dockerfile [1] and quickstart [2] are available on my Github repos. I would move them to Narayana account once I'm sure that everybody is happy with the implementation.

I've decided to use Supervisor [3] to manage transaction service, recovery manager, and same server processes which are named respectively: TM, RM, and NS.

Transaction manager and name server are accessible via ports 9998 and 9999. Additionally, ports 4711 and 4712 are exposed, which are default Narayana recovery ports.

More information about how to configure and run container is available in readme files.

Thanks in advance,

Gytis

[1] https://github.com/Gytis/narayana-docker/tree/master/jts

[2] https://github.com/Gytis/quickstart/tree/master-JBTM-2296/jts-docker

[3] http://supervisord.org

[tomjenkinson]

I had chance to kick the tyres of this now and think it looks to be heading in the right direction for me. I had to make a tweak to the quickstart to add the jboss.org repositories: https://github.com/tomjenkinson/quickstart/compare/Gytis:master-JBTM-2296...tomjenkinson:master-JBTM-2296?expand=1

I wonder if goldmann might have some opinion on whether it looks like the right kind of thing to get hosted at http://www.jboss.org/docker/? For example the use of supervisor to launch multiple processes in one docker.

[goldmann]

I had a (really) quick look at the Dockerfiles, but I have one comment. Please bear in mind that I do not know the Narayana project.

So, supervisord is not the best idea to use in a container. This is not necessarily how the Docker image should look. We should really avoid it and instead create multiple images that could be linked to each other (if required). If the three services do not talk to each other and just show 3 types of services - these should probably be 3 different images. Probably another, 4th should be introduced to give a base for all 3 images or just use CMD in a smart way to make it overrideable at boot and stick with one. Of course documentation is the key here.

Otherwise - I like the work you've done and I would be happy to create a repository to add narayana to jboss/* namespace on Docker HUB if this is what we want. Tom?

[tomjenkinson]

Thanks for the feedback Marek, we certainly want to get something on Docker HUB but we would like to try to ensure we have the right approach and philosophy here before pushing it further. Gytis is reviewing your comments and he will respond to you shortly. I know that part of the issue is that when running the transaction manager in a different machine to the name service it is best if we can be sure the name service is running first. Maybe we could look at creating a really lightweight feature pack of wfly-core to spin up the CORBA name service and JTS in one vm.

[goldmann]

Cool. You can ensure that naming service is running first by properly linking the containers: https://docs.docker.com/userguide/dockerlinks/

[gytis]

Hello Marek and thanks for your feedback. The reason we have three services is that all of them interact with each other. Name server process is used to expose transaction manager process for clients. And recovery manager process is used by transaction manager.

The reason I chose to use supervisor was firstly this article on Docker web page: https://docs.docker.com/articles/using_supervisord/. I assumed it is an acceptable technique to use. Additionally, my worry was that linking multiple containers would increase already complicated networking setup.

Do you reckon that our image could be rejected from Docker hub because of multiple process? Or would it be still possible to use supervisor? (I'm mainly asking to understand our options, I'm not trying to push the idea of supervisor).

[goldmann]

Hiya Gytis!

Nah, nobody said that because of supervisord someone will reject anything Using supervisord is an option of course. I just try to not involve it as much a possible since it's a bit of anti-pattern. The roots of Docker are: start a process in a container and do it right. Supervisord fights a bit with this introducing a init-like system. Sometimes it's good to use it, but in most cases it's not, since delegation of responsibility is a bit messed when you use supervisord.

There is no network set up overhead involved when using links. The only thing you need to do is to use the prepared environment variables to point to the other service.

[gytis]

Thanks, Marek. I will try splitting this into separate images an will report back on this thread.

[gytis]

Here are two new docker files: narayana-docker/jts-split at master · Gytis/narayana-docker · GitHub.

I've split the original docker file into two: name-server and transaction-service and linked them together. Additionally, I have modified transaction service's main method to start recovery manager in separate thread instead of using separate process. For now that change is in my repo and Narayana distribution is stored in my Dropbox.

Same test as before can be used for these images.

Looking forward to hear some feedback from you guys.

[tomjenkinson]

Hi Gytis,

Great work so far thanks. The overall look is close to what we need in my opinion.

I guess its in the process of development but just pointing out that I think the goal should be to remove narayana-docker/jts at master · Gytis/narayana-docker · GitHub and to move folder "jts-split" to just "jts" in order to make it clear of our expectations for the default setup.

The first thing I think we should tweak is that I think rather than mutating a jbossts-properties.xml on disk we should use system properties. So what I mean is remove:

narayana-docker/docker-entrypoint.sh at master · Gytis/narayana-docker · GitHub

https://github.com/Gytis/narayana-docker/blob/master/jts-split/transaction-service/docker-entrypoint.sh#L19

https://github.com/Gytis/narayana-docker/blob/master/jts-split/transaction-service/docker-entrypoint.sh#L22

Then change:

https://github.com/Gytis/narayana-docker/blob/master/jts-split/transaction-service/Dockerfile#L20

To include -D<OBJECT_STORE_LOC> -D<NAME_SERVICE_SETTING>

The other thing I think we should tweak is, if possible, consider using port 4711 (which appears to be the default JacORB port) and therefore not require the following change.

https://github.com/Gytis/narayana-docker/blob/master/jts-split/transaction-service/docker-entrypoint.sh#L25

I think you would just need to change the readme port opening stuff and this setting:

narayana-docker/Dockerfile at master · Gytis/narayana-docker · GitHub

Those changes are all about minimizing the delta from the defaults. Do you agree or do you feel strongly on the port/config file editing?

Thanks,

Tom

[gytis]

Tom,

you're right "jts-split" and "jts" are both there only temporally until we decide on the final design.

I agree with Narayana changes, they could be replaced with properties.

However, we will still need a way to tell jacorb to use a specific port (either by modifying config file or via system property), because by default it uses a random port. Also, 4711 will work for name server, but not for transaction service, since 4711 is taken by the recovery manager. For name server I picked 3528 since it is the same port used by WildFly.

Gytis.

[tomjenkinson]

Thanks for the response, Gytis.

How about changing:

https://github.com/Gytis/narayana-docker/blob/master/jts-split/transaction-service/Dockerfile#L5

To be 3528 as well then? I am trying to suggest that the ORB port on the name server and the JTS might be best to match if that is possible?

[gytis]

Ah fair enough, that sounds reasonable. Which port do you prefer? 3528 or 4710? As I mentioned before, 3528 is IIOP port used in WildFly. And I picked 4710 for transaction service, so that it would be next to default recovery manager ports (4711 and 4712)

[tomjenkinson]

Lets go with 3528 - it seems sensible to align with container ports already established by WFLY for the ORB. Its a mechanic of the ORB rather than the TM really which is just leveraging it as the transport.

[marklittle]

What is the solution to having a remote object store (not within the docker image)?

Will there be a quickstart/tutorial around this ?