Deployers Ordering

[ropalka]

I have some troubles with deployers ordering if multiple deployers
provide many inputs. So I'm trying to understand how deployers
ordering works internally.
I'd expect the following test to fail:
---

// copy paste from DeployerFlowUnitTestCase

public void testTransitionOrdering() throws Exception
{
DeployerClient main = createMainDeployer();
TestFlowDeployer deployer1 = new TestFlowDeployer("A");
deployer1.setInputs("3");
deployer1.setOutputs("4");
addDeployer(main, deployer1);

TestFlowDeployer deployer2 = new TestFlowDeployer("B");
deployer2.setInputs("1");
deployer2.setOutputs("2");
addDeployer(main, deployer2);

TestFlowDeployer deployer3 = new TestFlowDeployer("C");
deployer3.setInputs("2");
deployer3.setOutputs("3");
addDeployer(main, deployer3);

Deployment deployment = createSimpleDeployment("TransitionOrdering");
main.addDeployment(deployment);
main.process();

assertEquals(3, deployer1.getDeployOrder());
assertEquals(1, deployer2.getDeployOrder());
assertEquals(2, deployer3.getDeployOrder());
assertEquals(-1, deployer1.getUndeployOrder());
assertEquals(-1, deployer2.getUndeployOrder());
assertEquals(-1, deployer3.getUndeployOrder());

main.removeDeployment(deployment);
main.process();

assertEquals(3, deployer1.getDeployOrder());
assertEquals(1, deployer2.getDeployOrder());
assertEquals(2, deployer3.getDeployOrder());
assertEquals(4, deployer1.getUndeployOrder());
assertEquals(6, deployer2.getUndeployOrder());
assertEquals(5, deployer3.getUndeployOrder());

main.addDeployment(deployment);
main.process();

assertEquals(9, deployer1.getDeployOrder());
assertEquals(7, deployer2.getDeployOrder());
assertEquals(8, deployer3.getDeployOrder());
assertEquals(4, deployer1.getUndeployOrder());
assertEquals(6, deployer2.getUndeployOrder());
assertEquals(5, deployer3.getUndeployOrder());
}

---
I'd expect this test to fail because there's no deployer with "1" output?

[ropalka]

And I'd expect the following test to pass:
---

public void testWebServicesDeployersOrder() throws Exception
{
DeployerClient main = createMainDeployer();

TestFlowDeployer deployer1 = new TestFlowDeployer("FakeDeployer");
deployer1.setOutputs("WebServicesMetaData", "WebServiceDeployment", "JBossWebMetaData" );
addDeployer(main, deployer1);

TestFlowDeployer deployer2 = new TestFlowDeployer("WebServicesDeploymentTypeDeployer");
deployer2.setInputs("WebServicesMetaData", "WebServiceDeployment", "JBossWebMetaData" );
deployer2.setOutputs("DeploymentType", "JBossWebMetaData");
addDeployer(main, deployer2);

TestFlowDeployer deployer3 = new TestFlowDeployer("WebServiceDeployerEJB");
deployer3.setInputs("JBossWebMetaData", "DeploymentType");
deployer3.setOutputs("JBossWebMetaData");
addDeployer(main, deployer3);

TestFlowDeployer deployer4 = new TestFlowDeployer("WebServiceDeployerPreJSE");
deployer4.setInputs("JBossWebMetaData", "DeploymentType");
deployer4.setOutputs("JBossWebMetaData");
addDeployer(main, deployer4);

TestFlowDeployer deployer5 = new TestFlowDeployer("AbstractWarDeployer");
deployer5.setInputs("JBossWebMetaData");
deployer5.setOutputs("WarDeployment");
addDeployer(main, deployer5);

TestFlowDeployer deployer6 = new TestFlowDeployer("ServiceCL");
deployer6.setInputs("DeploymentType", "WarDeployment");
addDeployer(main, deployer6);

Deployment deployment = createSimpleDeployment("testWSDeploymentOrder");
main.addDeployment(deployment);
main.process();

System.out.println("D1: " + deployer1.getDeployOrder());
System.out.println("D2: " + deployer2.getDeployOrder());
System.out.println("D3: " + deployer3.getDeployOrder());
System.out.println("D4: " + deployer4.getDeployOrder());
System.out.println("D5: " + deployer5.getDeployOrder());
System.out.println("D6: " + deployer6.getDeployOrder());

assertEquals(1, deployer1.getDeployOrder());
assertEquals(2, deployer2.getDeployOrder());
assertEquals(3, deployer3.getDeployOrder());
assertEquals(4, deployer4.getDeployOrder());
assertEquals(5, deployer5.getDeployOrder());
assertEquals(6, deployer6.getDeployOrder());
assertEquals(-1, deployer1.getUndeployOrder());
assertEquals(-1, deployer2.getUndeployOrder());
assertEquals(-1, deployer3.getUndeployOrder());
assertEquals(-1, deployer4.getUndeployOrder());
assertEquals(-1, deployer5.getUndeployOrder());
assertEquals(-1, deployer6.getUndeployOrder());

main.removeDeployment(deployment);
main.process();

assertEquals(1, deployer1.getDeployOrder());
assertEquals(2, deployer2.getDeployOrder());
assertEquals(3, deployer3.getDeployOrder());
assertEquals(4, deployer4.getDeployOrder());
assertEquals(5, deployer5.getDeployOrder());
assertEquals(6, deployer6.getDeployOrder());
assertEquals(12, deployer1.getUndeployOrder());
assertEquals(11, deployer2.getUndeployOrder());
assertEquals(10, deployer3.getUndeployOrder());
assertEquals(9, deployer4.getUndeployOrder());
assertEquals(8, deployer5.getUndeployOrder());
assertEquals(7, deployer6.getUndeployOrder());

main.addDeployment(deployment);
main.process();

assertEquals(13, deployer1.getDeployOrder());
assertEquals(14, deployer2.getDeployOrder());
assertEquals(15, deployer3.getDeployOrder());
assertEquals(16, deployer4.getDeployOrder());
assertEquals(17, deployer5.getDeployOrder());
assertEquals(18, deployer6.getDeployOrder());
assertEquals(12, deployer1.getUndeployOrder());
assertEquals(11, deployer2.getUndeployOrder());
assertEquals(10, deployer3.getUndeployOrder());
assertEquals(9, deployer4.getUndeployOrder());
assertEquals(8, deployer5.getUndeployOrder());
assertEquals(7, deployer6.getUndeployOrder());
}


---
The test output is:
---

...
973 DEBUG [DeployersImpl] Fully Deployed testWSDeploymentOrder
D1: 1
D2: 4
D3: 2
D4: 3
D5: 5
D6: 6
...

[ropalka]

Cross reference: JBDEPLOY-201

[alesj]

I'd expect this test to fail because there's no deployer with "1" output?

Fail?
The order is trivial - 1,2,3,4.

[alesj]

And I'd expect the following test to pass:

Your test is wrong.
#2 takes JBWMD as input, where #3,#4 declare that as output.
So, this is properly ordered: 3,4,2

[alesj]

"alesj" wrote:

#2 takes JBWMD as input, where #3,#4 declare that as output.

Ah, they all declare JBWMD as input and output.
Then the name comparison kicks in - as the last resort on how to order.

[ropalka]

"alesj" wrote:

Then the name comparison kicks in - as the last resort on how to order.

This is fundamentally wrong IMHO.
Current deployers ordering algorithm is broken.
You can't implement deployers ordering by just comparing two deployers.
Two deployers are not comparable in general.

[ropalka]

Current deployers sorting is broken because:
- it doesn't take stages into account
- it doesn't validate inputs
- it sorts deployers on insert, but should sort them after deployers chain initialization
- two deployers are not comparable in general

[alesj]

What do you propose then?

If we have to resort to name comparison it either means:
* the order of how we process attachments doesn't matter
* you are describing/ordering your deployers wrong == lack of info

[ropalka]

In order to fix it the following is necessary to be done:
* deployer stages have to be comparable
* deployers have to be sorted on deployers chain retrieval (i.e. in method protected synchronized List getDeployersList(String stageName))
* sorting agorithm have to take stages, inputs, outputs and relative order into account
* different sorting algorithm have to be implemented (domino is wrong because two deployers are not comparable in general)

[ropalka]

* sorting agorithm have to take stages, inputs, outputs and relative order into account

And deployers name comparison as last ordering decision ;)

[ropalka]

Example of use case that shold fail but won't:

Deployer1:
- stage: PRE_REAL
- inputs: output1
Deployer2:
- stage: REAL
- outputs: output1

PRE_REAL is before REAL. Current domino sorting algorithm will not detect it.

[alesj]

* deployer stages have to be comparable

Why?
That's why they are called stages and they are already comparable - how else do you think they fit into our state machine. ;-)

* deployers have to be sorted on deployers chain retrieval (i.e. in method protected synchronized List<Deployer> getDeployersList(String stageName))

Why?
Based on what?
This would be complete waste of time/performance.

* sorting agorithm have to take stages, inputs, outputs and relative order into account

Why/how stages?
All the rest is already taken into account.

* different sorting algorithm have to be implemented (domino is wrong because two deployers are not comparable in general)

Sure.
But re-read my previous post:

"alesj" wrote:

If we have to resort to name comparison it either means:
* the order of how we process attachments doesn't matter
* you are describing/ordering your deployers wrong == lack of info

[alesj]

- it doesn't validate inputs

Validate how?

Order at runtime based on the actual attachment values?
Too much hassle for something that sounds useless in 99,9%.
Not to mention that it's not deterministic.

- it sorts deployers on insert, but should sort them after deployers chain initialization

What would you gain by this?

[alesj]

Example of use case that shold fail but won't:

Deployer1:
- stage: PRE_REAL
- inputs: output1
Deployer2:
- stage: REAL
- outputs: output1

PRE_REAL is before REAL. Current domino sorting algorithm will not detect it.

I never said creating a deployer is completely trivial.
I mean, the actual impl with all the existing helpers/abstract deployers, is trivial,
but how to properly set stage and inputs/outputs is more difficult task that it first appears.

But that's why we already provide you with Deployers statistics (off by default),
which show you
* deployer inputs/outputs
* deployer times
See DeployersImpl for more details.