11 Replies Latest reply on Oct 23, 2016 9:19 PM by rareddy

VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

kulbhushanc Oct 6, 2016 4:20 AM

Hi,

I am observing sometimes VDB got deleted automatically on restart.

I am following approach to restart VDB:

1. Fetch VDBMetaData of the VDB which I want to restart say its name is 'vdbMysql'.

2. Un-deploy the VDB 'vdbMysql'.

3. Redeploy VDBMetaData of 'vdbMysql'.

Sometimes if failed to deploy the VDB, I have pasted server logs. Let me know if you need additional information.

Server log:-

2016-10-06 12:47:19,029 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7)  WFLYSRV0027: Starting deployment of "SvcSourceVdb_MySQL510-vdb.xml" (runtime-name: "SvcSourceVdb_MySQL510-vdb.xml")
2016-10-06 12:47:19,038 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-3)  TEIID40118 VDB SvcSourceVdb_MySQL510.1 added to the repository - is reloading false
2016-10-06 12:47:19,038 INFO  [org.teiid.RUNTIME] (MSC service thread 1-3)  TEIID50029 VDB SvcSourceVdb_MySQL510.1 model "MySQL510" metadata is currently being loaded. Start Time: 6/10/16 12:47 PM
2016-10-06 12:47:19,038 INFO  [org.teiid.RUNTIME] (MSC service thread 1-3)  TEIID50029 VDB SvcSourceVdb_MySQL510.1 model "MySQL510Native" metadata is currently being loaded. Start Time: 6/10/16 12:47 PM
2016-10-06 12:47:19,082 INFO  [org.jboss.as.server] (management-handler-thread - 469)  WFLYSRV0010: Deployed "SvcSourceVdb_MySQL510-vdb.xml" (runtime-name : "SvcSourceVdb_MySQL510-vdb.xml")
2016-10-06 12:47:25,707 WARN  [org.teiid.RUNTIME] (Worker121_async-teiid-threads202)  TEIID50036 VDB SvcSourceVdb_MySQL510.1 model "MySQL510" metadata failed to load. Reason:TEIID11010 com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
2016-10-06 12:47:32,387 WARN  [org.teiid.RUNTIME] (Worker120_async-teiid-threads203)  TEIID50036 VDB SvcSourceVdb_MySQL510.1 model "MySQL510Native" metadata failed to load. Reason:TEIID11010 com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
2016-10-06 12:58:53,479 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-3)  TEIID40120 VDB SvcSourceVdb_MySQL510.1 will be removed from the repository
2016-10-06 12:58:53,479 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-3)  TEIID40119 VDB SvcSourceVdb_MySQL510.1 removed from the repository
2016-10-06 12:58:53,482 INFO  [org.teiid.RUNTIME] (MSC service thread 1-3)  TEIID50026 VDB "SvcSourceVdb_MySQL510.1[MySQL510{MySQL510=MySQL510, mysql5, java:/DSMYSQL20160926165600458}, MySQL510Native{MySQL510Native=MySQL510Native, mysql-native, java:/DSMYSQL20160926165600458}]" undeployed.
2016-10-06 12:58:53,482 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-3)  TEIID40120 VDB SvcSourceVdb_MySQL510.1 will be removed from the repository
2016-10-06 12:58:53,485 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-5)  WFLYSRV0028: Stopped deployment SvcSourceVdb_MySQL510-vdb.xml (runtime-name: SvcSourceVdb_MySQL510-vdb.xml) in 6ms
2016-10-06 12:58:53,492 INFO  [org.jboss.as.server] (management-handler-thread - 472)  WFLYSRV0009: Undeployed "SvcSourceVdb_MySQL510-vdb.xml" (runtime-name: "SvcSourceVdb_MySQL510-vdb.xml")
2016-10-06 12:58:53,498 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1)  WFLYSRV0027: Starting deployment of "SvcSourceVdb_MySQL510-vdb.xml" (runtime-name: "SvcSourceVdb_MySQL510-vdb.xml")
2016-10-06 12:58:53,501 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1)  MSC000001: Failed to start service jboss.deployment.unit."SvcSourceVdb_MySQL510-vdb.xml".PARSE: org.jboss.msc.service.StartException in service jboss.deployment.unit."SvcSourceVdb_MySQL510-vdb.xml".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment "SvcSourceVdb_MySQL510-vdb.xml"
2016-10-06 12:58:53,502 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 474)  WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "SvcSourceVdb_MySQL510-vdb.xml")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"SvcSourceVdb_MySQL510-vdb.xml\".PARSE" => "org.jboss.msc.service.StartException in service jboss.deployment.unit.\"SvcSourceVdb_MySQL510-vdb.xml\".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment \"SvcSourceVdb_MySQL510-vdb.xml\"
2016-10-06 12:58:53,502 ERROR [org.jboss.as.server] (management-handler-thread - 474)  WFLYSRV0021: Deploy of deployment "SvcSourceVdb_MySQL510-vdb.xml" was rolled back with the following failure message:
{"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"SvcSourceVdb_MySQL510-vdb.xml\".PARSE" => "org.jboss.msc.service.StartException in service jboss.deployment.unit.\"SvcSourceVdb_MySQL510-vdb.xml\".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment \"SvcSourceVdb_MySQL510-vdb.xml\"
2016-10-06 12:58:53,502 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1)  WFLYSRV0028: Stopped deployment SvcSourceVdb_MySQL510-vdb.xml (runtime-name: SvcSourceVdb_MySQL510-vdb.xml) in 0ms
WFLYCTL0186:   Services which failed to start:      service jboss.deployment.unit."SvcSourceVdb_MySQL510-vdb.xml".PARSE

Thanks,

Kulbhushan Chaskar.

1. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

shawkins Oct 6, 2016 8:35 AM (in response to kulbhushanc)

When you say redeploy the VDBMetaData, you are converting to xml and using Admin.deploy? How is that conversion performed? It would appear that the xml is invalid, but the error message doesn't seem very helpful.
Actions
2. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

kulbhushanc Oct 6, 2016 10:02 AM (in response to shawkins)

Following are steps I am performing using admin api:-
1.     VDBMetaData vdbMetaData = getVDB(vdbName, 1); //getVDB is the method where we fetch all VDBs and returns the vdb for which we request.
        Note: here vdbName is the previously stored VDB.
2.     I can get deploymentName using the call VdbHelper.getInstance().getVdbDeploymentName(vdbMetaData); //returns the deployment name.
3.     undeploy VDB using above fetched deploymentName
4.     Now I can deploy VDB using clientAccessor.getClient().deploy(deploymentName, new ByteArrayInputStream(vdbBytes));
        Where deploymentName can be any name post-fixes it with -vdb.xml. e.g. "myDeployment-vdb.xml".

In the logs above pasted VDB "SvcSourceVdb_MySQL510" is deleted automatically.

Thanks,
Kulbhushan Chaskar.
Actions
3. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

shawkins Oct 6, 2016 2:00 PM (in response to kulbhushanc)

How is vdbBytes created and can you print the string representation?
Actions

4. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

kulbhushanc Oct 7, 2016 10:20 AM (in response to shawkins)

I am creating vdbBytes using code-line

byte[] vdbBytes = vdbHelper.getVdbByteArray(theVDB);

1. where theVDB is the object of class VDBMetaData.

2. vdbHelper object is created using Code line "VdbHelper vdbHelper = VdbHelper.getInstance();"

VdbHelper is class from package com.bitwise.tdm.teiid.internal.util.VdbHelper

theVDB object String represetation:-

SvcSourceVdb_conn.1[conn{conn=conn, sqlserver, java:/DSSQLSERVER20161006161933573}, connNative{connNative=connNative, sqlserver-native, java:/DSSQLSERVER20161006161933573}]

vdbBytes byte array String representation:-

<?xml version="1.0" encoding="UTF-8"?>
<vdb name="SvcSourceVdb_conn" version="1">
   <description>VDB for: SvcSourceVdb_conn, Version: 1</description>
   <connection-type>BY_VERSION</connection-type>
   <property name="deployment-name" value="SvcSourceVdb_conn-vdb.xml" />
   <model name="conn" type="PHYSICAL" visible="true">
   <property name="importer.tableTypes" value="TABLE" />
   <source name="conn" translator-name="sqlserver" connection-jndi-name="java:/DSSQLSERVER20161006161933573" />
   </model>
   <model name="connNative" type="PHYSICAL" visible="true">
   <source name="connNative" translator-name="sqlserver-native" connection-jndi-name="java:/DSSQLSERVER20161006161933573" />
   </model>
   <translator name="sqlserver-native" type="sqlserver">
   <property name="SupportsDirectQueryProcedure" value="true" />
   </translator>
</vdb>

5. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

rareddy Oct 7, 2016 10:32 AM (in response to kulbhushanc)

When you say "restart" you mean restart of the server? or restartVDB method on the Admin API?
Actions
6. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

kulbhushanc Oct 10, 2016 1:16 AM (in response to rareddy)

I have tried 2 methods to fetch updated data using VDB:-
1.
     1.     Fetch VDBMetaData of the VDB which I want to restart say its name is 'vdbMysql'.
     2.     Un-deploy the VDB 'vdbMysql'.
     3.     Redeploy VDBMetaData of 'vdbMysql'.
2.
     Using restartVDB method on the Admin API.

In both the cases sometimes VDB got deleted automatically.
Actions
7. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

rareddy Oct 10, 2016 9:09 AM (in response to kulbhushanc)

Kulbhushan,

Deploy and Undeploy are asynchronous operations, I suspect the previous task is not completely finished/cleaned before you starting the new deploy. Can you try introducing couple seconds in between undeploy and redeploy and see if that helps?

Ramesh..
Actions
8. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

shawkins Oct 11, 2016 10:40 AM (in response to rareddy)

Since that vdb xml is valid it does seem to point to a different issue. Ramesh, if this is a timing issue - would that be a problem in our handling or a wildfly issue?
Actions
9. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

rareddy Oct 12, 2016 10:03 AM (in response to shawkins)

I am thinking some kind of WildFly issue, where the Service somehow gets into this state and does not recover. Or there are lingering corrupt content which is not cleaned up and the service is always looking at that. If there is way duplicate locally we can give it little bit more through examination.
Actions
10. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

kulbhushanc Oct 19, 2016 5:46 AM (in response to rareddy)

Can you please tell me , is this issue has been fixed in any of the wildFly-teiid server??
Actions
11. Re: VDB got deleted automatically throwing "org.jboss.msc.service.StartException"

rareddy Oct 23, 2016 9:19 PM (in response to kulbhushanc)

I do not think we have not identified the to begin with, so can not confirm about resolution of the issue. We need to be able replicate the issue locally as I mentioned, then only we can provide any solutions.
Actions

Go to original post