4 Replies Latest reply on Aug 3, 2017 10:09 AM by ketanmistry

    HAWKAPM600000: Maximum retry reached.

    ketanmistry

      Installed Hawkular APM 0.14.3 distribution on RHEL and started standalone server.

       

      Pointed my Zipkin instrumentation to it and generated some load - nothing. I set the standalone server in debug mode and can see this error:

       

      2017-05-11 12:09:09,647 WARN  [org.hawkular.apm.server.api.log] (Thread-189 (ActiveMQ-client-global-threads--1391354641)) HAWKAPM600000: Maximum retry reached. Last exception to occur ....: org.hawkular.apm.server.api.task.RetryAttemptException: URL is null, span id = 845E8F893976CA60

        at org.hawkular.apm.server.processor.zipkin.NodeDetailsDeriver.processOneToOne(NodeDetailsDeriver.java:69)

        at org.hawkular.apm.server.processor.zipkin.NodeDetailsDeriver.processOneToOne(NodeDetailsDeriver.java:39)

        at org.hawkular.apm.server.api.task.ProcessingUnit.handle(ProcessingUnit.java:146)

        at org.hawkular.apm.server.jms.RetryCapableMDB.process(RetryCapableMDB.java:191)

        at org.hawkular.apm.server.jms.RetryCapableMDB.onMessage(RetryCapableMDB.java:154)

        at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

        at java.lang.reflect.Method.invoke(Method.java:508)

        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)

        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82)

        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)

        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)

        at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)

        at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

        at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:254)

        at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:329)

        at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:53)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponentDescription$5$1.processInvocation(MessageDrivenComponentDescription.java:239)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)

        at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)

        at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)

        at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

        at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)

        at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)

        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

        at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)

        at org.hawkular.apm.server.jms.span.NodeDetailsDeriverMDB$$$view6.onMessage(Unknown Source)

        at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

        at java.lang.reflect.Method.invoke(Method.java:508)

        at org.jboss.as.ejb3.inflow.MessageEndpointInvocationHandler.doInvoke(MessageEndpointInvocationHandler.java:139)

        at org.jboss.as.ejb3.inflow.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:73)

        at org.hawkular.apm.server.jms.span.NodeDetailsDeriverMDB$$$endpoint1.onMessage(Unknown Source)

        at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage(ActiveMQMessageHandler.java:310)

        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:932)

        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)

        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1045)

        at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

        at java.lang.Thread.run(Thread.java:785)

       

      Any ideas?