Wildfly async logger handler help

[dannyruchman]

Hi,

We have an app that writes a lot of logs to the file system using async handler.

the async handler has a couple of interesting parameters:

1. Queue Length

2. Overflow Action

We could get much better performance results using Overflow Action=DISCARD.

As we understand it drops calls to the log writer handler if it's queue is full.

As we do not want to loose a lot of data, we tried increasing the size of the Queue Length and use BLOCK in Overflow Action value.

We could not get any better results.

can anyone help understanding those values and tune them for best performance without loosing a lot of data?

we have a 32GB RAM and 24 cores DPU. Memory is not an issue but we do have some issues of CPU when working with Overflow Action=BLOCK

Any help will do

Thanks!

[jamezp]

At this point the best you could do is have a high queue-length number. Setting the overflow-action to DISCARD you will definitely lose messages. Setting to BLOCK it will block synchronously until there is room on the queue to add the message.

--

James R. Perkins

[wdfink]

Is it possible that the filesystem where you write is slow? I saw people using shared filesystems which is not a good idea.

Maybe you use a fast (small) filesystem, log-file rotate and move/compress the old logs to a different store.

[dannyruchman]

It is not a shared directory. I don't think the hardware is the problem here too. We have parallel hardware servers that run php code that does more or less the same which does not have this issue.

[dannyruchman]

Hi James,

We would like to set this parameter to BLOCK to make sure data is not lost.

In our use case, those log files are the system output and we write a lot of them.

What is the Queue length parameter units? Is it bytes? kilo bytes?

How high can we set it and what can be the affect of high value?

We have 32GB RAM on our servers and java process is limited to 20 GB while never get full. We do not see full GC's never which means we do not have any memory allocation issues.

Can we give this Queue length parameter let's say - 1GB definition?

Thanks for your reply

Danny

[jamezp]

The queue-length is the number of messages to store before it blocks or discards messages. If you set the queue-length to 100 then it will store up to 100 messages before it either blocks or discards messages.

--

James R. Perkins

[dannyruchman]

Thank you