if you add 'border-collapse:collapse' to the table then the 'empty-cells:show' works fine.
Thank you zeltner, that is true regarding collapse. But this is still all a workaround. I would like the empty cells to have the boarders regardless if they are empty or not. As they used to be in previous RichFaces versions.