There were some different issues with editor scripts loading mechanism which could cause that - so at first please check if the problem exist at latest releases.
Anyway one still open confirmed problem exist [RF issue]. TinyMce that is rich:editor component base hardly depends from servlet directory structure to load it's resources. So usage of <base href="..."/> - could break editor loading.
We do not have plans on resolving tinyMCE issues as changing tiny codebase will make upgrades process almost impossible in future. So that issue should be addressed by tinyMCE team. And for now - avoiding base href definitions usage is the only official workaround.