Unfortunatelly tinyMCE used too specific resources loading mechanism which was not really compatible with JSF-RichFaces one. So we made much changes there. Besides - there are css resources correction we additionally applied to it.
So in general it possible but not an easy task. You should checkout component sources, gather all the differences from original tinyMCE and apply to new version. Than integrate the new version to component instead of original. But after that you should also check for any new functionality in tinyMCE which could require additional patches.