The validators are added to the component during view creation, i.e. the first GET request. Afterwards it is the job of the JSF state saving mechanism to preserve the state of the component tree and its attached resources, like validators. When the view is restored, new objects are created to represent the components and JSF calls their restoreState() method. In your case, the TextComponent also creates its validator. Then the state restore mechanism takes place, adding the previous instance(s) of the validator. So on the first POST there are two validators, etc.
I'd suggest one of the following:
1) Attach the validator in the normal way, even if the component and/or validator are custom:
<my:text ...> <my:validator ... /> </my:text>
2) Create a custom JSP/Facelets tag handler that creates a component and attaches the validator at the proper time (I believe the apply() method of the com.sun.facelets.FaceletHandler, but check docs/code to be sure).