Using ASYNC mode should have very minimal implications to the response time. Seems like you are using an older version of the AS where there were some known issues. I recommend upgrading to latest AS 7.2.0 (build yourself). You can also remove <file-store/> if you are not using it.
i removed the <file-store/> but the behavior still the same.
What version are you using? You might be hitting some long forgotten bug present in older versions. Could you try with 7.2.0 release?
(you will need to build yourself, see easy building instructions https://github.com/wildfly/wildfly/tree/7.2.0.Final )