The latency of your database connection certainly contributes to the latency of the JPA connector operations (e.g., a remote DB will be slower than a local DB, and a DB on a slower remote machine will be slower than a DB on a remote faster machine). But that's no different than any other database client application.
We are currently working on performance and have introduce several optimizations around startup and initialization, but haven't yet made andy changes that would improve the JPA connector performance. Thus 2.5.0.Beta1 will have some improvements, but probably not yet related to what you're seeing. So please stay tuned.
Finally, generally speaking all JCR repositories do better with a repository structure that is not flat. After all, JCR is inherently hierarchical, and your repository design should take that into account and, better yet, leverage that characteristic.