From my understanding the query (except findByPKey) will need to use the database (EJB specification).
Only load the entity by primary key will go to the cache.
Could you enable TRACE to see whether the query did not reach the DB direct?
I agree to what was said before.
But the server has to make sure that all cached objects are in sync with the database before the query is executed against the database.
But this can be influenced, have a look at http://docs.oracle.com/javaee/6/api/javax/persistence/Query.html#getFlushMode().