The reason for entrySet only showing the local available entries is for efficiency. In distribution, you could have several gigs spread in between diff nodes, you would not want to pull all those entries to a node where entrySet is called since it'd be slow and could lead to OOME.
Not sure how you're loading things, but preloading in particular happens in a local way, so instances are not distributed when preloaded. If you load them lazily, I think they'd spread.
Loading them lazily is definitely the better option.