This is really a Hibernate/JPA question, you might get more help from that forum. What I'd expect is happening is that due to your cascade level, refresh is attempting to refresh the database state of your detail entities. Since they don't exist, you get an error.
Perhaps you should enumerate the cascade options you want, and leave out refresh on your detail association.
Or clear the details collection before refreshing if you DO want to cascade on refresh.