You have to do the following:

 LineItem item = em.find(LineItem.class, itemId);
 item.getOrder().getItems().remove(item);
 em.remove(item);

This is because I mapped the one-to-many as an inverse relationship in Hibernate. I' ve changed this to be a non-inverse, so the removing from collection and merging should remove it from the collection.

Bill