You have to do the following:
LineItem item = em.find(LineItem.class, itemId);
item.getOrder().getItems().remove(item);
em.remove(item);
This is because I mapped the one-to-many as an inverse relationship in Hibernate. I' ve changed this to be a non-inverse, so the removing from collection and merging should remove it from the collection.
Bill