From the management portlet (page view), I clicked the button "Add instance" without entering anything in the text field. It created a blank portlet window. Since it has no names, there are no links associated with it. As a result, I can't delete it. I think this is a bug.
I created a jira issue for this. Thanks for reporting.