I would like this functionality as well. I have used other Portal vendors in the past, they had this functionality by setting a custom mode called SOLO.
I am also looking for similar functionality wherein user can see the preview of portlet before adding it to page.
Here is a way you can achieve opening a portlet in a popup window.
1. Link the Portlet to a Page.
2. Define a Page Property say WindowType and give it a value say "NewWindow".
3. In Tabs.jsp while rendering the link for this page (identified based on the WindowType page property) use target = "_new".
This should help you in achieving the objective.