you're basically limited to implement https://github.com/arquillian/arquillian-core/blob/master/container/spi/src/main/java/org/jboss/arquillian/container/spi/client/container/DeployableContainer.java interface in Arquillian Undertow container. I don't see too many common stuff in UnderTow deployment representation https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java/io/undertow/servlet/api/DeploymentInfo.java and ShrinkWrap Archive https://github.com/shrinkwrap/shrinkwrap/blob/master/api/src/main/java/org/jboss/shrinkwrap/api/Archive.java . Eventually, you could also implement UndertowDeploymentDescriptor as https://github.com/shrinkwrap/descriptors/blob/master/api-base/src/main/java/org/jboss/shrinkwrap/descriptor/api/Descriptor.java but that would metadata update difficult.
I'd go by implementing Assignable interface and require user to cast it to Archive<?> by hand. So it would look like following:
This approach would allow you to reuse Undertow builders and will not directly expose Archive<?> methods to user, so you can safely stub them with UnsupportedOperationException. I'm using similar way within MavenImporter in ShrinkWrap Resolvers - https://github.com/shrinkwrap/resolver/blob/master/impl-maven-archive/src/main/java/org/jboss/shrinkwrap/resolver/impl/maven/archive/importer/MavenImporterImpl.java. Other option I can think of is implementing an UndertowAsset and make it a root resource but this will be awkward to use imho.
Another open question is whether you actually want to support deploying JavaArchive/WebArchive/EnterpriseArchive to Undertow container.
My original idea was to use WebArchive for Servlets, Filters, well all DeploymentInfo artifacts, and use JarArchive for Handlers. And maybe I will implement as your suggestion to avoid users to use invalid methods.