yes, you are missing something, but not your fault...
EJBContext.lookup looks in the ENC of the EJB it is not a global JNDI lookup.
Have you bound the EJB within an ejb-ref? If not then you need to do a straight JNDI lookup of your EJB ref:
Good! Indeed, using a new InitialContext() solves the problem.
This is apparently not the case in JBoss versus the embedded version. How is it different?