I found an easy solution. Since I had already dumped richfaces in favour of Flex RIA, I simply removed all richfaces libraries from my deployed ear. This removed the ajax4jsf filter. This has fixed the streaming behaviour.
This also means that if you are using RichFaces, you are kinda screwed as far as handling large content is concerned. The RichFaces filter will get in the way and try to look for XML in your Blu-Ray quality video.