If you set the min and max heap sizes to different values, then when the JVM requires more heap space it will get it from the os, after which the JVM rarely lets go of that memory, even if the space required for those objects (the byte array in this case) is cleaned up during a garbage collection. Thus the behavior you are seeing in task manager does not indicate a memory leak - it is just how Java works.

To determine if a memory leak was taking place you would have to monitor the objects in the heap, either by monitoring the garbage collector (this lets you know if there is a possible leak but not where), or by using a tool like VisualVM which lets you take and compare heap dumps.