Hi,

I'm currently using Teiid to copy billions of rows from different sources to a Microsoft SqlServer  database. The records are related to patient files.

I'm doiing this using a small Java program that uses a JDBC connection to execute the desired statements (copying each time some ten thousands of lines). The statements are virtual procedures who are called in teiid. So I assume that when a procedure ends and the statement is closed, all related resources like temporary tables etc. are released. The program closes statements for each command and connections when a patient is complete.

After a lot of data is transfered (approximately 300 patient files or about 100 million lines) I sometimes get errors like this one:

2018-06-20 13:27:29,876 ERROR [org.teiid.BUFFER_MGR] (Worker16_QueryProcessorQueue1390153) 9FtM+9TQiuKE TEIID30002 Error persisting, attempts to read 4,805,248 992,126 later will result in an exception. : java.lang.AssertionError: Could not free space for pending write

        at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.evictFromMemoryBuffer(BufferFrontedFileStoreCache.java:1164)

        at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache$InodeBlockManager.nextBlock(BufferFrontedFileStoreCache.java:246)

        at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache$InodeBlockManager.getOrUpdateDataBlockIndex(BufferFrontedFileStoreCache.java:201)

        at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache$InodeBlockManager.allocateBlock(BufferFrontedFileStoreCache.java:348)

        at org.teiid.common.buffer.impl.BlockOutputStream.newBuffer(BlockOutputStream.java:51)

        at org.teiid.common.buffer.impl.ExtensibleBufferedOutputStream.ensureBuffer(ExtensibleBufferedOutputStream.java:51)

        at org.teiid.common.buffer.impl.ExtensibleBufferedOutputStream.write(ExtensibleBufferedOutputStream.java:57)

        at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)

        at java.io.ObjectOutputStream$BlockDataOutputStream.write(ObjectOutputStream.java:1808)

        at java.io.DataOutputStream.writeInt(DataOutputStream.java:200)

        at java.io.ObjectOutputStream$BlockDataOutputStream.writeInt(ObjectOutputStream.java:1943)

        at java.io.ObjectOutputStream.writeInt(ObjectOutputStream.java:797)

        at org.teiid.client.BatchSerializer$IntColumnSerializer.writeObject(BatchSerializer.java:559)

        at org.teiid.client.BatchSerializer$ColumnSerializer.writeColumn(BatchSerializer.java:530)

        at org.teiid.client.BatchSerializer.writeBatch(BatchSerializer.java:863)

        at org.teiid.client.BatchSerializer.writeBatch(BatchSerializer.java:839)

        at org.teiid.common.buffer.impl.BufferManagerImpl$BatchManagerImpl.serialize(BufferManagerImpl.java:332)

        at org.teiid.common.buffer.impl.BufferManagerImpl$BatchManagerImpl.serialize(BufferManagerImpl.java:187)

        at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.add(BufferFrontedFileStoreCache.java:664)

        at org.teiid.common.buffer.impl.BufferManagerImpl.evict(BufferManagerImpl.java:1034)

        at org.teiid.common.buffer.impl.BufferManagerImpl.doEvictions(BufferManagerImpl.java:997)

        at org.teiid.common.buffer.impl.BufferManagerImpl.persistBatchReferences(BufferManagerImpl.java:955)

        at org.teiid.common.buffer.impl.BufferManagerImpl.addMemoryEntry(BufferManagerImpl.java:1144)

        at org.teiid.common.buffer.impl.BufferManagerImpl.fastGet(BufferManagerImpl.java:1104)

        at org.teiid.common.buffer.impl.BufferManagerImpl$BatchManagerImpl.getBatch(BufferManagerImpl.java:359)

        at org.teiid.common.buffer.SPage.getValues(SPage.java:234)

        at org.teiid.common.buffer.SPage.search(SPage.java:135)

        at org.teiid.common.buffer.STree.find(STree.java:247)

        at org.teiid.common.buffer.STree.insert(STree.java:295)

        at org.teiid.query.processor.relational.EnhancedSortMergeJoinStrategy.createIndex(EnhancedSortMergeJoinStrategy.java:197)

        at org.teiid.query.processor.relational.EnhancedSortMergeJoinStrategy.loadRight(EnhancedSortMergeJoinStrategy.java:308)

        at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:257)

        at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)

        at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)

        at org.teiid.query.processor.BatchCollector$BatchProducerTupleSource.nextTuple(BatchCollector.java:90)

        at org.teiid.query.processor.relational.WindowFunctionProjectNode$1.nextTuple(WindowFunctionProjectNode.java:478)

        at org.teiid.query.processor.relational.WindowFunctionProjectNode.saveInput(WindowFunctionProjectNode.java:500)

        at org.teiid.query.processor.relational.WindowFunctionProjectNode.nextBatchDirect(WindowFunctionProjectNode.java:221)

        at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)

        at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)

        at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)

        at org.teiid.query.processor.relational.UnionAllNode.nextBatchDirectInternal(UnionAllNode.java:165)

        at org.teiid.query.processor.relational.UnionAllNode.nextBatchDirect(UnionAllNode.java:138)

        at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)

        at org.teiid.query.processor.relational.SelectNode.nextBatchDirect(SelectNode.java:100)

        at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)

....

This "space" the process is talking about, is this disk space? Or is it somekind of reserved space (e.g. buffer size) and can it be expanded?