ModeShape 4.1 and disk space
I've just started using ModeShape, but I've run into some issues. Essentially, I'm building a tree with approximately 136000 nodes in the entire tree structure. The data is originating from a relational database, but I'm trying to remove that datastore from the equation and use ModeShape as my source of record. The tree consists of 48 nodes at the root level of the tree, with the distribution as follows:
1: 6
2: 156
3: 117
4: 7
5: 8
6: 77
7: 28
8: 5
9: 16
10: 3
11: 8
12: 5
13: 2678
14: 8103
15: 2151
16: 96
17: 70
18: 103
19: 3
20: 9
21: 253
22: 30
23: 3
24: 5
25: 134
26: 10
27: 6
28: 3371
29: 331
30: 3
31: 9
32: 26
33: 10
34: 4
35: 4
36: 2652
37: 2726
38: 58
39: 6
40: 5
41: 81
42: 2051
43: 12
44: 5
45: 22
46: 110215
47: 3
48: 373
As I add new nodes into the structure, I'm noticing the Infinispan file size grow out of control.
| Record count | File size |
|---|---|
| 5000 | 250MB |
| 10000 | 761MB |
| 15000 | 1.5GB |
| 20000 | 2.0 GB |
| 40000 | 13 GB |
At that point, performance starts to visibly degrade, but given the size of the objects I'm trying to store I wouldn't ever think the file size could be that large. The raw data file size I have from an SQL query is only 7.5 kb.
Here's the code I'm using to build the nodes in my workspace:
@Named
@RequestScoped
public class Service {
@Inject
@Repository // Internal Qualifier
private Session session;
@Inject
private Logger log;
public void addComponent(final int record, final Component component) {
try {
/*
Component Object Structure consists of a component name, a type, and a key value
"Component" / "Component Type" / 1 / Properties:["C";"A";3;"Catalog"...]
*/
Node rootNode = session.getNode("/");
Node childNode = addNode(rootNode, component.getNodeLabel());
childNode = addNode(childNode, component.getComponentType());
childNode = addNode(childNode, component.getComponentKey() + "");
childNode.setProperty("child-type", component.getComponentChildType());
childNode.setProperty("parent-type", component.getComponentParentType());
childNode.setProperty("level", component.getComponentLevel());
childNode.setProperty("priority", component.getComponentPriorityOrder());
childNode.setProperty("source-primary-key", component.getSourcePrimaryKey());
session.save();
log.info("Added component #%d: %s",record , component.getComponentType());
} catch (RepositoryException e) {
log.error("Caught a repository exception trying to add component: %s [%s]",
component, e.getMessage());
}
}
private Node addNode(final Node rootNode, final String path) throws RepositoryException {
Node childNode = null;
try {
childNode = rootNode.getNode(path);
} catch (PathNotFoundException ex) {
log.debug("Need to add path for %s", path);
}
if (childNode == null) {
childNode = rootNode.addNode(path);
}
return childNode;
}
}
Here is my configuration in WildFly:
<subsystem xmlns="urn:jboss:domain:infinispan:2.0">
<cache-container name="modeshape" default-cache="sample" module="org.modeshape">
<local-cache name="sample">
<locking isolation="READ_COMMITTED"/>
<transaction mode="NON_XA" locking="PESSIMISTIC"/>
<eviction strategy="LRU" max-entries="10000"/>
<file-store shared="true" passivation="false" purge="false" path="modeshape/store/sample"/>
</local-cache>
</cache-container>
</subsystem>
<subsystem xmlns="urn:jboss:domain:modeshape:2.0">
<repository name="sample" anonymous-roles="admin">
<workspaces>
<workspace name="default"/>
<workspace name="other"/>
</workspaces>
</repository>
</subsystem>
Environment:
WildFly 8.2.0.Final with ModeShape 4.1
Centos 6.5
Java SE Runtime (HotSpot 1.7.0_55)
