Exception when trying to use Microsoft Office Document Sequencer

[jacobdoran]

Hi,

I'm to create a simple repository for storing and searching microsoft office documents and images but I'm getting a strange error when trying to sequence Microsoft Office Documents. I have the following code to create my repository. Exception handling is stripped:

     
     static final String repositoryName = "repository A";
     static final String repositorySource = "source A";
     
     static final String sourcePath =  "c:\\data\\modeshape\\";

     JcrConfiguration config = new JcrConfiguration();
     config.repositorySource(repositorySource)
          .usingClass(DiskSource.class)
          .setProperty("repositoryRootPath", sourcePath)
          .setProperty("updatesAllowed", true)
          .setDescription("The repository for our content")
          .setProperty("defaultWorkspaceName", workspaceName);
                    
         config.repository(repositoryName)
          .setSource(repositorySource);
         config.textExtractor("Tika Text Extractors")
                            .setDescription("Text extractors using Tika parsers")
                               .usingClass(org.modeshape.extractor.tika.TikaTextExtractor.class)
                              .setProperty("includedMimeTypes", "application/msword,application/vnd.oasis.opendocument.text");
          config.sequencer("Image Sequencer")
                 .usingClass("org.modeshape.sequencer.image.ImageMetadataSequencer")
                 .loadedFromClasspath()
                 .setDescription("Sequences image files to extract the characteristics of the image")
                 .sequencingFrom("//(*.(jpg|jpeg|gif|bmp|pcx|png|iff|ras|pbm|pgm|ppm|psd)[*])/jcr:content[@jcr:data]")
                 .andOutputtingTo("/images/$1");
               
               config.sequencer("Microsoft Office Document Sequencer")
                    .usingClass("org.modeshape.sequencer.msoffice.MSOfficeMetadataSequencer")
                    .loadedFromClasspath()
                    .setDescription("Sequences MS Office documents, including spreadsheets and presentations")
                    .sequencingFrom("//(*.(doc|xls|docx|xlsx)[*])/jcr:content[@jcr:data]")
                    .andOutputtingTo("/msoffice/$1");
               
               engine = config.build();
               engine.start();

Then to store a document I do the following:

          repository = engine.getRepository(repositoryName);               
          Session session = repository.login();
             JcrTools tools = new JcrTools();
             Node node = tools.findOrCreateNode(session, fileName, "nt:folder", "nt:file");
             // Upload the file to that node ...
             Node contentNode = tools.findOrCreateChild(node, "jcr:content", "nt:resource");
             contentNode.setProperty("jcr:lastModified", Calendar.getInstance());             
             
             Binary binaryValue = session.getValueFactory().createBinary(stream);
             contentNode.setProperty("jcr:data", binaryValue);
     
             session.save();

For image files it works pefectly but office documents I get the following error:

org.modeshape.repository.sequencer.SequencerException: java.lang.IllegalArgumentException: The bytes argument may not be null
     at org.modeshape.repository.sequencer.StreamSequencerAdapter.execute(StreamSequencerAdapter.java:198)
     at org.modeshape.repository.sequencer.SequencingService.processChange(SequencingService.java:498)
     at org.modeshape.repository.sequencer.SequencingService$RepositoryObserver$1.run(SequencingService.java:666)
     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
     at java.lang.Thread.run(Thread.java:662)
 

I'm a newbie so probably doing something stupid. I would really appreciate some pointers as to what I'm doing wrong.

Thanks in advance,

Jacob

[rhauch]

Are you sure the "stream" reference is not null? If it were null, then 'binaryValue' would be null and the "jcr:data" value would be set to null (rather, its a single-valued property, so it wouldn't be set), which would thus cause the StreamSequencerAdapter exception.

[jacobdoran]

Hi Randall, thanks very much for your answer. I'm positive the stream is not null. The stream comes from a file upload which works perfectly, I've checked it with the debugger. Also it works fine with uploading images which use a different sequencer but same code. Interestingly I just noticed it works fine with some documents but not others. Perhaps there is a bug in the sequencer where it throws an exception if certain metadata is missing? I will see if I can pindown which documents work and which don't and why. If anyone else can shed some light on this I would appreciate it a lot.

[hchiorean]

Is there any chance you can post one of the office files for which the streaming fails ? It may happen that it's a version/variant not supported by POI (and the error is misleading)

Thanks

[jacobdoran]

Attached is a document that causes the problem. Seems to be ok with word 2007/2010 documents. Thanks for your help.

[hchiorean]

Thanks for the attachment Jacob.

There's indeed a bug in the sequencer code, for which I've logged an issue: https://issues.jboss.org/browse/MODE-1502 that will be fixed in the next 2.x release.