BeanIO Writer arraylist of objects

[richardmoore]

Before I give up on this I wanted to ask about being able to use the chunk for the following scenario -

I have a case where we get a list of keys in a reader. From there the processor looks up rows in a database and checks the returned rows for instances where certain indicators determine if any of the returned rows need written or not. Can you pass a list of records to the BeanIO writer? If so, how would you write the beanio map?

Input has -

1. key1
2. key2

Processing -

• key1 look up returns 5 rows, after applying the business logic against all 5 rows and the values in each one it is determined that 4 of them need written out. Add the four to a list and pass the list to the writer.
• key2 look up returns 3 rows, after applying the business logic it is determined that nothing needs written out so pass null to writer.

[cfang]

If we forget about BeanIOItemWriter for a moment, the unusual part of your case is that the item processor may produce multiple items from one single reader item.  Any item processor can already filter out reader items (in your case, the reader item key2 will be filtered out and will not have any items out of the item processor).  The mismatch is that the item processor API returns one single item, which will be accummulated by JBeret batch runtime.  If your item processor returns a List<Record>, JBeret will think the type of items is List<Record>, while you intend it to be Record.

The other option is to keep your intelligence inside item reader, such that the item reader still returns a single item, pass it to the optional item processor (your may not need the item processor any more).  The item reader may need to cache the multiple spawned items and return them one at a time.  This will fit into the typical chunk processing pattern.

[richardmoore]

The problem is that I can't select just what I need off the tables, a part of the business logic is to do some field compares in order to determine which records need to be written for this customer. My plan was to select off all the keys needed, then let the item processor get all the rows involved for the given key, compare and apply the business logic to determine which if any of the records need written then pass a List of records.

I achieved this by creating a new writer from the BeanIOItemWriter and modifying the following method -

@Override

  public void writeItems(final List<Object> items) throws Exception {

  for (final Object e : items) {

  System.out.println(e.getClass().getName() + " -> " + e);

  if (e instanceof Collection) {

  for (Object o : ((Collection<?>) e)) {

  beanWriter.write(o);

  }

  } else {

  beanWriter.write(e);

  }

  }

  }

[cfang]

Good to know you found the solution for your case.  Just adding a note here that in other cases, exploding a collection into individual items may not be the right choice, because a collection may indeed form a single item.  For instance, a list of strings forms multiple fields in a CSV row: {"John", "Smith", "M", "30"}.

[cfang]

You can also attach a ItemWriteListener to the chunk processing, and inside its beforeWrite(List<Object>) method expand and replace those special container collections.

ItemWriteListener (Java(TM) EE 7 Specification APIs)

This can shield this type of special handling from the item writer class.

[richardmoore]

I like the ItemWriteListener idea. I set one up and have a question.

1. Reads input, which contains 2 records.
2. The item processor takes each input records and performs additional business logic to create a List of records to be written. The first input record becomes a List of 4 records, the second becomes a List of 2 records.
3. The write listener (below) converts the container/list and the writer writes a total of 6 records, as expected.
4. The step listener display the metrics but it only shows a WRITE_COUNT of 2, I was expecting 6.

Is there something I can do to ensure the write count matches the file's record count?

Step listener's afterStep() contains the following -

for (Metric metric : stepContext.getMetrics()) {

   log.info("Metric " + metric.getType() + "=" + metric.getValue());

}

import java.util.ArrayList;

import java.util.Collection;

import java.util.List;

import javax.batch.api.chunk.listener.ItemWriteListener;

public class CollectionTestWriteListener implements ItemWriteListener {

  @Override

  public void afterWrite(List<Object> arg0) throws Exception {

  }

  @Override

  public void beforeWrite(List<Object> arg0) throws Exception {

  List<Object> list = new ArrayList<Object>();

  for (Object o : arg0) {

  if (o instanceof Collection) {

  list.addAll((Collection<?>) o);

  } else {

  list.add(o);

  }

  }

  arg0.clear();

  arg0.addAll(list);

  }

  @Override

  public void onWriteError(List<Object> arg0, Exception arg1) throws Exception {

  }

}

[cfang]

step metrics should correctly reflect the number of items to be written.

do you cache step metrics or the like inside your step listener class?

[cfang]

1. JBERET-351

Step metrics WRITE_COUNT does not correctly reflect the number of items to be written

[richardmoore]

I do cache any metrics but rely on the following -

Step listener's afterStep() contains the following -

for (Metric metric : stepContext.getMetrics()) {

   log.info("Metric " + metric.getType() + "=" + metric.getValue());

}

[cfang]

Your code looks right.  The above JIRA issue (JBERET-351) will be fixed soon to address this problem.