5 Replies Latest reply on Aug 31, 2016 10:54 AM by cfang

jdbcItemReader with POJO beanType case problem

richardmoore Aug 29, 2016 10:11 AM

I have a DB2 jdbcItemReader and jacksonCsvItemWriter chunk step that is getting -

2016-08-29 09:03:43.940 [jberet-1 ] jberet - [ERROR] ProcessingInfo{count=0, timerExpired=false, itemState=RUNNING, chunkState=RUNNING, checkpointPosition=-1, readPosition=0, failurePoint=null}

2016-08-29 09:03:43.976 [jberet-1 ] jberet - [WARN ] JBERET000028: Failed to serialize exception: java.lang.IllegalArgumentException: Unrecognized field "CAT_CODE" (class com.awginc.jberet.examples.RPEItem), not marked as ignorable (30 known properties: "retail_DEPT", "awg_CREATE_DATE", "whse_CODE_DESC", "brand", "whse_CODE", "shipper_FLAG", "retail_UPC", "vvd_DESC", "item_CD", "whi_CPY_CD_2", "loq", "itm_DESC", "retail_DEPT_DESC", "discnt_DATE", "itm_PLCD", "cat_DESC", "subcat_DESC", "whi_CPY_CD", "sub_CATEGORY", "ordr_BOOK_INDX_CD", "facl", "desc_SHORT", "ibu_MPCK", "itm_STRATEGY", "cat_CODE", "store_PACK", "case_UPC", "brand_2", "brand_TYPE_DESC", "itm_SIZE"])

at [Source: N/A; line: -1, column: -1] (through reference chain: com.awginc.jberet.examples.RPEItem["CAT_CODE"])

As indicated in the error message, my POJO field is declared as - 'private String CAT_CODE;'

I have tried all lowercase - cat_code, and - cat_CODE and it still fails here. What am I doing wrong? Could this be due to DB2 returning the column names as all uppercase?

1. Re: jdbcItemReader with POJO beanType case problem

cfang Aug 29, 2016 10:45 AM (in response to richardmoore)

is this error from jdbcItemReader? Have you configured columnMapping batch property?

JdbcItemReader (jberet-support 1.3.0.Beta2 API)

columnMapping property maps the column names in sql statement to POJO fields.

YOu can also experiment with various jackson annotations for mapping data columns to pojo fields.
Actions
2. Re: jdbcItemReader with POJO beanType case problem

richardmoore Aug 29, 2016 1:41 PM (in response to cfang)

I added the columnMapping but still having problems. Below is what I have, all of the setters are showing in the log as being called but then fails on what appears to be the writer (I am only pulling one row from my table).

Do you have an example you can point me to on github?

Step -

<chunk>
<reader ref="jdbcItemReader">
<properties>
<property name="dataSourceLookup" value="db2"/>
<property name="sql" value="..." />
<property name="beanType" value="com.awginc.jberet.examples.RPEItem" />
<property name="columnMapping" value="
...,
STORE_PACK,
LOQ,
AWG_CREATE_DATE,
DISCNT_DATE,
ITM_STRATEGY,
VVD_DESC,
ORDR_BOOK_INDX_CD
"/>
</properties>
</reader>
<writer ref="jacksonCsvItemWriter">
<properties>
<property name="resource" value="#{jobProperties['deliveryFilename']}" />
<property name="writeMode" value="overwrite" />
<property name="columnSeparator" value="|" />
<property name="beanType" value="com.awginc.jberet.examples.RPEItem" />
<property name="useHeader" value="false" />
<property name="csvGeneratorFeatures" value="ALWAYS_QUOTE_STRINGS=false, STRICT_CHECK_FOR_QUOTING=true"/>
<property name="columns" value="
...,
STORE_PACK STRING,
LOQ STRING,
AWG_CREATE_DATE STRING,
DISCNT_DATE STRING,
ITM_STRATEGY STRING,
VVD_DESC STRING,
ORDR_BOOK_INDX_CD STRING
"/>
</properties>
</writer>
</chunk>

POJO snippet -

@JsonGetter("ORDR_BOOK_INDX_CD")
public String getORDR_BOOK_INDX_CD() {
log.info("get ordr_book_indx_cd");
return ORDR_BOOK_INDX_CD;
}

@JsonSetter("ORDR_BOOK_INDX_CD")
public void setORDR_BOOK_INDX_CD(String oRDR_BOOK_INDX_CD) {
log.info("set ordr_book_indx_cd");
ORDR_BOOK_INDX_CD = oRDR_BOOK_INDX_CD;
}

Exception -

2016-08-29 12:31:39.751 [jberet-1    ]                         jberet - [ERROR] ProcessingInfo{count=0, timerExpired=false, itemState=RUNNING, chunkState=RUNNING, checkpointPosition=-1, readPosition=0, failurePoint=null}
2016-08-29 12:31:39.762 [jberet-1    ]                         jberet - [ERROR] item-count=10, time-limit=0, skip-limit=-1, skipCount=0, retry-limit=-1, retryCount=0
2016-08-29 12:31:39.772 [jberet-1    ]                         jberet - [ERROR] JBERET000007: Failed to run job RPEItemIntegration, createFile, org.jberet.job.model.Chunk@3c57f41a
java.lang.NullPointerException
at org.jberet.support.io.ItemReaderWriterBase.validate(ItemReaderWriterBase.java:92) ~[jberet-support-1.3.0.Beta2-20160808.jar:1.3.0.Beta2]
at org.jberet.support.io.JdbcItemReader.readItem(JdbcItemReader.java:298) ~[jberet-support-1.3.0.Beta2-20160808.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.ChunkRunner.readItem(ChunkRunner.java:359) ~[jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.ChunkRunner.readProcessWriteItems(ChunkRunner.java:305) ~[jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.ChunkRunner.run(ChunkRunner.java:201) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:226) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:147) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:164) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.CompositeExecutionRunner.runJobElement(CompositeExecutionRunner.java:128) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:203) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:164) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:88) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:60) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at org.jberet.spi.JobExecutor$1.run(JobExecutor.java:99) [jberet-core-1.3.0.Beta2.jar:1.3.0.Beta2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
Actions
3. Re: jdbcItemReader with POJO beanType case problem

cfang Aug 30, 2016 8:04 PM (in response to richardmoore)

I'll look into it in more details. I was off work the last couple of days.
Actions
4. Re: jdbcItemReader with POJO beanType case problem

cfang Aug 30, 2016 10:12 PM (in response to cfang)

from the line where the NPE occurred, it was invoking bean validator. You can configure the reader to skip bean validation by setting skipBeanValidation batch property to true on jdbcItemReader.

The default bean validator should be available if running inside a EE 7 environment. Not sure why it got NPE. But if you don't need bean validation, then just disable it with skipBeanValidation batch property.
Actions
5. Re: jdbcItemReader with POJO beanType case problem

cfang Aug 31, 2016 10:54 AM (in response to richardmoore)

Even in standalone environment, the validator should still be available as long as you have a bean validation provider in the classpath, e.g., hibernate bean validator.

Even if the provider jar is missing, you will get errors like this:

javax.validation.ValidationException: Unable to create a Configuration, because no Bean Validation provider could be found. Add a provider like Hibernate Validator (RI) to your classpath.

For example app, you can look at the test app: jsr352/JdbcReaderWriterTest.java at master · jberet/jsr352 · GitHub

and its associated job.xml: jsr352/org.jberet.support.io.JdbcReaderTest.xml at master · jberet/jsr352 · GitHub

These tests read data from csv, writes to db; and then use jdbcItemReader to read from db.

In the tests, jdbcItemReader is configured to read data in as either POJO, List, or Map. The pojo class is jsr352/StockTrade.java at master · jberet/jsr352 · GitHub
1 of 1 people found this helpful
Actions

Go to original post

	@JsonGetter("ORDR_BOOK_INDX_CD")
	public String getORDR_BOOK_INDX_CD() {
		log.info("get ordr_book_indx_cd");
		return ORDR_BOOK_INDX_CD;
	}

	@JsonSetter("ORDR_BOOK_INDX_CD")
	public void setORDR_BOOK_INDX_CD(String oRDR_BOOK_INDX_CD) {
		log.info("set ordr_book_indx_cd");
		ORDR_BOOK_INDX_CD = oRDR_BOOK_INDX_CD;
	}