[sebersole]

This is misunderstanding number 1.

The persistence is not a live source of data. It is a recovery log in the event of a crash.
Selectors are run against the objects in memory not the database.

That was in response to a JIRA issue that I thought I saw (either one of your links or linked to from one of you links) requesting that selectors be applied in the database. My bad...

Even still though, makes it harder to recover *efficiently* when you gotta pull full BLOBs from the database...

[adrian.brock]

Even still though, makes it harder to recover *efficiently* when you gotta pull full BLOBs from the database...

Yes, the idea was to put (selected) headers in a separate column such that when
reloading from the db you don't have to deserialize the whole blob (message body).

[adrian.brock]

So after reading some of this background, it sounds like JBossMQ (messaging module) is basically in maintenance/bug-fix mode. So I'll leave it up to y'all: given that fact, what (if any) work makes sense here in relation to Hibernate and this module?

1) nothing
2) new Hibernate-based PM
3) replace existing PMs with a new Hibernate-based PM
4) other?

For JBossMQ, option (2). The old jdbc2 still needs to work for people with existing
configuration/data.

[sebersole]

OK, so then back to my original question :)

Should I just "copy" the schema used by the jdbc3 stuff? Or do you want to look at the "more efficient recovery"?

[adrian.brock]

jdbc3 is experimental and not supported.

I don't like the way jdbc2/3 uses a char[] for the destination
there is a thread somewhere where I said it would be better to create a separate
JMS_DESTINATIONS table
id -> destination string
with a cache inside the persistence manager

So I'd say do whatever you think is best. You are creating an independent
implementation of the interfaces.

[adrian.brock]

So I'd say do whatever you think is best. You are creating an independent
implementation of the interfaces.

I'd also say why bother. We are putting our
development resources into the new jms NOT JBossMQ.

[sebersole]

I'd also say why bother. We are putting our
development resources into the new jms NOT JBossMQ.

Fine with me, I'll gear towards the new codebase then.

I was under the impression that y'all wanted to get out of maintaining the database-specific scripts. Other than that I completely agree with you that it does not make much sense to add new capabilities to something in bug-fix mode...

[starksm64]

Yes, the new codebase should be the target. Once that is done someone is free to look at porting it to 4.0 to get rid of the db script maintence issue.

[ovidiu.feodorov]

2) At how high of a priority should I place inter-operability with the current schema used by the JDBC-based state/persistence-managers? Should it strictly use the same schema already in use by the JDBC counterparts; or is it ok to do what seems reasonable here if that differs? Two things in particular from my initial look:

a. I noticed that the JMS_MESSAGE is using a composite primary key; I?d prefer to use surrogate keys.

b. Based on the definition of the JMS_MESSAGE table, I would assume that the entire javax.jms.Message gets serialized into the MESSAGEBLOB column. Based on that assumption, I would further assume then that **all** messages need to get loaded from the database in order to apply message selectors. It should be possible to store the parts of the message individually; is it ok to explore that path? Or do the internals already assume that the message selectors get applied to
in-memory instances?

We've already using a different schema. Serializing the whole message and storing it as a BLOB is not efficient enough. Message headers should be stored separately. We want to be able to push message selectors to the database, for example. For more on this discussion, see http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3891897

The schema is not settled down yet, so your input is welcome. A good start point to see the current state of affairs is jms/src/main/org/jboss/messaging/core/persistence/HSQLDBPersistenceManager.java

[ovidiu.feodorov]

FYI - (jira issues + related threads)
http://jira.jboss.com/jira/browse/JBAS-1348
http://jira.jboss.com/jira/browse/JBAS-1341
http://jira.jboss.com/jira/browse/JBAS-1336

Adrian, how about transferring these to Messaging? It would be easier for the Messaging team to manage them.

[adrian.brock]

FYI - (jira issues + related threads)
http://jira.jboss.com/jira/browse/JBAS-1348
http://jira.jboss.com/jira/browse/JBAS-1341
http://jira.jboss.com/jira/browse/JBAS-1336

Adrian, how about transferring these to Messaging? It would be easier for the Messaging team to manage them.

They are still JBossMQ issues, so no.
I would hope that at least some of them are non-issues
from the start in the new messaging because they have been thought about up-front.

[sebersole]

Yes, I've started looking at jms\src\main\org\jboss\messaging\core\persistence\HSQLDBPersistenceManager.java to get an idea of the schema. So here's what I've gleaned so far:

DELIVERIES (
 CHANNELID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

MESSAGE_REFERENCES (
 CHANNELID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

TRANSACTIONAL_MESSAGE_REFERENCES (
 CHANNELID VARCHAR,
 TXID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

MESSAGES (
 MESSAGEID VARCHAR,
 RELIABLE BOOLEAN,
 EXPIRATION BIGINT,
 TIMESTAMP BIGINT,
 COREHEADERS OBJECT,
 PAYLOAD OBJECT,
 TYPE INT,
 JMSTYPE VARCHAR,
 PRIORITY INT,
 CORRELATIONID OBJECT,
 DESTINATIONISQUEUE BOOLEAN,
 DESTINATION VARCHAR,
 REPLYTOISQUEUE BOOLEAN,
 REPLYTO VARCHAR,
 JMSPROPERTIES OBJECT
)

Couple of things:
1) Solely based on the column naming, I assume that CHANNELID, STOREID, and TXID are supposed to be foreign keys? Where are the tables they are supposed to reference?
2) Any particular reason for chosing VARCHAR as the primary key column data type? The reason I ask is that indexes on character data types are more expensive to build and store than those on integral data types. But if this is mandated by something, then it is what it is.
3) I'm not very familiar with HSQLDB. What is its OBJECT data type? Is it some form of LOB/binary-data?
4) If you want a fully normalized schema, you might consider combining the common elements of MESSAGE_REFERENCES and TRANSACTIONAL_MESSAGE_REFERENCES.
5) Based on the stated desire to be able to resolve selector restrictions as SQL restrictions and the fact that properties can be referenced in selectors, we'd need to seperate the MESSAGES.JMSPROPERTIES column into another table/series-of-tables with a many-to-one relation back to the MESSAGES table. Similiar for MESSAGES.COREHEADERS (I assume that is storing the JMS message header data).

[ovidiu.feodorov]

1) Solely based on the column naming, I assume that CHANNELID, STOREID, and TXID are supposed to be foreign keys? Where are the tables they are supposed to reference?

There are no corresponding tables. CHANNELID, STOREID and TXID are identifiers used for recovery. CHANNELID is the name of the destination, for instance.

2) Any particular reason for chosing VARCHAR as the primary key column data type? The reason I ask is that indexes on character data types are more expensive to build and store than those on integral data types. But if this is mandated by something, then it is what it is.

This is what message IDs are. This is how they are generated. I don't see any problem to store them as an integral type.

3) I'm not very familiar with HSQLDB. What is its OBJECT data type? Is it some form of LOB/binary-data?

Yes. The corresponding java type is java.lang.Object.

4) If you want a fully normalized schema, you might consider combining the common elements of MESSAGE_REFERENCES and TRANSACTIONAL_MESSAGE_REFERENCES.

Agreed.

5) Based on the stated desire to be able to resolve selector restrictions as SQL restrictions and the fact that properties can be referenced in selectors, we'd need to seperate the MESSAGES.JMSPROPERTIES column into another table/series-of-tables with a many-to-one relation back to the MESSAGES table. Similiar for MESSAGES.COREHEADERS (I assume that is storing the JMS message header data).

Agreed. Storing COREHEADERS and JMSPROPERTIES as OBJECTS was just a first attempt to get a functional version out. Separating those as many-to-one tables is what we should do, eventually.

[timfox]

Hi Steve-

First thing to mention, I think Ovidiu has already mentioned it too, is that the schema is still very much in flux, what you see is basically the first pass that Ovidiu did a while back.

Before we optimise the physical schema we really need to get the logical schema more settled, at the moment it isn't.

The logical schema should should begin to settle once we work through our remaining issues in core.

Once we've got the nice normalised logical schema sorted we should think about optimising and denormalising (if necessary) for the real world.

see JBMESSAGING-144

Yes, I've started looking at jms\src\main\org\jboss\messaging\core\persistence\HSQLDBPersistenceManager.java to get an idea of the schema. So here's what I've gleaned so far:

DELIVERIES (
 CHANNELID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

MESSAGE_REFERENCES (
 CHANNELID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

TRANSACTIONAL_MESSAGE_REFERENCES (
 CHANNELID VARCHAR,
 TXID VARCHAR,
 MESSAGEID VARCHAR,
 STOREID VARCHAR
)

MESSAGES (
 MESSAGEID VARCHAR,
 RELIABLE BOOLEAN,
 EXPIRATION BIGINT,
 TIMESTAMP BIGINT,
 COREHEADERS OBJECT,
 PAYLOAD OBJECT,
 TYPE INT,
 JMSTYPE VARCHAR,
 PRIORITY INT,
 CORRELATIONID OBJECT,
 DESTINATIONISQUEUE BOOLEAN,
 DESTINATION VARCHAR,
 REPLYTOISQUEUE BOOLEAN,
 REPLYTO VARCHAR,
 JMSPROPERTIES OBJECT
)

Couple of things:
1) Solely based on the column naming, I assume that CHANNELID, STOREID, and TXID are supposed to be foreign keys? Where are the tables they are supposed to reference?

They are logical foreign keys but don't map to anything in the db currently.

2) Any particular reason for chosing VARCHAR as the primary key column data type? The reason I ask is that indexes on character data types are more expensive to build and store than those on integral data types. But if this is mandated by something, then it is what it is.

We're not mandated to use varchar

3) I'm not very familiar with HSQLDB. What is its OBJECT data type? Is it some form of LOB/binary-data?

Yes I think so

4) If you want a fully normalized schema, you might consider combining the common elements of MESSAGE_REFERENCES and TRANSACTIONAL_MESSAGE_REFERENCES.

Yes, I think these can be combined.
There is a JIRA task to consider this JBMESSAGING-143

5) Based on the stated desire to be able to resolve selector restrictions as SQL restrictions and the fact that properties can be referenced in selectors, we'd need to seperate the MESSAGES.JMSPROPERTIES column into another table/series-of-tables with a many-to-one relation back to the MESSAGES table. Similiar for MESSAGES.COREHEADERS (I assume that is storing the JMS message header data).

Yes, we need to do something like this

[starksm64]

A related topic is how to avoid leaking the hibernate dependencies into the user space: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=70438