-
1. Re: Porting JBoss Mail Server to another container
acoliver Apr 12, 2006 11:57 AM (in response to esriram)Our plan is to do this AFTER the microcontainer comes out (too painful otherwise as we have pushed changes to the appserver).
We'd be interested in the storage component. I'm thinking of adding one similar. DB storage perf is actually good depending on the database your using.
For user transaction:
send patches and put them in JIRA (cvs -u format please) -
2. Re: Porting JBoss Mail Server to another container
acoliver Apr 13, 2006 11:47 AM (in response to esriram)Hey guys the more I think about it... do you think you guys could try the microcontainer and port to IT? It just makes more sense than duplicating work. Then if the changes are sane we can sync with the main branch (or prossibly a post 1.0 branch since 1.0 is June). There will be work on the microcontainer but you'll only have to do it once and it won't be throw away code.
-
3. Re: Porting JBoss Mail Server to another container
esriram Apr 14, 2006 7:59 AM (in response to esriram)I am developing a POP/SMTP grid. Around a dozen PCs which work together to give a highly available and scalable POP/SMTP service.
For scalability reasons I dont want to use a database. Example: if a mail is deleted, I will have to delete or update a row from a table, and deletions/update will lock the whole table. (READ_COMMITED isolation level) No one else will be able to delete/insert into the table. This will slow down the whole system. Therefore NO database and all data will be stored in a remote (replicated) file server.
For data availability reasons I will not save anything in the local disk. Everything will be saved in a replicated file server.
The default implementation of StoreMBean, MailServiceImpl, MessageData does'nt fit in my architecture. So I gave my own implementation. We have our own kernel with no JNDI, JMS, EJB over which the mail server will be deployed.
I will surely give the code, which I feel can be integrated with jboss-mail-server. As I make progress I will post any ideas, issues and requirements I have in this forum. -
4. Re: Porting JBoss Mail Server to another container
acoliver Apr 14, 2006 9:19 AM (in response to esriram)Humm. I always thought READ_COMMITTED was about not locking the table and that READ_SERIALIZED was table locks. Even in DB2 I was under the impression that the worst you got were page locks.
A replicated vs non-replicated filesystem is just a filesystem. Mostly I'm interested in a file-based store implemnetation. It sounds like the rest of your architecture is too unique to be general enough for JBMS. -
5. Re: Porting JBoss Mail Server to another container
lhoriman Apr 14, 2006 5:04 PM (in response to esriram)"esriram" wrote:
For scalability reasons I dont want to use a database. Example: if a mail is deleted, I will have to delete or update a row from a table, and deletions/update will lock the whole table. (READ_COMMITED isolation level) No one else will be able to delete/insert into the table. This will slow down the whole system. Therefore NO database and all data will be stored in a remote (replicated) file server.
You really don't understand how databases work. This is incorrect.
It's true that one table type (MyISAM) of one RDBMS (MySQL) works does table locking for writes. However, nobody in their right mind would recommend this particular configuration for your purposes.
Nearly all RDBMS implementations today use page-level or row-level locking. This allows a very high degree of concurrency.
BTW, the isolation level defines a client view of the database, not an implementation strategy. Table locking is hardly necessary for READ_COMMITTED (or even SERIALIZABLE) and, as I've mentioned, almost no RDBMSes use this strategy.
Whether or not a network fileserver is better than a database for your purposes may still be an open issue for debate, but the justification you mention here is not correct. The world is full of examples of database-backed architectures that have orders of magnitude more throughput than anything you or I are likely to implement.
Jeff Schnitzer -
6. Re: Porting JBoss Mail Server to another container
acoliver Apr 14, 2006 9:18 PM (in response to esriram)Might I also note that at least as I understand databases with MVCC supporting DBs the row locks are rather short at the very end of the transaction (don't use DB2 or Informix).
That being said, I would like to see a file-system Store implementation. However the core contributors won't likely have time to write one for 1.0. -
7. Re: Porting JBoss Mail Server to another container
esriram Apr 17, 2006 4:47 AM (in response to esriram)Am using MySql server 4.1.7 with INNODB engine.
You are right the whole table does not lock when a delete/update/insert is done. But I see that this is true only when the PK is involved in the update/delete query. Please correct me if I am wrong.
Consider MyTable having 2 columns MyPK (varchar) and MyCol(varchar). MyPK is the PK. Both the columns are of varchar type in my test scenario.
Locking example #1
Trx-1 : Delete from MyTable where MyCol='something'
Trx-2 : Insert into MyTable values ('xxx','yyy')
The insert (Trx-2) will block till Trx-1 completes.
Non Locking example #1
Trx-1 : Delete from MyTable where MyPK='AAA' (Note that the PK is involvedin the criteria)
Trx-2 : Insert into MyTable values ('xxx','yyy')
Trx-2 does NOT block till Trx-1 completes.
Locking example #2
Trx-1: Update MyTable set MyCol='after' where MyCol='before';
Trx-2: Insert into MyTable values ('xxx','yyy');
The insert (Trx-2) will block till Trx-1 completes.
Jeff: I want to avoid these kind of blcking things. That is the reason I dont want a DB. Please correct me if I am wrong anywhere. Please let me know if there is a better way for me to proceed. -
8. Re: Porting JBoss Mail Server to another container
lhoriman Apr 17, 2006 12:37 PM (in response to esriram)"esriram" wrote:
Am using MySql server 4.1.7 with INNODB engine.
You are right the whole table does not lock when a delete/update/insert is done. But I see that this is true only when the PK is involved in the update/delete query. Please correct me if I am wrong.
I'm an odd person to be offering MySQL support, but I'll try.
First, are you sure your tables are InnoDB? Try "show table status;" in the mysql client.
Jeff -
9. Re: Porting JBoss Mail Server to another container
acoliver Apr 17, 2006 1:02 PM (in response to esriram)Not sure on MySQL specifics (I switched to Postgresql which is better for JBMS). but MVCC does not mean at the VERY END of the commit there won't be a lock. It just means there is no lock needed at the beginning. you work on your copy until the end...at the end it has to lock to sync. Not sure I get your example. Postgresql is a more appropriate database for JBMS btw though some folks have gotten great numbers with MySQL.
-
10. Re: Porting JBoss Mail Server to another container
esriram Apr 18, 2006 3:51 PM (in response to esriram)First, are you sure your tables are InnoDB? Try "show table status;" in the mysql client.
I am 100% sure. It is INNODB. Output of show table status given below.
Name: GridConfiguration
Engine: InnoDB
Version: 9
Row_format: Dynamic
Rows: 62
Avg_row_length: 264
Data_length: 16384
Max_data_length: NULL
Index_length: 0
Data_free: 0
Auto_increment: NULL
Create_time: 2006-04-18 19:36:35
Update_time: NULL
Check_time: NULL
Collation: utf8_general_ci
Checksum: NULL
Create_options:
Comment: InnoDB free: 0 kB
but MVCC does not mean at the VERY END of the commit there won't be a lock. It just means there is no lock needed at the beginning. you work on your copy until the end...at the end it has to lock to sync. Not sure I get your example
1. Create the table and populate a few rows.
2. Open 2 mysql clients and set auto commit to 0 in both.
3. (TX#1) In mysql-client-1 run this query: Delete from MyTable where MyCol='something'
4. (TX#2) In mysql-client-2 run this query: Insert into MyTable values ('xxx','yyy')
You will notice that the insert query will block. TX#2 will complete only after TX#1 completes.
This is not a-little-bit-of-locking done before commit. The insert query itself does not return.
In a query like "delete from table where PrimaryKey=value", it is possible to lock that row (instead of the whole table) because the query itself contains the PK. Other transactions can delete/insert without any blocking.
But in a delete/update query which does not contain the PK...... Locking the whole table seems to be the simplest implementation to me. Maybe this MVCC is something more sophisticated. -
11. Re: Porting JBoss Mail Server to another container
lhoriman Apr 18, 2006 6:24 PM (in response to esriram)"esriram" wrote:
1. Create the table and populate a few rows.
2. Open 2 mysql clients and set auto commit to 0 in both.
3. (TX#1) In mysql-client-1 run this query: Delete from MyTable where MyCol='something'
4. (TX#2) In mysql-client-2 run this query: Insert into MyTable values ('xxx','yyy')
You will notice that the insert query will block. TX#2 will complete only after TX#1 completes.
This is not a-little-bit-of-locking done before commit. The insert query itself does not return.
This appears to be a MySQL-specific "feature". Firebird does not exhibit the same behavior. I doubt PostgreSQL does either.
Two solutions:
1) Put an index on the field you are trying to delete from. If you're worried about scalability, table scans will kill you with or without locking. Matching on a non-indexed field is the equivalent of grepping your filesystem - and if you want slow, try doing that on your network fileshare :-)
2) Use PostgreSQL or Firebird.
Jeff -
12. Re: Porting JBoss Mail Server to another container
acoliver Apr 19, 2006 7:02 PM (in response to esriram)Wow.. learn something new. I had read in serveral places that MySQL Innodb supported MVCC. Now it appears that is partially true. It supports MVCC for reads but requires row locks for updates/writes (in read committed mode). So its not really MVCC.. If I'm reading correctly. Its kind of a hybrid between MVCC and the DB2 style locking model. Like he said. Use PostgreSQL (can't vouch for firebird). It supports large objects poperly MySQL gives you ByteArrayInputStream from Blob.getInputStream() so we have to jump hoops. Oracle should also work well if you can pay.