M2-final release available

[acoliver]

http://jboss.org/jbossBlog/blog/acoliver/?permalink=73D125255677F18B14FB800D2807C435.txt

I have finally provided a relief of weight on my posterior and produced our blessed spawn. I'll be updating the schedule on the wiki. I'm probably going to change the goal for M3 as I've not had time to set up a proper load testing environment. The goal for M3 will be instead features and memory consumption reduction.

I'd like it if someone started thinking real hard about a mail API. Meaning a simple scripting/workflow way of routing and processing mails. I lean towards exposing things via Javascript. Our target audience for said API is comprised of developers and power-adminstrators.

Sorry that I've been lax. I've been on the road to frozen places for the last month. Should have some more time in the coming weeks to pitch in more meaningfully. Keep up the good work! I can't wait to install M2 on my server (its running something in between M1 and M2) and I REALLY can't wait till M3.

Does anyone have the cycles to liason with the MySQL guys and figure out what needs to be done to add decent blob support (meaning really streaming or blocking rather than copying all in one big fat byte array). Moreover, does anyone know if postgresql or any of the other mainstream open source dbs (or even not-open source but stuff we can get our hands on for free) support real blob streaming?

Thanks!

[jsight]

I believe that Mckoi supports streamable blobs.

[kryptontri]

Keep up the good work.

[mikezzz]

PostgreSQL has an interface called the LargeObjectAPI:

http://www.postgresql.org/docs/8.0/interactive/largeobjects.html

I have built an implementation of the Store that talks directly to it and from my tests it works really well (128MB in 1MB chunks in about 12s). They don't have support for the JDBC3 Blob interface, well not yet:

http://archives.postgresql.org/pgsql-jdbc/2005-03/msg00033.php

Oracle is also another that we should definitely support. Once I get the store committed I will start a Wiki page with information regarding status of the various dbs that I have tested.

Mike

[acoliver]

I'll post more, I talked to the mysql guy who wrote their JDBC driver at JBossWorld (shameless plug for next year I guess? ;-)) and he told me how to work around this with MySQL simulated sql locators.

[mikezzz]

Hi Andy,

Before you embark on that, can I ask you to have a look at the current org.jboss.mail.store implemenation (the old one will suffice, but it will be updated this weekend). I went through most of the pain of using the MySQL emulated locators when I had my very first attempts at implementing the store. In fact the Blob[Input/Output]Stream classes were written to solve this very problem.

Also MySQL blobs are very buggy, see this thread:

http://www.jboss.org/index.html?module=bb&op=viewtopic&t=59236

Apparently they have fixed Bug 8096, but haven't indicated which release it will be in. Also they suck when it comes to performance. Try loading a 128MB blob in 1MB pieces and watch the MySQL process eat itself (takes around 15 minutes). From the work I have done so far it looks like most of the problems are not so much with the JDBC driver by the database. I should really get on the mailing list and bitch a little louder about blob support.

Regards,
Mike.

[acoliver]

Ahh..I'll hook you up with Mark and see what we can come up with.

[mikezzz]

My orginal development with MySQL was using connector 3.1.5-gamma. I have retested with 3.1.7 (final). Much better, bug 8096 is fixed (bug 7745 is still outstanding) and performance is now at acceptable levels.

[iwadasn]

Guys, why rely on MySQL? It is well known (and well documented) that MySQL is a VERY weak database.

For instance, see this link....

http://benchw.sourceforge.net/benchw_results_open3.html

On any meaningful workload (especially with large objects), Postgres is faster, and it's way less buggy. Save yourselves the trouble, and use a real DB. I'm not in any way connected to Postgres, but it just kills me to see perfectly good projects rendered almost useless by a bad choice of backend DB.

I also hear that JBoss has some sort of special deal with MySQL (they are planning to use MySQL to back their portal for the jboss.org website), and I would like to point out that this is not wise. Wikipedia (less than a month ago) came very close to losing everything (modulo tape backups) because they used MySQL, which isn't truly ACID.

Sorry about the rant, but friends don't let friends trust their data to MySQL.

[acoliver]

Bla bla bla. You don't like MySQL. I don't like database XYZ. We will support as many dbs as is feasible and at the highest performance feasible. For now, I'd like that to be Hypersonic (for testing purposes), BerkleyDB JE, MySQL, PostgreSQL, and Oracle. Eventually I'd like to add MS SQL Server, Informix and Sybase to that. Once we get that far then we'll add support on an as needed basis. Besides, most comes for free with Hibernate....its the blobby stuff that doesn't.

MySQL is a JBoss Inc. partner. They are thoroughly nice and helpful guys. And this mail you so carefully used to disparage them is already served from MySQL. The JBoss Forums have been stored using MySQL pretty much from the beginning. (Yes there have been occassional problems because of Blob support and we weren't using InnoDB because we had to migrate, but life involves pain).

In my experience, MySQL is a more than adeuate DB though it has its issues like any other. In my experience, all databases have bad remote protocols and horrid blob support :-) so I grant you the bar isn't high.

Honestly I'm reasonably agnostic, I just want to see us support (in JBoss Mail Server) what people are already using.