TransactionalFileIO

    Transactional File I/O in Java.

     

    Summary

    Transactions are often used to structure activities within reliable software applications. In Java EE, business logic typically involves accessing transactional resource managers (databases, message queues) within boundaries denoted by calls to the JTA (begin/commit/rollback). The resource managers work with the transaction manager to perform e.g. locking, logging and recovery transparently to the application programmer. However, this separation of concerns is broken with regard to one important resource: the file system. Java's file I/O library does not support transactions, a situation which requires application programmers to implement such support manually in their programs. This project aims to develop a transaction aware resource manager for file I/O in Java. This library will provide application programmers with access to a filesystem that offers ACID semantics.

     

     

    Example use cases

     

    use case: transactional changes within a single file.

     

    A file on disk contains a number of business data entries,

    each of which must be read and processed. At the end of the

    processing for each entry, results are sent via JMS and the

    entry in the file is marked as done. There may be lots of

    entries and processing them can take a long time. The system

    may crash during processing. We never want to miss

    processing an entry, nor to process it more than once.

     

    Solution: within an XA transaction, read an entry from the

    file, process it, mark it as done and enqueue the JMS

    message. Two-phase commit of the transaction ensures that

    even if the system crashes we still get the 'mark entry as

    done' and the 'send JMS message' as an single atomic update.

     

    Even better: allow multiple transactions to be active on the

    file simultaneously, provided they are updating different

    parts of the file. This allows concurrent processing of

    entries on multi-processor systems.

     

    use case: batch changes on group of files.

     

    A directory contains a number of files. Each file must be

    read, some processing done and the result written back to

    another file in the same directory. The source file is then

    deleted. As a side effect of the processing, one or more

    database updates may occur. We want to ensure each file is

    processed exactly once.

     

    Solution: within an XA transaction, read the source file,

    update the database and create the output file. The

    transaction ensures that the source file is never deleted

    before the output file is created, nor is the source file

    left intact after the output file is created. Likewise the

    database is never updated unless the other operations also

    complete.  Even better: multiple transactions on the

    directory simultaneously, so different files can be

    processed in parallel.

     

    use case: software upgrade.

     

    A new version of a software package includes additional

    files, moves some existing files, edits others and deletes

    some. We would like an upgrade installer that will ensure

    that if any of these operations fails, the upgrade can be

    backed out smoothly, leaving the original version of the

    software in a usable state. Under no circumstances should an

    upgrade that fails part way through leave the installed

    software unusable.

     

    Solution: Perform all the installation operations within an

    XA transaction. If any of them fails, rollback the

    transaction, which returns the filesystem to its original state.

     

    use case: clustered transactional file storage.

     

    A web based application provides for the upload and storage

    of user images. There are a large number of users, such that

    we need a cluster of application servers to support them.

    Some files may be shared by multiple users, such as in a

    business workgroup. The server cluster may perform

    asynchronous operations (crop, rotate, etc) on the image

    files in response to user requests.  Simultaneous

    manipulations of the same image should never be allowed to

    corrupt the file.

     

    Solution: provide access to a shared network file system

    through a transactional API, such that all nodes in the

    cluster have the same view of the filesystem state and can

    lock files to prevent interference.

     

    Summary of required functionality:

     

    Manipulate the contents of a single file transactionally.

    Conceptually this is very similar to having multiple

    concurrent transactions on a database. As long as they don't

    try to update the same data (file: blocks, db: rows of

    table) they should not interfere.

     

    Allow these manipulations to be committed or rolledback as

    part of an XA transaction's 2PC, so that other updates e.g.

    JMS or SQL can be committed or rolledback consistently.

     

    Manipulate the contents of a directory in the same way:

    add or remove files in a transaction, such that the

    operations can be undone together if needed.

     

    Allow changes to file contents and directories in the same

    transaction. This basically combines the points above.

     

    Allow transactions to span multiple JVMs, to allow for

    greater scaling. Conceptually this in similar to moving from

    an in-process main memory database to one that is accessed

    by multiple clients using a network protocol. JCA may be

    useful here.

     

    Discussion

     

    Implementing the above is likely to involve changes to some

    java.io. code. Fortunately it's open source. We don't want

    to change too much: ideally just the low level code, not the

    things like Streams that layer on top of it. Perhaps

    subclassing java.io.File is one approach. Shame it's a class

    not an interface!  Taking that approach, Streams etc don't

    need to change, they still see a File, though it's actually

    our new XAFile?  The way JDBC uses Connection vs.

    XAConnection is worth looking at too.

     

    The ideas being discussed in JSR-203 may useful.

     

    The apache commons transaction project also has done a

    little work on this in their file module, but it's not very

    impressive last time I looked. Microsoft have some transactional NTFS work that may be relevant.

     

    http://commons.apache.org/transaction/

     

    http://www.infoq.com/news/2008/01/file-systems-transactions

     

     

    In term of executing the project, I'd anticipate step one to

    cover use in a single JVM only, with step two then adding

    distribution so multiple JVMs can share transactional access

    if there is enough time.

     

    This work relates only to the JVM's view of the filesystem.

    Indeed, it may relate only to the view of threads within the

    JVM that are in a transaction.  Other processes on the

    machine will not access the filessystem through the

    transactional API, so they won't be affected. This has

    benefits and drawbacks. It will be hard to deal with the

    case where the filesystem is changed by an external process

    whilst a transaction is running. But it will be easy to keep

    a virtual view of the modified version of the filesystem in

    memory for the benefit of ongoing transactions. The new vfs used in AS 5.0 may be handy?

     

    Some kind of logging mechanism will be needed to record

    changes to the filesystem for use in crash recovery. Perhaps

    a 'shadow' directory for each transaction, in some specially

    designated out of the way place on the file system.  Similar

    to what a database does with transaction log files or

    rollback segments.

     

    Although what's being implemented here is basically a resource manager, part of the JBossTS transaction manager may be reusable, particularly the locking and logging support provided by ArjunaCore.

     

    Thoughts on the apache commons transaction project

     

    This focuses on multi level locks, transactional collections and transactional file access. The transactional collections are not of interest to this project. The locking overlaps the capabilities of ArjunaCore to some extent, but is less mature. The transactional file access focuses mainly on manipulation of filesystems i.e. the 'java.io.File as dir structure' view rather than 'java.io.File as byte{FOOTNOTE DEF  }' view. It does not support XA, so it won't interoperate with a standard transaction manager unless we provide an adaptor layer. There is no crash recovery support either. In short, it may provide a useful source of ideas and some reusable code, but its not a workable solution by itself at present.

     

    Thoughts on transactional access to File as byte{FOOTNOTE DEF  }

     

    Conceptually a single file is an array of bytes. There may be business level structure layered on top of this e.g. fixed length records, csv, lines of text, serialized objects.  Most of what we know about tx concurrency control comes from the database community, where transactions work in terms of tuples, data blocks or tables. For example, row level locking attaches transaction meta data (locks) to data at the tuple level.  To do meaningful concurrency control on a file, we need to adapt these ideas a bit.

     

    How to break down a file into smaller units so we can do fine grained cc? Take a 'black box' approach where we know nothing about the business data structure and just lock on byte blocks of fixed size, or byte ranges according to what a is read/written inside the scope of a tx? Or provide hooks through which the business logic can provide some structure meta-data so we can lock at the business record level e.g. new RecordLengthSpecifier(long numberOfBytesPerRecord)

     

    Also access patterns for a file may differ from a db. In a db, reads outnumber writes considerably. With a file, an app may read once, then write once or not at all. What implications does that have for the design? Overhead is likely to come from writing locks or versioned blocks/records to disk, so we want to minimise that.

     

    Where records are not fixed length, a write in the middle of a file involves moving all subsequent records back or forwards, or leaving a gap. That's potentially a lot of data copying. Also if locks are expressed in terms of byte indexes and the data moves to different indexes, we have a problem...  However, how common is this use case really?

     

    To my mind transactional access to a single file breaks down into three cases:

    • manipulation of fixed length records e.g. read a record, process it, change a 0->1 to mark it done. record boundaries are thus fixed, so we can do concurrency control easily

    • conditional append: we want to append a log record to the end of a file for each tx, or perhaps just the successful tx. Ordering may or may not be important. We can't use afterCompletion sync to do the write because that's volatile and won't survive a crash. Locking here is very different to the earlier case, particularly if we can write in commit order rather that tx start time order.

    • everything else i.e. manipulation of variable length records. This is hard.

    Perhaps we have special cases for the first two, so we do more efficient things where the business logic indicates its running one of those more limited use cases.

     

    Thoughts on XA in a object oriented, connectionless world.

     

    The XA spec is not object oriented, so its mapping into Java through the JTA already has some interesting issues. XA is mainly used by JCA/JMS/JDBC, which are Connection oriented. File access is not, which leaves us either a) further stretching JTA to work in a connectionless model or b) providing a JCA like filesystem provider that forces a connection centric model on file I/O.  That is not too big a stretch, since Streams are already close to Connections in essence, plus users are used to a 'connection to a remote file server' approach to storage. The choice here also feeds in to work on distributed (multi JVM) transactional access, since there a Connection based model works well. With a connectionless API we would need to hide the distribution bit from the user rather than allowing it to be exposed.