-
1. Re: NBF and all that.
zhfeng Jul 25, 2011 10:11 PM (in response to fhitchen)In actually, the NBF buffers is xml format on the wire.
We use XSD to validate these buffers, and get type information of every element from schema which will been used in btXattribute functions.
Yes, it must be very carefully to handle with the buffer string.
Anyway, this xml buffer could be integrated with others, such as content based routing.
Now, I'm working on using JBossESB to do CBR with NBF buffer.
If you are interest , you can find a cbr example on blacktie trunk.
Amos
-
2. Re: NBF and all that.
fhitchen Jul 26, 2011 10:04 AM (in response to zhfeng)I'm more interested in emulating legacy TUXEDO functions, i.e. XML to simple C struct with minimal overhead. Performance is key for my applications.
-
3. Re: NBF and all that.
tomjenkinson Jul 26, 2011 12:15 PM (in response to fhitchen)Hi Francis,
Which tool do you use to convert the FML buffer into the C struct?
Out of interest, can you explain why you needed to use FML and could not get away with using say X_C_TYPE as your buffer type instead? X_C_TYPE allows you to interact with structs by default.
Tom
-
4. Re: NBF and all that.
fhitchen Jul 27, 2011 6:58 PM (in response to tomjenkinson)The FML library had a function Fvftos which takes the field view table and converts the FML buffer to a C struct.
We have Java and Sybase clients that need a neutral buffer format. Going forward we may drop Sybase but will need Java client support.
-
5. Re: NBF and all that.
zhfeng Jul 27, 2011 10:23 PM (in response to fhitchen)I'm trying to understand your application,
in Java client you use FML buffer with JOLT, such as:
...
fmlbuf.setString("NAME", "zhfeng");
fmlbuf.setINT("ID", "1234");
session.tpcall(...);
in Server use C, such as:
struct employee {
char name[32];
int id;
};
// this struct from view file such as
VIEW Employee
$
#type cname fbname count flag size null
string name - 1 - 32 -
long id - 1 - - -
END
void service(TPSVCINFO* rqst) {
FBFR* fml = (FBFR*) rqst->data;
struct employee buf;
Fvftos(fml, &buf, "employee");
...
do something with buf
}
you do not want to use fml function in server code as performance is key for your applications.
Is this clear ?
-
6. Re: NBF and all that.
fhitchen Jul 28, 2011 12:04 PM (in response to zhfeng)In the Java code Jolt directly supports formatting the FML. We generate a jave fields mapping from the view that is also used by the C server side so you are mostly correct.
As this is all mostly generated code driven off initial C struct's that are then converted to FML views which can easily be converted to XSD, what do you recommend as the most efficient way to replace the Jolt/FML layer in BlackTie?
-
7. Re: NBF and all that.
zhfeng Jul 28, 2011 11:46 PM (in response to fhitchen)1. convert FML views to our buffer XSD
2. we have a tool to convert XSD schema to JavaBean, you can see example under
<BLACKTIE_SRC>/jatmibroker-xatmi/src/test/java/org/jboss/narayana/blacktie/jatmibroker/nbf/TestNBFToJavaBean.java
you can run mvn test -Dtest=TestNBFToJavaBean to see the generate JavaBean code in target/test-classes/EmployeeBuffer.java
you can consider this as a java fields mapping from the views
3. On the C server, you can only use btXattribute functions to access NBF buffer currently.
In terms of performance, XML maybe not a good choice. I have some interest in google protoc buffer. If you think it is OK, we can work together to support this buffer in blacktie.
-
8. Re: NBF and all that.
tomjenkinson Jul 29, 2011 4:01 AM (in response to zhfeng)Amos Feng wrote:
3. On the C server, you can only use btXattribute functions to access NBF buffer currently.
It is our plan to support mapping structs to the XML wire format though. We plan to support this for X_COMMON, X_C_TYPE and NBF buffers.
This means the programmer interacts with a struct, but on the wire it is XML so that it can be routed by ESB.
-
-
10. Re: NBF and all that.
fhitchen Jul 29, 2011 11:01 AM (in response to tomjenkinson)Tom, Amos,
thanks for your inputs. I looked at the google protoc buffer, it does not support mapping to C structs, and I'm concerned that if we stick with XML on the wire, the parsing overhead will be too great. We call millions of TUX services a day and expect sub millisecond response times on some of them. The google doc quotes this statistic "When this message is encoded to the protocol buffer binary format (the text format above is just a convenient human-readable representation for debugging and editing), it would probably be 28 bytes long and take around 100-200 nanoseconds to parse. The XML version is at least 69 bytes if you remove whitespace, and would take around 5,000-10,000 nanoseconds to parse."
-
11. Re: NBF and all that.
tomjenkinson Jul 29, 2011 11:10 AM (in response to fhitchen)Hi Francis,
Thanks for reviewing the google buffer. It would be great to work with you on providing a suitably performant wire protocol implementation for BlackTie.
Currently we support a binary wire format with zero compression (this could work faster than a compressed buffer format for certain classes of message as the compression algorithm is omitted).
We also have the XML format which is more useful when you wish to take advantage of routing services for tpcall invocations as the ESB can easily parse the XML. This is what we are finishing off now.
We could introduce a third protocol which deals with larger buffers and compresses them for efficient transfer.If you would like to work with us on this and have some ideas on what you believe would be the most appropriate wire protocol, that would be excellent. BlackTie has a pluggable transport layer so the replacement of the buffer encoder is a relatively painless operation.
Tom
-
-