-
1. Re: HORNETQ-189 Twitter Support
timfox Feb 12, 2010 12:20 PM (in response to brgorrie)Thanks Brian.
Sorry, haven't forgotten about you, just very busy. Will get around to looking at this soon
-
2. Re: HORNETQ-189 Twitter Support
timfox Feb 17, 2010 4:22 AM (in response to timfox)Brian, I've had a quick look at your pdf, here are a few initial comments:
1) I don't think this is appropriate to implement as a protocol. Protocols in HornetQ deal with communication between clients and HornetQ servers.
Here we are talking about communication between a HornetQ server and a Twitter server, which is a different thing.
2) Looking at the twitter4j api, i don't see why you'd need to create any new APIs like you mention in your pdf:
http://twitter4j.org/en/code-examples.html
Can't we just use these for sending/receiving tweets?
BTW, can we avoid things like pdfs or docs for designs? Best to just use the forums, or create a wiki page if it's something more elaborate
-
3. Re: HORNETQ-189 Twitter Support
brgorrie Feb 17, 2010 8:31 AM (in response to timfox)Good points, don't worry I'll stay away from the pdfs generated from google docs :-)
I'm glad you preferred option 1, twitter 4j. Its what I've been messing around with since the initial post ;-).
Would implementing it as a bridge between hornetq and twitter be a more appropriate solution?
That way the user just configures a twitter-bridge which as a destination is a twitter account, for example:
<twitter-bridge name="my-bridge">
<queue-name>jms.queue.sausage-factory</queue-name>
<twitter-account>
<username>xxx</username>
<password>yyy</password>
</twitter-account>
<rate-limited>
<updates>1000</updates>
<direct-messages>250</direct-messages>
<api-requests>150</api-requests>
</rate-limited>
<use-streaming>yes</use-streaming>
</twitter-bridge>How is that or am I barking up the wrong tree?
Cheers,
Brian.
-
4. Re: HORNETQ-189 Twitter Support
timfox Feb 17, 2010 10:35 AM (in response to brgorrie)Yes, that's the kind of thing I would envisage for outgoing tweets, for incoming tweets you would also need to specify an address, that the incoming tweets would get sent to as messages, e.g.:
<twitter-connector name="my-bridge">
<!-- Messages are consumed from this queue and sent to twitter as tweets -->
<queue-name>jms.queue.sausage-factory</queue-name>
<twitter-account>
<username>xxx</username>
<password>yyy</password>
</twitter-account>
<rate-limited>
<updates>1000</updates>
<direct-messages>250</direct-messages>
<api-requests>150</api-requests>
</rate-limited>
<use-streaming>yes</use-streaming><!-- Incoming tweets are routed to this address -->
<incoming-address>jms.topic.MyTopic</incoming-address>
</twitter-connector> -
5. Re: HORNETQ-189 Twitter Support
brgorrie Mar 3, 2010 9:14 AM (in response to brgorrie)Just to let you know that I haven't disappeared.
Have had a few entertaining issues with my home dev machine relating to a DNS resolver issue with Ubuntu 9.10 - got it working again of a sort (provided I'm patient). Long story short - I can access the internet reliably (albeit slowly) again from my dev machine so I'm happy :-) In the next day or so it will return to normal speed as I finish fixing it. There more info then you needed to know ;-).
So now back to business.
Here is a quick pick list of the tasks I've set myself to complete in the next week:
- Under hornetq-tests in the package org.hornetq.tests.integration.jms.bridge create the Twitter JMS Bridge tests.
- Check to make sure Twitter4J is accessible through Maven, if not well I'll worry about that when I get there. Haven't made any use of Maven as yet other then to build HornetQ.
- Create a new implementation of JMSBridge, JMSTwitterBridgeImpl
- Create a new implementatin of JMSBridgeControl for the Twitter Bridge, JMSTwitterBridgeControlImpl.
- Add a new section to the jms-bridge part of the documentation to cover off the jms twitter bridge.
- Create a jms-twitter-bridge example under hornetq-javaee-examples (incase people go there instead of to hornetq-jms examples)
- Create a twitter-bridge example under hornetq-jms-examples
- Make sure the test cases compile and run.
- Do a clean build and deploy of hornetq
- Bridge a queue to twitter on my own hornetq instance
- Bridge a topic to twitter on my own hornetq instance
- Once I'm happy, package the patch up and send it through to Tim for review.
Scenarios to cover off in testing:
- Source - Queue, Destination - Same Queue
- Source - Queue, Destination - Different Queue
- Source - Queue, Destination - Topic
- Source - Topic, Destination - Queue
- Source - Topic, Destination - Same Topic
- Source - Topic, Destination - Different Topic
Any thoughts/concerns/issues with that task list let me know.
Cheers,
Brian.
-
6. Re: HORNETQ-189 Twitter Support
timfox Mar 6, 2010 4:42 PM (in response to brgorrie)Brian,
I don't think modelling it after the jms bridge is the right thing to do, since this is nothing to do with JMS.
What we need is a component that consumes from a core destination and can also send to twitter, in some ways like our core bridges, but don't take the "bridge" analogy too far.
-
7. Re: HORNETQ-189 Twitter Support
brgorrie Mar 9, 2010 9:13 AM (in response to brgorrie)Alrighty ... not modelling it after the Bridge then ...
Are you ok with the name Twitter Bridge?
Bridge fits as we are bridging the queue/topic through to twitter. But you might have a better name for it.TwitterConnector? TwitterStream?
reduce the examples to one example under
hornetq-javaee-examples
no examples anywhere else ...
on a side note, should there be a separate hornetq-examples module for examples that aren't core, j2ee, or jms related?
the acutual code to go under the package
org.hornetq.integration.twitter
tests will go under same package but in the hornetq-tests module.
Interfaces to be created (names may change depending on answer to question at top of message):
TwitterBridge
TwitterBridgeControl
and associated ImplementationsHow does that sound?cheers,Brian. -
8. Re: HORNETQ-189 Twitter Support
timfox Apr 9, 2010 7:55 AM (in response to brgorrie)Brian, did you get anywhere with this?
-
9. Re: HORNETQ-189 Twitter Support
brgorrie Apr 15, 2010 7:49 AM (in response to timfox)My apologies Tim, I haven' t been able to make time to work on this beyond the information in this thread due to other commitments.
If you need the twitter support completed in the next couple of months it might be better for you to take the information we worked out here and assign it onto someone else.
If it isn't urgent leave it assigned to me and when I've got time spare I'll chip away at it.
Cheers,
Brian.
-
10. Re: HORNETQ-189 Twitter Support
timfox Apr 15, 2010 10:05 AM (in response to brgorrie)No worries Brian.
thanks for your efforts anyway
-
11. Re: HORNETQ-189 Twitter Support
igarashitm Apr 26, 2010 12:16 PM (in response to timfox)Hello,
Now I'm trying to attack this task. I've wrote a prototype of basic function. I've separated the incoming and outgoing handler.
---- Sample configs ---------------------------------
<incoming-twitter-connector name="my-incoming-tweets">
<queue-name>jms.queue.incomingQueue</queue-name>
<twitter-account>
<username>xxx</username>
<password>yyy</password>
</twitter-account>
<interval-minutes>10</interval-minutes><!-- not yet support
<rate-limited>
<updates>1000</updates>
<direct-messages>250</direct-messages>
<api-requests>150</api-requests>
</rate-limited>
<use-streaming>yes</use-streaming>
-->
</incoming-twitter-connector><outgoing-twitter-connector name="my-outgoing-tweets">
<queue-name>jms.queue.outgoingQueue</queue-name>
<twitter-account>
<username>xxx</username>
<password>yyy</password>
</twitter-account>
</outgoing-twitter-connector>--------------------------------------------------
---- Classes ---------------------------------
[Created]
- org.hornetq.core.twitter.TwitterConstants
- org.hornetq.core.twitter.TwitterConnectorService
- org.hornetq.core.twitter.impl.TwitterConnectorServiceImpl
- IncomingTweetsHandler { poll and retrieve new tweets, and then put it into queue }
- OutgoingTweetsHandler { set MessageHandler to consumer and tweet from onMessage() }
- org.hornetq.core.config.TwitterConnectorConfiguration
[Modified]
- HornetQServerImpl { start TwitterConnectorService }
- FileConfigurationParser
--------------------------------------------------
---- Next step ---------------------------------
- Test and debug above basic function
- Change OutgoingTweetsHandler to implement core Consumer like a Bridge
- Add rate-limit support
- Add streaming API support
- Write unit test, exec test and debug
- Write example
--------------------------------------------------
I don't know the maening of rate-limited/updates,direct-messages settings, so I should check twitter API and Twitter4j documentation. Also should study streaming API.
Any suggestions would be appreciated.
Thanks
igarashitm
-
12. Re: HORNETQ-189 Twitter Support
clebert.suconic Apr 30, 2010 10:23 AM (in response to igarashitm)You said you wrote a basic prototype for this... maybe you could attach it to a JIRA... or to a thread here?
We could then later provide you SVN access.. and start working in a branch until the task is done.
I believe what we need is a TwitterBridge. Something that will transfer messages back and forth between HornetQ and Twitter.
-
13. Re: HORNETQ-189 Twitter Support
igarashitm May 1, 2010 10:53 AM (in response to clebert.suconic)I'm sorry to be late, my prototype is here. Any feedback would be appreciated!
I believe what we need is a TwitterBridge. Something that will transfer messages back and forth between HornetQ and Twitter.
OK, I'll read a BridgeImpl more and get an idea. To do about it that I've already realized is, OutgoingTweetsHandler should implement org.hornetq.core.server.Consumer.
Thanks
igarashitm
-
14. Re: HORNETQ-189 Twitter Support
timfox May 2, 2010 6:38 AM (in response to igarashitm)Basically what we need is pretty simple.
We need a "bridge" that just consumes from a queue and forwards to twitter - take a look at the core bridge.
And for the other direction we just need a component that consumes from twitter and forwards to a configurable address.
That's basically it. Nothing fancy. Don't worry about rate limiting. Let's get something basic working first.
Also, please can you attach your first attempt to the JIRA?