Skip navigation
2015
ozizka

Windup 2.2.0 Released

Posted by ozizka Mar 27, 2015

A new veresion of Windup was released.

What's new? Windup is faster, rules are shorter, few bugfixes, a bit nicer API.

Download here.

 

Enjoy!

 

 

 

   Bug  WINDUP-517

Application Index page lacks technology list and effort information

   Bug  WINDUP-518

XML Handlers do not support nested rules

   Bug  WINDUP-524

Fix the cyclic dependency between windup and windup-rulesets

   Bug  WINDUP-511

Quickstarts tests fail with a NPE

   Task  WINDUP-512

Rename Implicit to Dependent

   Enhancement  WINDUP-508

WindupRuleProvider should contain a location/origin field (to indicate where it was defined)

   Bug  WINDUP-510

Build fails with BOM missing

   Enhancement  WINDUP-470

Classes in java.lang package may differ in how they are parsed

   Enhancement  WINDUP-506

Identify Archives Ruleset: Improve memory footprint

   Enhancement  WINDUP-409

Allow rules metadata to be specified with annotations - @Rules( phase = ..., ...)

   Feature Request  WINDUP-402

Change ruleset category metadata to tag-like system (make it a set of strings)

   Feature Request  WINDUP-521

Rule Externalization - Create XML Handlers for Rule metadata

   Task  WINDUP-530

Rename the module "utils" to "windup-utils"

   Task  WINDUP-513

Phase pseudo-RuleProviders to have ~Phase suffix

   Bug  WINDUP-516

[Performance] Multiple valueStores even if the rule does not use parameters

   Feature Request  WINDUP-522

Rule Externalization - Create user-interface for selecting source/target platforms

   Enhancement  WINDUP-520

Rule Externalization - Introduce a Rule, RuleProvider and RulesetMetadata API

   Bug  WINDUP-542

Fix severe regression in AST performance

   Bug  WINDUP-544

StackOverflowException when generating Application report

   Feature Request  WINDUP-523

Rule Externalization - Allow filtering of RuleProviders by tags (categories)

   Enhancement  WINDUP-515

Add Severity to Hint

   Sub-task  WINDUP-464

WINDUP-463 Expand all addons with simple layout to use full api/impl/tests structure

   Sub-task  WINDUP-465

WINDUP-463 Review all rules, organize into an external repository, and add useful hint information with links to knowledge articles

Interesting news. If I understood it correctly:

Datastax bought the company behind graph database Titan (last version is 0.5.4).

They announced that they will finish Titan 1.0 and donate it to the Apache Foundation.

Then they will work on newly rewritten implementation of graph database, integrated with their products.

And that will be opensource too.

Not sure about licensing for comercial purposes.

 

http://www.zdnet.com/article/datastax-snaps-up-aurelius-and-its-titan-team-to-build-new-graph-database/

ozizka

Windup just sped up 10x

Posted by ozizka Mar 15, 2015

Good news! Windup does not run for 16 hours anymore... when refactoring, we omitted to sync one constant between addons... and it was a name of indexed graph DB property. As a result, windup ran 3 times slower on SSD disk and 10 times slower on HDD.

 

Now I fixed that and voila - you don't have to wait 20 hours for Windup to finish your 300 MB app

 

So that was [WINDUP-529] Performance issue: Query requires iterating over all vertices [(ClassificationModel:classification = ...)] .

Additionally, I've increased Titan database cache size to 9 % of available heap, which I experimentally ascertained to give best results with our current default heap size.

But my notion is we take wrong approach for some modeling cases and we should use edges rather than indexed properties. That could give some more performance gain, and not insignificant.

That's [WINDUP-532] Performance: Change storing of classification to edges instead of vertex property .

 

Enjoy the recovered performance and stay tuned for more

There are two main technologies from the old ages of internet that are (were) used for mass distribution of assynchronous posts: mailing lists and newsgroups (NNTP).

 

Mailing lists are, well, just a list of recipients that will get a copy of a mail sent to a special mail on a mailing list server.

Newsgroups are "topic IDs" (such like "org.jboss.windup.users") to which you can post, and it gets distributed over NNTP protocol to newsgroups servers all over the world. It's not copied to users' inboxes - it waits until they request it from the server.

 

I hate mailing lists (and I am not alone by far). They are just a poor substitution for newsgroups (NNTP protocol). For many reasons:

 

1) Subscribe/unsubscribe nonsense

Useless overhead. Why should anyone subscribe? Why not just authenticate / get authorized and access the content? Subscribing is a waste of time.

NNTP server, on the other hand, allows authentication and authorization, and "subscription" is done in your client - you simply favorite some groups and read them at will.

 

2) Downloading everything

In a mailing list, you simply get all of the trafic to your inbox. If someone sends 5 MB file, you get 5 MB file. And have to delete it. (IMAP helps a bit, but you still get a copy to your inbox).

NNTP server, on the other hand, passively waits until you _want_ to read the post, and sends it on your request.

Of course, pre-downloading in favorited groups may happen.

 

3) Waste of disk space

Again, if someone sends a 5 MB file, and the mailing list happens to be company-wide list with 4000 recipients, then we get around 40 GB wasted (5 MB attachment is encoded into roughly 10 MB in MIME format). That's really sick. And no - if you doubt the mail servers are so dumb and they only keep one copy of the file somewhere - they really don't, at least not the one we use(d) internally (confirmed by it's admin).

NNTP server, on the other hand, really does keep just one single copy of the post, and sends it to individuals on demand. The same goes for cross-posting.

 

4) Silly sorting to folders using filters

If you are in a company running a lot of mailing lists, you have to create a filter for each mailing list, if you don't want to end up with 1500 mails a day in your inbox. And not just you - everyone has. Typically, everyone needs to go to the impractical web UI of mail server ZYX and do the same stuff, figuring out which headers are the right to filter by. There's usually no sharing of these filters. And yet when finished, it still doesn't work well - sometimes you get stuff you have nothing to do with in your inbox, just because some header goes wrong. Or, very often, when a mail is cross-posted, it goes to one folder, but not to the other. And if it was copied to multiple folders, we're back to #3.

Newsgroups posts, on the other hand, are sorted by definition - each posts goes to one or more groups. And that's where you find them, period. No need for everyone to waste hours to create his/her own filters, almost identical to what others have.

 

5) Cross-posting

With mailing lists, cross-posting is real hell. Okay, the mailing lists server can prevent sending the mail to you twice. But not two servers - they have no idea about each other's internal IDs, so if someone sends a mail to a-list@foo.org and b-list@bar.org, and you happen to be subscribed to both - you end up with 2 mails. Now, to which of them are you going to reply? Both? If not both, some people will not get your reply. And worse: You need to watch both of the copied threads; otherwise, If someone replies to just one of them (i.e. not subscribed to the other), you will miss it.

NNTP protocol, on the other hand, is distributed, the IDs are shared, and one post, no matter to what group or what server, is the same, and you get just one single copy of that post. It _will_ appear in multiple groups, but it will be the same post, and if you reply to it, it will get distributed to all newsgroup servers, and nobody will miss any reply in any of the groups the conversation goes in (cross-posted groups).

 

6) Missing posts when not subscribed

When you're not subscribed, you don't get the posts. And after you subscribe - you still don't get the old posts. That's really one of biggest disadvantages - which other asynchronous channel does this? Forums let you see the old posts. Google groups do. Stack overflow does. Twitter does. Facebook does. Youtube does. Wherever you "subscribe" to some content, you get the old stuff. Except mailing lists.

Of course, there are archives. But how convenient is that? Leave your mail client and go search in the ugly Mailman UI from 90's? Thanks, but no thanks.

NNTP protocol, needless to say, allows you to download whatever you want, just what the server keeps in it's archive (some delete old posts).

 

7) Security reasons

You want to bring a mail to broader audience. But wait! Is the original list private list? Public list? And what about the 2nd list? How the hell should I know? Well, you need to go check the domain name. And even then, you still don't know - different list, different policy. You have to remember. If you forward a mail to another list, nothing prevents you to do so, or better said, nothing reminds you not to do so.

NNTP, on the other hand, supports mechanisms to refuse forwarding of a post based on the ID's of the posts contained in the post's headers. Sure, you could do it with mails, too; but it's up to mail client if it forwards the MIME headers or not. It's not mandatory for the protocol. For NNTP, it is, so a good client should send them, and your SMTP server (often the same instance as your local NNTP server) will refuse to send the post.

 

On par

There are areas where NNTP is just like mails. It's the same format (MIME), after all. And NNTP is quite similar to SMTP.

For example, most mainstream email clients support NNTP: ThunderBird, Evolution, Outlook (even the old Outlook Express did), ...

Basically you work with NNTP the same way as with mails; only, the addresses are group names such like "org.jboss.windup.users".

And you can even mix recipients - a post can go as mail to some users, and to a newsgroup.

 

Let's revive NNTP

With the above said, I really don't get why companies run mailing lists. What advantage mailing lists give? Waste of storage on server, waste of bandwidth, waste of user's disk space, waste of CPU on mail servers, waste of people's time. It's not even simpler or anything. It's just that people are used to it, or rather, don't know about newsgroups at all.

 

Let's spread the word about newsgroups. It's one of the best things that came from internet's stone ages, and should not be forgotten.

Share this article if you agree, and let me know in comments.

For a little while, Windup Quickstarts could not build as they got out of sync with latest API.

 

That's over and now they build. Well, if you skip tests One of rules fails and I had no time to figure out why yet. But at least you can take the quickstarts as a basis for your rule.

 

Go and try: windup/windup-quickstarts · GitHub

 

Here's a script to get everything you need to start playing with Windup rules in Java syntax: Windup lab scripts

Good news!

 

Windup rules, when written in Java syntax (instead of XML), is now much nicer, after we did [WINDUP-409] in PR #380 @Rules( phase = ..., .... by OndraZizka .

 

Before, metadata were provided by callback methods. Combined with our insisting on space-wasting formatting, each piece of metadata required 5 lines in class source.

 

public class SkipArchivesRules extends WindupRuleProvider
{
    private static final Logger log = Logging.get(SkipArchivesRules.class);

    @Override
    public void enhanceMetadata(Context context)
    {
        super.enhanceMetadata(context);
        context.put(RuleMetadata.CATEGORY, "Java");
    }

    @Override
    public List<Class<? extends WindupRuleProvider>> getExecuteAfter()
    {
        return asClassList(IdentifyArchivesRules.class, SkipArchivesLoadConfigRules.class);
    }

    @Override
    public List<Class<? extends WindupRuleProvider>> getExecuteBefore()
    {
        return asClassList(DecompileArchivesRuleProvider.class);
    }

    @Override
    public Class<? extends RulePhase> getPhase()
    {
        return Implicit.class;
    }
...

 

Now it's just one line with annotations:

 

@RuleMetadata(tags = "java", after = {IdentifyArchivesRules.class, SkipArchivesLoadConfigRules.class}, before = {DecompileArchivesRuleProvider.class}, phase = DependentPhase.class)
public class SkipArchivesRules extends AbstractRuleProvider
{
    private static final Logger log = Logging.get(SkipArchivesRules.class);
...

 

Isn't it great improvement?

Filter Blog

By date:
By tag: