Changing urls so that search engines can crawl

[prophecy]

Does Nukes have anything to handle this problem?

[julien1]

I am thinking about it , any ideas ?

[techn9ne]

Google can crawl all sites and consists of the vast majority of traffic from "search engines".

Directories like yahoo and dmoz ( which powers aol ) are by far the largest market share.

I think it would be a cool feature for some of the other search engines but shouldn't be a priority.

[prophecy]

I am mostly referring to google. Google can't crawl too far into most cms's and judging by how this looks on the jboss site, I can probably safely assume the same with this one.

it's because that index.html?modules=blah&blah......

[jcooley]

Zope uses urls like
moduleName/op
So you get URLs like
help/index_html?parameter1=x
The idea is that each module is a Zope (python) object and you call methods on it. It might be worth having a look at how they do it.

James

[julien1]

that is very similar to nukes, each op is an mbean operation called on the mbean.

julien

[cnovara]

There is another, simpler way to achieve the same result :
create some "dummy pages" presenting a flat view of database content, organised with "clean" (friendly) URL links.
Robots can easily follow those links to index the full content.

A batch process can compute thos pages each night for example ...

Another way : detect bots crawl and presents "custom pages to them. thi pages presents only friendly URLs.
At the same time, we could log the web crawl for futur analyses ...

I'm working on it[/img]