 | spiderSettings Properties |
[This is preliminary documentation and is subject to change.]
The spiderSettings type exposes the following members.
Properties| Name | Description | |
|---|---|---|
 | Comment | Customization comments |
| doEnableCrossLinkDetection | shoulw crawler perform crosslink detection on each iteration |
| doEnableDLC_BlockTree | autoconfigurable :: switching page block decomposing on DLC level |
| doEnableDLC_TFIDF | autoconfigurable :: switching semantic record tracking (TF-IDF) if required by a module |
| flags | |
| FRONTIER_doLinkHarvest |
true - enables link harvest from the loaded page, recommanded value is true
|
| FRONTIER_doLinkResolver |
true - enables link resolving after the harvest -- resolver will unify links that point to the same target
|
| FRONTIER_harvestNature |
Flags enabling harvest links of specified linkNature |
| FRONTIER_harvestScope |
Flags enabling link harvest for specified linkScope
|
| FRONTIER_PullDecayModes |
Flags that turn on one of Target auto removal modes
|
| FRONTIER_PullTakeMode |
How it makes the final selection of the targets
|
| limitIterationNewLinks |
LoadTake - LT
|
| limitIterations |
Maximum allowed spider iterations per web site profile
|
| limitTotalLinks |
Total maximum of links to be loaded
|
| limitTotalPageLoad |
Limits number of pages allowed to be loaded by spider
|
| primaryPageSetSize |
Target size of primary page set
|
| SignatureSufix | Signature sufix used for experiment crawler ID |
| TermExpansionForContent | SEMANTIC_DOCUMENT_CONTRACTION |
See Also