Back to mnoGoSearch site

URLSelectCacheSize

Name

URLSelectCacheSize -- sets URLs cache size for indexer

indexer.conf

Synopsis

URLSelectCacheSize {num}

Description

Defines the number of URLs to load from the database as crawler targets at once. The bigger number makes crawling performance better, however in case when indexer crashes or is killed or is terminated using Ctrl-Break, some documents will get stuck in crawling queue and won't be available for crawling for 4 hours.

Note: To make stuck documents available for crawling use indexer -a flag, in combination with suitable subsection control option, like -s, -u, etc.

The default value is 128.

Examples


URLSelectCacheSize 1024
      

See also

URLSelectSkipLock.