Disallow

Name

Disallow -- disallows indexing defined URLs

indexer.conf

Synopsis

Disallow [Match | NoMatch] [Case | NoCase] [String | Regexp] {arg...}

Description

Use this command to disallow URLs that matches/doesn't match given argument.

The meaning of the first three optional parameters is exactly the same as in Allow command.

You can use several arguments for one "Disallow" command.

This command takes global effect for configuration file.

Examples


# Disallow URLs not in udm.net domains using "String" match:
Disallow NoMatch *.udm.net/*
# Disallow any except known extensions and directory index using "regex" match:
Disallow NoMatch Regex \/$|\.htm$|\.html$|\.shtml$|\.phtml$|\.php$|\.txt$
# Exclude cgi-bin and non-parsed-headers using "string" match:
Disallow */cgi-bin/* *.cgi */nph-*
# Exclude anything with '?' sign in URL. Note that '?' sign has a
# special meaning in "string" match, so we have to use "regex" match here:
Disallow Regex  \?