Provided by: inn2_2.6.4-2build4_amd64 
NAME
innfeed.conf - Configuration file for innfeed
DESCRIPTION
The configuration file innfeed.conf in pathetc is used to control the innfeed(8) program. It is a fairly
free-format file that consists of three types of entries: key:value, peer and group. Comments are from
the hash character "#" to the end of the line.
key:value entries are a keyword and a value separated by a colon (which can itself be surrounded by
whitespace). For example:
max-connections: 10
A legal key starts with a letter and contains only letters, digits, and the "_" and "-" characters.
There are 5 different types of values: integers, floating-point numbers, characters, booleans, and
strings.
Integer and floating-point numbers are as to be expected, except that exponents in floating-point numbers
are not supported. A boolean value is either "true" or "false" (case is not significant). A character
value is a single-quoted character as defined by the C-language. A string value is any other sequence of
characters. If the string needs to contain whitespace, then it must be quoted with double quotes, and
uses the same format for embedding non-printing characters as normal C-language string.
Peer entries look like:
peer <name> {
# body ...
}
The word "peer" is required. The <name> is the same as the site name in INN's newsfeeds configuration
file. The body of a peer entry contains some number (possibly zero) of key:value entries.
Group entries look like:
group <name> {
# body ...
}
The word "group" is required. The <name> is any string valid as a key. The body of a group entry
contains any number of the three types of entries. So key:value pairs can be defined inside a group, and
peers can be nested inside a group, and other groups can be nested inside a group.
key:value entries that are defined outside of all peer and group entries are said to be at "global
scope". There are global key:value entries that apply to the process as a whole (for example the
location of the backlog file directory), and there are global key:value entries that act as defaults for
peers. When innfeed looks for a specific value in a peer entry (for example, the maximum number of
connections to set up), if the value is not defined in the peer entry, then the enclosing groups are
examined for the entry (starting at the closest enclosing group). If there are no enclosing groups, or
the enclosing groups do not define the key:value, then the value at global scope is used.
A small example could be:
# Global value applied to all peers that have
# no value of their own.
max-connections: 5
# A peer definition. "uunet" is the name used by innd
# in the newsfeeds configuration file.
peer uunet {
ip-name: usenet1.uu.net
}
peer vixie {
ip-name: gw.home.vix.com
max-connections: 10 # Override global value.
}
# A group of two peers which can handle more connections
# than normal.
group fast-sites {
max-connections: 15
# Another peer. The "max-connections" value from the
# "fast-sites" group scope is used. The "ip-name" value
# defaults to the peer's name.
peer data.ramona.vix.com {
}
peer bb.home.vix.com {
max-connections: 20 # He can really cook.
}
}
Given the above configuration file, the defined peers would have the following values for the max-
connections key:
uunet 5
vixie 10
data.ramona.vix.com 15
bb.home.vix.com 20
innfeed ignores key:value pairs it is not interested in. Some configuration file values can be set via a
command-line option, in which case that setting overrides the settings in the file.
Configuration files can be included in other configuration files via the syntax:
$INCLUDE filename
There is a maximum nesting depth of 10.
For a fuller example configuration file, see the supplied innfeed.conf.
GLOBAL VALUES
The following listing show all the keys that apply to the process as whole. These are not required
(compiled-in defaults are used where needed).
news-spool
This key requires a pathname value and defaults to patharticles in inn.conf. It specifies where the
top of the article spool is. This corresponds to the -a command-line option.
input-file
This key requires a pathname value. It specifies the pathname (relative to the backlog-directory
value) that should be read in funnel-file mode. This corresponds to giving a filename as an argument
on the command-line (i.e. its presence also implies that funnel-file mode should be used).
The default is unset; innfeed then runs in channel or batch mode.
pid-file
This key requires a pathname value and defaults to innfeed.pid. It specifies the pathname (relative
to pathrun in inn.conf) where the pid of the innfeed process should be stored. This corresponds to
the -p command-line option.
debug-level
This key defines the debug level for the process. Default is 0. A non-zero number generates a lot
of messages to stderr, or to the config-defined log-file. This corresponds to the -d command-line
option.
If a file named innfeed.debug exists in the pathlog directory (as set in inn.conf), then debug-level
is automatically set to 1. This is a cheap way of avoiding continual reloading of the newsfeeds file
when debugging. Note that debug messages still go to log-file.
debug-shrinking
This key requires a boolean value and defaults to false (the debug file is allowed to grow without
bound). If set to true, this file is truncated when its size reaches a certain limit. See backlog-
limit for more details.
initial-sleep
This key requires a positive integer. The default value is 2. It defines the number of seconds to
wait when innfeed (or a fork) starts, before beginning to open connections to remote hosts.
fast-exit
This key requires a boolean value and defaults to false. If set to true, when innfeed receives a
SIGTERM or SIGQUIT signal, it will close its listeners as soon as it can, even if it means dropping
articles.
use-mmap
This key requires a boolean value and defaults to true. When innfeed is given file names to send (a
fairly rare use case) instead of storage API tokens, it specifies whether mmaping should be used if
innfeed has been built with mmap(2) support. If article data on disk is not in NNTP-ready format
(CR/LF at the end of each line), then after mmaping, the article is read into memory and fixed up, so
mmaping has no positive effect (and possibly some negative effect depending on your system), and so
in such a case this value should be "false", which corresponds to the -M command-line option.
log-file
This key requires a pathname value and defaults to innfeed.log. It specifies where any logging
messages that could not be sent via syslog(3) should go (such as those generated when a positive
value for debug-value is used). This corresponds to the -l command-line option.
This pathname is relative to pathlog in inn.conf.
log-time-format
This key requires a format string suitable for strftime(3). It is used for messages sent via
syslog(3) and to the status-file. Default value is "%a %b %d %H:%M:%S %Y".
backlog-directory
This key requires a pathname value and defaults to innfeed. It specifies where the current innfeed
process should store backlog files. This corresponds to the -b command-line option.
This pathname is relative to pathspool in inn.conf.
backlog-highwater
This key requires a positive integer value and defaults to 5. It specifies how many articles should
be kept on the backlog file queue before starting to write new entries to disk.
backlog-ckpt-period
This key requires a positive integer value and defaults to 30. It specifies how many seconds elapse
between checkpoints of the input backlog file. Too small a number will mean frequent disk accesses;
too large a number will mean after a crash, innfeed will re-offer more already-processed articles
than necessary.
backlog-newfile-period
This key requires a positive integer value and defaults to 600. It specifies how many seconds elapse
before each check for externally generated backlog files that are to be picked up and processed.
backlog-rotate-period
This key requires a positive integer value and defaults to 60. It specifies how many seconds elapse
before innfeed checks for a manually created backlog file and moves the output backlog file to the
input backlog file.
dns-retry
This key requires a positive integer value and defaults to 900. It defines the number of seconds
between attempts to re-lookup host information that previously failed to be resolved.
dns-expire
This key requires a positive integer value and defaults to 86400. It defines the number of seconds
between refreshes of name to address DNS translation. This is so long-running processes do not get
stuck with stale data, should peer IP addresses change.
gen-html
This key requires a boolean value and defaults to false. It specifies whether the status-file should
be HTML-ified.
status-file
This key requires a pathname value and defaults to innfeed.status. An absolute pathname can be used.
It specifies the pathname (relative to pathhttp when gen-html is true; otherwise, pathlog as set in
inn.conf) where the periodic status of the innfeed process should be stored. This corresponds to the
-S command-line option.
connection-stats
This key requires a boolean value and defaults to false. If the value is true, then whenever the
transmission statistics for a peer are logged, each active connection logs its own statistics. This
corresponds to the -z command-line option.
host-queue-highwater
This key requires a positive integer value and defaults to 10. It defines how many articles will be
held internally for a peer before new arrivals cause article information to be spooled to the backlog
file.
stats-period
This key requires a positive integer value and defaults to 600. It defines how many seconds innfeed
waits between generating statistics on transfer rates.
stats-reset
This key requires a positive integer value and defaults to 43200. It defines how many seconds
innfeed waits before resetting all internal transfer counters back to zero (after logging one final
time). This is so a innfeed process running more than a day will generate "final" stats that will be
picked up by logfile processing scripts.
initial-reconnect-time
This key requires a positive integer value and defaults to 30. It defines how many seconds to first
wait before retrying to reconnect after a connection failure. If the next attempt fails too, then
the reconnect time is approximately doubled until the connection succeeds, or max-reconnect-time is
reached.
max-reconnect-time
This key requires an integer value and defaults to 3600. It defines the maximum number of seconds to
wait between attempt to reconnect to a peer. The initial value for reconnection attempts is defined
by initial-reconnect-time, and it is doubled after each failure, up to this value.
stdio-fdmax
This key requires a non-negative integer value and defaults to 0. If the value is greater than zero,
then whenever a network socket file descriptor is created and it has a value less than this, the file
descriptor will be dup'ed to bring the value up greater than this. This is to leave lower numbered
file descriptors free for stdio. Certain systems, Sun's in particular, require this. SunOS 4.1.x
usually requires a value of 128 and Solaris requires a value of 256. The default if this is not
specified, is 0.
Special keys for imapfeed
The following keys are used with imapfeed to authenticate to a remote host. Several parameters may be
included at global scope:
deliver-authname
The authname is who you want to authenticate as.
deliver-password
This is the appropriate password for authname.
deliver-username
The username is who you want to "act" as, that is, who is actually going to be using the server.
deliver-realm
In this case, the "realm" is the realm in which the specified authname is valid. Currently this is
only needed by the DIGEST-MD5 SASL mechanism.
deliver-rcpt-to
A printf(3)-style format string for creating the envelope recipient address. The pattern MUST
include a single string specifier which will be replaced with the newgroup (e.g. "bb+%s"). The
default is "+%s".
deliver-to-header
An optional printf(3)-style format string for creating a To: header field to be prepended to the
article. The pattern MUST include a single string specifier which will be replaced with the newgroup
(e.g. "post+%s@domain"). If not specified, the To: header field will not be prepended.
GLOBAL PEER DEFAULTS
All the key:value pairs mentioned in this section can be specified at global scope. They may also be
specified inside a group or peer definition. Note that when peers are added dynamically (i.e. when
innfeed receives an article for an unspecified peer), it will add the peer site using the parameters
specified at global scope.
Required keys
No keys are currently required. They all have a default value, if not present in the configuration file.
Optional keys
The following keys are optional:
article-timeout
This key requires a non-negative integer value. The default value is 600. If no articles need to be
sent to the peer for this many seconds, then the peer is considered idle and all its active
connections are torn down.
response-timeout
This key requires a non-negative integer value. The default value is 300. It defines the maximum
amount of time to wait for a response from the peer after issuing a command.
initial-connections
This key requires a non-negative integer value. The default value is 1. It defines the number of
connections to be opened immediately when setting up a peer binding. A value of 0 means no
connections will be created until an article needs to be sent.
max-connections
This key requires a positive integer value. The default value is 2 but may be increased if needed or
for large feeds. It defines the maximum number of connections to run in parallel to the peer. A
value of 0 specifies an unlimited number of maximum connections. In general, use of an unlimited
number of maximum connections is not recommended. Do not ever set max-connections to zero with
dynamic-method 0 set, as this will saturate peer hosts with connections.
close-period
This key requires a positive integer value and defaults to 86400. It is the maximum number of
seconds a connection should be kept open. Some NNTP servers do not deal well with connections being
held open for long periods.
dynamic-method
This key requires an integer value between 0 and 3. The default value is 3. It controls how
connections are opened, up to the maximum specified by max-connections. In general (and
specifically, with dynamic-method 0), a new connection is opened when the current number of
connections is below max-connections, and an article is to be sent while no current connections are
idle. Without further restraint (i.e. using dynamic-method 0), in practice this means that max-
connections connections are established while articles are being sent. Use of other dynamic-method
settings imposes a further limit on the amount of connections opened below that specified by max-
connections. This limit is calculated in different ways, depending of the value of dynamic-method.
Users should note that adding additional connections is not always productive -- just because opening
twice as many connections results in a small percentage increase of articles accepted by the remote
peer, this may be at considerable resource cost both locally and at the remote site, whereas the
remote site might well have received the extra articles sent from another peer a fraction of a second
later. Opening large numbers of connections is considered antisocial.
The meanings of the various settings are:
0 (no method)
Increase of connections up to max-connections is unrestrained.
1 (maximize articles per second)
Connections are increased (up to max-connections) and decreased so as to maximize the number of
articles per second sent, while using the fewest connections to do this.
2 (set target queue length)
Connections are increased (up to max-connections) and decreased so as to keep the queue of articles
to be sent within the bounds set by dynamic-backlog-low and dynamic-backlog-high, while using the
minimum resources possible. As the queue will tend to fill if the site is not keeping up, this
method ensures that the maximum number of articles are offered to the peer while using the minimum
number of connections to achieve this.
3 (combination)
This method uses a combination of methods 1 and 2 above. For sites accepting a large percentage of
articles, method 2 will be used to ensure these sites are offered as complete a feed as possible.
For sites accepting a small percentage of articles, method 1 is used, to minimize remote resource
usage. For intermediate sites, an appropriate combination is used.
dynamic-backlog-low
This key requires a floating-point value between 0 and 100. It represents (as a percentage) the low
water mark for the host queue. If the host queue falls below this level while using dynamic-method 2
or 3, and if 2 or more connections are open, innfeed will attempt to drop connections to the host.
An Infinite Impulse Response (IIR) filter is applied to the value to prevent connection flap (see
dynamic-filter). The default value is 20.0. This value must be smaller than dynamic-backlog-high.
dynamic-backlog-high
This key requires a floating-point value between 0 and 100. It represents (as a percentage) the high
water mark for the host queue. If the host queue rises above this level while using dynamic-method 2
or 3, and if less than max-connections are open to the host, innfeed will attempt to open further
connections to the host. An Infinite Impulse Response (IIR) filter is applied to the value to
prevent connection flap (see dynamic-filter). The default value is 50.0. This value must be larger
than dynamic-backlog-low.
dynamic-backlog-filter
This key requires a floating-point value between 0 and 1. It represents the filter coefficient used
by the Infinite Impulse Response (IIR) filter used to implement dynamic-method 2 and 3. The default
value of this filter is 0.7, giving a time constant of 1/(1-0.7) articles. Higher values will result
in slower response to queue fullness changes; lower values in faster response.
max-queue-size
This key requires a positive integer value. The default value is 20. It defines the maximum number
of articles to process at one time when using streaming to transmit to a peer. Larger numbers mean
more memory consumed as articles usually get pulled into memory (see the description of use-mmap).
streaming
This key requires a boolean value. Its default value is true. It defines whether streaming commands
are used to transmit articles to the peers.
no-check-high
This key requires a floating-point number which must be in the range [0.0, 100.0]. When running
transmitting with the streaming commands, innfeed attempts an optimization called "no-CHECK mode".
This involves not asking the peer if it wants the article, but just sending it. This optimization
occurs when the percentage of the articles the peer has accepted gets larger than this number. If
this value is set to 100.0, then this effectively turns off no-CHECK mode, as the percentage can
never get above 100.0. If this value is too small, then the number of articles the peer rejects will
get bigger (and your bandwidth will be wasted). The default value of 95.0 usually works pretty well.
no-check-low
This key requires a floating-point number which must be in the range [0.0, 100.0], and it must be
smaller that the value for no-check-high. When running in no-CHECK mode, as described above, if the
percentage of articles the remote server accepts drops below this number, then the no-CHECK
optimization is turned off until the percentage gets above the no-check-high value again. If there
is small difference between this and the no-check-high value (less than about 5.0), then innfeed may
frequently go in and out of no-CHECK mode. If the difference is too big, then it will make it harder
to get out of no-CHECK mode when necessary (wasting bandwidth). Keeping this to between 5.0 and 10.0
less than no-check-high usually works pretty well. The default value is 90.0.
no-check-filter
This is a floating-point value representing the time constant, in articles, over which the
CHECK/no-CHECK calculations are done. The default value is 50.0, which will implement an Infinite
Impulse Response (IIR) filter of time constant 50. This roughly equates to making a decision about
the mode over the previous 50 articles. A higher number will result in a slower response to changing
percentages of articles accepted; a lower number will result in a faster response.
port-number
This key requires a positive integer value. It defines the TCP/IP port number to use when connecting
to the remote. Usually, port number 119 is used, which is the default value.
force-ipv4
This key requires a boolean value. By default, it is set to false. Setting it to true is the same
as setting bindaddress6 to "none" and removing bindaddress from "none" if it was set.
drop-deferred
This key requires a boolean value. By default, it is set to false. When set to true, and a peer
replies with code 431 or 436 (try again later), innfeed just drops the article and does not try to
re-send it. This is useful for some peers that keep on deferring articles for a long time to prevent
innfeed from trying to offer the same article over and over again.
min-queue-connection
This key requires a boolean value. By default, it is set to false. When set to true, innfeed will
attempt to use a connection with the least queue size (or the first empty connection). If this key
is set to true, it is recommended that dynamic-method be set to 0. This allows for article
propagation with the least delay.
no-backlog
This key requires a boolean value. It specifies whether spooling should be enabled (false, the
default) or disabled (true). Note that when no-backlog is set, articles reported as spooled are
actually silently discarded.
backlog-limit
This key requires a non-negative integer value. If the number is 0 (the default), then backlog files
are allowed to grow without bound when the peer is unable to keep up with the article flow. If this
number is greater than 0, then it specifies the size (in bytes) the backlog file should get truncated
to when the backlog file reaches a certain limit. The limit depends on whether backlog-factor or
backlog-limit-highwater is used.
This parameter also applies to the debug file when debug-shrinking is set to true, and has the same
effect on this file as the one has on backlog files.
backlog-factor
This key requires a floating-point value, which must be larger than 1.0. It is used in conjunction
with the peer key backlog-limit. If backlog-limit has a value greater than zero, then when the
backlog file gets larger than the value backlog-limit * backlog-factor, then the backlog file will be
truncated to the size backlog-limit.
For example, if backlog-limit has a value of 1000000, and backlog-factor has a value of 2.0, then
when the backlog file gets to be larger than 2000000 bytes in size, it will be truncated to 1000000
bytes. The front portion of the file is removed, and the trimming happens on line boundaries, so the
final size may be a bit less than this number. If backlog-limit-highwater is defined too, then
backlog-factor takes precedence. The default value of backlog-factor is 1.1.
This parameter also applies to the debug file when debug-shrinking is set to true, and has the same
effect on this file as the one has on backlog files.
backlog-limit-highwater
This key requires a positive integer value that must be larger than the value for backlog-limit. The
default value is 0.
If the size of the backlog file gets larger than this value (in bytes), then the backlog file will be
shrunk down to the size of backlog-limit. If both backlog-factor and backlog-limit-highwater are
defined, then the value of backlog-factor is used.
This parameter also applies to the debug file when debug-shrinking is set to true, and has the same
effect on this file as the one has on backlog files.
backlog-feed-first
This key requires a boolean value. By default it is set to false. When set to true, the backlog is
fed before new files. This is intended to enforce in-order delivery, so setting this to true when
initial-connections or max-connections is more than 1 is inconsistent.
bindaddress
This key requires a string value. It specifies which outgoing IPv4 address innfeed should bind the
local end of its connection to. It must be an IPv4 address in dotted-quad format (nnn.nnn.nnn.nnn),
"any", or "none". If not set or set to "any", innfeed defaults to letting the kernel choose this
address. If set to "none", innfeed will not use IPv4 for outgoing connections to peers in this scope
(i.e. it forces IPv6).
If not set in innfeed.conf, innfeed defaults to the value of sourceaddress from inn.conf (which by
default is unset).
bindaddress6
This key requires a string value. It behaves like bindaddress except for outgoing IPv6 connections.
It must be in numeric IPv6 format (note that a value containing colons must be enclosed in double
quotes), "any", or "none". If set to "none", innfeed will not use IPv6 for outgoing connections to
peers in this scope.
If not set in innfeed.conf, innfeed defaults to the value of sourceaddress6 from inn.conf (which by
default is unset).
username
This key requires a string value. If the value is defined, then innfeed tries to authenticate by
AUTHINFO USER and this value used for user name. password must also be defined, if this key is
defined.
password
This key requires a string value. The value is the password used for AUTHINFO PASS. username must
also be defined, if this key is defined.
PEER VALUES
As previously explained, the peer definitions can contain redefinitions of any of the key:value pairs
described in the section about global peer defaults above. There is one key:value pair that is specific
to a peer definition.
ip-name
This key requires a word value. The word is either one of the host's FQDNs, or the dotted-quad IP
address of the peer for IPv4, or the colon-separated IP address of the peer for IPv6. If this value
is not specified, then the name of the peer in the enclosing peer block is taken to also be its ip-
name.
RELOADING
If innfeed gets a SIGHUP signal, then it will reread the configuration file. All values at global scope
except for backlog-directory can be changed (although note that bindaddress and bindaddress6 changes will
only affect new connections).
Any new peers are added and any missing peers have their connections closed.
The log file is also reopened.
EXAMPLE
For a comprehensive example, see the sample innfeed.conf distributed with INN and installed as a starting
point.
Here are examples of how to format values:
eg-string: "New\tconfig\tfile\n"
eg-long-string: "A long string that goes
over multiple lines. The
newline is kept in the
string except when quoted
with a backslash \
as here."
eg-simple-string: A-no-quote-string
eg-integer: 10
eg-boolean: true
eg-char: 'a'
eg-ctrl-g: '\007'
HISTORY
Written by James Brister <brister@vix.com> for InterNetNews. Converted to POD by Julien Elie.
Earlier versions of innfeed (up to 0.10.1) were shipped separately; innfeed is now part of INN and shares
the same version number. Please note that the innfeed.conf format has changed dramatically since
version 0.9.3.
$Id: innfeed.conf.pod 10179 2017-09-18 20:13:48Z iulius $
SEE ALSO
inn.conf(5), innfeed(8), newsfeeds(5).
INN 2.6.4 2018-01-28 INNFEED.CONF(5)