Ubuntu Manpage: fetchnews - fetch news via NNTP

NAME

       fetchnews - fetch news via NNTP

SYNOPSIS

       fetchnews [-q] [-v] ... [-x number] [-l] [-n] [-f] [-P] [-w]

DESCRIPTION

       Leafnode  is  a USENET package intended for small sites, where there are few users and little disk space,
       but where a large number of groups is desired.

       The design of leafnode is intended to self-repair after problems, and to require no manual maintenance.

       Fetchnews (formerly called "fetch") is the program which submits and retrieves new articles  to  or  from
       the upstream NNTP server.

OPTIONS

-q Be quiet. Suppresses some warnings. This flag cancels any previously encountered -v flags.

-v Be verbose. This flag cancels any previously encountered -q flags. A maximum of four -v can be
used (the more v, the more verbose):

-v Warn about non-critical errors.

-vv Be verbose whenever a group command occurs.

-vvv Be verbose whenever an article command occurs.

-vvvv For debugging purposes. Don't expect the additional output to make sense.

-w Wait mode. This causes the XOVER update (which runs after the article fetch phase) to run in the
foreground. By default, the XOVER update runs in the background, detached from the shell, on
systems that offer a working fork(2) call. Helpful for debugging and to avoid terminal clobbering
when fetchnews is run manually.

-x number
Go number articles back and try to re-fetch what's missing. This can be useful to pull data from
broken servers that have either overview and group data out of synch or create articles out of
article number order.

-l Don't use supplementary servers (-l is supposed to mean "local"). This does speed up the
operation quite a bit.

-n Do not expire newsgroup subscriptions. Technically: do not unlink files in the interesting.groups
directory.

-f Erase the active.read file, thus force redownloading the whole active file from all upstream
servers. This is usually occurring only very infrequently because it takes a long time. See
timeout_active in leafnode(8). DO NOT use this option in cron jobs or otherwise regularly.

-P Only post articles that have accumulated in the out.going directory but do not fetch articles from
upstream servers.

EXIT STATUS

       fetchnews  will  exit  with code 0 if it could connect to all servers and encountered no other errors. It
       exits with code 2 when it was unable to connect to at least one server, and it exits with code 1 on other
       errors.

FILES AND CONFIGURATION

       /var/spool/news/leaf.node/active.read
              is a file to track when the active files have  been  successfully  downloaded  from  the  upstream
              servers  the  last time. If it is missing, this has the same effect as giving the -f option. It is
              only created after the active file of each upstream servers has been downloaded successfully.

       See leafnode(8) for the list of other files affected, and for information on configuring fetchnews.

ENVIRONMENT

       LN_SKIP_GROUPS=pattern[,pattern[...]] (since v1.9.53)
              If set, all groups that match one of the wildmat(3) patterns will be  skipped  during  the  fetch.
              This can be used as a quick way of skipping binary groups during daytime, for instance, example:

              env 'LN_SKIP_GROUPS=*binary*,alt.*' /usr/sbin/fetchnews -nq

       LN_LOCK_TIMEOUT
              This  variable  is  parsed  as an unsigned integer value and determines how many seconds fetchnews
              will wait when trying to obtain the lock file from another  leafnode  program.  0  means  to  wait
              indefinitely. This variable takes precedence over the configuration file.

       LN_SUPPRESS_DATE
              (since  v1.11.7)  If this variable is defined (regardless of its value, including empty), leafnode
              will skip checking the upstream server's time by means of the NNTP DATE command.

ALGORITHM

Here is a brief description of what fetchnews does.

First, after connecting to the upstream NNTP server, all articles that have the u+r permission (0400)
flag set in /var/spool/news/out.going are posted using POST. If a posting fails, that article is moved
to /var/spool/news/failed.postings which an administrator has to clean up once in a while. Moving a file
back to /var/spool/news/out.going is sufficient to retry the post it. The u+r flag is checked to prevent
posting an (incomplete) article that leafnode(8) is still receiving at the same time. If a posting
succeeds, the article is removed from out.going in order to not post it twice into a moderated newsgroup
on a different server (which would send multiple copies of the article to the moderator). Note: older
fetchnews versions tried to post on all servers first, but failed to do so when one of the servers could
not be reached. If you need to work around a dodgy upstream server that is fast but doesn't forward your
posts reliably, prohibit posting to this server using the nopost server option in the configuration file,
see leafnode(8).

Second, the upstream server's list of newsgroups (obtained using LIST and LIST NEWSGROUPS) is merged into
leafnode's (if timeout_active has expired, the active.read file is not present or the active file is
empty) or updated (with NEWGROUPS).

Third, fetchnews iterates over the list of newsgroups, performing a GROUP, an XOVER (if supported) or
XHDR Message-ID and a number of HEAD and BODY or ARTICLE commands for each group which has been read
recently. It assumes that for the particular group it is looking at, "maxage" was set to the groupexpire
or global expire. As a consequence, it makes only sense to set maxage lower than or equal to expire.

NOTES

       Fetchnews does not use NEWNEWS.  NEWNEWS is not used because it is often  very  expensive  and  therefore
       also disabled at many sites.

       After  some  experiments,  I  decided  that  it  was  better to fetch all the articles in a group than to
       optimize away some; users would complain too loudly when the optmizer guessed wrongly.

       The only way to control what newsgroups are accessible to local users is to  control  what  the  upstream
       server  lets  fetchnews  read,  and  that  will  not keep out all crossposts (endemic in e.g. the alt.sex
       groups).  If your site is big enough that you need to control newsgroup access,  consider  using  a  news
       server which is designed for a big site.

       Fetchnews  sets its real and effective uid to "news" when started as root and refuses to run if it cannot
       get this user ID.

BUGS

       Fetchnews ignores the "Supersedes:" and "Control:" headers.

HISTORY

       Older versions of fetchnews posted any articles found in out.going. Since v1.9.23,  fetchnews  will  only
       post  article  files  that  have  their  user  executable  bit  set.  Leafnode sets this bit after it has
       successfully received a post. This behaviour avoids posting incomplete articles when leafnode receives  a
       post at the same time fetchnews is posting.

AUTHOR

       Written  by  Arnt  Gulbrandsen  <agulbra@troll.no> and copyright 1995\-–\-96 Troll Tech AS, Postboks 6133
       Etterstad, 0602 Oslo, Norway, fax +47 22646949.

       Modified    by    Cornelius    Krasel    <krasel@wpxx02.toxi.uni-wuerzburg.de>,    Markus     Enzenberger
       <enz@cip.physik.uni-muenchen.de>   and   Randolf   Skerka   <Randolf.Skerka@gmx.de>.   Copyright  of  the
       modifications 1997\-–\-1999.  Modified by Matthias  Andree  <matthias.andree@gmx.de>,  copyright  of  the
       modifications 2001\-–\-2004.