Cyrus NNTP
**********


Overview
========

Note:

  The NNTP support in Cyrus is still relatively young in the grand
  scheme of things, and has not been tested under a heavy Usenet load.
  That being said, the code appears to be stable and is currently
  running in production serving 50-60 newsgroups with a volume of
  about 6000 messages per day.

Cyrus includes a NNTP server. The NNTP works a lot like the IMAP
server, it uses the same authentication and authorization. It also
stores messages like IMAP. This means the messages are stored in one
file per post.

Cyrus has the ability to export Usenet via IMAP and/or export shared
IMAP mailboxes via NNTP.

This "diagram" shows how the various NNTP components interact.


NNTP Setup
==========

Decide where the newsgroup mailboxes will reside: either at the
toplevel of the hierarchy (eg, comp.mail.imap) or rooted elsewhere
(eg, netnews.comp.mail.imap). If the newsgroup mailboxes are not at
the toplevel of the hierarchy, then the parent must be specified using
**newsprefix** in imapd.conf(5).

1. Make sure that Cyrus is built with NNTP support using the "--
   enable-nntp" switch. This builds nntpd and the associated
   utilities.

      In many pre-packaged versions of Cyrus, the NNTP server is in
      its own package (Example: Debian has a cyrus-nntpd-2.2 package
      that must be installed in addition to the rest of Cyrus).

2. Edit "/etc/cyrus.conf".

      In order to receive usenet articles, make sure that the Cyrus
      nntpd service is enabled in cyrus.conf. The
      master/conf/normal.conf and master/conf/prefork.conf sample
      configs both include entries for nntpd (disabled by default).

      Add a new line in the SERVICES section with something like this:

         nntp: cmd="nntpd"  listen="nntp" prefork="0"

      This defines a new service that will run the nntpd command (man
      nntpd for more info) on the nntp port (defined in
      /etc/services). Since the prefork is listed as 0, Cyrus will not
      start a new process for this.

3. Edit "/etc/imapd.conf".

      There are many directives for news that can be placed in
      /etc/imapd.conf. Check out imapd.conf(5) to learn more.

      At minimum, something like this:

         # News:
         partition-news: /var/spool/cyrus/news
         newsprefix: ournews

      This tells Cyrus:

      * Where the news partition should be stored (should be similar
        to the value in **partition-default**)

      * What part of the Cyrus tree should be treated as the location
        for news groups.

      The **newsprefix** is a path in the IMAP directory hierarchy
      (that is, the hierarchy used with cyradm). User accounts, for
      instance, are often in the "users" hierarchy. Here, we are
      telling it to put the news in the "ournews" hierarchy.

4. Restart your server.


Defining Newsgroups
===================

Create a mailbox for each newsgroup to receive/export. If some groups
are private, be sure to set the ACLs accordingly. The mknewsgroups(8)
script can be used to help facilitate mass creation of newsgroup
mailboxes. When using this script, be sure to add posting rights for
'anyone' (eg. "mknewsgroups -a 'anyone +p' ...") so that articles can
be fed/posted.

News groups can be defined in two ways:

1. Use mknewsgroups(8), which is a very simple wrapper for creating
   groups

2. Use cyradm(8), and create a new mailbox for news.

Some distributions such as debian do not include mknewsgroups.


Using Cyradm
------------

Create a new group like creating a new mailbox:

   $ cm ournews/mynewsgroup

Next, set some permissions. (See cyradm(8) for more info on the cm and
sam commands.)

   $ sam ournews/mynewsgroup anyone lrsp

The above gives **L** Lookup, **R** Read, **S** Seen, and **P** Post
permissions (basically read/write -- see the man page for more) to
anyone logged into the server. More restrictive permissions are
possible. (Note: This example assumes *unixhierarchysep* is being used
in imapd.conf(5))


Configuration
=============


Push (traditional) feeds
------------------------

If the usenet peer will be pushing articles to the server, no further
configuration is necessary, beyond providing peer access to the Cyrus
server on port 119 (nntp).


Pull (suck) feeds
-----------------

If pulling articles from the peer is preferred (and the provider
allows it), then use the "fetchnews" utility which will retrieve
articles from the peer and feed them to the Cyrus server. If supported
by the peer, fetchnews will use the NEWNEWS command, otherwise it will
fallback to keeping track of the high water mark of each group.
Configure fetchnews as an EVENT in cyrus.conf(5) to be called
periodically (eg, once an hour, every 15 minutes, etc).


imapfeed
--------

Alternatively, if there is an INN v2.3 server in-house use the
included "imapfeed" utility (written by the authors of Cyrus) to feed
articles to the Cyrus server via LMTP. Consult the INN documentation
for further details.


Control Messages
----------------

Control messages are accepted, parsed and delivered to the
corresponding "control.*" pseudo-group (eg, control.newgroup,
control.cancel, etc) if it exists, so that they may be reviewed by an
administrator.

Automatic execution of control messages is only performed if the
newsmaster (default = "news") user has the proper access control for
the given mailbox. For example, to allow cancel control messages to be
performed for "misc.test" articles, give the "news" user the 'd' right
on "misc.test". To allow newgroup, rmgroup and mvgroup control
messages to be performed on the "misc" hierarchy, give the "news" user
the 'c' right on "misc".

NOTE: No sender or PGP verification of control messages is currently
implemented.


Reading/Posting articles
------------------------

In order to have articles posted by your local users propagate to the
outside world, you must specify the name of your usenet peer(s) with
the newspeer option in imapd.conf(5). This is the host(s) that nntpd
contacts to feed outgoing articles. Depending on the configuration of
the newspeer option, articles will be fed to the upstream server(s)
using either the POST or IHAVE command. Also note that you may specify
an optional wildmat to filter which groups will be fed (see
imapd.conf(5) for details).

Newsgroups can also be gatewayed to email by setting "/vendor/cmu
/cyrus-imapd/news2mail" mailbox annotations to the corresponding email
addresses.


News clients
------------

If anonymous logins are disabled (default) in imapd.conf(5), then your
news clients will have to be configured to login with a username and
password, otherwise they will not be allowed to post. Furthermore, if
plaintext logins are disabled in imapd.conf, then you might have to
configure your news clients to use SSL/TLS and enable the nntps
service in cyrus.conf(5).

If you want to allow your news clients to use the NNTP NEWNEWS
command, you will have to enable the **allownewnews** option in
imapd.conf.


Email clients
-------------

If you are exporting Usenet via IMAP, and your users' messaging
clients are not savvy enough to reply to and post articles via NNTP,
then you will have to configure your server so your users can reply to
and post articles via SMTP.

To help facilitate this, you can set the **newspostuser** option to a
pseudo user which will be used to construct email delivery addresses
for each incoming article. These addresses are inserted into a Reply-
To: in the article. For example, if set to "post", an article posted
to comp.mail.imap will have an address of "post+comp.mail.imap"
inserted into the Reply-To: header. This will allow a user to easily
reply to an article via email. Otherwise, the users will have to learn
the correct email address format for posting and replying to articles.

In order for these email messages to be fed into your news server (and
subsequently to the outside world) you need to use an email to news
gateway, such as lmtp2nntp. You need to configure your MTA (Sendmail,
Postfix, etc) so that lmtp2nntp is used as the local mailer whenever
it receives a news article. A simple rule for doing this in Sendmail
is shown below:

   # mail addressed to post+ goes to lmtp2nntp@localhost
   LOCAL_RULE_0
   Rpost + $+ < @ $=w . >              $#lmtp2nntp $@ localhost $: $1

For other configurations, consult the lmtp2nntp and documentation and
your MTA documentation.

NOTE: If anonymous logins are disabled (default) in imapd.conf(5),
then you should configure lmtp2nntp to use its "feed" operation mode.


Expiring articles
-----------------

Expiration of articles is done by the cyr_expire(8) utility. Control
over when articles are expunged is accomplished with the "/vendor/cmu
/cyrus-imapd/expire" mailbox annotation. This annotation sets the
number of days that messages should be kept in the mailbox before they
expire. All entries in the duplicate deliver database that correspond
to these messages are also kept for the same number of days before
they are purged (overriding the "cyr_expire -E" option).

Setting the expire time to 0 (zero) for a mailbox will ensure that
neither the messages nor the corresponding database entries will ever
be expired. This can be useful for shared mailboxes (e.g. mailing list
archives) which are being exported via NNTP. Note that this will cause
the duplicate delivery database to consistently grow in proportion to
the number of messages in such mailboxes.

If a mailbox does not have an expire time set on it, then the messages
will never be expunged, but the corresponding database entries WILL be
expired after the default number of days ("cyr_expire -E" option).

Note that the "/vendor/cmu/cyrus-imapd/expire" mailbox annotation is
inherited by child mailboxes, so that you may control expiration on an
entire mailbox/newsgroup hierarchy simply by setting the annotation on
the root of the hierarchy. For example, if you set the annotation on
"comp", then ALL of the newsgroups in the "comp" hierarchy will be
expired at the same time. Similarly, if you set the annotation on
"alt.binaries", all of the binary newsgroups under "alt" will be
expired at the same time (independently from "comp").


Further information
===================

This thread from the info-cyrus mailing list may be of use: Cyrus and
Usenet
