Modular listserv

I use this for student classes. Since I own a domain name, I also own all email addresses whatsoever aliased to an account where the email for the domain is received. I tell a class of students that a listserv and website are available for them at, for example,

economics@domain.zone
and
http://domain.zone/economics/

The set of files below will work for any number of simultaneous classes or listservs. It will require a single recipe in the normal ".procmailrc" file at the account to call another procmail script, "serv-main", to complete the processes of subscription, email verification, cleanup, redistribution, and archiving. Since writing these notes there have been a few changes, not noted here.

Subdirectories are created and populated with the first email received.

"Serv-main" is one of four scripts. "Serv-main" is a procmail script; the other three are shell scripts, which are called from "serv-main". The 4 scripts are modular, so they can be used without modification for any number of differently named listservs at the same domain.

"serv-main"
The main file which redistributes the email after checking a number of things, and may call other scripts for separate purposes.
"serv-subscribe"
Adds a new email address to the list of users, sends an initial help-file to the new user, and sends a report to the owner.
"serv-cleanup"
This cleans up the email, mainly removing email-quoted text, before redistribution. HTML, multipart/mixed and multipart/alternative can also be cleaned up, that is, converted to plain text. These filters are shown below also.
"serv-archive"
The final step after deliver, this script html-izes the text of the email, extracts a 'name', and updates web files.

A few environmental variables are set in the ".procmailrc" file before initiating the "serv-main" script. These are (others can be added):


	# variables for "serv" files
		X_FROM=`formail -x "From:"`
		X_SUBJECT=`formail -x "Subject:"`
		TODAY=`date +%y-%m-%d` 


A last environmental variable to be set is the value of $SERV; this will happen when a recipe in the ".procmailrc" file recognizes an email addressed to the listserv.

The following inclusion in the ".procmailrc" file will then start up a set of files which will handle the email addressed to 'economics' (any other email addresses could be added). This should probably be preceeded by spooler delivery of email which already includes the "X-Loop:" header.


	# -- deliver X-Loop mail to spooler
		:0w
		* X-Loop
		${SPOOLER}
	# -- economics listserv
		:0w
		* ^To:.*economics@domain.zone
		* ! X-Loop
		{
		PW="whatever"
		SERV="economics"
		INCLUDERC=${HOME}/scripts/serv-main
		}

Here we are using, as an example, 'economics@domain.zone' -- as if that were the name of the class I was teaching. The $SERV environmental variable will thus be set to "economics". This recipe calls for the 'inclusion' of the file "serv-main" detailed below. This recipe also requires a password of "whatever" -- see the notes below.

The Script "serv-main"

"serv-main" is a procmail script, located in the subdirectory "scripts/", but it could be anywhere. A transcription of the "serv-main" as follows (numbered notes below):


   # VERBOSE can be set here (note 1)
	VERBOSE=no
   # A separate logfile is used (note 2)
	LOGFILE=$HOME/log.${SERV}.${TODAY}
   # The logfile entries separation (note 3)
	LOG="
	"
   # set variables for "serv" files
        X_FROM=`formail -x "From:"`
        X_SUBJECT=`formail -x "Subject:"`
        X_TO=`formail -x "To:"`
   # set site variable 
        SITE=${HOME}/public_html/${SERV}
   # make directory and files (note 3a)
        :0 cw:serv-dir.lock
        * ! ? test -d ${SITE}
        | ${HOME}/scripts/serv-dir
   # Bounce if not using password (note 3b)
        :0hw:no-pw.lock
        * PW ?? (.)
        *$ ! ^Subject:.*$PW
        | (formail -r -I "X-Loop: no pw" \
        -I "From: errors@domain.zone" \
        -I "Reply-To: help@domain.zone"; \
        echo -e "email refused, password required \n"; \
                 ) | $SENDMAIL -oi -t
   # Save incoming email unfiltered (note 4)
	:0wc
	${SERV}-asis.${TODAY}
   # LOOKUP extracts the email address (note 5)
	LOOKUP=`echo ${X_FROM} \
	| sed -e 's/.*<\(.* @ .*\)>.*/\1/' \
	| sed -e 's/ *\(.* @ .*\) .*/\1/' \
	| sed -e' s/'\"'//g' \
	| sed -e's/'\''//g'`
	| tr A-Z a-z `
   # INLIST checks to see if LOOKUP is a user (note 6)
	INLIST=`cat ${HOME}/lists/${SERV} | grep -is ${LOOKUP} `
   # Handle subscriptions (note 7)
	:0wc
	* ! INLIST ?? (.)
	{
		:0w:serv-subscribe.lock
		| ${HOME}/scripts/serv-subscribe
   # Notification is returned 
		:0hw:serv-help.lock
		| (formail -I "X-Loop: subscribe" \
		-I "To: ${LOOKUP}" \
		-I "Reply-To: help@domain.zone" \
		-I "From: ${SERV}@domain.zone" \
		-I "Precedence: junk"; \
		echo " ${LOOKUP} subscribed"; \
		echo " "; \
		cat ${HOME}/scripts/serv-help; \
			) | $SENDMAIL -oi -t
	}
   # Refuse email if size exceeds 32K (note 8)
	:0hw:toobig.lock
	* > 32000
	| (formail -r \
	-I "X-Loop: ${SERV} refusal" \
	-I "Reply-To: help@domain.zone" \
	-I "From: errors@domain.zone" \
	-I "Precedence: junk"; \
	echo -e "email refused, too big - ${SIZE} bytes"; \
		) | $SENDMAIL -oi -t
   # Tag-line added to each outgoing email (note 9)
	TAG="
	
==================================
	 Replies to ${SERV}@domain.zone  
	 Limit email to 32K bytes. Website
	 at http://domain.zone/${SERV} 
	==================================
	"
   # clean up multipart- e2t extract (note 10)
        :0 w
        * ^Content-Type:.*multipart
        {
                :0 fhbw:serv-e2t.lock
                | ${HOME}/scripts/serv-e2t
        }
   # clean up quoted/printable
        :0w
        * ^Content-Type:.*text/plain
        * ^Content-Transfer-Encoding:.*quoted-printable
        {
                :0 fbw:serv-quoted.lock
                | ${HOME}/scripts/serv-quoted
        }
   # clean up base64 plain text encoding (UTF-8)
        :0w
        * ^Content-Type:.*text/plain
        * ^Content-Transfer-Encoding:.*base64
        {
                :0 fbw:serv-decode64.lock
                | ${HOME}/scripts/serv-decode64
        }
   # html text - lynx dump - no delivery
        :0 w
        * ^Content-Type:.*text/html
        {
                :0 fbw:serv-html.lock
                | ${HOME}/scripts/serv-html
        }
   # Posting script - tests and rewrites (note 10a)
	:0 w
	* INLIST ?? (.)            
	{
   # Rewrite headers of outgoing email (note 11)
		:0 fhw:rewrite.lock    
		| formail \
		-I "X-Loop: ${SERV} post" \
		-I "From: ${X_FROM}" \
		-I "Reply-To: ${SERV}@domain.zone" \
		-I "To: ${SERV}@domain.zone" \
		-I "Content-Type: TEXT/PLAIN; charset=US-ASCII" \
		-I "Cc:" \
		-I "Precedence: junk"; 
   # Cleanup for redistribution (note 12)
		:0 fbw:serv-cleanup.lock
		| ${HOME}/scripts/serv-cleanup
   # Add tag line (note 13)
		:0 fbw
		| cat - ; echo "${TAG}"
   # Un-archived copy deliver to cat list (note 14)
		:0 cw
		! `cat ${HOME}/lists/${SERV}`
   # Deliver very last copy to archive (note 15)
		:0bw:serv-archive.lock
		| ${HOME}/scripts/serv-archive
	}
   # Fallthrough to spooler (note 16)
		:0 
		${SPOOLER}

Notes on "serv-main"

Note 1: VERBOSE can be set here

Verbose logging results in a lot of information written to the file. If set here it will not do verbose logging of the other recipies, since that is controlled by the ".procmailrc" file.

Note 2: A separate logfile is used

A separate log file is not needed, but might be of use if the normal logfile gets a lot of writes during the day. Notice that a new logfile is started for each day with the use of $TODAY

Note 3: The logfile entries are separated by blank lines

Just a nicety of adding a blank line between log entries. You could also write a visible marker like ...

	-=-=-= new log entry =-=-=-
... or use available environmental variables to indicate which email is being handled - since at this point in the environmental variables $X_FROM and $X_SUBJECT are available.

Note 3a: Make directory and files

The website directory and the required files do not have to be created. As long as the 'name' is listed in the main ".procmailrc" file, the script serv-dir will make the directory, name it, and add the base files. It is a very short script, as follows:


	mkdir ${SITE}
	touch ${SITE}/list
	touch ${SITE}/next
	echo "1" > ${SITE}/next
	cp ${HOME}/scripts/serv-help ${SITE}/README.TXT
	chmod -R a+rx ${SITE}

Note 3b: Bounce if not using password

Generally not needed, but recommended if the listserv is public. On one listserv which has generated 2500 postings over 4 years, we have had only 3 attempts at spam. The reply "password required" always shuts them up. If a password is used, include it with the ".procmailrc" script. Since this is a environmental variable, it has to be reset (to a blank) with every additional listserv recipe. That will be as simple as

PW=

Note 4: Save incoming email unfiltered, just in case

Not needed, but strongly suggested, especially when you are dealing with users new to the listserv world. Saved in separate files for each day. This will include emails which fall through the script.

Note 5: LOOKUP extracts the email address from "From:"

The "From:" header might take any number of forms ...

The first two sed commands extracts the bare email address; the next two remove any single or double quote mark as a matter of security; the last line changes it to lower case.

Note 6: $INLIST checks to see if $LOOKUP is a user

The value of $LOOKUP is compared line by line with the values of the list of users. This list has the same name as $SERV, and in this example would be 'economics' in the subdirectory "lists/" - but could be anywhere convenient. If the email address is already included, then $INLIST takes a value, otherwise it does not. This is used for new subscriptions, and could also be used for the redistribution section.

Note: I originally used "egrep" instead of "grep". Egrep doesn't work properly.

Note 7: Handle subscriptions

Subscriptions are handled with this recipe. Read "* ! INLIST ?? (.)" as, "if not $INLIST has an assigned value" -- meaning the $LOOKUP email address was not found in the list of subscribers. IF $INLIST has no value, the steps needed to add a subscriber are taken. These include...

Note 8: Refuse email if size exceeds 32K

The limit is here set at 32K which ought to be large enough for just about anything except some bloated-format HTML email. A "Reply-To:" header reads "help@domain.zone" and the "From:" header is set to "errors@domain.zone". Note that the email body will specify the size of the email which was received.

Note 9: Tag-line added to each outgoing email

The tag line will not be added until after "serv-cleanup" has been called. The tag line uses the equal sign as the score. This allows "serv-cleanup" to remove the tags from previous emails. The "serv-archive" script removes it again. $SERV is of course substituted with the value (name) of the current listserv.

Note 10: Dealing with Multipart, Quoted/printable, Base64 encoding, UTF-8 character set, and HTML format

If you do not disallow HTML-ized email, or multipart email, then the bulk of these will pass unmolested. The Archiving script, however, will have problems with HTML-ized and multi-part texts, but some measure can be taken to clean up weird email.

Additionally, fitering emails will radically reduce their size and make them universally readable.

What we have in the above is four separate recipes using four additional body filters, staggered so as to catch even the recursive stupidly formatted email and reduce it to plain text. Saves a lot of grief.

I will not present the filters here, since these consist of a Perl script (serv-e2t, 4686 bytes), a Lynx dump (serv-html, 224 bytes), a two Perl Mime decoders (serv-quoted, 200 bytes, and serv-decode64, 172 bytes). If you need to know, ask me. The set has worked consistantly on a dozen listservs for years.

However, because an extensive Perl library is not always available on some systems, you would be advised to visit [http://raf.org/textmail/] and check out the file 'textmail'. Textmail is larger, 49,964 bytes, and is a Perl script, but does not require any special Perl modules. It can be installed as a filter.

Note: the download is a selfcontained script, but a browser might fold the long lines, so for a download use ..

lynx -source http://raf.org/textmail/textmail > /usr/local/bin/textmail

.. (or place it where you want it, like /usr/local/share, or whatever is in use on your operation system). Add permission to execute as required. Access a manpage with "textmail -m".

... after the filters

Note that in all the cases the "Content-Type:" header has to be rewritten after application of these filters in order to keep the mail reader of the recipient from screwing up. The inclusion of

         "Content-Type: TEXT/PLAIN; charset=US-ASCII" 
          and 
	 "Content-Transfer-Encoding: 7bit"

when the headers are being rewritten (above) will accomplish this.

Note 10a: Posting script - a series of tests and rewrites

This recipe checks (again) if the "From:" address is in the user list for the listserv. That condition is not really needed.

Note 11: Rewrite headers of outgoing email

This is a "filter", that is, changes are made to the email, but it is not delivered. The changes consist of rewriting some of the headers. Notice that by inserting a "Cc:" header the original "CC" is overwritten. This protects against spurious deliveries.

The "From:" header takes the value of the email address of the sender, the "Reply-To:" header is of course the listserv. The "To:" header is also overwritten to remove multiple addresses. The "Content-Type:" header is discussed below.

Note 12: Cleanup for redistribution

Cleanup of the email is accomplished with a shell script. See "serv-cleanup", below.

Note 13: Add tag line

The TAG is added after a cat of the complete email body.

Note 14: Un-archived copy deliver to cat list

The email is now delivered to the cat of the SERV list. The number of names is limited by the allowed size of the command line, and probably is good for about 80 or 90 addresses. After that a normal list-alias will need to be instituted. A copy is made for a secondary delivery to the archive script.

Note 15: Deliver very last copy to archive

See the "serv-archive" below details.

Note 16: Fallthrough to spooler

Everything else falls through to the spooler.

The Script "serv-subscribe"

"Serv-subscribe" is a shell script which adds the bare email address from the passed $LOOKUP environmental variable to the list of users. A copy of the email could be passed along to the rest of the "serv-main" script - so that everyone gets notification of the new user. Add a "c" to the first line to accomplish that (shown in the script above).

Alternately, the first email could be suppressed, by removing the "c" from the first line of the recipe (note 7 in the main script). However, forwarding the first email to the rest of the users might start some dialog. "Serv-subscribe" goes like this...


   #!/bin/sh
   # script/serv-subscribe 
   # LOOKUP is passed as an e.v.
	cp ${HOME}/lists/${SERV}  ${HOME}/lists/foo
	echo "${LOOKUP}"  >> ${HOME}/lists/foo
	cat ${HOME}/lists/foo | sort | uniq > ${HOME}/lists/${SERV}

If, as the owner, you want to be notified of new subscriptions, add the following two lines to the "serv-subscribe" script


   echo -e "${SERV} new subscribe: ${X_FROM}" > ${HOME}/update
   mail -s "${SERV} list update" me@localhost < ${HOME}/update

Not shown is the ability to unsubscribe.

The Script "serv-cleanup"

"Serv-cleanup" is a shell script which acts only as a filter. The concern here is to reduce redistributed email to a reasonable size by removing all email-quoted text, tag lines, and duplicate blank lines. Not all of these need to be removed, and some can be rewritten. If you don't care about the look of the archive, leave the left hooks in place, and watch text blow.


   #!/bin/sh
   # serv-cleanup -- filter only
   # following establishes a new-line as an e.v. for sed
	NL="
	"
   # email body gets filtered:
	cat - \
	| sed -e '/^>/d' \
	| sed -e'/(---Original)|(--- Original)/,$d' \
	| sed -e'/===/,$d' \
	| sed -e'/___/,$d' \
	| sed -e 's/^[[:space:]]*$//g' \
	| sed -e 's/[[:cntrl:]]/ /g' \
	| sed -e'/^$/{'"$NL"'N'"$NL"'/^\n$/D'"$NL"'}'

Filtering is accomplishes as follows:

  sed -e '/^>/d'
Removes email-quoted texts, but I have had so many complains, that I started to substitute QUOTE for the left hook. The reason should be obvious: I cannot deal with hooks in a html-ized archive. And the word QUOTE in caps makes it clear to the users that the email has not been cleaned up.
  sed -e'/(---Original)|(--- Original)/,$d'
Removes from line starting with "--- Original", this removes everything (to the end) that represents quoted text. There are 3 or 4 varieties of this format in use. I catch most of them, but not all. I have only shown one form here.
  sed -e'/===/,$d'
Removes to the end from any ==== score, which is the listserv tag.
  sed -e'/___/,$d'
Removes to the end from any tag line, but does not catch all of them.
  sed -e 's/^[[:space:]]*$//g'
Removes leading blank space, including tabs,
  sed -e 's/[[:cntrl:]]/ /g'
Removes any control characters,
  sed -e'/^$/{'"$NL"'N'"$NL"'/^\n$/D'"$NL"'}'
Removes duplicate blank lines.

Upper ASCII (8-bit) characters can also be removed with..

  | sed -e 's/[^[:graph:]]/ /g'\

The Script "serv-archive"

"Serv-archive" is a shell script. It will require a few comments. First, you need not set up a directory at your web pages. The initial email will do that, populate the directory with the needed files, and set permissions (for which, see above). Permissions for the directory are 'drwxr-xr-x'. The files need '-rw-r--r--' permission.

The students will not get a default index file to look at, but will get a directory listing (generally the default for an Apache server, but ask the admins if this is not so).

In the 'economics' directory will be a list of files..


                               Index of /economics
 Icon   Name                    Last modified      Size  Description
__________________________________________________________________________
 [DIR]  Parent Directory                             -
 [DIR]  bin/                    12-Jul-2003 04:29    -
 [TXT]  July.htm                30-Jul-2003 14:08   12K
 [TXT]  June.htm                30-Jun-2003 23:29  9.5K
 [   ]  calendar                26-Jun-2003 01:28  470
 [TXT]  handouts                29-Jun-2003 21:02  711
 [   ]  next                    30-Jul-2003 14:08    3
 [TXT]  notes.htm               26-Jun-2003 01:28  453
 [TXT]  syllabus.htm            26-Jun-2003 03:23  8.9K
__________________________________________________________________________
README.TXT text follows this line....


To explain this set of files...

Obviously the directory has to be pre-populated with at least the following files to make it work...

We will assume that the "economics" directory, as with all the other web files is located in the "public_html" subdirectory of the user account. Here is the script which does the archiving (lettered notes below)...

The script "serv-archive"


   #!/bin/sh
   # serv-archive  -- generic version for procmail use 
   # Set env variables (note a)
	ARCHIVE=${HOME}/public_html/${SERV}/`date +%B`.htm
	NL="
	"
   # Increment the post number (note b)
	echo "`cat ${HOME}/public_html/${SERV}/next` 1 + p" | dc \
	> ${HOME}/figure 
	cp ${HOME}/figure ${HOME}/public_html/${SERV}/next
	NEXT=`cat ${HOME}/public_html/${SERV}/next`

   # Extract a name from the email address (note c)
	NAME=`echo ${X_FROM} \
	| sed -e's/^<\(.*\)@.*>$/(\1)/' \
	| sed -e's/<.*@.*>//' \
	| sed -e's/^\(.*\)@.*$/(\1)/' \
	| sed -e 's/^.*@.* //' \
	| sed -e 's/(\(.*[ ]+.*\))/\1/' \
	| sed -e 's/"//g'`

   # Clean up the Subject line (note d) 
	SUBJ=`echo ${X_SUBJECT} \
	| sed -e 's/[[:cntrl:]]/ /g' \
	| sed -e 's/[\/><]/ /g' \
	| sed -e' s/^[[:space:]]*//' \
	| sed -e' s/[[:space:]]*$//' \
	| sed -e' s/'\"'//g' \
	| sed -e's/'\''//g'`

   # Htmlize the header (note e)
	echo -e "\n</UL><P><A HREF=\"#${NEXT}\">[down]</A>" >> ${ARCHIVE} 
        echo -e "<P><HR><UL>" >>${ARCHIVE}
        echo -e "<LI>received: `date`" >> ${ARCHIVE}
        echo -e "<LI>${NAME} writes:" >> ${ARCHIVE}
        echo -e "<LI><I>${SUBJ}</I> \n</UL><P>" >> ${ARCHIVE}

   # Cleanup for archive (note f)
	cat - \
	| sed -e '/===/,$d' \
	| sed -e '/___/,$d' \
	| sed -e 's/@/ at /g' \
	| sed -e 's/[[:cntrl:]]/ /g'\
	| sed -e 's/[^[:graph:]]/ /g'\
	| sed -e 's/^[[:space:]]*$//g' \
	| sed -e '/^$/{'"$NL"'N'"$NL"'/^\n$/D'"$NL"'}' \
	| sed -e 's/^$/<\/UL><P>/g' \
	| sed -e '/<P>$/{'"$NL"'N'"$NL"'s/\n//'"$NL"'}'\
	| sed -e 's/<P>[[:space:]]*"/<P><UL>"/' \
	| sed -e 's/^[[:space:]]*-/<BR> -/g' \
	| sed -e 's/http:\/\/[[:graph:]\.\/]*/<A  HREF="&">[&]<\/A> /g' >> ${ARCHIVE}

   # Add to existing archive web files (note g)
	echo -e "</UL><P><A NAME=\"${NEXT}\"></A>" >> ${ARCHIVE}

   # Make readable (note h)
	chmod a+r  ${ARCHIVE}

Notes on "serv-archive"

Note a: Set env variables

The variable $ARCHIVE is set to keep down the length of the lines of the script. $ARCHIVE is the growing archive of posts for the current month. A new $ARCHIVE is written when the month changes. $NL is a new-line used by sed.

Note b: Increment the post number

Every archived email is numbered. This is done for record keeping, and to allow jumping down from one email post to the next, and aids searching if the posts get lenghty. Additionally, a script can be used to remove any post by its "number" since the number occurs in a HREF and NAME anchor which enclose the post, for example...


	#!/bin/sh
	# remove $1 $2 (Month_file, Post_number)
        cp $1 foo
        sed -e '/\"#'"$2"'\"/,/\"'"$2"'\"/d' foo > $1

Note c: Extract a name from the email address

The $NAME variable is used in writing the header for a post in the form "${NAME} writes:... etc"

The email address "name" is either the readable name suplied with the email address of the sender or, if there is no plain name, the account name. The following will work with any form of a legitimate email address in use (tested on over 100,000 addresses). It works. Extracting $NAME involves... (line by line):

  sed -e's/^<\(.*\)@.*>$/(\1)/'
save the portion following an angle bracket at the beginning up to a @ sign,
  sed -e's/<.*@.*>//'
delete anything including a @ sign if within angle brackets,
  sed -e's/^\(.*\)@.*$/(\1)/'
save anything from the beginning up to a @ sign,
  sed -e 's/^.*@.* //'
delete anything from the start which includes a @ sign,
  sed -e 's/(\(.*[ ]+.*\))/\1/'
Save anything from the start which includes a blank space,
  sed -e 's/"//g'
delete any quotes.

Note d: Clean up the Subject line

The Subject line is cleaned of weird characters, leading and closing spaces, slashes and hooks, and protected against hacking. If a Subject line password is used, this would be a good place to remove it. The sed commands, line by Line:

  sed -e 's/[[:cntrl:]]/ /g'
control characters (often from Mac users) are substituted with blanks. Upper Ascii characters should really also be removed.
  sed -e 's/[\/<>]/ /g'
Hooks and slashes will not pass in HTML format.
  sed -e' s/^[[:space:]]*//' 
  sed -e' s/[[:space:]]*$//'
Opening and closing space are deleted for aesthetic purposes
  sed -e' s/'\"'//g'  
  sed -e' s/'\''//g'
Included single and double quotes are deleted

Note e: Html-ize the header

Posts are separated with a score, and preceeded with a HREF anchor. The text will be followed by a NAME anchor with the same $NEXT value. The header is a dot list of "when", "who", and "what".

The current local machine date and time are used, because senders are amazingly unreliable in getting the time right on their own machines. "Who" consists of a name from the "From:" header or the account name. The Subject is a cleaned up version of the original "Subject:" header.

Notice that every P tag is preceeded by a /UL tag. This is because the UL tag is used to indent paragraphs, but there is no way to close the UL section.

All of the header is written to $ARCHIVE.

Note f: Cleanup for archive

Quoted text, tag lines, and duplicate lines are removed. In addition the text is HTML-ized as follows..

The result of the cleanup is added to $ARCHIVE. Step by step as follows...

  sed -e '/===/,$d'  
  sed -e '/___/,$d'
Everything from a tag line to the end of the text is deleted.
  sed -e 's/@/ at /g'
The "at" sign is replaced by (space)at(space) to keep email addresses from being harvested.
  sed -e 's/[[:cntrl:]]/ /g'  
  sed -e 's/[^[:graph:]]/ /g'
  sed -e 's/^[[:space:]]*$//g'
Control characters, non-printing characters are replaced by blanks; lines consisting of blanks and tabs are reduced to zero space.
  sed -e '/^$/{'"$NL"'N'"$NL"'/^\n$/D'"$NL"'}'
Duplicate blank lines are removed.
  sed -e 's/^$/<\/UL><P>/g'
Zero space lines are replaced by /UL,P tags.
  sed -e '/<P>$/{'"$NL"'N'"$NL"'s/\n//'"$NL"'}'
The /UL P tag lines are placed at the beginning of the following line.
  sed -e 's/<P>[[:space:]]*"/<P><UL>"/'
The UL tag is introduced at lines (actually paragraphs) starting with a leading quote mark; leading spaces are deleted.
  sed -e 's/^[[:space:]]*-/<BR> -/g'
The BR tag is introduced at lines starting with a hyphen and leading space is reduced to a single space.
  sed -e 's/http:\/\/[[:graph:]]*/<A HREF="&">[&]<\/A> /g'
Href anchors are created from included URLs.

Note g: Add to existing archive web files

A NAME anchor is added at the end of the text, that is, it is added to $ARCHIVE.

Note h: Make readable

Public read permission needs to be set on these files.

The help file, README.TXT

Following is a typical text...


 Directions for use: send email to "economics at domain.zone"

 (1) These emails will be redistributed to everyone on the list, 
 (see the list) and are archived here.
 (2) Your first email will add your address to the list.
 (3) No links point to this website - it is invisible to the world.

If the site is intruded on by an outsider we can add passwords,
and disable the auto-subscribe feature.

- Email has to be smaller that 8K, or will be refused.
- Clean up your mail before replying, and sign your name.
- A "Reply-to:" header is added, it will respond to everyone.
- The "From:" header is retained - it is the person posting.

In order to have the emails readable on the web page, the text 
is HTML-ized. Since all empty space is ignored by browsers,
- Skip a line between paragraphs
- Start each item of a list with a hyphen (ascii 45) at the left
- To indent a paragraph start with a quote (ascii 34) at the left
- start links (URLs) with 'http://' (and end with a blank)
- If you have further questions send email to "help".


[] Website Provider: Outflux.net, www.Outflux.net
URL: http://jnocook.net/geek/serv.htm