Configure a Cron Job with a Wiki

I have some periodic cron jobs that need extra configuration.  For example, one of them generates a report on bug statistics on a code branch basis.  So I need to tell it which code branches to process.  I could just put the list of branch tags on the command line of the report generator, and just use "crontab -e" while logged in to modify it.  However, I want anybody to be able to maintain the list, without having to know my password or the syntax for crontab.

It turns out that we installed Mediawiki locally for our own internal wiki.  So I created a wiki page with a table listing the code branches that are active.  Then I wrote a script which uses "curl" to fetch that wiki page and parse out the branches.  This gives me a nice web-based GUI interface to the tool that everybody is already familiar with.  Everybody here knows how to use Wikipedia, so anybody can go in and change the list of branches.

After doing some additional development, I wanted to be able to include additional configuration for the cron job which I didn't particularly want displayed on the wiki page.  You can use <!--HTML-style comments--> with Mediawiki and it won't display it on the page.  Unfortunately, it completely withholds the comment from the html of the page.  I.e. you can't see it even if you display source of the page.  You only see the comment when you edit the page.

So here's what I ended up with:

#!/bin/sh

# all.sh

# Generate QA report for all active releases.  Runs via cron nightly.

date

# Read wiki page and select rows in the release table ("|" in col 1).

# Uses "action=edit" so that are included (for option processing).

curl 'http://localwiki/index.php?title=page_title&action=edit' | egrep "^\|" >all.list

# read the contents of all.list, line at a time.  Each line is an entry in the table of active releases.

while read ILINE; do :

    # Extract target milestone (link text of web-based report page).

    TARGET_MILESTONE=`echo "$ILINE" | sed -n 's/^.*_report.html \(.*\)\].*$/\1/p'`

    # Extract the (optional) set of command-line options.

    OPTS=`echo "$ILINE" | sed -n 's/^.*--OPTS:\([^:]*\):.*$/\1/p'`

    eval ./qa_report $OPTS \"$TARGET_MILESTONE\" 2>&1

done <all.list

The "sed" commands use "-n" to suppress printing of lines to stdout.  Adding a "p" suffix to the end of a sed command forces a print, if the command is successful.  So, for example, the line:
    OPTS=`echo "$ILINE" | sed -n 's/^.*--OPTS:\([^:]*\):.*$/\1/p'`
If the contents of $ILINE does not contain match the pattern (i.e. does not have an option string), the "s" command is not successful and therefore doesn't print, leaving OPTS empty.

One final interesting note: the use of "eval" to run the qa_report.sh script.  Why couldn't you just use this?
    ./qa_report $OPTS "$TARGET_MILESTONE" 2>&1

Let's say that $TARGET_MILESTONE is "milestone" and the contents of $OPT is:

    -a "b c" -d "e f"

If you omit the "eval", you would expect the resulting command line to be:

    ./qa_report -a "b c" -d "e f" "milestone" 2>&1

I.e. the qa_report tool will see "b c" as the value for the "-a" option, and "e f" as the value for the "-d" option.  But the shell doesn't work this way.  The line:

    ./qa_report $OPTS \"$TARGET_MILESTONE\" 2>&1

will expand $OPTS, but it won't group "b c" as a single entity for -a.  Without "eval", the "-a" option will only see the two-character value "b (with the quote mark).  I found a good explanation for this; the short version is that the shell does quote processing before it does symbol expansion.  So essentially, the thing you need to do is have the shell parse the command line twice.

The "eval" form of the command works like this:
    eval ./qa_report $OPTS \"$TARGET_MILESTONE\" 2>&1
First the shell looks at this command line and parses it with "eval" as the command and the rest as "eval"s parameters.  It does the symbol substitution.  Thus, the thing that gets passed to "eval" is:
    ./qa_report -a "b c" -d "e f" "milestone"
What does the eval command do with that?  It passes it to the shell for parsing!  In this pass, "./qa_report" is the command, and the rest are the parameters.  Since the shell is parsing it from scratch, it will group "b c" as a single entity, letting the "-a" option pick it up as a single string.