Natural Order Development

With a few code reviews and tweaks to SCT Tools basic captcha support is now online. I still need to integrate the captcha call into the user registration system.

The nice part is now email addresses are hidden unless a captcha check is done. Further captcha checks time out after a short period. Is this open to some kind of mining pass? Possibly a replay attach. Need to check that out. Of course Django is supposed to be pretty solid. Need to see about coping cookies around a little.

Oh what a relief, I was trying to figure out why none of the static elements of the site were working. Well not really. I was actually trying to determine how the templates get all wrangled up when I noticed a lot of 404 error messages in the log files. Which is when I figured out why some of the site's interfaces were not working exactly right.

Turned out that although the template loaders for Django have a nice "go find the template relative to the application asking for it" feature. The static stuff like CSS pages, images and other elements do not have that ability. Instead they all ask for the /media/ directory.

So what was breaking was two fold:

I had a typo in the alias url definition I forgot the trailing slash. Curd I have been bitten by this in Apache and now in lighttpd too. If you alias a directory say "/media/" then you need to specify where to look for that directory and that needs to have a matching trailing slash. So I had this:

    alias.url = (
        "/media/" => "/srv/nodsw.com/media",
    )

when what I was supposed to do was:

    alias.url = (
        "/media/" => "/srv/nodsw.com/media/",
    )

Big time dooh! I even knew better. Oh well. It is fixed now. So thing are looking much better.

Back at the job and code crawling. Got more done this weekend on my site project then I did in the last year. Of course a new kid will always crush your schedule for a couple of years. So not really complaining.

The main page now shows the site goals. I think I need to expand on those a little bit. But first I need to figure out how to integrate a source code control system. I want to be able to get at my source code from anywhere and I definitely want to be able to publish parts of it here.

I found a great code highlighting engine and there is a Wiki page on that now. (**Edit this with the right links**). LOL this will be a test to show I managed to get re-edit ability active for posts that YOU own. I am thinking something like a re-edit with change log should be about right.

Oh what a quagmire of crud to walk through. All of the Django docs on FCGI and lighttpd are pretty sparse and the examples are not what I would recommend for a production service. So first step was to improve the FCGI tie in for Django. This turned out better then I thought. I had to do a lot of guessing and testing. But now I have it set up so that when the lighttpd service is started and stopped the FCGI processes are also started and stopped. In other words the web service application actually controls its own subprocesses including the FCGI linking to python for the Django service.

I'll have to create a wiki pages on how I configured this place. But to save you time if you want to see my script here it is:

-- BEG start-fcgi.sh --

#!/usr/bin/env bash

NOW=`date +%Y-%m-%d_%T`
MY_LOG_DIR=/tmp
[ -n "${OUT_LOG}" ] && MY_LOG_DIR=${OUT_LOG%/*}
[ -n "${DJANGO_LOG_DIR}" ] && MY_LOG_DIR=${DJANGO_LOG_DIR}
[ ! -d $MY_LOG_DIR ] && mkdir -p  $MY_LOG_DIR
[ ! -d $MY_LOG_DIR ] && echo "$0 ERROR log dir $MY_LOG_DIR is not a directory" >&2 && exit -1
MY_LOG=${MY_LOG_DIR}/start_fcgi_${NOW}_$$.out
rm -rf $MY_LOG
echo "$0 process $$ started log file = $MY_LOG"
echo "$0 process $$ started" >> ${MY_LOG}
MY_DIR=${0%/*}
cd $MY_DIR
echo "command line: $0 $@" > ${MY_LOG}
echo "----- BEG ENV -----" >> ${MY_LOG}
env >> ${MY_LOG}
echo "----- END ENV -----" >> ${MY_LOG}
PIDFILE="pidfile=${MY_LOG_DIR}/django_fcgi_$$.pid"
[ -n "${DJANGO_PID}" ] && PIDFILE="pidfile=${DJANGO_PID}"

# if socket is not specified the fcgi process handling will negotiate the correct
# socket file via the fcgi API so best to NOT specify it
SOCKET=""
[ -n "${DJANGO_SOCKET}" ] && SOCKET="socket=${DJANGO_SOCKET}"
METHOD="method=prefork"
[ -n "${DJANGO_METHOD}" ] && METHOD="method=${DJANGO_METHOD}"

# stdout and stderr will go to the http server logs if not otherwise specified
OUTLOG=""
[ -n "${OUT_LOG}" ] && OUTLOG="outlog=${OUT_LOG}"
ERRLOG=""
[ -n "${ERROR_LOG}" ] && ERRLOG="errlog=${ERROR_LOG}"

CMD="./manage.py runfcgi $METHOD $SOCKET $PIDFILE $OUTLOG $ERRLOG"

echo "executing : exec ${CMD}" >> ${MY_LOG}

exec ${CMD}

-- END start-fcgi.sh --


You will note that this is controlled via the lighttpd.conf file. Here is the relevant section of mine:

-- BEG lighttpd.conf subsection --

## Special fastcgi handling for each domain under django python
# If the hostname is '*.nodsw.com' or 'nodsw.com' ...
$HTTP["host"] =~ "(^|\.)nodsw.com" {
    fastcgi.server += (
        "/mysite.fcgi" => (
            "main" => (
                "socket" => "/tmp/nodsw.socket", # This will get auto-incremented by fcgi
                "check-local" => "disable",
                "bin-path" => "/django/nodsw.com/start_fcgi.sh",
                "bin-environment" => (
                    "DJANGO_LOG_DIR"         => "/django/logs",
                    #"ERROR_LOG"              => "/django/logs/nodsw.django_error.log",
                    #"OUT_LOG"                => "/django/logs/nodsw.django_out.log",
                    #"DJANGO_PID"             => "/django/logs/nodws.django.pid"
                    #"DJANGO_SOCKET"          => "/tmp/nodsw.socket", # don't use this unless you are sure
                    #"PYTHONPATH" => "/django/greydragon/project"
                )
            )
        )
    )
    alias.url = (
        "/media/" => "/django/nodsw.com/media/",
    )

    url.rewrite-once = (
        "^(/media.*)$" => "$1",
        "^/favicon\.ico$" => "/media/favicon.ico",
        "^(/.*)$" => "/mysite.fcgi$1",
    )
}

-- END lighttpd.conf subsection --

The database is running. Django seems happy. I have managed to get things kind of set up. Still need to figure out a lot of little things. But it is a start. I hope to have Natural Order Software up and rolling as well as Greydragon off of the same code base. We'll see.