1. Recent Leeland's Side Notes

• 2009-06-22 12:00:37: Recovering Lost Data From Page Firefox Cache
  Big company, large IT department, huge wiki server used by all. Do they have a recent backup? If you count 6 months old as recent then sure. Of course for a project started since the last back up that put 100% of the documentation into the corporate wiki when the server breaks and they have to go back to the back-ups this sucks.
  
  Now of course being a tech-head I have ultra paranoid cleanup settings on my cache, browse history and other things. But, managers and office types often don't care nearly as much.
  
  Enter the very cool Firefox feature about:cache now that is a handy little magic URL. And thankfully some people just don't bother with making sure their cache is cleaned out on exit.
  0 Comments
• 2009-05-19 16:02:49: Long Pause
  Wow, cannot believe so much time has passed. So what happened? Why a child of course. My son hit a coming to awareness level where he wasn't just a feed -> sleep -> poop machine but started talking, playing and of course the running.
  
  This of course is the point where most parents disappear from everyone's lives for a year or two.
  
  Now my desk is buried in piles of papers, my books have a nice layer of dust, and the house needs a year's worth of maintenance. Oh and I am trying to remember where I left off with this place. I see nothing has changed.
  0 Comments

Read The Blog

2. What Is Going On

The original idea was for NOD SW to be about Natural Order Development software projects, programming (process, techniques and patterns) best practices for software projects. A kind of specialized collection of resources, tools and information. But that seems so dry now that I am actually trying to build it and seeing it come into being.

There have been a few things I wanted to do where I have said to myself "self, that really wont fit with the NOD only theme." So after debating with my self a lot ( "self, then there would be less interesting things, it can still be themed but not so tightly constricted" , "hmm, you have a point their self. OK maybe I can open it up. Lets expand the to do list with more of the community items and then see what that looks like." , "self, you got a deal." -- I am sometimes very easily swayed by myself.)

The best thing to do is to make NOD SW a simple collection of resources, tools and information We ( *as in YOU and everyone one else who comes here* ) find useful with no overall theme other then being about programming. This is a place to explore what can be done with modern programming techniques and modern languages like Java, Python, Jython and shell scripting. I expect the usefulness or even originality to be a bit hit and miss but that's not a bad thing.

Of course this will be a lot more interesting if more people toss in their ideas and occasionally stand on soap boxes (of various sizes). So please don't hesitate to join up (its free and no spam) use the site and of course discuss, comment, question and/or criticize (politely of course) without hesitation.

I expect once I get the features below set up this main page will be replaced with the blog. In the mean time this page will keep track of what is happening.

3. Site News

• 2008-06-26 14:25:27: FCGI and output streams
  OK so this took more then a while to figure out. I had to go read the FAST CGI specification (http://www.fastcgi.com/devkit/doc/fcgi-spec.html), the lighttpd FastCGI Interface docs (http://trac.lighttpd.net/trac/wiki/Docs:ModFastCGI), the Django FastCGI docs (http://www.djangoproject.com/documentation/fastcgi/) and a number of other forum posts in a number of groups.
  
  All this just to answer the question: How do I get my Python Django applications to print to the the lighttpd error logs when using Lighttpd + FastCGI?
  
  At last I found the answer. At least partially...
  
  The crux of this problem is that FastCGI is a multi-threaded and multiplexed communication protocol. Just calling print is not enough. The output streams need to be encoded and directed to the right channel and in a way that the FCGI service can untangle them to know where they should be routed.
  
  This is a messy problem due to the complexity, the poor documentation, the incomplete state of the specifications and also the incomplete state of the FCGI supporting modules on both the server and client ends.
  
  So here is the basic trick.
  
  You can not do this from a settings file. The streams are not available UNTIL there is a call that causes the FCGI instance to initiate a request. The streams will be available via the request call. They might be set up in the environment (there are supposed to a couple of hooks for you to look for and then snag onto). I haven't finished getting all this figured out yet. But, I have succeeded in having a multi-threaded call to a single FCGI channel successfully process all the data into responses with resulting messages ending up in the expected logs.
  
  Once you have the request it is not too difficult to do this. For example within the views of a Django application you can do this:
  
  

  if 'wsgi.errors' in request.META:
      request.META['wsgi.errors'].writelines("An error message for the server's error logs.\n")
      request.META['wsgi.errors'].flush()
  

  
  
  Now that I know where to find these channels and I know how to use them correctly (using them correctly is the key). I can experiment with a few other tricks I know to see if I can make this as easy as:
  
  

  if isFCGI():
      dup2StdStreams()
  
  print "This should go to the access log."
  print >>sys.stderr "This should go to the error log."
  

  
  
  If not then I can at least hook into the python logging and set up the logging to go to the server's access and error logs.
  0 Comments
• 2008-06-17 00:50:49: Captcha is online
  With a few code reviews and tweaks to SCT Tools basic captcha support is now online. I still need to integrate the captcha call into the user registration system.
  
  The nice part is now email addresses are hidden unless a captcha check is done. Further captcha checks time out after a short period. Is this open to some kind of mining pass? Possibly a replay attach. Need to check that out. Of course Django is supposed to be pretty solid. Need to see about coping cookies around a little.
  0 Comments

View All News Articles

4. Features This site will have:

This site will be based on Django and initially starting with Sphene Community Tools as the foundation. Everything else will be engineered to overlay on top of that set of initial tools. Yes, I will be contributing back into both of those when I find issues, enhancements, etc.

• Items are in the order to be work.
• Items marked with are not yet being worked on.
• Items marked with are considered completed.
• Items marked with are currently being worked on.

1. User System (UserSystem)
   • enhance the core captcha framework to allow for settings_local.py control
   • optional captcha for registration
   • login using user name (aka primary display name) or email address
   • Multiple Display names (for use in various areas TBD)
   • email availability settings (Google like with a captch between initial display and full display of email names)
   • Auto assigning groups based of various criteria (number of posts, number of submissions to tags, weighted posting [replying with acknowledge helpful advice counts as more] and subscription levels)
   • Multiple Signatures (for use in various areas TBD)
   • Graphical Signatures (based on authorization / user roles)
   • Push the captcha into a pluggable module.
2. Blog
   • Blog uses BBCode markup style
   • Blog posts have a WSIWG editor built that provides easy use of all allowed markup (including any allowed HTML)
   • Multiple Named Blogs supported displayed by list or locked down specified in an area
   • If authorized administrative ability to edit all blog articles
   • If authorized user ability to re-edit their own blog articles
   • If authorized administrative ability to edit all blog comment
   • If authorized user ability to re-edit their own blog comments
   • Comment time stamps are settable and editable with authorization
   • Blog article time stamps are settable and editable with authorization
   • Comments or blog articles with a future time stamp are not shown until the time set
   • Users with authorization may activate, deactivate and set a viewing time window to include future articles
3. Tags
   • tag cloud spanning all site items (source, files, images, blogs, etc.)
   • auto-focus reduction tag-cloud (meaning as you drill in there will be the ability to have a focused tag-cloud for the subsection you are in)
4. Wiki
   • Ability to break it down by areas
   • ability to grant users various rights within sections
   • Tied into the tags
   • On demand tables of content
   • ability to embed HTML if needed
   • Source code highlighting for all the languages I do (bash, Java, Jython, Python, HTML, CSS, XML, Javascript, C, etc.) See CodeHighlighting
5. Subversion Repository Browsing
   • tied into license systems for bundled access based on roles
   • Source code highlighting for all the languages I do (bash, Java, Jython, Python, HTML, CSS, XML, Javascript, C, etc.) See CodeHighlighting
6. Trouble Ticket System (TroubleTickets)
   • ability to grant users various rights within sections
   • If authorized ability to edit all parts of the ticket post
   • Source code highlighting for all the languages I do (bash, Java, Jython, Python, HTML, CSS, XML, Javascript, C, etc.) See CodeHighlighting
   • ability to link it one or more wiki pages, blogs, forum discussions and source code in the system
7. Forums
   • ability to grant users various rights within sections
   • ability to move conversations to other forums
   • ability to hide / show forums based on permissions / roles / groups
   • ability to allow a forum posts to be re-edited by the poster (or an authorized admin) with full logging of the change (in a selectable change log forum and/or Wiki like using a DB table)
8. Search
   • A complete context search engine across everything including the source code in the subversion sections
9. Shopping Cart
   • Support for subscriptions (represented as Roles that are activated, renewed or deactivated automatically)
   • Support for bundled licenses (lifetime, time limited and limited number of accesses)
10. News
    • Similar to blogs but represented more like snippets (show the headline and the intro, click on it to see the whole story.)
    • included on the pages in subsections of some number
    • named news feeds with identifying tags for CSS overriding
11. Subprojects
    • A definable subsection of the site with one or more of the above elements all to itself.
12. All the other bells and whistles (SiteTechNotes)
    • A WSIWG editor for anything that takes text
    • Award winning layout
    • Fast, fast, fast
    • fully automated
    • Backed up (with snapshots)
    • Repeatable, documented installation & recovery process