emson…

I was recently challenged to write a quine in the language of my choice. If you are unsure what a quine is you can find out from http://en.wikipedia.org/wiki/Quine_(computing). However basically in our case it is a piece of code that when executed creates a copy of itself. This copy can then be executed and ad infinitum.

There are some official rules about how a quine should be implemented:

• A quine must not take any input, as this would allow the source code to be fed in from the keyboard.
• Following on from the previous rule a quine, should not open it’s own file and display it’s contents.

Anyway here is my solution in Ruby…

    lambda{|x| puts x + x.inspect}.call "lambda{|x| puts x + x.inspect}.call"

Here is a brief description of what is happening. The lambda creates an anonymous function that takes one parameter and outputs it’s value

Officially I think passing a value into the lambda x attribute violates one of the rules – but non-the-less it is quite a clear example and may be useful to get an idea of how a quine works.

Links

Here is a list of interesting links that might help, good luck:

• An excellent overview of quine’s with a Ruby slant: http://blog.onideas.ws/ruby_quine
• Understanding the concepts of a quine: http://research.swtch.com/2010/03/zip-files-all-way-down.html
• Using a quine as an interview question: http://www.skorks.com/2010/03/an-interview-question-that-prints-out-its-own-source-code-in-ruby/
• Shortest Ruby quine: http://stackoverflow.com/questions/2474861/shortest-ruby-quine
• More quine links: http://www.nyx.net/~gthompso/quine.htm

The concepts of a Proc and a lambda in Ruby are subtly different and this post aims to try and explain the differences.

Procs and Lambdas

Firstly why do we need Procs and lambdas? In order to answer this you need to understand what a block is in Ruby.

Understanding what a block is?

A block is simply a piece of code, which is usually attached to a method, Proc or lambda. It is important to note that this block of code is NOT an object (yet…), it is code that is defined either within braces {} or within a do end container.

The code in a block is just logic that hasn’t been wrapped into an object yet. When this block is passed to a method and that method has a yield statement then Ruby will wrap that block into a special type of object. That type of object is a Proc object. Once the block becomes a Proc Ruby can use it and the block code ‘comes alive’.

Therefore we need a Proc object in order to handle blocks. The next question is how does a lambda fit in.

A lambda is a type of Proc object, it is almost completely identical to a Proc apart from two differences. This little code snippet shows that a Proc and a lambda are derived from the same superclass, Proc.

  def what_are_lambdas_and_procs
    puts "--- #{__method__} ---"
    lam = lambda { puts 'lam variable assigned a lambda' }
    lam.call
    puts "lam is a class of: #{lam.class.name}"
    puts "lam method count: #{lam.methods.size}"
    puts
    prc = Proc.new { puts 'prc variable assigned a Proc' }
    prc.call
    puts "prc is a class of: #{prc.class.name}"
    puts "prc method count: #{prc.methods.size}"
    ""
  end

This will output:

  --- what_are_lambdas_and_procs ---
  lam variable assigned a lambda
  lam is a class of: Proc
  lam method count: 55

  prc variable assigned a Proc
  prc is a class of: Proc
  prc method count: 55

So what are the subtle differences between Proc and lambda?

As mentioned there are two differences, and these differences are :

1. in how a Proc/lambda assigns arguments, and
2. in what happens, when a Proc/lambda calls return within a caller method.

Proc, lambda argument assignment

When a lambda is called with more or less than the required number of arguments Ruby will throw an ArgumentError.

However when a Proc is called with more arguments, no error is thrown and the extra values are simply thrown away. When it is called with less arguments these parameters are simply assigned a value of nil.

These two code snippets demonstrate this:

  def lambda_args
    puts "--- #{__method__} ---"
    lam = lambda { |a, b| puts "lambda with 2 arguments a:#{a}, b:#{b}" }
    lam.call( "first", "second")
    puts "lambda called with three arguments..."
    begin
      lam.call("first", "second", "third")
    rescue
      puts "   ArgumentError: wrong number of arguments (3 for 2)"
    end
    puts "lambda called with no arguments..."
    begin
      lam.call()
    rescue
      puts "   ArgumentError: wrong number of arguments (0 for 2)"
    end
  end

  def proc_args
    puts "--- #{__method__} ---"
    prc = Proc.new { |a, b| puts "Proc with 2 arguments a:#{(a.nil?) ? "nil" : a }, b:#{(b.nil?) ? "nil" : b }" }
    prc.call( "first", "second")
    puts "Proc called with three arguments..."
    prc.call("first", "second", "third")
    puts "Proc called with no arguments..."
    prc.call()
  end

This will output:

  --- lambda_args ---
  lambda with 2 arguments a:first, b:second
  lambda called with three arguments...
     ArgumentError: wrong number of arguments (3 for 2)
  lambda called with no arguments...
     ArgumentError: wrong number of arguments (0 for 2)

  --- proc_args ---
  Proc with 2 arguments a:first, b:second
  Proc called with three arguments...
  Proc with 2 arguments a:first, b:second
  Proc called with no arguments...
  Proc with 2 arguments a:nil, b:nil

Proc and lambda return behaviour

When a lambda returns a value and this lambda is called within a method then method will simply continue it’s execution until it itself returns a value.

However when a Proc returns a value and this Proc is called within a method this containing method will be exited WITHOUT any further execution. This is slightly surprising behaviour, and consequently a Proc should be handled with care.

These two code snippets demonstrate this behaviour:

  def lam_return
    puts "--- #{__method__} ---"
    puts "simple function that calls a lambda and then returns it's own string."
    my_lam = lambda { return 'Lambda has returned' }
    my_lam.call 
    return 'lam_return method string returned'
  end

  def proc_return
    puts "--- #{__method__} ---"
    puts "simple function that calls a Proc, \nBUT instead of returning it's own string returns the Proc's string."
    my_proc = Proc.new { return 'Proc string returned' }
    my_proc.call # this function returns and finishes here
    return 'proc_return method string returned' # the return is never called
  end

This will output the following:

  --- lam_return ---
  simple function that calls a lambda and then returns it's own string.
  lam_return method string returned

  --- proc_return ---
  simple function that calls a Proc, 
  BUT instead of returning it's own string returns the Proc's string.
  Proc string returned

Proc return work around

There is however a very simple work around to avoid a Proc from returning out of it’s caller method. Simply removed the return statement, because Ruby always returns the last line this last line will be returned automatically.

  def proc_return_work_around
    puts "--- #{__method__} ---"
    puts "simple function that calls a Proc, \nhowever the method string still returns."
    my_proc = Proc.new { 'Proc string returned' }
    my_proc.call # this function returns and finishes here
    return "#{__method__} method string returned" # the return is never called
  end

This will output:

  --- proc_return_work_around ---
  simple function that calls a Proc, 
  however the method string still returns.
  proc_return_work_around method string returned

Conclusion

As you have seen there are only two small differences between a Proc and a lambda but these differences have quite surprising behaviours. Therefore following the Ruby principle of “…least surprise”, it is best practice to use lambdas over Procs unless you absolutely need the Proc behaviour.

For more information please see this article: http://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Method_Calls

I recently did code kata 4 (data munging), and here is my solution to the problem. Like anything it can be improved but I think it is relatively simple to understand and yet flexible enough to work for a number of different formatted .dat files.

The premiss is to write a program that will work out the deltas ( difference ) between two columns in .dat files and return the lowest values.

These .dat files have different numbers of columns and these columns can be in different order. The two .dat files used for this program supplied weather temperature ranges and football (soccer) result scores.

Anyway here is my solution in the Ruby language and a description of how it works:

# data_processor.rb

module DataProcessor

  def process_dat(filename, regex=/^\s*\d+\.?\s+/)
    items = []
    IO.foreach(filename) do |line|
      if line =~ regex
        num, max, min = yield line
        delta = max.to_i - min.to_i
        items << [num, delta] if items.size == 0
        current_items = items[0][1]
        if delta <= current_items
          items.clear if delta < current_items
          items << [num, delta]
        end
      end
    end  
    items
  end

end

if __FILE__ == $0

  include DataProcessor

  puts "-------- Weather  --------"
  list = process_dat('weather.dat') { |line| tokens = line.split(' ') }
  list.each { |item| puts "Lowest Day:#{item[0]} Delta:#{item[1]}"}

  puts "\n-------- Football --------"
  list = process_dat('football.dat') { |line| tokens = line.split(' ');[tokens[1], tokens[6], tokens[8]] }
  list.each { |item| puts "Team:#{item[0]} Delta:#{item[1]}"}
end

The code above was created as a module as this will allow the process_dat method to be mixed-in to other objects, allowing these objects to execute the code without having to instantiate another class to get this functionality.

The process_dat method takes a filename and an optional regular expression. The regular expression (regex) is used to find the data in the text file for processing. In this case any line that starts with 0 or more spaces followed by at least one number and then an optional ‘.’ after that number followed by at least one space. This regular expression could have been used to extract the data from each row as well – but it actually became much easier to simply split each row and tokenise it around a space, as we are processing space separated files.

Using the IO.foreach method will iterate over each line of the .dat file allowing the logic to process that line. This is much more memory efficient, as only one line is loaded into memory at each iteration of the loop – thus allowing this method to process very large data files.

Next the method checks to see if the line matches the regex (if line =~ regex) if it does it can be processed.

Processing each line is handled by a Ruby block, (as a yield statement is used), in the “weather.dat” case the block simply splits the line around a white space. In the “football.dat” case the block splits the line around the white space but re orders the returned columns. Incidentally it is useful to know that if the number of tokens in the array exceeds the number of possible assignments (num, max, min = yield line, where yield line returns a 10 item array for instance) Ruby conveniently ignores the other values after assigning the first lot.

The next couple of lines simply calculate the delta and assigns it to a temporary array (items) if items is empty. The items array simply holds the lowest values as a collection of lowest values with the same value. Note that the items array is an array of arrays. The item in the items array consists of a number and a delta.

Finally we get the lowest item from the items array and check to see if the current line delta is lower if it is, clear the items array and add this new item. If it equals the same value simply add this item to the items array. Thus the items array will contain the same lowest values.

I hope you find this helpful – if you have an alternative solution, why don’t you add it to the comments at the bottom.

Part I

The best practice suggestions listed here are summaries from Douglas Crockford’s book: JavaScript: The Good Parts

By following these principles you will write better JavaScript code.

JavaScript Best Practices

1. Commenting

Avoid using /* */ for commenting out your code instead use the // method. The reason being that */ can occur in regular expressions and if you were to comment out a regular expression containing this, you would get a syntax error.

2. Object Value Retrieval

For an object such as var myObj = {key:"value"}; An object value can retrieved in two ways:

1. myObj["key"]; // “value”
2. myObj.key; // “value”

However the second version using ‘.’ notation is preferred as it reads better.

3. Avoid Global Variables

Global variables will reduce the resiliency of your JavaScript programs so try to keep these to a minimum. A best practice strategy for this is to use a single global variable for your program and simply assign objects to it. e.g.

var MYAPP = {};

MYAPP.myObj = {
  key:"value"
};

4. Capitalize Constructor Functions

Functions that are created as constructor functions should be capitalized. Think of a constructor functions as a factory creating cloned functions. Strictly this is not classical inheritance but prototypical inheritance. This is essentially where a function is cloned and contains all the properties of it’s parent.

var Shape = function (name) {
  this.name = name;
};

var square = new Shape("square");

This style of constructor functions is not recommended as if you forget to use new then the this object gets bound to the global variable.

5. Avoid Recursion

JavaScript does not offer tail recursion optimization, therefore functions that recurse too deeply will eventually fail.

6. Use Function Scope To Protect Against Namespace Collisions

JavaScript does not have block scope, i.e. variables declared in a block statement (e.g. with in curly braces) are accessible from outside of that block. However functions do have function scope, variables declared within a function cannot be accessed from outside of that function.

Best practice is to declare the variables used in a function at the top of the function.

7. Use Functions To Create Modules

A Module is a function or object that presents an interface but hides its state and implementation. This therefore avoids the use of global variables.

var myModule = function() {
  var call = '';

  return {
    setCall: function(myCall) {
      call = String(myCall);
    },
    doCall: function() {
      return 'Hello ' + call;
    }
  };
}();  // the () will immediately invoke this function

var myCaller = myModule();
myCaller.setCall = 'Ben';
alert( myCaller.doCall() );

8. Use Cascades

If you have a method that changes state, be sure to return this as opposed to undefined. The reason is that you can then chain these methods together. This is how JQuery works.

getElement('myDiv').changeColour('red').setFont('Times').setStyle('italic');

9. For Constructor That Take Many Parameters Use Object Specifiers

If an object takes many parameters it can be tricky ensuring that these parameters are in the correct order. Therefore create a single object instead that holds all the parameters as properties:

// Tricky
var myObj = someObject(a, b, c, d);

// Better
var myObj = someObject({ first: a, second: b, third: c, fourth: d});

General JavaScript Notes

Strings are immutable, in that once a string has been created that string object cannot be changed. However note that you can easily append and modify a string, the outcome just generates a new string.

JavaScript provides a prototype linkage function, which when used allows one object to ‘inherit’ the properties of the linked object.

A try catch statement only allows a single catch block, to catch all exceptions. If you need to differentiate between exceptions then add code to inspect the name of the exception and handle it appropriately.

Objects

Object creation methods:

1. var myObj = {};

Set a default value if an object property doesn’t exist:

var myObj = {one:"1"};
myObj.two || "2" // returns "2"

Assigning a value to an object property that is undefined will simply augment (or create) that property in that object and assign it.

Objects are passed by reference, they are never copied.

Every object is linked to a prototype object from which it can inherit properties. This object is Object.prototype

The prototype link is only used in the retrieval of properties from objects, if the first object does not have the property then JavaScript will use this prototype link to get the next object up the chain and try and retrieve the property from this object. If no property exists then undefined is returned.

When using for in for enumerating over properties in an object, be aware that it will also enumerate the properties of objects further up the prototype chain. If you don’t want this simply just use for,

Functions

As Douglas Crockford says:

Functions are the fundamental modular unit of JavaScript. Use functions for code reuse, information hiding and composition. Functions are used to specify the behaviour of objects.

Functions are objects, but they have a prototype link to Function.prototype, and then this root function object in turn links to Object.prototype.

A function can take 0 or more specified parameters, however functions get two extra hidden parameters this and arguments.

A method is simply a function assigned (bound) to an objects property. e.g.

var myObj = {
  myMethod: function (name) {
    alert("Hello " + name);
  };
};

A function always returns a value. If no return value is defined then the function will return undefined. However if a function is constructed with the new prefix and the return value is not an object the new object is returned.

JavaScript Gotcha’s

typeof Returns incorrect types

Using the typeof method will return six results:

1. number
2. string
3. boolean
4. undefined
5. function
6. object

The problem occurs with typeof ['1'] or typeof null which both return object which is incorrect.

Function Invocation Binds ‘this’ To The Global Object

When a function is created by assigning it to a global variable, and the contents of that function contains a this statement then the this is bound to the Global Object

When a function, contains an internal function the this of the internal function is bound to the Global Object and not the this of the parent function. To fix this simply assign this to a that variable and use that within your internal function.

var myObj = {
  value: 0;

  double: function() {
    var that = this;
    var helper = function() {
      that.value = that.value + that.value;
    };
    helper();
  };
};

arguments Is An Array Like Object Not An Array

All functions have a parameter arguments which gives a function access to all the arguments supplied with it’s invocation – including excess arguments that are not assigned to a parameter. However note that arguments is in fact a special object with a length method and not an Array object. So arguments will lack methods that an Array object will have.

This article outlines how to set up your new Django site from scratch. I tend to use Dreamhost Web Site Hosting so there will be a few notes specific to them.

If you are unsure how to set up Python and Django using Dreamhost then checkout my previous article.

I hope you find it useful and drop me a comment if you have any questions.

Create the new site

Create a project directory

$ mkdir -p mysite-project/mysite.com
$ cd mysite-project/mysite.com

Create you new site:

$ django-admin.py startproject mysite

Now create the following directories:

$ mkdir -p mysite-project/settings
$ mkdir -p mysite-project/mysite.com/tmp/
$ mkdir -p mysite-project/mysite.com/public/appmedia
$ mkdir -p mysite-project/mysite.com/mysite/scripts
$ mkdir -p mysite-project/mysite.com/mysite/templates

Initialise git

$ cd mysite-project/mysite.com
$ git init
$ touch .gitignore

Add the following:

.DS_Store
*.pyc
packages/
work/

Add WSGI file

The following should be carried out from this directory:

$ cd mysite-project/mysite.com

Add the passenger_wsgi.py file to mysite-project/mysite.com directory:

import sys, os
if sys.hexversion < 0x2060000: os.execl("/home/myuser/opt/python261/bin/python2.6", "python2.6", *sys.argv)

sys.path.append(os.getcwd())
# START
# required if you are using VirtualEnv, to get Passenger to use local Python environment.
# see: http://wiki.dreamhost.com/Passenger_WSGI
INTERP = "/home/myuser/virtual/bin/python2.6"
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)
# END
os.environ['DJANGO_SETTINGS_MODULE'] = "mysite.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Create the public/appmedia directory in this directory to. Create tmp/restart.txt

Set up the VirtualEnv Environment

Move to the mysite-project directory. Ensure that you have virtualenv installed on your local system:

$ cd mysite-project
$ pip install virtualenv

Create the virtual/ directories using virtualenv:

$ virtualenv ./virtual

Create a symbolic link to the activate executable.

$ ln -s ./virtual/bin/activate ./activate

NB: if you rename your mysite-project directory you may well need to delete the virtual directory and activate symbolic link and recreate them using the previous virtualenv steps.

You can check which Python site packages being used with this command:

$ python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Create Local Dev Database

Change directory to the scripts directory in mysite-project/mysite.com/mysite:

$ cd mysite-project/mysite.com/mysite/scripts
$ touch scripts/create-db.sql
$ touch scripts/drop-db.sql

create-db.sql

create database mysite_com_dev;
grant all on mysite_com_dev.* to 'root'@'localhost';

drop-db.sql

drop database mysite_com_dev;

Now create the database

$ mysql -u root < scripts/create-db.sql

Set up the Django Project Files

Make the following changes to the settings.py file.

import sys
import os
import os.path
import socket

PROJECT_ROOT = os.path.realpath(os.path.dirname(__file__))
sys.path.insert(0, os.path.join(PROJECT_ROOT)) # needed because of shared hosting PYTHONPATH problems

# MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'public/appmedia/')
MEDIA_ROOT = '/full/local/path/to/appmedia/'
# MEDIA_URL = '/appmedia/'
MEDIA_URL = 'http://media.mysite.com/appmedia/'
ADMIN_MEDIA_PREFIX = 'http://media.mysite.com/media/'

TEMPLATE_DIRS = ( os.path.join(PROJECT_ROOT, 'templates').replace('\\','/'), )

INSTALLED_APPS = (
    'django.contrib.admin', # add this
    'django.contrib.admindocs', # and this
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

# Add this to the bottom to import your local settings
try:
    from settings_local import *
except ImportError:
    pass

Now we need to create a settings_local.py file in the same directory as the main settings.py file:

$ touch settings_local.py

Then add your default settings (most probably your dev settings):

DATABASE_ENGINE = 'mysql'     
DATABASE_NAME = 'mysite_com_dev'
DATABASE_USER = 'root'         
DATABASE_PASSWORD = ''         
DATABASE_HOST = 'localhost'    
DATABASE_PORT = ''

As we will be using multiple environments we need to have different configurations for each environment. There are a number of ways to do this, however the DRYist and easiest is to have a main settings file that imports different settings according to the environment.

Change directory to the mysite-project/settings directory, and create the appropriate settings files – we will have a dev, staging and production file as well as the main one:

$ cd mysite-project/settings
$ touch settings_dev.py; touch settings_staging.py; touch settings_production.py

Now add the appropriate settings to these files database settings. Note that these settings will overwrite the settings in the main settings file.

Adding South to Your Application

For detailed installation instructions see http://south.aeracode.org/wiki/Download.

Note, for MySQL databases, you should install South to a new instance of your MySQL database and not an existing or legacy database. This is because South won’t update it’s self when you run: python manage.py migrate

Quick installation.

Add South (Note the capital ‘S’) to your pip requirements.txt file, and pip install it:

  pip install -r requirements.txt

Or just run:

  pip install South

Install the south (Note lowercase ’s’) into your project settings.py file. e.g.:

  INSTALLED_APPS = (
      'django.contrib.admin',
      'django.contrib.admindocs',
      'django.contrib.auth',
      'django.contrib.contenttypes',
      'django.contrib.sessions',
      'django.contrib.sites',
      'south',
  )

Now syncdb with your database so that your App knows about South.

  $ python manage.py syncdb

You should get an output similar to this:

      Syncing...
      Creating table south_migrationhistory

      Synced:
       > django.contrib.admin
       > django.contrib.auth
       > django.contrib.contenttypes
       > django.contrib.sessions
       > django.contrib.sites
       > django.contrib.markup
       > south
       > tagging

      Not synced (use migrations):
       - 
      (use ./manage.py migrate to migrate these)

Finally you need to tell South what Apps you want it to manage (in terms of migrations). So for example say you have an App called ‘firepages’, then run:

  python manage.py startmigration firepages --initial

This will create a migrations directory and will allow South to manage all the models in that App:

  Creating migrations directory at '/mypath/mysite-project/mysite.com/mysite/firepages/migrations'...
  Creating __init__.py in '/mypath/mysite-project/mysite.com/mysite/firepages/migrations'...
   + Added model 'firepages.Group'
   + Added model 'firepages.Category'
   + Added model 'firepages.Page'
  Created 0001_initial.py.

Note that because you will be uploading this new migrations directory you shouldn’t need to run this command on the server.

You will now need to execute the following command to migrate forward:

  python manage.py migrate

Now if you want to change your database do the following:

1. Modify your model. e.g. add / remove fields
2. Tell South to create a migration: python manage.py startmigration myapp name-of-my-migration --auto
3. Execute the migration: python manage.py migrate

Change urls.py

Use the following imports:

from django.conf import settings
from django.contrib import admin

admin.autodiscover()

At the bottom add a new URL path so that the site can get access to the appmedia, NB don’t forget to add a staging and production version to:

# simple trick to use local css files for development.
# (Just make sure DEBUG=False in production!!!)
if settings.DEBUG:
    urlpatterns += patterns('', 
        url(r'^appmedia/(?P<path>.*)$', 'django.views.static.serve', { 
            'document_root': settings.MEDIA_ROOT }),
        )

Now add the following URLs:

urlpatterns = patterns('',
    #Admin URLS
    # (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    (r'^admin/', include(admin.site.urls)),

    #Home URL
    (r'^$', include('mysite.foo.urls'),
)

You may want to change the admin URL pattern to be something else. This will make the admin area more obscure thus helping to prevent hacking. e.g. mylogin

Add a views.py file

In the mysite-production/mysite.com/mysite/ directory add a views.py file with the following:

from django.http import HttpResponse

import settings

def hello(request, path='none'):
    """quick hello test"""
    return HttpResponse("Quick test, PROJECT_ROOT: %s<br/>URL path: %s" % (settings.PROJECT_ROOT, path))

Sync the DB and Start the Server

Navigate to the mysite-project/mysite.com/mysite directory

Start the virtual environment:

$ . ../../activate
(virtual)$ python manage.py syncdb
** create a user **
(virtual)$ python manage.py runserver

Now open a browser and point it at this URL:

$ open http://localhost:8000

It should display your hello view results.

Requirements

Add a requirements.txt file to the following directory:

$ touch mysite-project/mysite/requirements.txt

Add in any requirements e.g. django-tagging

echo mysite-project/mysite/requirements.txt << django-tagging

Execute and add requirements:

pip install -r requirements.txt 

Local Apps Directory

Create an apps directory in the root of your project folder. Then tell your settings.py file about it:

  sys.path.insert(0, os.path.join(PROJECT_ROOT, "apps"))

Now any apps that you want in here and not in your “pip install” directory just add them and they will be found.

RequestContext on All Views

All your views should implement a RequestContext variable, but first you will need to add the following CONTEXT_PROCESSORS to the settings.py file:

TEMPLATE_CONTEXT_PROCESSORS = (
    'django.core.context_processors.auth',
    'django.core.context_processors.debug', # passes the DEBUG variable to the template
    'django.core.context_processors.i18n',
    'django.core.context_processors.media', # passes the MEDIA_URL to the template
    'django.core.context_processors.request', # passes the request object to the template
)

Now Make sure that you pass the RequestContext to the render.

def home_page(request):
    """display home page"""
    template = "page_home.html" 
    return render_to_response(template, { 'foo':'bar' }, context_instance=RequestContext(request))

This will allow you to use generic templates, and also get a handle on certain variables in the template.

Page Specific Menus

See this article for more information: http://gnuvince.wordpress.com/2007/09/14/a-django-template-tag-for-the-current-active-page/

Django doesn’t out of the box provide a current_page variable like Ruby on Rails – but it is easy enough to implement.

You will need to have the ‘django.core.context_processors.request’, setting set in the template context processors as specified above.

You will also need to create a module called templatetags with a init.py file in it. Now add a file called active_menus.py

    from django import template

    register = template.Library()

    @register.simple_tag
    def link_to(request, pattern, url, name):
        import re
        if re.search(pattern, request.path):
            return '<span class="active">%s</span>' % name
        return '<a href="%s">%s</a>' % (url, name)

Now in your template load this template tag and call it like this:

    {% load active_menus %}
    <li>{% link_to request "^/$" "/" "home" %}</li>
    <li>{% link_to request "^/software/$" "/software/" "software" %}</li>
    <li>{% link_to request "^/books/$" "/books/" "books" %}</li>

Template Includes

When creating an include template file give it a name starting with an underscore e.g. _nav.html

Add a Sitemap to Your Site

Add the following to your settings.py file in the INSTALLED_APPS section:

    django.contrib.sitemaps

Now check that you have the following installed also.

This should be in the TEMPLATE_LOADERS:

  'django.template.loaders.app_directories.load_template_source',

Make sure you’ve installed the sites framework. Also make sure that you have set the site domain name and description correctly in the sites project in the Admin Panel. It should not be example.com, but something like: myproject.com

Create a sitemaps.py file in your project. This file should contain something like:

from django.contrib.sitemaps import Sitemap
from myproject.models import ProductPage, ProductCategory

class ProductPageSitemap(Sitemap):
    changefreq = "never"
    priority = 0.5

    def items(self):
        return ProductPage.objects.all().filter(published=True)


class ProductCategorySitemap(Sitemap):
    changefreq = "never"
    priority = 0.5

    def items(self):
        return ProductCategory.objects.all()

Now add the following to your urls.py file:

    from myproject.sitemaps import *
    # from django.contrib.sitemaps import FlatPageSitemap, GenericSitemap

    sitemaps = {
        'product_pages': ProductPageSitemap,
        'product_categories': ProductCategorySitemap,
    }

    urlpatterns = patterns('',
      (r'^sitemap\.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps})
    )

Adding a new App

$ python manage.py startapp myappname

Add myappname to your settings.py file INSTALLED_APPS Now create some models in your myappname/models.py file.

Then run a syncdb:

$ python manage.py syncdb 
OR to see the SQL
$ python manage.py sql myappname

Now if you want to add South migrations:

$ python manage.py startmigration myappname --initial

Installing PIL

PIL should not be installed as part of the requirements.txt file, as you will need to compile libjpeg first.

This article describes how to install and set up Python and VirtualEnv on Dreamhost web site hosting.

Some of these notes are based on this article: URL: http://blog.localkinegrinds.com/2007/08/20/custom-python-installation-for-django-on-dreamhost/

Set up

Log into your shared server and do the following:

$ mkdir -p opt/python261
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.6.1/Python-2.6.1.tgz
$ tar xvzf Python-2.6.1.tgz

Note that: According to the Filesystem Hierarchy Standard, the /opt dir “is reserved for the installation of add-on application software packages.”

Now configure where your new Python instance will be installed:

$ cd Python-2.6.1
$ ./configure --prefix=$HOME/opt/python261
$ make
$ make install > install.txt

NB: The install.txt file is useful to locate files if you wish to uninstall Python later. NB: you might wish to use:

$ ./configure --prefix=$HOME/opt/python261  --enable-unicode=ucs4

But Dreamhost does uses ucs2 by default (Python’s default to), and there can be issues if you try to load packages that are compiled using a different Unicode setting.

Now edit your ~/.bash_profile file to specify the new Python install located in: ~/opt/

export PATH=$PATH:$HOME/opt/python261/bin

You might also want to add an alias so that instead of typing python2.6 to access the python shell, you can just type python in the command line.

alias python='python2.6'

Don’t forget to tell bash to load this file after you have made your changes:

$ source ~/.bash_profile
$ python -V  # will display your new version 2.6.1

Installing VirtualEnv

$ cd downloads/
$ curl -O http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.5.tar.gz
$ tar -xvzf virtualenv-1.4.5.tar.gz
$ python virtualenv-1.4.5/virtualenv.py $HOME/virtual --no-site-packages

NB: This binds your new install with virtualenv and creates a virtual directory in your user home directory. You will need to use the –no-site-packages option so that virtualenv doesn’t use any of the original Dreamhost Python install packages.

Update your ~/.bash_profile file so that it puts the virtualenv packages onto the Python path, and activate(s) your virtual environment:

export PATH="$PATH:$HOME/opt/python261/bin:$HOME/virtual/bin"
source $HOME/virtual/bin/activate

Check that the packages directory is in your new virtual site_packages directory:

$ python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Now activate your virtual environment, by executing the .bash_profile file:

$ source ~/.bash_profile

Or manually activate it by:

$ . ~/virtual/bin activate

Installing MySQL Python Bindings

(virtual)$ cd downloads
(virtual)$ wget http://internap.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.3c1.tar.gz
(virtual)$ tar xvzf MySQL-python-1.2.3c1.tar.gz
(virtual)$ cd MySQL-python-1.2.3c1
(virtual)$ python setup.py install

Check that you can import the database correctly (NB: do this from a different directory from the one where MySQL-python was installed, as you will get Module already imported warnings):

(virtual)$ cd ~
(virtual)$ python
>>> import MySQLdb

If you don’t get an error all succeeded OK.

Install Django

(virtual)$ pip install Django

If you build web sites and host (or want to host) them on Dreamhost, you will be well aware that at some stage you will be using SSH to log in and out of your hosting server. If you have many sites then this gradually becomes tiresome – especially if you use tools like Capistrano to deploy your applications, as you will constantly have to re-enter your passwords.

Fortunately you can certify your local computer, such that it will automatically enter the SSH password for you. Here are some quick tips on how to do it.

Generate Your Security Keys

Generate your private/public key pair on your local machine, from your home directory:

ssh-keygen -d

Now DON'T enter a password - just press return, otherwise you 
will still need to enter a password, when you log in with SSH.

Set Privileges And Copy The Public Key To Your Server

Use the following command to copy your newly created ~/.ssh/id_dsa.pub file up to the server you wish to remote log in to. Note that this command below will set the permissions both locally and remotely. The remote .ssh/* needs to be set to 0600.

N.B. Replace USERNAME and MY_DOMAIN with your own username and domain.

ssh USERNAME@ftp.MY_DOMAIN.COM 'test -d .ssh || mkdir -m 0700 .ssh ; cat >> .ssh/authorized_keys && chmod 0600 .ssh/*' < ~/.ssh/id_dsa.pub

  Enter the normal USERNAME password.

Log In Using SSH

Now log in using:

ssh USERNAME@ftp.MY_DOMAIN.COM

You shouldn't neet to enter a password.

Taking It Further

To simplify things even further you can modify the ~/.ssh/config file such:

Host NICKNAME
Hostname MY_DOMAIN.COM
User USERNAME

Now you only need to type:

ssh NICKNAME

And you will be automatically logged in.

This is just a quick post. I’ve been playing around with the Sinatra micro web framework, and have to say that it is excellent. It really is the perfect tool for rapidly creating small and useful Ruby micro-sites.

Being small and lightweight it doesn’t have many of the features of larger frameworks like Ruby on Rails. One of the features I’ve begun to need is the handling of Rails like partials.

Notably I need to pass both local variables and collections to the partial code and have the partials rendered accordingly. I started implementing my own version, and then came across a nice little Github project: Sinatra-Helpers by sbfaulkner.

Anyway here is the code he uses for Sinatra partials:

def haml_partial(name, options = {})
  item_name = name.to_sym
  counter_name = "#{name}_counter".to_sym
  if collection = options.delete(:collection)
    collection.enum_for(:each_with_index).collect do |item,index|
      haml_partial name, options.merge(:locals => {item_name => item, counter_name => index+1})
    end.join
  elsif object = options.delete(:object)
    haml_partial name, options.merge(:locals => {item_name => object, counter_name => nil})
  else
    haml "_#{name}".to_sym, options.merge(:layout => false)
  end
end

You can use his project or if you just need this feature just pop this code into your Sinatra helpers file application_helper.rb:

helpers do

  def partial(name, options = {})
    item_name = name.to_sym
    counter_name = "#{name}_counter".to_sym
    if collection = options.delete(:collection)
      collection.enum_for(:each_with_index).collect do |item, index|
        partial(name, options.merge(:locals => { item_name => item, counter_name => index + 1 }))
      end.join

    elsif object = options.delete(:object)
      partial name, options.merge(:locals => {item_name => object, counter_name => nil})
    else
      haml "_#{name}".to_sym, options.merge(:layout => false)
    end
  end
end

Now you can call the partial from a your page e.g. index.haml

= partial("mypartial", :locals => {:var1 => "value 1", :var2 => "value 2 and so on"})

Create a partial in the same directory as the calling page. Note that this code doesn’t like filenames truncated with a ‘-’, so avoid this if possible. Also like in Rails your partials should start with an underscore. e.g. _mypartial.haml

To access the variable from this partial simply call the local variable name in your code:

%h2 
  This is var1:
  = var1

%p 
  And this is var2:
  = var2

Handling collections, again, is similar to the way Rails handles partial collections. From your page call the partial with a collection such as:

= partial("listitem", :collection => ["item 1", "item 2 and so on"])

Now in the _listitem.haml partial code simply provide the partial name as a variable or the partialname with _counter for the increment number e.g.

%li
  = listitem
  is the 
  = listitem_counter
  in this array

Anyway all pretty basic stuff but I hope it helps the casual Sinatra surfer.

Goodluck.

Using bash scripts to become a more efficient web developer

Here are a few scripts, that I find really useful for speeding up my web development time.

I’ve been building up this list as I needed to use them – so they maybe a little raw.

For example often clients send me images with filenames that don’t match my naming standard, so running the appropriate script really helps keep me focussed on the job in hand and not waste too much time reformatting filenames etc.

Finally if you have any useful little bash scripts why don’t you add them to the comments below?

Bash script to append a .txt extension to a filename

Iterate over the current directory, get all files with .log and append .txt to the end of the entire filename:

for i in *.log*; do mv "$i" "$i.txt"; done

Script to make filenames lowercase

Converts all the file names in a directory and converts them to lowercase.

echo version:

for i in *.log*; do echo mv \"$i\" \"`echo $i| tr [A-Z] [a-z]`\"; done

real version:

for i in *.txt; do mv "$i" "`echo $i| tr [A-Z] [a-z]`"; done

Quick note on how to use grep to search for a string in file

-o print Only the matched parts of a matching line, with each part on a separate line. -H print the filename for each match. -n prefix each line of output with the 1 based line number. -R, -r recursively read all files under subdirectories.

grep -oHnr "some pattern"  *.txt

{txt,log} this globs the file extensions together. 2>/dev/null this passes all errors to a blackhole, so that they won’t be displayed.

grep 123741 ./adapter_logs/*.{txt,log} 2>/dev/null

Find all occurrences of 123741 in all .txt and .log files in all subdirectories.

for i in 'find . -type d'; do grep 123741 $i/*.{txt,log}; done 2>/dev/null

List all files that match this regular expression in the current direcory

All files with the number 5 in their name.

ls | grep -e .*5.*

Bash script to check whether a tag exists in a subversion repository

svn ls http://www.mysvnserver.co.uk/myproject/tags | grep mytag-0.7.0.1/

The [ ] is known as a Test, by default it tests integers Also to end an if statement you use fi

if [ 1 = 1 ] ; then echo YES; else echo NO; fi

if [ "`svn ls http://mysvnserver/mysvnrepository/myproject/tags | grep R-9.20.400.0/`" = "R-9.20.400.0/" ]; then echo YES; else echo NO; fi

Note need the / in the evaluation part of “it-0.9.0.7/”, don’t really need it in grep it-0.9.0.7/

$ if [ "`svn ls http://www.mysvnserver.co.uk/myproject/tags | grep it-0.7.0.1/`" = "it-0.7.0.1/" ]; then echo YES; else echo NO; fi

Quick note on how to increment a variable, in a bash script

j=0; j=$(($j+1))

incrementing in a loop

j=0;for i in *.txt; do echo "some_file_"$j".txt"; j=$(($j+1)); done

Using a bash for loop to list all text files and append whoopee

for i in *.txt; do echo $i"whoopee"; done

Change all filenames to be a specific incremented name, starting from file 16

j=16;for i in *.jpg; do mv "$i" "gallery_"$j".jpg"; j=$(($j+1)); done;ls

Script to loop through all .txt filenames and make them lower case

for i in *.txt; do mv "$i" "`echo $i| tr [A-Z] [a-z]`"; done

Get all JPG files and create the appropriate HTML list tags for them and add them to a file

Create an HTML version:

for i in *.jpg; do echo "\t<li>\r\t\t<img src='images/$i' alt='' />\r\t</li>"; done > list_items.html

Create a HAML version:

for i in *.jpg; do echo "\t%li\r\t\t%img{ :src => 'images/$i', :alt => '' }\r"; done > imgs.haml

Batch change a misspelled filename of certain files, using string replacement

for i in aples*.jpg; do mv $i ${i/aples/apples} ; done

Script to batch create files based on filenames in a file

Create a file called something like “my_files.txt” with the following content:

archive_error_112480_0040.txt archive_error_114390_0043.txt etc…

Commands:

for i in `sed -n -e 'p' my_files.txt`; do mkdir a_test/$i; done
or
while read x; do touch $x; done < my_files.txt
or
cat my_files.txt | xargs touch
(xargs passes each line as an argument into EACH touch)

Using bash Expansions, and allow escapes

-e is an undocumented flag that allows escapes Expansion ${command/parameter/substitution} everything else needs to be escaped.

echo -e ${PATH//\:/\\n}

Executing a .bash_script after making changes

source .bash_profile
or
. .bash_profile

NB: you can ‘include’ other bash .sh scripts in a .bash_profile file by using ’source’ or ‘.’

Quick tip on using curl for downloading files off the web

Use -O to name the remote file, curl will save it locally but remove the rest of the path.

curl -O http://rubyforge-files.ruby-forum.com/rubygems/rubygems-0.9.0.tgz

Bash command for testing the existence of files and directories

If a directory does NOT exist create it

[ -d "/var/cache/git" ] || mkdir /var/cache/git

If a file exists echo a message:

[ -f "./new_file.txt" ] && echo "its there"

Bash command to remove spaces from filenames

for i in *.html; do mv "$i" "`echo $i| tr -d ' '`"; done

replace spaces for underscores

for i in *.html; do mv "$i" "`echo $i| tr ' ' '_'`"; done

Vim regex to replace spaces in images file names of ah HTML page

:%s:\(images\/\)\(\w\+\)\s\(\w\+\.jpg\):\1\2_\3:g

Copy text to the clipboard

echo 'hello world'|pbcopy
cmd + v

Conclusion

Some of these are quite basic but useful none-the-less, please add your own to the comments below. Many thanks, and good luck.

Updated…

Script function for .bash_profile to cd to the last opened finder location

This is a script written by a colleague. James Power. Add the following function to your .bash_profile in Apple OSX. Now open a directory in Finder, then open terminal and type cdf, your terminal will now change directory to the same one as the last Finder opened.

# function to change directory to the one set in the last opened finder.
cdf () {
   currFolderPath=$( /usr/bin/osascript <<"         EOT"
       tell application "Finder"
           try
               set currFolder to (folder of the front window as alias)
           on error
               set currFolder to (path to desktop folder as alias)
           end try
           POSIX path of currFolder
       end tell
            EOT
   )
   echo "cd to \"$currFolderPath\""
   cd "$currFolderPath"
}

Also for a much more comprehensive list of commands check out: http://www.commandlinefu.com/commands/tagged/34/bash

Ruby Rush command-line tool

If you use Ruby maybe check out Rush, it allows you do use Ruby in a bash command-line like way.

Jenson Button has just got pole position ready for the Australian Formula One race tomorrow. This is pretty exciting as on either side of his helmet is the Halcyon Events Ltd. logo.

Halcyon Events is a company that specialises in high end bespoke Formula One events. Through their contacts they can take you and your clients behind the scenes to areas not normally permitted, and give you a unique experience into Formula One (F1).

The web site was built by me, the design was created by Scamper Branding and the logo on Jenson’s helmet was designed by Firefly Creative.

Anyway fingers crossed for Jenson Button to win the Australian Grand Prix!.

– Update – Fantastic Jenson Button won the Melbourne Grand Prix! How exciting.