SaltyCrane Blog — Notes on JavaScript and web development

Django Blog Project #11: Migrating from Django 0.96 to SVN Trunk

I've been using the Django 0.96 release for this blog, but I've been thinking about switching to the SVN trunk version since it is recommended by the Django community. Django 1.0 alpha was released a couple weeks ago, so now seems like a good time to migrate.

Here are the changes I had to make. There were suprisingly few changes required-- probably because I'm not using a lot of the Django functionality. For a complete list of changes, see the Backwards-incompatible changes documentation.

Note, I am using trunk revision 8210. (2 weeks post Alpha).

Model changes

The admin definitions have been decoupled from the model definitions. Also, the prepopulate_from database field has been moved to the new admin class. See here for more information. Also maxlength was changed to max_length.

~/src/django/myblogsite/myblogapp/models.py:
import re
from django.db import models
from django.contrib.auth.models import User
from django.contrib import admin

class Post(models.Model):
    author = models.ForeignKey(User)
    title = models.CharField(max_length=200)
    slug = models.SlugField(max_length=200,
                            prepopulate_from=['title'],
                            unique_for_month='date_created')
    date_created = models.DateTimeField(auto_now_add=True)
    date_modified = models.DateTimeField(auto_now=True)
    tags = models.CharField(max_length=200, help_text="Space separated.")
    body = models.TextField()
    body_html = models.TextField(editable=False, blank=True)
    lc_count = models.IntegerField(default=0, editable=False)

    def get_tag_list(self):
        return re.split(" ", self.tags)

    def get_absolute_url(self):
        return "/blog/%d/%02d/%s/" % (self.date_created.year,
                                      self.date_created.month,
                                      self.slug)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-date_created"]

    class Admin:
        pass

class PostAdmin(admin.ModelAdmin):
    prepopulated_fields = {'slug': ('title',)}

admin.site.register(Post, PostAdmin)
Modify URLConf

See here for more information.

~/src/django/myblogsite/urls.py:
from django.conf.urls.defaults import *
from django.contrib import admin
from django.contrib.comments.models import FreeComment
from iwiwdsmi.myblogapp.views import *
from iwiwdsmi.feeds import *
from iwiwdsmi.views import *

admin.autodiscover()

feeds = {
    'latest': LatestPosts,
}

urlpatterns = patterns(
    '',
    (r'^site_media/(?P.*)$', 'django.views.static.serve', {'document_root': '/home/sofeng/src/django/iwiwdsmi/media'}),

    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^admin/(.*)', admin.site.root),
    (r'^feeds/(?P.*)/$', 'django.contrib.syndication.views.feed', 
     {'feed_dict': feeds}),
    (r'^comments/', include('django.contrib.comments.urls.comments')),
    
    (r'^myview1/$', myview1),
    (r'^$', rootview),
    (r'^blog/$', frontpage),
    (r'^blog/(\d{4,4})/(\d{2,2})/([\w\-]+)/$', singlepost),
    (r'^blog/(\d{4,4})/$', yearview),
    (r'^blog/(\d{4,4})/(\d{2,2})/$', monthview),
    (r'^blog/tag/([\w\-]+)/$', tagview),
)
Use the safe template filter

In the Django templates, Django SVN now escapes HTML by default to protect against cross-site scripting. To display my HTML blog posts, I needed to use the safe filter.

Excerpt from ~/src/django/myblogsite/templates/listpage.html:
    {{ post.body|safe|truncatewords_html:"50" }}
Use new admin templates

Finally, I copied over the new Django SVN templates from trunk/django/contrib/admin/templates.

Notes on moving Ubuntu Wubi to a standard ext3 partition using LVPM

Here are my notes for moving my wubi Ubuntu install to a dedicated ext3 partition. I used the Loopmounted Virtual Partition Manager (LVPM) to do the transfer. From the webpage:

The Loopmounted Virtual Partition Manager allows users to upgrade their existing Wubi or Lubi installation to a standard Ubuntu system by transferring all data, settings, and applications from the original install to a dedicated partition. The advantages of upgrading using LVPM are better disk performance and reliability, and the ability to replace the original operating system with Ubuntu.

Add 2 new partitions

My hard disk had only 1 partition containing Windows XP.

  • Boot to Xubuntu 8.04 Hardy Heron Live CD
  • Go to System, Partition Editor
  • Click the existing partition, and "Resize".
  • Click "Apply".
    • Note 1: the first time I tried this, I got ERROR: Extended record needed (1712>1024), not yet supported. Please try to free less space. So I slid my partition divider a little more to the right and tried again and it worked.
    • Note 2: This took a few minutes when it worked.
  • In the unallocated section, create a new partition. I made this my swap partition. It was 1065 MB, Primary Partition, Filesystem: linux-swap. It became sda2. Click "Add".
  • Then I created my main Linux partition. Used the remaining size, filesystem: ext3, Primary Partition. This is sda3. Click "Add".
  • Click "Apply".
Install and run LVPM
  • Boot into the Wubi Ubuntu installation.
  • Go to http://sourceforge.net/project/showfiles.php?group_id=198821, download, save, install, and run it.
  • Select "Transfer" and then the new main linux partition (sda3)
  • Reboot
  • When I selected Ubuntu in the Grub boot loader, I got the following error mesage: Error 17: Cannot mount selected partition.
  • To fix this, I hit "e" to edit the command, and changed root ()/ubuntu/disks to root (hd0,2). Then I hit "b" to boot. hd0 means the first hard disk, and 2 means the 3rd partition.
  • After booting into the new ext3 Ubuntu install, I edited the /boot/grub/menu.lst file to change root ()/ubuntu/disks to root (hd0,2)
  • All pau.

How to use gnip-python to retrieve activity from Twitter, Delicious, Digg, etc.

  • Create an account at http://www.gnipcentral.com/
  • Download gnip-python from github.com.
  • Unpack it:
    $ tar -zxvf gnip-gnip-python-028364a70bd40dda0069ecdd3e7f6fff23bb985e.tar.gz
    
  • Move it to your example directory:
    $ mkdir ~/src/python/gnip-example
    $ mv gnip-gnip-python-028364a70bd40dda0069ecdd3e7f6fff23bb985e/*.py ~/src/python/gnip-example
  • Create an example file called ~/src/python/gnip-example/gnip-example.py:
    #!/usr/bin/env python
    
    from gnip import *
    
    gnip = Gnip("[email protected]", "yourpassword")
    
    for publisher in ["twitter", "digg", "delicious"]:
        activities = gnip.get_publisher_activities(publisher)
        print
        print publisher
        for activity in activities[:5]:
            print activity
    
  • Run it:
    $ python gnip-example.py

    And get the following results:
    twitter
    [derricklo, 2008-08-01T22:49:59+00:00, tweet, http://twitter.com/derricklo/statuses/875165550]
    [sam_metal, 2008-08-01T22:50:01+00:00, tweet, http://twitter.com/sam_metal/statuses/875165564]
    [lalatina, 2008-08-01T22:49:59+00:00, tweet, http://twitter.com/lalatina/statuses/875165544]
    [Nochipra, 2008-08-01T22:50:01+00:00, tweet, http://twitter.com/Nochipra/statuses/875165562]
    [jmcgaha, 2008-08-01T22:50:01+00:00, tweet, http://twitter.com/jmcgaha/statuses/875165556]
    
    digg
    [DAlexopoulos, 2008-08-01T18:50:08+00:00, submission, http://digg.com/health/CLA_Conjugated_Linoleic_Acid_Explained]
    [EradicateIV, 2008-08-01T18:50:05+00:00, submission, http://digg.com/pc_games/Pittco_Iron_Storm_9]
    [vivianpetman, 2008-08-01T18:49:58+00:00, submission, http://digg.com/business_finance/Dub_Me_Now_Turns_Business_Cards_into_Bits_and_Bytes_Small]
    [portia7896, 2008-08-01T18:49:53+00:00, submission, http://digg.com/world_news/Radioactive_water_yum]
    [hhdepot, 2008-08-01T18:49:52+00:00, submission, http://digg.com/2008_us_elections/Why_WE_are_democrats_in_2008_Original_Video]
    
    delicious
    [hanasama1, 2008-08-01T22:49:55+00:00, delicious, http://www.birchmere.com/]
    [Chrmftcotl, 2008-08-01T22:49:54+00:00, delicious, http://www.shaunlow.com/a-definitive-stumbleupon-guide-driving-traffic-to-websites/]
    [shankar, 2008-08-01T22:49:53+00:00, delicious, http://www.guardian.co.uk/books/2008/jul/26/salmanrushdie.bookerprize]
    [grzyweasel, 2008-08-01T22:49:53+00:00, delicious, http://patterntap.com/]
    [metaffect, 2008-08-01T22:50:05+00:00, delicious, http://dean.edwards.name/]
    Pretty cool.

This is just scratching the surface of what you can do with gnip. You can also filter by time or user. Or get XML output. Or you can publish to gnip yourself. See the gnip-python README for more python examples or the Gnip API for more detailed information. Also, here is a list of Gnip publishers.

If you get a ImportError: No module named iso8601 error, install iso8601.

Django Blog Project #10: Adding support for multiple authors

Here is a quick post on how I added support for multiple users on my blog.

Modfiy the model
Excerpt from ~/src/django/myblogsite/myblogapp/models.py:
import re
from django.db import models
from django.contrib.auth.models import User

class Post(models.Model):
    author = models.ForeignKey(User)
    title = models.CharField(maxlength=200)
    slug = models.SlugField(maxlength=200,
                            prepopulate_from=['title'],
                            unique_for_month='date_created')
    date_created = models.DateTimeField(auto_now_add=True)
    date_modified = models.DateTimeField(auto_now=True)
    tags = models.CharField(maxlength=200, help_text="Space separated.")
    body = models.TextField()
    body_html = models.TextField(editable=False, blank=True)
    lc_count = models.IntegerField(default=0, editable=False)

    def get_tag_list(self):
        return re.split(" ", self.tags)

    def get_absolute_url(self):
        return "/blog/%d/%02d/%s/" % (self.date_created.year,
                                      self.date_created.month,
                                      self.slug)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-date_created"]

    class Admin:
        pass
Update the database
  • List the SQL commands Django would use the create the database tables:
    $ cd ~/src/django/myblogsite/
    $ python manage.py sqlall myblogapp
    BEGIN;
    CREATE TABLE "myblogapp_post" (
        "id" integer NOT NULL PRIMARY KEY,
        "author_id" integer NOT NULL REFERENCES "auth_user" ("id"),
        "title" varchar(200) NOT NULL,
        "slug" varchar(200) NOT NULL,
        "date_created" datetime NOT NULL,
        "date_modified" datetime NOT NULL,
        "tags" varchar(200) NOT NULL,
        "body" text NOT NULL,
        "body_html" text NOT NULL,
        "lc_count" integer NOT NULL
    );  
    CREATE INDEX myblogapp_post_author_id ON "myblogapp_post" ("author_id");
    CREATE INDEX myblogapp_post_slug ON "myblogapp_post" ("slug");
    COMMIT;
  • Enter the sqlite shell:
    $ sqlite3 mydatabase.sqlite3

    and enter the following statement:
    sqlite> ALTER TABLE myblogapp_post ADD COLUMN author_id integer REFERENCES auth_user (id);
    sqlite> .exit
Update the template
Excerpt from ~/src/django/myblogsite/templates/singlepost.html:
  <h3>{{ post.title }}</h3>
  {{ post.body }}
  <hr>
  <div class="post_footer">
    Author: {{ post.author.first_name }}<br>
    Date created: {{ post.date_created.date }}<br>
    {% ifnotequal post.date_modified.date post.date_created.date %}
      Last modified: {{ post.date_modified.date }}<br>
    {% endifnotequal %}
    Tags: 
    {% for tag in post.get_tag_list %}
      <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
    {% endfor %}
    <br>
    <a href="/admin/myblogapp/post/{{ post.id }}">Edit post</a>
  </div>

Now you should be able to go in to the Admin interface select a user to associate with each post. Unfortunately, it does not automatically associate the logged in user with the post.

Here is a snapshot screenshot of what I'm calling version 0.1.1. Yeah, I know, I skipped 0.1.0-- I consider that to be the point where I said Goodbye Blogger and Hello Saltycrane.

How to set the default directory in Claws Mail

I've decided to use Claws Mail because it looked pretty good and mainly because I respect Adam Gomaa's opinion a lot. My first customization is to change the default directory used to store the messages. I couldn't find this in the documentation, but found the solution after Googling. The solution is: edit the path in your ~/.claws-mail/folderlist.xml file. Now to figure out how to show new message notifications in my wmii status bar!

How to set up Django with MySql on Ubuntu Hardy

Here are my notes on installing Django with MySql. Almost all of this was taken from Zeth's article: Baby Steps with Django - part 2 database setup.

Install Django and MySql

Note: during the installation of mysql-server, you will be prompted for a root password. Use this in the section below.

$ sudo apt-get install python-django
$ sudo apt-get install mysql-server
$ sudo apt-get install python-mysqldb
Set up a MySql database and user

Note, use the password you entered when installing MySql

$ mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.51a-3ubuntu5.1 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> CREATE DATABASE django_db;
Query OK, 1 row affected (0.01 sec)

mysql> GRANT ALL ON django_db.* TO 'djangouser'@'localhost' IDENTIFIED BY 'mypassword';
Query OK, 0 rows affected (0.03 sec)

mysql> quit
Bye
Create a Django Project
$ django-admin startproject mysite
Edit the Django database settings
Edit mysite/settings.py:
DATABASE_ENGINE = 'mysql'           # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'ado_mssql'.
DATABASE_NAME = 'django_db'             # Or path to database file if using sqlite3.
DATABASE_USER = 'djangouser'             # Not used with sqlite3.
DATABASE_PASSWORD = 'mypassword'         # Not used with sqlite3.
DATABASE_HOST = ''             # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.
Use Django to create the database tables
$ cd mysite
$ python manage.py syncdb
Creating table auth_message
Creating table auth_group
Creating table auth_user
Creating table auth_permission
Creating table django_content_type
Creating table django_session
Creating table django_site

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use 'sofeng'):    
E-mail address: [email protected]
Password: 
Password (again): 
Superuser created successfully.
Installing index for auth.Message model
Installing index for auth.Permission model
Loading 'initial_data' fixtures...
No fixtures found.
Run the development server
$ python manage.py runserver
Validating models...
0 errors found.

Django version 0.96.1, using settings 'mysite.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[30/Jul/2008 16:37:23] "GET / HTTP/1.1" 404 2053

Point your browser at http://127.0.0.1:8000 and you should see the Django It worked! page.

Install wmii snapshot

These are my notes for installing wmii snapshot in my home directory on Ubuntu 8.04 Hardy.

  • Download the latest snapshot (wmii+ixp-20080520) from the wmii homepage and save it to ~/incoming.
  • Untar the snapshot.
    $ cd ~/incoming
    $ tar -zxvf wmii+ixp-20080520.tgz
  • Install prerequisites:
    $ sudo apt-get install build-essential xorg-dev
    $ sudo apt-get install dwm-tools
  • Build: (I left the defaults for everything during make config except for the path. I changed this to a temporary directory, ~/tmp/wmii.)
    $ cd ~/incoming/wmii+ixp-20080520
    $ make config
    $ make 
    $ make install
  • Move bin, etc, lib, and share to a new wmii directory.
    $ mkdir ~/lib/wmii
    $ mv ~/tmp/wmii/* ~/lib/wmii
  • Create links in my ~/bin directory:
    $ cd ~/bin
    $ ln -s ../lib/wmii/bin/wihack wihack
    $ ln -s ../lib/wmii/bin/wmii9rc wmii9rc
    $ ln -s ../lib/wmii/bin/wmii9menu wmii9menu
    $ ln -s ../lib/wmii/bin/wmii.rc wmii.rc
    $ ln -s ../lib/wmii/bin/wmii.sh wmii.sh
  • Create a .xinitrc script:
    xmodmap ~/.Xmodmap
    gnome-screensaver&
    urxvt&
    
    until wmii; do
        true
    done
  • And link ~/.xsession to it:
    $ ln -s ~/.xinitrc ~/.xsession
  • Log out of the current window manager and then select X client script as the session and log in.

New PC setup notes

Here is how I set up my new work PC with Xubuntu, wmii, and conkeror using the wubi windows installer and my home mercurial repository of dot files. Notes: ti is a python script I use to interface with the remote machine that holds my mercurial repositories. I also have another python script which concatenates my configuration files in ~/etc with my local machine-specific configuration files in ~/local/localmachine/etc and puts them in my home directory. crayola is my home machine. cotton is my new work machine. (Actually I haven't changed the name yet.)

  • I used the wubi installer to install Xubuntu as part of the Windows filesystem.
  • I restarted the computer and selected to boot into Xubuntu. (I actually had to restart twice because the first time it was configuring stuff.)
  • I ran the Update Manager.
  • I installed Mercurial:
    $ sudo apt-get install mercurial
  • I cloned my home directory repository and set it up:
    $ cd /home
    $ sudo mv sofeng sofeng-old
    $ sudo hg clone ssh://[email protected]//home/sofeng
    $ sudo mkdir -p ~/local/cotton/etc
    $ sudo mkdir -p ~/local/cotton/bin
    $ sudo chown -R sofeng:sofeng /home/sofeng
    $ cp ~/local/crayola/etc/.* ~/local/cotton/etc
    $ . ~/.bashrc
  • I installed some stuff:
    $ sudo apt-get install wmii
    $ sudo apt-get install rxvt-unicode
    $ sudo apt-get install emacs-snapshot
    $ sudo apt-get install lastfm
    $ sudo apt-get install conky
  • I logged out of Xfce and logged into wmii. Mod4+Enter to get a urxvt terminal.
  • I cloned my conkeror repository:
    $ cd ~/lib
    $ ti clone conk

Emacs ruby-mode

To install ruby-mode for emacs, I followed the directions at the digital sanitation engineering blog. I pretty much did exactly the same thing-- I don't know why I wrote my own post-- I suppose it is good just for my records.

sofeng@crayola:~/etc
$ export SITE_LISP=~/etc/.emacs.d/site-lisp
sofeng@crayola:~/etc
$ svn export http://svn.ruby-lang.org/repos/ruby/trunk/misc $SITE_LISP/ruby
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/rubydb2x.el
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/rubydb3x.el
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/ruby-mode.el
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/ruby-electric.el
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/inf-ruby.el
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/README
A    /home/sofeng/etc/.emacs.d/site-lisp/ruby/ruby-style.el
Exported revision 18207.
sofeng@crayola:~/etc
$ emacs -batch -f batch-byte-complile $SITE_LISP/ruby
Loading 00debian-vars...
No /etc/mailname. Reverting to default...
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...
Skipping dictionaries-common setup for emacs-snapshot
Loading /etc/emacs/site-start.d/50psvn.el (source)...
Symbol's function definition is void: batch-byte-complile
sofeng@crayola:~/etc
$ ll ~/etc/.emacs.d/site-lisp/ruby
total 92
-rw-r--r-- 1 sofeng sofeng   369 2008 04/10 04:36 README
-rw-r--r-- 1 sofeng sofeng 14393 2008 01/06 07:49 inf-ruby.el
-rw-r--r-- 1 sofeng sofeng  6747 2007 02/12 15:01 ruby-electric.el
-rw-r--r-- 1 sofeng sofeng 44473 2008 07/21 17:51 ruby-mode.el
-rw-r--r-- 1 sofeng sofeng  1798 2008 04/22 06:15 ruby-style.el
-rw-r--r-- 1 sofeng sofeng  4481 2007 02/12 15:01 rubydb2x.el
-rw-r--r-- 1 sofeng sofeng  4613 2007 02/12 15:01 rubydb3x.el

Then added the following to my .emacs:

(add-to-list 'load-path "~/.emacs.d/site-lisp/ruby")
(autoload 'ruby-mode "ruby-mode"
    "Mode for editing ruby source files")
(add-to-list 'auto-mode-alist '("\\.rb$" . ruby-mode))
(add-to-list 'interpreter-mode-alist '("ruby" . ruby-mode))
(autoload 'run-ruby "inf-ruby"
    "Run an inferior Ruby process")
(autoload 'inf-ruby-keys "inf-ruby"
    "Set local key defs for inf-ruby in ruby-mode")
(add-hook 'ruby-mode-hook
    '(lambda ()
        (inf-ruby-keys)))
;; If you have Emacs 19.2x or older, use rubydb2x                              
(autoload 'rubydb "rubydb3x" "Ruby debugger" t)
;; uncomment the next line if you want syntax highlighting                     
(add-hook 'ruby-mode-hook 'turn-on-font-lock)

How to search C code for division or sqrt

The following Python script searches through C code for division or sqrt and prints the line of code and the line number. It skips C comments. To use, run python find_divides.py filename.c

#!/usr/bin/python

"""find_divides.py

usage: python find_divides.py filename
"""

import re
import sys

def main():
    filename = sys.argv[1]
    text = open(filename).read()
    lines = text.splitlines()
    lines = ["%4d: %s" % (i, line) for (i, line) in enumerate(lines)]
    text = "\n".join(lines)
    text = remove_comments_and_strings(text)

    for line in text.splitlines():
        if ("/" in line) or ("sqrt" in line):
            print line

def remove_comments_and_strings(text):
    """ remove c-style comments and strings
        text: blob of text with comments (can include newlines)
        returns: text with comments and strings removed
    """
    pattern = r"""
                            ##  --------- COMMENT ---------
           /\*              ##  Start of /* ... */ comment
           [^*]*\*+         ##  Non-* followed by 1-or-more *'s
           (                ##
             [^/*][^*]*\*+  ##
           )*               ##  0-or-more things which don't start with /
                            ##    but do end with '*'
           /                ##  End of /* ... */ comment
         |                  ##  -OR-  various things which aren't comments:
           (                ## 
                            ##  ------ " ... " STRING ------
             "              ##  Start of " ... " string
             (              ##
               \\.          ##  Escaped char
             |              ##  -OR-
               [^"\\]       ##  Non "\ characters
             )*             ##
             "              ##  End of " ... " string
           |                ##  -OR-
                            ##
                            ##  ------ ' ... ' STRING ------
             '              ##  Start of ' ... ' string
             (              ##
               \\.          ##  Escaped char
             |              ##  -OR-
               [^'\\]       ##  Non '\ characters
             )*             ##
             '              ##  End of ' ... ' string
           |                ##  -OR-
                            ##
                            ##  ------ ANYTHING ELSE -------
             (.              ##  Anything other char
             [^/"'\\]*)      ##  Chars which doesn't start a comment, string
           )                ##    or escape
    """
    regex = re.compile(pattern, re.VERBOSE|re.MULTILINE|re.DOTALL)
    goodstuff = [m.group(5) for m in regex.finditer(text) if m.group(5)]
    return "".join(goodstuff)

if __name__ == "__main__":
    main()