Django and multiple joins on the same table > Post > Jan 22, 2010 01 a.m.

I picked up this trick back using Wordpress, which I always kinda liked, you scenario: you have a Content model and a Date model, but the Date model is generically-related:

#django models
class Date(models.Model):
    # GR stuff
    content_type = models.ForeignKey(ContentType, null=True, blank=True)
    object_id = models.PositiveIntegerField(null=True, blank=True)
    content_object = generic.GenericForeignKey()

    # actual data
    key = models.CharField(max_length=75)
    date = models.DateTimeField()

class Content(models.Model):
    title = models.CharField(max_length=75)
    body = models.TextField()

    dates = generic.GenericRelation(Date)

The reason you would do this is so you could have as many dates related to the content as you wanted (you don't have to use a generic relation for this, but I like to use dates on lots of things).

What fun

I'm a little eccentric, I like to do wierd things, like with Django's .extra() method on querysets, you can do some funky stuff, like pull several of the generically-related dates based on their 'key' value, and add that 'key' value directly to the Content object as though it were a native attribute. Something like this:

>>> from myapp.models import Content, Date
>>> from datetime import datetime
>>> content = Content(title="Hello", body="My content :)")
>>> content.save()
>>> post_date = Date(key='post_date', date=datetime.now())
>>> post_date.save()
>>> content.dates.add(post_date)
>>> modified_date = Date(key='modified_date', date=datetime.now))
>>> modified_date.save()
>>> content.dates.add(modified_date)

Now the trick here is to join the myapp_date table on to our content twice, which is easy enough:

>>> content = Content.objects.all()
>>> content.filter(dates__key='post_date')
>>> content.filter(dates__key='modified_date')

But now we have a problem

Because django is so wonderful, it does all the work of joining tables and keeping track of those pesky aliases for you, but how do we select the dates from the joined tables (using .extra(select={xxx})) if we don't know the aliases? Well we can spit out the query, as mentioned in the django docs, and that works fine (for now):

>>> print content.query # live on the edge...
# the result is something like:
SELECT "myapp_content"."title", "myapp_content"."body" FROM "myapp_content" INNER JOIN "myapp_date" ON ("myapp_content"."id" = "myapp_date"."object_id") INNER JOIN "myapp_date" T2 ON ("myapp_content"."id" = T2."object_id") WHERE ("myapp_date"."key" = "post_date" AND T2."key" = "modified_date")
# Disclaimer: I wrote this query by hand, it's an approximation, yours may differ, also note: it's pretty damn close.

Now if you were just feeling adventurous, you could simply nab the alias names from that query and grab the extra data as needed:

>>> content.extra(select={'post_date': 'myapp_date.date', 'modified_date': 'T2.date'})
>>> print content.query
# the result is something like:
SELECT (myapp_date.date) AS post_date, (T2.date) AS modified_date, "myapp_content"."title", "myapp_content"."body" FROM "myapp_content" INNER JOIN "myapp_date" ON ("myapp_content"."id" = "myapp_date"."object_id") INNER JOIN "myapp_date" T2 ON ("myapp_content"."id" = T2."object_id") WHERE ("myapp_date"."key" = "post_date" AND T2."key" = "modified_date")
# extra(select={}) just pops a couple extra selects in there, like it should, but yuck, static references ie: T2
>>> content[0].post_date
datetime.datetime(2010, ...)
>>> content[0].modified_date
datetime.datetime(2010, ...)

Fun huh?

But, there is a gotcha...

I had been employing this method in an application, and arbitrarily decided to upgrade my django. Lo' and behold, the alias names changed! Whooda thought? So after much turmoil and source-code-diggery, I found the way to get those pesky aliases (and therefore perform the query) without all the guilt. The trick was I had to find the place in django.db.models.sql.query.Query where the table alias (T2) was related to the part in the where clause that was related to the field (post_until). The answer was in the content object's where clause, something like:

>>> content.query.where.children
[(<django.db.models.sql.where.constraint object="object" at="at">, 'exact', True, 'post_date'),(</django.db.models.sql.where.constraint><django.db.models.sql.where.constraint object="object" at="at">, 'exact', True, 'modified_date'),...]
>>> content.query.where.children[0].alias # < the constraint holds the key, literally
'myapp_date'
>>> content.query.where.children[1].alias
'T2'
</django.db.models.sql.where.constraint>

And there you have it

So my quick solution to keep the aliases produced by they code referenced dynamically, I just build a quick little dict out of the 'where.children' so I could reference the aliases properly later...

# summed up 
>>> content = Content.objects.all()
>>> content.filter(dates__key='post_date')
>>> content.filter(dates__key='modified_date')
>>> alias_map = {}
>>> for condition in content.query.where.children:
...    # alias_map['modified_date'] = 'T2'
...    alias_map[condition[3]] = condition[0].alias 

>>> content.extra(select={ \
...    'post_date': '%s.date' % alias_map['post_date'], \
...    'modified_date': '%s.date' % alias_map['modified_date']})

And with that, the keys end up as they should. Nothing spectacular, it just took some digging to find where the join "black magic" was happening in django. Thanks to jtiai on #django for the pointers.

Hope that really helps someone down the line,

Aaron

Flex Django and AMF > Post > Dec 15, 2009 10 p.m.

Been working back in Flex again a bit, using Vim in Linux for a while, but I been really impressed with the short compile times in FlashDevelop for Windows.

Besides the point

Having become increasingly frustrated with the shortcomings of Django-amf, I've decided to give pyamf another try. I tried it the first time around, and got hung up on an error that made me bail out on it:

Cannot find a view for the path ['/gateway/cardservice/echo'], 'DjangoGateway' object has no attribute '__name__'

Some poking around in the source didn't help any, but the fix turned out to be easy. I stumbled on this post on StackOverflow, the author answered his own question in his comment. Turns out, leaving the django-amf middleware turned on in the settings.py file messed up the operation.

I don't have enough clout on StackOverflow to tell the author thanks there, so I'll do it here...

Aaron

Foot of snow and 30 below > Photo > Dec 13, 2009 05 p.m.

That's all I have to say about that

Adbusters + Pioneer General Store = Whoa! > Post > Dec 09, 2009 03 p.m.

I nearly shat myself today. And there is nothing glorious about that!

I was standing at the checkout of "the Pioneer" (the little general store smack in the middle of Caroline, AB, population 500 [including the cats]) and I glanced across a magazine that stuck out in the middle of the tabloids (the usual mental staple of Carolinites) like a sore hip > lo and behold a copy of Adbusters!

Shock set in

I says to the lady at the till (as I set the magazine down for purchase), I says, "this magazine is a little progressive for poor, little, innocent (read:hick) Caroline, isn't it?" :O

Well, it was a wasted effort, she didn't get it. Still made my day, though.

Af

Google Chrome for Linux > Post > Dec 09, 2009 02 p.m.

I just installed Google Chrome for Linux yesterday, and decided to try it out. It's been bugging me lately that my application set in Linux (vim, svn, irssi, etc) are all snappy, and yet Firefox is still a little slow (even with most of my plugins shut off). I've been craving a faster browser, and some of the others just don't make the cut.

Thank you Google

I signed up for the Chrome release mailing list and got the email yesterday:

Hello everybody out there using Linux -

Google Chrome is go for beta on Linux! Thanks to the many Chromium and WebKit developers who helped make Google Chrome a lean, mean browsing machine. Here are a few fun facts from us on the Google Chrome for Linux team:

60,000 lines of Linux-specific code written
23 developer builds
2,713 Linux-specific bugs fixed
12 external committers and bug editors to the Google Chrome for Linux code base, 48 external code contributors

Thanks for waiting and we hope that you enjoy using Google Chrome!


Google Chrome Team

Installed like a snap, and it sure is snappy. The developer tools are looking quite promising also...

Sorry Firefox, looks like Chrome may be here to stay...

Af

Vim and ^M Dos characters > Post > Dec 04, 2009 01 p.m.

I've been working on a Flex project recently, one that I used to compile in Windows, but have since moved over to Linux with the Flex SDK. One of the problems I run into is that Dos appears to leave these characters in my files at the end of each line

// a comment in my flex file^M

The solution

It's painful to have to delete them all manually, so a simple command in vim will do it:

:1,$ s/^M//

To get the ^M you can hit 'Ctrl+v' and then 'Enter'

Aaron

Dreamhost, Django, Premature end of script headers: dispatch.fcgi > Post > Oct 28, 2009 03 p.m.

This problem comes up from time to time on my Dreamhost account, I get a long page load, finally ending with a 500 error in the browser. Tail'ing the apache error_log shows:

...
[Wed Oct 28 13:48:37 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:48:37 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:48:40 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:49:18 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:49:21 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:49:58 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
[Wed Oct 28 13:50:39 2009] [error] [client 198.53.10.99] Premature end of script headers: dispatch.fcgi
...

But where's the real error?

A little trick for you: go to the folder where your dispatch.fcgi file is and type:

./dispatch.fcgi

Basically you are executing the script, similar to an incoming request. You should then see the original traceback.

Hth,

Aaron

Dreamhost and Django (fcgi) caching > Post > Oct 28, 2009 12 p.m.

Every once in a while (while dabbling with Django on Dreamhost) I run into the problem that Dreamhost appears to cache my site. I've even canned the db or deleted the settings file to try and provoke an error. But the site ticks on (for a bit anyway), and I want to see results now! damnit...I am after all hacking on the live site instead of my staging server! Talk about living on the edge!

The cure

Thwart the so-called caching with a mighty blow to the process itself, it will be restarted at the next request:

killall -USR1 dispatch.fcgi

Thanks to the dreamhosters forum for the tip.

Af

Files where Ubuntu menus are stored > Post > Oct 21, 2009 12 p.m.

Note to self - Ubuntu menus are stored here:

/usr/share/applications 

I keep forgetting this one...

af

svn command line commit wierdness and Windows XP > Post > Sep 29, 2009 10 p.m.

I thought I was entering into Windows twilight zone, svn wouldn't let me commit on my XP console. Turns out it doesn't like single quotes, so this wouldn't work:

svn ci -m 'This is my message'

I had to do this:

svn ci -m "Windows XP sucks bobo bigtime"

Hope that helps someone keep from pulling their hair out...

Aaron

XP and dv9500 > Post > Jul 24, 2009 11 a.m.

I ran into a problem with my new(ish) laptop, I've been running Ubuntu 9 on it since I got it, and previous owner put the Windows 7 RC (no link intended) against my wishes. So the RC finally expired, and I'm left without Windows on my laptop. Normally this wouldn't be a big deal, but I have a Mackie Onxy 400f and Canon XL2 which can both connect and record on the laptop via the IEEE1394 (firewire) interface.

The problem

So I own a copy of XP Professional (from when it first came out), and I try to install it. Problem is, the drives are SATA and the XP installed doesn't recognize there's hard drives in the machine. I looked at a couple guides and opted not to build my own XP, and instead downloaded Vista Premium, for which there is a key on the bottom of the laptop. I've been resisting this to the bitter end, even trying to get a copy of OSx86 running instead (to no avail).

So I finally install Vista (actually my brother Dan did), update all the garbage, install the software, do all the typical 'fresh OS' stuff, and run my software to interface with the XL2, and find out there's a bloody bug in Vista that the firewire doesn't work properly or something stupid. The computer (rather, Vista) won't recognize my damn camera. Not much of a remote monitor for shooting then, is it. We've really made technological advancements when we sell people a feature like 'Firewire' on a laptop, and the operating system can't even utilize it. Great work, Microsoft.

The fix

So I downloaded XP Dark which promised to have SATA drivers for my laptop. I'm not into pirating Microsoft stuff, but if the bastards can't give me a working Vista, and won't let me install my XP Pro (it was $1000 US new), then they can kiss my arse.
XP Dark installed like a dream (it gave me an install option for SATA), it was way faster than a standard XP install, and relatively shiny. The last problem I ran into was a whole whack of missing drivers. Like, missing drivers for just about everything. No ethernet, wireless or otherwise. The only thing that did work was the silly webcam!

Anyway, I rummaged around a bit, and tried finding drivers for some stuff, but was having limited success. I found this page with link to a Super Awesome XP Driver Pack specifically for me laptoppy! All the drivers installed perfectly (thanks again, Dan) and we're now on our way to on-location shooting heaven!

Cheers,

Aaron

XBMC video problem > Post > Jul 13, 2009 06 a.m.

I just discovered XBMC media center while setting up a computer for a friend of mine, and it's pretty awesome. I couldn't figure out how to get it to load some episodes into the video Library, XBMC has missing or hiding the videos, and it turns out, the episodes need to have the episode number formatted like so: 1x01 in order for it to be recognized. I don't know if it matters where that is in the name, but 01-01 didn't work.

af

Fluxbox keys stopped working > Post > Jun 23, 2009 04 p.m.

I was mucking around trying to hide some files in vim's netrw, and all of a sudden fluxbox did some wierd blinky thing, and I had to reset my machine. None of the menus would work, I couldn't switch workspaces, nothing.

Resetting brought me back into fluxbox, but my theme had been reset to the default, which was wierd. So I set it back to the one I downloaded and started to go back to work...but...now none of my hotkeys were working. I jumped on #fluxbox and didn't get anything useful so I did a little digging around and found this post on linuxquestions.org. Turns out the session.keyFile setting in ~/.fluxbox/init was set back to the default which is /etc/X11/fluxbox/keys on my machine. So I put it back to my user file:

" ~/.fluxbox/init
session.keyFile: ~/.fluxbox/keys

After a quick restart, everything is back to normal again...oh, except I have to set up more workspaces, I lost 3 of them. Hmmm, wish I could figure out what the hiccup was.

Maybe next time.

Af out.

'Find in files' in vim > Post > Jun 14, 2009 03 p.m.

I have been enjoying vim quite a bit recently, it is really starting to speed up my workflow. The learning curve is constant, however. The latest thing I have needed to know (and use quite a bit) is a 'find in files' feature. Vim has this built in with a plugin called 'vimgrep'. The documentation was pretty straight forward (:h vimgrep) but it wasn't immediately obvious how to do recursive searching. The solution was what vim calls 'starstar-wildcard' (essentially **)

This simple example produces a recursive search:

:vimgrep /myPattern/ **

The above example will search through everything though, including binary files (like pngs and such), so a little more instruction is needed.

:vimgrep /myPattern/ **/*.html

# this is also equivalent
:vim /myPattern/ **/*.html

Vim then gives you the first of the matches, you can cycle between them with :cnext/:cn and :cp/:cprevious.

If you have a lot of files to search through, :grep is faster.

:grep -Ir /myPattern/ *

Not so hard after all, af

Dreamhost and SMTP server > Post > Jun 10, 2009 08 p.m.

I was monkeying around a little bit for a friend earlier, trying to set them up on one of our outgoing mail servers, since our ISP server has been a little funky lately. I just wanted to note that Dreamhost SMTP server works on port 587 and not 25. When you try and send mail, it will not authenticate on port 25.