Django snippet — Template tag: split list to n sublists

This is the code for a Django template tag I adapted (also posted here). It was based on this snippet, adapted to split a list into n number of sublists, e.g. split a list of results into three evenly-divided sublists to enable display of the results in three columns on one page with CSS.

Tag: {% list_to_columns your_list as new_list number_of_columns %}

You don't have to know the exact number of items in the list, just the number of columns (sublists) you want to split the list into. For example, given this list:

monty = ['Eric', 'Graham', 'John', 'Michael', 'Terry', 'Terry']

using the tag like this:

{% list_to_columns monty as full_monty 3 %}

will produce a new object called full_monty containing three new sublists:

['Eric', 'Graham']
['John', 'Michael']
['Terry', 'Terry']

The remainders will be evenly distributed from the first list on over, e.g.:

monty_plus_one = ['Eric', 'Graham', 'John', 'Michael', 'Neil', 'Terry', 'Terry']

{% list_to_columns monty_plus_one as full_monty 3 %}

will produce a full_monty object that contains three sublists:

['Eric', 'Graham', 'John']
['Michael', 'Neil']
['Terry', 'Terry']

Example template usage:

{% load list_to_columns %}
{% list_to_columns people as list 3 %}
	{% for l in list %}
		(cycle through your div and ul code)
			{%for p in l %}
				(cycle through your list items)
			{% endfor %}
		(end ul and div tags)
	{% endfor %}

The list_to_columns.py code:

"""Splits query results list into multiple sublists for template display."""

from django.template import Library, Node
    
register = Library()

class SplitListNode(Node):
    def __init__(self, results, cols, new_results):
        self.results, self.cols, self.new_results = results, cols, new_results

    def split_seq(self, results, cols=2):
        start = 0
        for i in xrange(cols):
            stop = start + len(results[i::cols])
            yield results[start:stop]
            start = stop

    def render(self, context):
        context[self.new_results] = self.split_seq(context[self.results], int(self.cols))
        return ''

def list_to_columns(parser, token):
    """Parse template tag: {% list_to_colums results as new_results 2 %}"""
    bits = token.contents.split()
    if len(bits) != 5:
        raise TemplateSyntaxError, "list_to_columns results as new_results 2"
    if bits[2] != 'as':
        raise TemplateSyntaxError, "second argument to the list_to_columns tag must be 'as'"
    return SplitListNode(bits[1], bits[4], bits[3])
    
list_to_columns = register.tag(list_to_columns)

As usual, save list_to_columns.py in your templatetags directory, wherever you have that directory located.

A little tip/weirdness/FYI for others who run into this problem: I made my templatetags directory as a subdirectory of my project, e.g. myproject.templatetags, but had to load just 'myproject' into the INSTALLED_APPS part of settings.py to get it to work. I thought I needed to load myproject.templatetags there (as you do for apps), but that just raised an error: 'list_to_columns' is not a valid tag library: Could not load template library from django.templatetags.list_to_columns, No module named list_to_columns. Anyway, loading 'myproject' rather than 'myproject.templatetags' solved the error for me.

References:

8 Comments

  1. Thanks for sharing. The templatetag is really useful.

    The templatetag has some performance issues when splitting querysets though. A large number of SQL queries are executed. To reduce the number of queries you could evaluate the queryset prior to splitting. I posted one approach on the django-snippets page.

    Thanks again for sharing.

  2. Herself says:

    Glad you found it useful, Kjell. :) I'm trying really hard to document things that took me a while to figure out or find answers to while working on my current projects, 'cuz goodness knows I'd be lost sometimes without other kind souls who have done the same thing. *vbg*

    Now on to your suggested change (and this is my lack of knowledge of python showing, as I'm still learning it), how does adding the "tmplist=tuple(list)" bit help reduce the number of queries?

    (I must admit to a great deal of frustration with how the queries and results from django are structured, as they just don't make sense to me yet. Largely because I'm not really comfortable with the ins and outs of OOP, I think. I only just discovered the connection.queries trick to see the raw sql last week, and am still looking for a way that will display the results of the queries and their object names, because that would help me immensely in my debugging. Ah, the joys of a new language/framework. *g*)

  3. The reason for the high number of queries is due to the fact that Querysets are lazy. They are only evaluated when needed. So when you slice your list with code like len(list[i::cols]) you hit the database multiple times. For instance, determining the length of the queryset requires one query. The results are then thrown away and a new query is necessary for the next iteration. Likewise, extracting a slice requires a new query.

    With the code "tmplist=tuple(list)" all items in the list are retrieved from the database with a single query and stored in a list. So writing len(tmplist) now does not require a database query. Neither do slicing. The downside is that all items are stored in memory, but that is usually not a big problem.

    I hope that the above explanation makes things clearer. If you are interested in looking at the SQL-queries I recommend trying the "Django Debug Toolbar" (http://rob.cogit8.org/blog/2008/Sep/19/introducing-django-debug-toolbar/) That's how I found out about the "problem" with your template tag. A page that used your tag required more than hundred queries. After I did the tuple/list trick the number of queries was reduced to eight.

    PS. I used tuple instead of list to avoid confusion with your variable named list.

  4. Herself says:

    Thanks for the pointer to Rob's debugging toolbar! (Very nice, though it still doesn't have the one thing I really need, of course. *g*)

    However, I'm not seeing any change in the number of queries performed (one for each try) on the pages I'm using the template tag as written versus the change you've suggested. (Though it makes more sense to me know why you're suggesting that now, thanks. *g*)

    Also, I've changed the variables used so list isn't a variable name. Didn't think about that when I started working on this. *g*

  5. Herself says:

    Oh wait, I take that back. The debug toolbar does sort of have one of the things I was looking for; it was under templates, which I hadn't looked at yet. (The request context info toggle is what I'm looking at, though it still doesn't give me exactly what I want, but it's a start. *g*)

  6. In an earlier comment I said that the change reduced the number queries from over a hundred to eight. That was not completely true. I did a few denomalization tricks that also reduced the number of queries a lot.

    The reason that you don't see any changes may be that you are already passing an evaluated queryset to the templatetag. If you only use two columns the extra overhead will not be that large either. In my case I split it into four columns.

    The debug toolbar is indeed useful. It can be a bit slow for complex pages, but it is a great tool for spotting bottlenecks.

  7. Matthew Woitaszek says:

    Thanks, this is really useful!

    I know I'm a bit behind the discussion, but I found a change that you might consider useful. If you replace the line:

    context[self.new_result s] = self.split_seq(context[self.results], int(self.cols))

    with

    context[self.new_results] = self.split_seq(Variable(self.results).resolve(context), int(self.cols))

    then you can take advantage of Django's automatic context variable resolution feature. This will allow you to use the template tag to split lists that are elements of hashes, etc., instead of just top-level lists in the context dictionary.

    I ran into a problem with the code as-is when I wanted to split a list that was an element of a hash in the context. That is, the list was created in the view as variable["hashname"]=[], and the entire hash variable was returned through context. When specified in the template this turned into things like {{for element in variable.hashname}} — which worked fine. But when passed to list_to_columns, it failed with a key lookup. Back in Python, context["variable.hashname"] doesn't exist — you have to know that it's a hash and use the bracket syntax. The Django django.template.Variable object can take care of this type of lookup automatically.

    Thanks again!

  8. Steve says:

    Thanks for sharing. The templatetag is really useful.

    The templatetag has some performance issues when splitting querysets though. A large number of SQL queries are executed. To reduce the number of queries you could evaluate the queryset prior to splitting. I posted one approach on the django-snippets page.

    Thanks again for sharing.

Leave a Reply