DeclarativeFunctional vs. imperative style in Python
Which is better for Python? An imperative / procedural style or a declarative / functional style? Why? I put some examples I've encountered below.
Update 2010-04-10: As dan and pete pointed out, the examples below are not declarative, so I replaced declarative with functional. I've also added a better method for example 1 suggested by deno.
| Imperative | |
|---|---|
Example 1: For each key in a list of S3 keys, download the associated file and return a list of the filenames. | |
def download_files(keylist):
filelist = []
for key in keylist:
filename = '%s/%s' % (TEMP_DIR, key.name)
key.get_contents_to_filename(filename)
filelist.append(filename)
return filelist
| def download_files(keylist):
def get_file(key):
filename = '%s/%s' % (TEMP_DIR, key.name)
key.get_contents_to_filename(filename)
return filename
return [get_file(key) for key in keylist]
|
Here is a better method suggested by deno: def download_files(keylist):
for key in keylist:
filename = '%s/%s' % (TEMP_DIR, key.name)
key.get_contents_to_filename(filename)
yield filename
| |
Example 2: Given a dict, strip the percent character from the value whose key is 'df_use_percent' and return the dict. | |
def post_process(data):
newdict = {}
for k, v in data.iteritems():
if k == 'df_use_percent':
v = v.rstrip('%')
newdict[k] = v
return newdict
| def post_process(data):
def remove_percent(k, v):
if k == 'df_use_percent':
v = v.rstrip('%')
return (k, v)
return dict([remove_percent(k, v)
for k, v in data.iteritems()])
|
Example 3: Parse a ~/.ssh/config file and return a dict of ssh options for a given host | |
def get_ssh_options(host):
def get_value(line, key_arg):
m = re.search(r'^\s*%s\s+(.+)\s*$'
% key_arg, line, re.I)
if m:
return m.group(1)
else:
return ''
mydict = {}
for line in file(SSH_CONFIG_FILE):
line = line.strip()
line = re.sub(r'#.*$', '', line)
if not line:
continue
if get_value(line, 'Host') != host:
continue
if get_value(line, 'Host') == '':
k, v = line.lower().split(None, 1)
mydict[k] = v
return mydict
| def get_ssh_options(host):
def remove_comment(line):
return re.sub(r'#.*$', '', line)
def get_value(line, key_arg):
m = re.search(r'^\s*%s\s+(.+)\s*$'
% key_arg, line, re.I)
if m:
return m.group(1)
else:
return ''
def not_the_host(line):
return get_value(line, 'Host') != host
def not_a_host(line):
return get_value(line, 'Host') == ''
lines = [line.strip() for line in file(SSH_CONFIG_FILE)]
comments_removed = [remove_comment(line) for line in lines]
blanks_removed = [line for line in comments_removed if line]
top_removed = list(itertools.dropwhile(not_the_host, blanks_removed))[1:]
goodpart = itertools.takewhile(not_a_host, top_removed)
return dict([line.lower().split(None, 1) for line in goodpart])
|
Example 4: Summation | |
total = 0
for item in item_list:
total += item.value
| total = sum(item.value for item in item_list)
|
See also
Comments
I don't really see how your "declarative" code snippets are declarative. They are certainly examples of functional programming in Python, but declarative..?? From the wikipedia definition of declarative programming, which you linked: "...expresses the logic of a computation without describing its control flow...", yet the code snippets DO describe control flow. In fact, the only difference is that it decomposes the code into closures which can then be mapped accross lists (sing list comprehension) - textbook examples of functional programming, but nothing declarative.
If you want an example of what I would consider delcarative programming in Python, take a look at the django or Elixer ORM's.
Having said that, I think this post makes a nice introduction to the differences in imperative and functional programming styles in Python. Congrats. Though I would perhaps replace declarative for functional in the text above.
Does it really matter given that the two approaches are so similar?
Personally I find the imperative style a little more straightforward and easier to read, whereas the declarative style smacks of trying to be too clever for your own good.
I tend to use both. Sometimes it's easier to read a solution using imperative style and other times it's easier using a more functional style. Use what feels better in a given situation.
I can't speak for others, but when I use functional programming in python I'm definitely not trying to "be too clever". Sometimes it's just simply the better solution.
Interesting comparison. Personally, I'd go with the imperative. I would assume the declarative approach would benefit from using fewer closures and more previously defined functions, but that is sort of a problem in terms of maintainability. The functional style presented lends itself to better organization of the code through closures, but if that were slower, then moving the closures out makes things harder to read and understand. If you start organizing functions in objects or even dicts, it helps, but most programmers (for better or worse) would probably be taken aback by the format. Time learning a new coding style definitely counts in my book. That is just my opinion though.
My point here is not so much an argument for the imperative style, but settling on the realization that Python doesn't optimize for a functional style. The impact of this decision means you can be penalized for calling more functions than necessary and the community at large will find functional code difficult to understand.
That said, there is a time and place for everything, so it is good to be well versed in both styles in order to use the best techniques where they apply. Thanks for making folks think about it!
I really this kind of cool coding tricks in Python. But I'm just thinking; doesn't go against the Zen of Python?
"There should be one - and preferably only one - obvious way to do it."
Steven: the approaches aren't actually that similar, given that these are extremely short snippets. The declarative examples all have more function boundaries.
Smaller functions are not always good, but a good heuristic for lower-complexity code (http://www.google.com/buzz/105203334711953366503/SCVDA9evKuA/Bad-article-I-cant-believe-Im-reading-this-Just#1269808103349000)
Python has preferred way to write this kind of code:
def some_func(aList):
for elem in aList:
# do something with elem
yield elem
Indeed this functional code is not at all declarative. A declarative language is one in which you specify what you want, and the backend decides how to give it to you. Popular examples include SQL and Prolog.
The "functional" code is not very functional, either. It's graceless and verbose, and not suitable for comparison to imperative-style code.
Here are cleaner implementations of the first two:
The last one is more complicated, but can be simplified. The mess of assigned comprehensions at the bottom is the furthest thing from FP imaginable. What you really want are functions (perhaps with functools.partial) which do all the work for you, then:
return dict(line.lower().split(None, 1) for line in not_a_host(not_the_host(remove_blanks(remove_comments(file(SSH_CONFIG_FILE))))))
The performance also sucks, because each list comprehension is an O(N) traversal. Use itertools.imap / itertools.ifilter and generators for better performance.
Thanks everyone for the feedback!
dan: Thanks for the correction. I made the correction above. Thanks for the reference to some example declarative Python code. I will take a look at that.
Eric: Thanks for the good ideas. I agree Python might not be the best place to do too much functional programming.
rubayeet: This is a concern of mine also.
Kartik: Interesting discussion... I think the small functions can be better for maintainability. I am continually refactoring code also.
deno: Thanks for the better solution. I added it above.
Ian: Thanks for your solutions. The first one doesn't work for me because key.get_contents_to_filename doesn't return the filename. The second one is good-- I did not think of copying and updating the dict. I'll admit I did not consider performance in these examples. If you could demonstrate using itertools.imap and itertools.ifilter, I would be very interested.
Create a function f that given a list l and a group length gl, returns a list of lists with length gl.
a=[1,2,3,4,5]
f(a, 2) # -> [ [1, 2], [3, 4], [5] ]
f(a, 3) # -> [ [1, 2, 3], [4, 5] ]
functional
def groupper_functional( lst, count ):
def sublist( lst, count ):
return lst[0:count]
if len(lst) <= count:
return [lst]
return [sublist(lst, count)] + groupper_functional(lst[count:], count)
imperative
def groupper_imp( lst, count ):
subLst = []
result = []
for elem in lst:
subLst.append( elem )
if len(subLst) == count:
result.append( subLst )
subLst = list()
if subLst:
result.append(subLst)
return result
Eliot, I updated the Gist.
It's not really possible to write a meaningful FP implementation of your first method, if I understand what you're saying. You are relying on side-effects and global state, which run directly counter to functional programming.
A functional implementation would take e.g. a list of URLs, and return a list or dict containing the original URL and some sort of object representing the remote resource. You would then have the chance to write it to disk or use it in memory or pass it to another service… i.e. it would be _composable_, you could reuse the function which fetches the resources in many different ways.
Tiago: Thank you for another example comparing the approaches. To anyone else, feel free to leave your examples as well.
Ian: Thank you for updating the gist! That is some good stuff-- I will have to study up on itertools and functools some more. Thanks for the notes on functional programming also. I am new to functional programming so I have much to learn. The comments on this post are helping with that though!
"I agree Python might not be the best place to do too much functional programming."
Interestingly enough, a lot of Python programmers seem to slowly transition to more and more functional programming style code. I'm told that one of the most sold OReilly books at PyCon is a book on Haskell and going by my own (and other people who I've spoken to about it) experience, it seems quite common. I myself used Python almost exclusively for a while before eventually more-or-less jumping ship to more functional languages (Clojure, Yeti and a little F# and OCaml for me).
Both are an important thing in Python learning that we must learn. With these two things, we able to do different platforms of computer programs that help us to build strong domains. Also, it able to helps us about computer programs that are highly responsive and effective. Good thing that you have able to put some important tips about programming here.
disqus:3151938239