You Can be a Lisper Too!

"Really?" I hear you saying, "you can’t be serious!  Python and Ruby, that’s one thing, but Lisp?  Seriously?!"

About Lisp

Briefly, Lisp is a family of programming languages that date back to 1958.  It is characterized by a very simple syntax, garbage collection, reference semantics, and very powerful language extension and self introspection capabilities.  Many features in common programming languages today were first pioneered in dialects of Lisp.

Common Lisp is the most commonly available and "general purpose" Lisp, and has been out there since 1984.  It supports object-oriented, imperative, functional, and generic programming techniques.

For information on the history of Lisp, please check out the paper The History of Lisp by Lisp’s original designer, John McCarthy — there are a lot of references in that paper to get you started with Lisp’s history.

For information on the present of Lisp, such as learning Common Lisp, I will again recommend Practical Common Lisp by Peter Seibel, available in print form or via said website for no cost.

I thought Lisp was like, esoteric or a dead language or something?

I see a reoccurring barrier to conversation when I mention Lisp — when I mentioned I was learning Python and Ruby, people generally responded positively.  But when I mentioned Lisp, I get incredulous responses, as if I mentioned Intercal or something. 

I think Lisp is being taught *wrong*.  Hidden behind the instruction of recurive function definitions and continuations, there’s a useful imperative programming language there too, and that you can start learning Lisp with its familiar constructs before moving on to the esoteric features. 

Don’t get me wrong, I think the more esoteric features are important!  But you don’t start learning mathematics in multi-variable calculus.

Lets try some imperative examples.

(defun factorial (x)
  "calculate factorial the *boring* way"
  ;; create a local variable
  (let ((accum 1))
    ;; note: from 2 to x includes 2 and x
    (loop for current-factor from 2 to x do
          ;; multiply to variables and assign to accum
          (setf accum (* current-factor accum)))

(defun fibonacci (x)
  "calculate fibonacci the *boring* way"
  ;; create 3 locals variables
  (let ((a 1) (b 1) (tmp nil))
    ;; loop x times
    (loop repeat x do
          (setf tmp a)
          (setf a b)
          (setf b (+ tmp a)))

Oh no you didn’t!  Two functions notorious for recursive definition, written in imperative style.  Imperative style! A beginner (like me) has to start somewhere, and the best way to start with programming is with working code you can understand and then build on.  So if you’ve never used Lisp, then…

Try it!

First, get yourself a Common Lisp implementation.  There are a variety of them out there.  I used Steel Bank Common Lisp on Windows for this example — it’s marked experimental on Windows, but I haven’t run into any problems yet tinkering.  They also have stable versions for Linux, BSD varieties. 

Installation is simple — they have an MSI, but I’d recommend just unzipping it.  Add it to your path or create an alias for sbcl.exe.  Invoke SBCL and you should see:

$ sbcl
This is SBCL 1.0.22, an implementation of ANSI Common Lisp.
More information about SBCL is available at <>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.

This is experimental prerelease support for the Windows platform: use
at your own risk.  "Your Kitten of Death awaits!"

Which means you’re in business.  Copy all the complete code snippets seen here into ‘memo.lisp’, and then you can load it at the SBCL prompt, represented by star (*).

* (load "example.lisp")

* (loop for x upto 30 collecting (fib x))

(1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711
 28657 46368 75025 121393 196418 317811 514229 832040 1346269)

Loop is a sort of minilanguage in Common Lisp for doing various looping tasks — this one iterates over x from zero to 30, collecting into a list the results of invoking (fib x) for each value.  This is roughly equivalent to Python "[ fib(x) for x in range(21) ]".  Don’t worry about the details for now, just copy and paste for now.

Lastly if you run into any sort of error, you may see a strange prompt, such as:

* (fiib 2)


restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

("bogus stack frame")

Ack!  Don’t worry, this is the debugger, and it seems that every Lisp environment comes with one that is on by default.  To run away, just enter either :q or :a… one of these is bound to escape back to the REPL in just about every Lisp I’ve tried so far.  You can then pick up where you left off.

Eventually learning a debugger will be valuable, but to begin with, just try taking it apart one statement at a time. 

0] :a

* (fib 2)


Back in business.  When done, execute (quit) or press Ctrl-Z

Where to get Lisp?

I’m currently running all of the following on my dev laptop:

  • Common Lisp
    • SBCL – Steel Bank Common Lisp, as mentioned earlier.
    • ECL – available on almost everything it seems.  Distributed in source form only, but I was able to build it on Windows with Microsoft Visual C++ Express 2008 SP1 in about 5 or 10 minutes on a dinky netbook.
    • GCL – Gnu Common Lisp, available on Windows and Linux.
  • Scheme – (slightly different syntax, will have a post in the future)
    • Chicken Scheme – of interest because it compiles to C using CPS and uses the C stack instead as free-store!
    • MzScheme – available on Windows and Linux.  I understand it also contains graphical ui in addition to traditional command line.
    • IronScheme – an implementation of Scheme for the CLR and DLR runtimes, interoperates with C# and other managed components.  Not sure if it runs on Mono, but runs fine on Windows.

I had one other Lisp implementation installed earlier this week, it was a space hog.  The ones listed above are in the 10-50MB range, so you really have no excuse for not installing one.

Chicken and ECL can both compile to C.  SBCL compiles to native code, and despite being dynamically typed, can compete favorably with some statically typed languages, for example the language shootouts of SBCL against C# and Java.

Pick one, install it, learn Lisp, you’re set.  Make it a new-years resolution 🙂

And don’t forget Practical Common Lisp to get you going.

Memoizer 4b – Common Lisp

Continuing this series, in Common Lisp…


As before, we’ll be walking through a memoizing function — for review, it’s a function that will transforms an existing function into one that remembers previous results instead of recomputing them.

;;; memoizer
(defun memoizer (f)
  (let ((store (make-hash-table)))
      ((replace-f (self n)
                  (let ((found (gethash n store)))
                    (if found
                      (setf (gethash n store) (funcall f self n))))))

The first part of memoizer uses the Common Lisp function ‘make-hash-table‘ to create a new hash table, and assign it to a new local variable ‘store‘.  You might also see this refered to as a ‘lexical variable‘, which simply means that you can only refer to the variable in that block of the program — a variable with the same name in a different block is a different variable.

(let ((store (make-hash-table)))
  ... )

Next, we’ll create a new replacement function nested in memoizer, named ‘replace-f‘.  As in the Lua and Python examples, nested functions can refer to the outer lexical scope as well, so it can make use of ‘store‘ from within this replacement. 

  ((replace-f (self n)

Replace-f works like f — it’s a function that takes two arguments, the first being a function to use for recursion, and the second is the value ‘n‘ for which we’re memoizing.  This version will check for a cached value corresponding to ‘n‘, and if not found, recurse on the original function ‘f‘.

Lastly, we recursively call through the variable ‘f‘, the original function.  In Common Lisp, variables and functions are separate namespaces, so to call through a variable, we use the funcall operator (similar to the .call syntax in Ruby):

(funcall f self n) ; call f with (self n) as arguments

Abstracting recursion

You may have noticed something up with how recursion is being done — instead of calling ‘(f n)‘ i’m calling ‘(f something n)‘.  In the Python example recall, ‘fib’ was first defined to call ‘fib’ recursively, but since the name ‘fib’ is resolved dynamically in Python, by the time it gets called ‘fib’ now points to a different function — it’s no longer auto-recurive.

Alternately, ‘fib’ could be specified to take the replacement as an argument — and I’m doing that here.  Rewriting fib to take itself as an argument:

;;; fixed-point version of fibonacci function
(defun fib-fixed (self n)
  (if (< n 2)
    (+ (funcall self self (- n 1))
       (funcall self self (- n 2)))))

In this case, you can invoke "(fib #’fib 30)" to calculate the 30th fibonacci number, but it will take a while.  We can now memoize this function though, without replacing the original function, by modifying the recursive ‘self’ parameter.

(defun fib (n)
  (let ((memo-fib (memoizer #'fib-fixed)))
    (funcall memo-fib memo-fib n)))

This function will create a new memo of fib, and then invoke with that memo. 

If this seems inside out, you can always use Common Lisp’s dynamic capabilitied to redefine fib like was done in the earlier examples.  Proof by example:

;;; simpler memoizer, doesn't require
;;;  function to provide recursion parameter
(defun simple-memoizer (f)
  (let ((store (make-hash-table)))
      ((replacement (n)
                    (let ((found (gethash n store)))
                      (if found
                        (setf (gethash n store) (funcall f n))))))

;;; define a global named *fib2* that is a function
  (lambda (n)
    (if (< n 2)
      (+ (funcall *fib2* (- n 1))
         (funcall *fib2* (- n 2))))))
;;; replate *fib2* with a memoized copy of the current
;;;  value of *fib2*
*fib2* (simple-memoizer *fib2*))

;;; wrapper function
(defun fib2 (n) (funcall *fib2* n))

Try it out!

Copy all the complete code snippets seen here into a file named ‘memo.lisp’, and then you can load it at the REPL prompt, represented here by star (*).

* (load "memo.lisp")

* (loop for x upto 50 collecting (fib x))

(1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711
 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578
 5702887 9227465 14930352 24157817 39088169 63245986 102334155 165580141
 267914296 433494437 701408733 1134903170 1836311903 2971215073 4807526976
 7778742049 12586269025 20365011074)

And there ya go!