Lisp in The Real World: Using A Sophisticated Language For Mundane Tasks

Lisp in the
Real World [TM]
using a sophisticated language

for mundane tasks
No rocket science today
• No cool new library

• No clever algorithms
• Just war stories
“I’m just a simple application programmer.” [Kenny Tilton]

What’s happened so far
• Lisp (not CL) at university in late 80s,
not impressed
• Freelance hacker since 1998 - PHP (yuk!),
Perl, C, Java, you name it
• Emacs Lisp 1999, then CL around 2000,
became hooked
• First paid Lisp job 2002, Lisp exclusively
since 2004
How to get a Lisp job,
version #1
• Find customers who don’t care

(about the language)
• Just do it!
• [A good Lisp implementation helps]
How to get a Lisp job,
version #2
• The Lisp world is small, so use that to

your advantage.
My first Lisp library
CL-PPCRE
• Started with a bet

• Written in a café in Hamburg
• Core code: 10 Latte Macchiatos
• Perl idiosyncrasies: 20 Latte Macchiatos
• Optimization: 50+ Latte Macchiatos
• Dark corners in the code...
First Lesson learned:
• Write documentation!
Second Lesson learned:
• First rule of optimization:

DON’T DO IT.
• Second rule of optimization:
DON’T DO IT.
• Third rule of optimization (for experts):
DON’T DO IT... YET.
“I cons deliberately.” [Luke Gorrie]

The Hunchentoot story
• Football sweepstakes for friends

(1st version 2000 in, cough, PHP)
• Rewritten using CMUCL/mod_lisp
• Used a bit of “Ajax” in 2002 already
• Earned money with it in 2004
The Hunchentoot story
(cont’d)
• Released “TBNL” (2004) because a

client wanted a documented version
• 2005: Bob Hutchinson patches for
different front-ends Hunchentoot
• (Writing Chunga was fun)

• Too popular now... ☹
Mostly a plain old
online shop
• First PHP, then (2003) CL

• My first “public” website in Lisp
• One app, several websites
• Home-grown CMS
• HTML-TEMPLATE
• Complex product customizations
PHP to the rescue
• Not enough time to continue/extend.

• It’s in PHP (TYPO3) again!
• But they left out all the complicated
stuff...
Lesson learned:
• Everyone puts their pants on

one leg at a time.
Online polls
• For medical specialists

• Particular requirements
• Real-time statistics for the customer
• Graphical representation CL-GD
• Unique amongst competitors (2003)

Lesson learned:
• Niche markets are good.

My life as a web scraper
• Automated data gathering from

unsuspecting websites
• Questionable, but legal
(or so I was told)
• Resulted in the evolution of Drakma
Assembling addresses
• Lots of different sources

• Cookies, user agent variations, etc.
• Mostly CL-PPCRE for “parsing”
• Windows app generating CSV
• 250,000 data sets
Online betting in Asia -
client side
• CL-HTML-PARSE
• Javascript! (Solved by brute force.)
• Volatile
• Bad connections
• MP
Online betting in Asia -
server side
• Monitoring sites in parallel

• Identify identical matches
• Summary: “similar” bets, different odds
• [Analyze and propose bets]
• Make bets observing restrictions
• One “GUI” for all sites
Tariff comparison
• Only one website, but captchas!

• Development based on ca. 100 samples
• Read with CL-GD, home-grown
“cleaning”, statistical analysis
• Success rate 20% - good enough
• One weekend, “anonymous” server
50,000 data sets
Lessons learned:
• The REPL is your friend.

• Make sure you have a great IDE and
learn how to use it.
Talking to cell phones
the good, the bad, the ugly
• European market leader (2002) for

online PIMs
• 200,000+ customers
• Customized versions for T-Online etc.
• Lots of developers (mostly PHP)
• Wanted to sync with cell phones
• SyncML (i.e. XML)

• Sync4j (i.e. Java)
• Don’t buy, adapt - had to be generic
• Half-assed, undocumented, ever-
changing extension interface
• Several phones not fully compatible

• Sync4j ignored some information
• Proxy in Lisp
• CL-WBXML and FLEXI-STREAMS
Lesson learned:
• “Java is great - as long as someone else

writes the code...” [Hans Hübner]
Extending existings apps
• Don’t touch old code

• maintainability
• “black box”
• too horrible
• Lisp is well-suited for this
The Regex Coach
Lesson learned:
• Nobody cares if it’s written in Lisp.

What I wanted
• CL-PPCRE maintained separately

• Track how regex engine works:
• show parse tree
• show partial matches
• step through matching process
How it was done
• Wrap several functions (defadvice)

• Extend one class
• Stepper hooks “reporters” into closure
chain and runs scan in its own process
• Less than 800 lines of code
Aircraft analysis
• Airbus, Boeing, McDonnell Douglas,

MTU, Daewoo, ...
• First design steps of A380
• Since ~1990
• Mac (MCL) app to be ported to
Windows
• More info at this year’s ECLM!
Porting effort
• Core code is voodoo

• Simulate MCL’s QuickDraw with CAPI
• Simulate parts of IDE
• No packages!
• CL-DONGLE
A graphical tool to
design database apps
• GoldHill on Windows 3.x

• Russian C (?) hackers
• Paradox / direct (FFI) Win16 calls
• Now runs in Win16 box
• 50,000+ lines of code
• More than 15 years old and still in use
It works, but...
• NO indentation
• NO documentation
• C-like variable names (A, X, Y, TMP, ...)
• MAPC the only iteration construct
• Home-grown "CLOS” based on
symbols and property lists
[No comment]
Lessons learned:
• Don’t do that!
• Use Emacs.
Back to the future
(Requirements)
• Old code must run unmodified

• Win32 (CAPI)
• But keep non-standard widgets
• SQL
Lessons learned:
• Turing equivalence is just theory.

• Even Lisp can be joyless sometimes.
Lisp as a
scripting language
• FileMaker
• C-API for plug-ins (well...)
• FM-PLUGIN-TOOLS:
• generate FFI defs from C
• create shared libraries (Win/Mac)
• example: “Maut” (road charge)
Lesson learned:
• Lisp is more fun than C.

Simulate signal
processing
• Needed error correction algorithms

(Hamming, BCH, RS)
• (with-galois-arithmetic (m p)
• ODD-STREAMS
• Much more about this at the ECLM
Lesson learned:
• I ♡ Gray streams...
There’s more...
• Google Earth (KML)
• Hifidelio (MP3)
• Self-modifying DB “scripted” with XSL-FO templates
• Semi-automated time-tracking and billing
• Bulk-extract data from PDF into CSV
• Convert Excel files to S-expressions
• Library usage at ITA
• My wife’s website
• My own tax return
Lesson learned:
• “Don't assume Lisp is only useful for Animation and Graphics, AI,
Bioinformatics, B2B and E-Commerce, Data Mining, EDA/
Semiconductor applications, Expert Systems, Finance, Intelligent
Agents, Knowledge Management, Mechanical CAD, Modeling and
Simulation, Natural Language, Optimization, Research, Risk Analysis,
Scheduling, Telecom, and Web Authoring just because these are the
only things they happened to list.” [Kent Pitman]
Thank you!
Hamburg, 2009-09-13
http://weitz.de/eclm2009

Lisp in The Real World: Using A Sophisticated Language For Mundane Tasks

Uploaded by

Copyright:

Available Formats

Lisp in The Real World: Using A Sophisticated Language For Mundane Tasks

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lisp in The Real World: Using A Sophisticated Language For Mundane Tasks

Uploaded by

Copyright:

Available Formats

Lisp in the

Real World [TM]

using a sophisticated language

• No cool new library

• Just war stories

“I’m just a simple application programmer.” [Kenny Tilton]

• Find customers who don’t care

• The Lisp world is small, so use that to

• Started with a bet

• First rule of optimization:

“I cons deliberately.” [Luke Gorrie]

• Football sweepstakes for friends

• Released “TBNL” (2004) because a

• (Writing Chunga was fun)

• First PHP, then (2003) CL

• Not enough time to continue/extend.

• Everyone puts their pants on

• For medical specialists

• Unique amongst competitors (2003)

• Niche markets are good.

• Automated data gathering from

• Lots of different sources

• Monitoring sites in parallel

• Only one website, but captchas!

• The REPL is your friend.

• European market leader (2002) for

• SyncML (i.e. XML)

• Several phones not fully compatible

• “Java is great - as long as someone else

• Don’t touch old code

• Nobody cares if it’s written in Lisp.

• CL-PPCRE maintained separately

• Wrap several functions (defadvice)

• Airbus, Boeing, McDonnell Douglas,

• Core code is voodoo

• GoldHill on Windows 3.x

• Old code must run unmodified

• Turing equivalence is just theory.

• Lisp is more fun than C.

• Needed error correction algorithms

You might also like