It's not that there is anything particularly special about what you can do in Lisp than what you can do in, say, C#. It's that Lisp provides facilities for metaprogramming that most other languages lack. Most other languages make metaprogramming hard enough, or require it all be done at runtime, that it is very discouraging against the concept in general. Lisp creates a culture around metaprogramming that fundamentally changes how you approach programming forever after.
You stop writing programs. You start writing programs that write programs. It's a lever that multiplies the power of the programmer. That can be both good and bad. For a hacker coding in earnest on her own, it can be extremely good.
Yes, if you are the type of beginner or intermediate programmer that still struggles to just write programs, advanced metaprogramming in any language is not for you. If you're an intermediate programmer looking to become an expert programmer, learn Lisp, where metaprogramming is easy, and take the lessons on to whatever other languages you eventually end up using.
I've never had to change my mind about Lisp -- but to me, Common Lisp's condition system kicks the crap out of every other language because it allows you to handle conditions without unwinding the stack.
And whenever you get dropped into the debugger, you can edit and reevaluate the broken code or values or whatever -- and then continue on with the task.
In other words, you can begin a task with imperfect code -- and refine it as you go along -- figuring out how to handle corner cases as you go. All without starting the task over from the beginning.
I know of only one other language that allows you to explore a problem space like that.
Besides, the parentheses make the code less jagged looking and jagged looking code imo, is more tiring to look at. Lisp is simply prettier.
Smalltalk also drops you into a debugger on an error, letting you inspect the state of the system, correct your code, and resume. Check out Pharo Smalltalk.
Smalltalk probably supports this as well, but what's cool about Common Lisp's condition system is that it doesn't depend on user interaction or debugger to work. Conditions let you separate the code making the decision about how to handle an error from actual error handling. The decision doesn't have to involve interactive debugger, you can have your code making the choice about which restart (unit of error handling code) to invoke. The common use case is that you install various restarts as you go down the call stack; then, when an exception happens, control goes up the stack until appropriate decision code, which selects restart, and control goes back down the call stack to the selected restart block.
It's an incredibly flexible system that makes exception handling much more useful.
Tcl I heard has string-based homoiconicity. Otherwise, I'm yet to see a proof you can get an actual, usable and not incredibly annoying macro system working without turning your language into a Lisp. People keep trying, but it's always some inelegant subset bolted onto the core syntax.
I love using Julia, but for me writing Julia macros came with a steeper learning curve than in Lisp / Scheme, due in part to the syntax. I kept wishing Julia was a layer on top of a Lisp, and that I can just write Lisp macros. (I know the parser is currently in Scheme, but you can't exploit that easily.)
I tend to end up writing `Expr` objects directly when I'm building larger macros as I find them much easier to reason about. It's clearly not as convenient/clean as Lisp though. (David Moon actually sent a PR last year to make hygiene much easier.. unfortunately it got stuck for a while, but the PR was recently rebased and hopefully will be merged eventually).
Regarding the learning curve: we rewrote the first half of the Metaprogramming section of the manual a few months ago to try to provide a more gradual introduction, especially for people without a Lisp background. Objectively I don't know if the change has helped, but I tell myself that the number of macro-related questions has gone down :)
We would really like to make these tools accessible to people both with and without a formal CS background (probably the majority of Julia users will fall in the latter category). So, if you have any suggestions for doc/pedagogy improvements in this area, I would be very happy to hear them!
We talk sometimes about having an additional reader to support another syntax ... but so far, it isn't something where someone's actually volunteered to step up and help. It'd be interesting to play with some ideas from David Moon's PLOT as well ...
Take a look at Converge, Nemerle, TH and many other similar languages - they're doing just fine without any homoiconicity. All you need is a decent quasiquotation.
Thanks for the pointers, I never heard of those languages before.
I am actually looking at the Converge documentation on macros now, and I found the perfect quote to highlight the problem I see with those approaches:
Quasi-quotes allow ITree's to be built using Converge's normal concrete syntax. Essentially a quasi-quoted expression evaluates to the ITree which represents the expression inside it. For example, whilst the raw Converge expression 4 + 2 prints 6 when evaluated, [| 4 + 2 |] evaluates to an ITree which prints out as 4 + 2. Thus the quasi-quote mechanism constructs an ITree directly from the users' input - the exact nature of the ITree is of immaterial to the casual ITree user, who need not know that the resulting ITree is structured along the lines of add(int(4), int(2)).
That is, quasi-quotes take some code and turn it into a tree object. This tree object can then somehow get compiled into actual code later. Compare that with Lisp approach, in which code is data. You don't think about the code as something separate from the tree that represents it. The code is the tree. There is no ITree object.
It may seem like just "semantics", but I find this to be a significant cognitive difference.
> Compare that with Lisp approach, in which code is data.
It's not any different. It does not matter if your `(,a ,b) is compiled into (list a b), or your ` \a\ = \b\` is compiled into make_binary('=',a,b) - both are constructors, essentially.
Sure it is. All parts of a Smalltalk program are present as objects that can be manipulated by sending them messages. Classes, methods, variable bindings, closures, stack frames, even individual messages are objects.
As a practical matter, methods also have a textual representation, but that's not strictly necessary. Ten or fifteen years ago, the Squeak method browser had a mode where you could write Smalltalk code using tiles, kind of like in Etoys, no parsing necessary. That was a hassle to use, though, so people stuck with writing methods using text.
By the way, Lisp has the same sort of "impurity". S-expressions are very simple, but they are syntax. It's surely possible to create a tool that would let one construct cons cells in memory directly, but that would be a hassle. Instead we use parentheses.
Maybe not the same, but I feel like nim (http://nim-lang.org/docs/manual.html#templates) can have a comparable power ot redefining itself. But I'm absolutely not an expert neither in any lisp nor in nim so I can't really tell.
After learning about Elixir (http://elixir-lang.org/) and playing with it some, I now want an Elixir job. (I'm skilled in Ruby, and Ruby already back in the day wanted me to get a Ruby job... which I finally did)
One nice thing about Elixir for Lisp lovers is that you get "real" macros (full AST access within a quoting context) AND actual syntax. I am not sure there are any other languages which feature that combination currently.
You get full AST access in Erlang as well. I don't know how it is in Elixir, but in Erlang you don't want to touch that feature with a ten foot pole. It's hairy and incredibly annoying to work with. I'm having a hard time to imagine how it couldn't be without homoiconicity and with that "actual syntax". I've been reading about macros in Scala recently, and they seem to suffer from the same problem - they just don't fit well with the rest of the language.
> but in Erlang you don't want to touch that feature with a ten foot pole. It's hairy and incredibly annoying to work with
It's pretty much the exact opposite in Elixir. The only thing you really have to wrap your head around is the switch between the quoting context and the unquoting context... which is pretty much no different from understanding macros period. The definition syntax looks just like a method definition, except it's "defmacro" instead of "def", and the macro body receives an AST instead of (or in addition to) your usual arguments. But I'm probably not doing it justice...
Yes, I was riffing on Lisp not really having a "syntax."
The fact that Erlang's syntax is (disclaimer: subjective) awful just adds further grist to the Elixir mill, assuming you think Elixir's syntax is (disclaimer: subjective) sweet.
Which a lot of people seem to be coming to the same conclusion on.
In Elixir, I have the full power of Lisp-level macros, in a functional immutable pattern-matching language, with a ridiculous level of concurrency (you can spawn a million or so processes in a second on your typical laptop), hot software upgradability (https://www.youtube.com/watch?v=96UzSHyp0F8 for a cool demo), access to the entire repository of already-existing battle-tested Erlang code...
... AND it's readable. :P
Lisp with its powerful macro facility has had literally dozens of years to find acceptance and still struggles (argumentum ad populum notwithstanding, a userbase with critical mass brings a plethora of other advantages). Ruby found enough of a niche that I think there is something to be said for Ruby's style of syntax. Elixir gives you both, and then some.
I was talking about Erlang and Elixir, not Lisp and Elixir. I don't need an introduction to Erlang/OTP, as I've used it for quite a while. You sound a lot like you're in the hyper-enthusiastic newbie phase, though.
I... guess I am. Is that OK? :) I like Erlang too... but I was one of the folks for whom the syntax turned me off originally. I can't explain why, especially if you're one of those developers (bless their pure-engineering hearts) who thinks syntax is irrelevant once you grok it. The best I can explain it is that some brains interpret computer code as a very abstract form of writing and some don't (or don't need to), and I may be one of the former, and that causes some syntaxes to "feel better" or "feel worse". It's... not very rational, sigh.
Why?