"CL went through a spike of interest in the mid-2000s; where did those people go . . ."
I don't know about others, but I was part of that spike and I went . . . far away from Common Lisp.
Learning Lisp absolutely blew my mind. I felt like every page of Peter Siebel's Practical Common Lisp was some new kind of glorious revelation. I was going to write everything in Common Lisp.
I did write some fairly serious things in CL, but in the end, it came down to what felt like a broken, barely documented, half-assed ecosystem for doing most of the things that ordinary working programmers want to do.
I realize that sounds harsh and unfair (though I'm hardly the first person to put it in those terms), but the problem is relative. It's not a total disaster as such, but if you're used to the ecosystems of languages like Java, Python, Ruby, PHP, Javascript -- almost anything else (including, in recent years, Haskell) -- you'll probably find yourself frustrated with CL. I did, anyway.
So what did I do? I abandoned CL and took my knowledge over to languages like JavaScript (cum Underscore) and Clojure.
The OP ends that quote above by asking "What can we learn from that?" I don't know. I don't know how a group of brilliant programmers working in one of the most brilliant languages of all time end up turning enthusiastic newbs into people who feel frustrated and disappointed. But I'll say it: It for damn sure ain't the language or the skills of the people who work with it.
Maybe give CL another try? Since the mid-2000's the ecosystem has definitely improved exponentially. (ala Quicklisp, Quickdocs, and related portable libraries)
Threading has a de-facto implementation as well now.
Also part of that group - Paul Graham's essays drove me to Lisp. I am very happy with what I've learned over ~5 years of near-exclusive CL usage. I agree - while CL-the-language is amazing, CL-the-ecosystem just doesn't have any momentum. It's spread too thin.
I think I'll be putting my chips on Julia from now on. It should be a near-ideal language for data science and AI research. I like that it's focused on one thing (numerical computations), so it should avoid the lack of focus that plagued most Lisps.
I also got into lisp (Lush lisp) because of Graham, and still consider it a glimpse of the promised land. The thing is, if you haven't screwed around in an APL language, you don't realize what you're missing. For numerics, the array part of the language is much more important than the rest of the thing. When I read the J sources, I recognized what Yann and Leon were trying to do in Lush, and the combination with a decent lisp repl was gold.
Now all the array parts are in Torch7/Lua, along with threading, the bleeding edge of Deep Learning research, better C++ support and CUDA. Lua ain't pretty like lisp, but these are practical men, and it has everything one needs.
Julia: no thanks.
I gave Lush serious consideration, but at this point, it already feels like Julia has more momentum (might be all perception, though). I actually haven't tried Julia yet, what problem do you see with it?
The thing is, if you haven't screwed around in an APL language, you don't realize what you're missing
Please tell us more! Give examples.
The -idx- system in Lush is a pretty good example. Like I said above, this has now made its way into Torch7/Lua.
Anyone who works with arrays and matrix math for a living needs to look at an array based language to see how it is done properly. J is the one I've been futzing with. It's a rough learning curve, as the language looks like line noise to the uninitiated. The upside is it teaches you a lot about how the array parts of other languages should work; vectorizing R or Matlab certainly becomes more obvious. As a bonus, it can be amazingly fast: single threated SVD in native J matches lapack speeds on big problems. Stuff like J encapsulates decades of insight into programming, particularly with arrays, that is mostly forgotten in modern programming languages. Learning an APL is the same sort of experience as learning Lisp: "why doesn't everything work this way?" Definitely a productivity boost if you can do a project in J; you can replace pages of Java with a line of J. That line might take a while to write if you haven't been J-ing for a decade, but it's still a productivity boost. As a bonus, J comes with a decent columnar database built in. It isn't concurrent yet, but it's pretty good for medium sized problems, and unlike something like Redshift, you can do some fancy math on the data directly using J verbs.
I don't want to crap all over Julia, but it's pretty obviously a student project. I'm sure its compiler is better than that of Lush, but Lush was designed for people needing to solve real problems. Julia definitely has more momentum, but frankly, I'll stick with R/Torch7/J until Julia has proved useful for something.
I was amazed by lisp, also encouraged by Paul Graham's work. When I came to questions of packaging for use by end-users, I didn't like what I saw. Explorations into my options led me to Haskell. Since then I've started asking more interesting questions and haven't looked back.
OP here. Thanks for taking the time to read my post.
I am quite bad about maintaining conversations on HN, so I would appreciate it if you would email me if you have any thoughts on those questions -- why did you leave CL, what could have been done to improve the community or ecosystem?
I am not promising to fix these things, but I would like to give it another try.
The reason I have stuck with Common Lisp, rather than bailed to Clojure or something else, is that the language is so fundamentally good -- Mature, robust, standarized, performant, compiled, gradually-typed -- that it is worth sticking with it, it is worth working to expand its ecosystem.
Regarding the ecosystem, Common Lisp has improved a lot over the recent years. We have Clack (Equiv. of Python's WSGI/Ruby's Rack), a bunch of web frameworks, an ORM, a small set of web-related tools built around those. It's certainly nowhere near the size of the ecosystems of Ruby and Python, but I think it's acceptable given the size of the community.
I do, however, think there has to be a movement like Fare's "Lisp library consolidation"[0] to, if not expand, improve the existing libraries: Build Github Project Pages, Sphinx documentation, a better-looking wiki. Just porting old READMEs to Markdown would be an improvement.
the language is so fundamentally good -- Mature, robust, standarized, performant, compiled, gradually-typed -- that it is worth sticking with it, it is worth working to expand its ecosystem
This is why I created https://wukix.com/mocl. CL has a ton of merits. It's unfortunate that people pass over CL because it's "old" or whatever (a worthless criterion if there ever was one). The language is great, and with a little house cleaning, CL can seriously kick ass.
Wow, I haven't seen the licensing model to use a language in a while. Many researchers are migrating away from MATLAB for that reason, and new projects are often not initiated in MATLAB unless it incorporates legacy code. How's this business model holding up?
Nitpick: It's not licensing for a language, it's licensing for a specific tool. Common Lisp itself is not proprietary (having a standard) and code can be portable across implementations.
To answer your question, I would say that the business model (of charging for licenses) is fine in principle, but developer tools is a very difficult business to be in.
Have you considered a student price, I like the idea of having CL on my phone (and eventually tablet) but the price is a bit step for me (and presumably other students) especially to get source code access.
I'm emboldened by comments like these, and I don't mean to be harsh or dismissive. Maybe time to give it another try. But I'd like some acknowledgement of the problems, which I think is what the OP was after.
I do acknowledge (Or try to) that Common Lisp has plenty of problems. But aside from some design flaws, I think the majority of these problems are social, which in a way is a good thing: Fundamental characteristics of a language cannot be changed, but presentation and impression can and should be improved. What is needed is the right people making the right appeals to the community.
I don't know about others, but I was part of that spike and I went . . . far away from Common Lisp.
Learning Lisp absolutely blew my mind. I felt like every page of Peter Siebel's Practical Common Lisp was some new kind of glorious revelation. I was going to write everything in Common Lisp.
I did write some fairly serious things in CL, but in the end, it came down to what felt like a broken, barely documented, half-assed ecosystem for doing most of the things that ordinary working programmers want to do.
I realize that sounds harsh and unfair (though I'm hardly the first person to put it in those terms), but the problem is relative. It's not a total disaster as such, but if you're used to the ecosystems of languages like Java, Python, Ruby, PHP, Javascript -- almost anything else (including, in recent years, Haskell) -- you'll probably find yourself frustrated with CL. I did, anyway.
So what did I do? I abandoned CL and took my knowledge over to languages like JavaScript (cum Underscore) and Clojure.
The OP ends that quote above by asking "What can we learn from that?" I don't know. I don't know how a group of brilliant programmers working in one of the most brilliant languages of all time end up turning enthusiastic newbs into people who feel frustrated and disappointed. But I'll say it: It for damn sure ain't the language or the skills of the people who work with it.