Hey there :)

Hey, Arjuna!

> Can you maybe give a small tutorial on how to use the new compiler? (especially compiling to javascript)?

When it's ready for public consumption, I can write such a tutorial.

> Is it in a state, that it could be used?

It is almost there.
I've been focusing on just tying-up loose ends and finishing the myriad strands of work I foolishly opened for myself in all the time I've been working on v0.6.

My goal is to have a release-quality version of the new pure-Lux rewritten JVM compiler by the end of June (so, in about a month).
July will then be dedicated to finishing the JS compiler.

The subsequent months will be dedicated to the other non-JS compilers (i.e. Lua, Python, Ruby, etc.).

I'm not sure that I'll be able to deliver all of them, as I've run into some issues. But I plan to time-box the work on each of them to 2 weeks.
If I can't finish one in that time-span, I'll just leave it for a later release and move on.

Small Status Update:

I think I've got the new JVM compiler in a release-ready state.

The next few days will be dedicated to packaging it, publishing it and making sure the Leiningen plugin can work with the new compiler.

July will then be dedicated to finishing the JS compiler.

A small status update:

I think I'm close to being done with work on the JS compiler.
I'll still need a few days to fix a few remaining bugs and iron out a few kinks, but I think the remaining days in the month should be enough to do everything I need.

I'm also very happy with the results I'm getting.
You see, the main problem with the JS compiler was that it was extremely slow. The reason is because I needed to be able to run the JS code I generated when doing macro-expansion, in order for macros to be fully integrated into the language (unlike ClojureScript, where macros have to be defined in JVM Clojure and then imported in a special way because ClojureScript doesn't have the means to run the JS it generates).

The way I did it was by using the Nashorn JS interpreter on the JVM.
However, as it turns out, Nashorn is a very slow JS interpreter, and, as a consequence, running the Lux JS compiler would be an extremely long and tedious process that could even run up to an hour or more depending on the size of the code-base you were compiling.

My strategy to get around that problem became to use the JS compiler to compiler its own source code in the hopes that I could then run the resulting JS program on Node and use the normal JavaScript "eval" function instead of the Nashorn interpreter to do all the necessary macro-expansion.

I didn't know if it would be fast enough, but now I believe it might.

So, I've been successful in getting the JS compiler to compile itself and I've gotten some positive initial results.

I compiled the same program with both the JVM compiler, the JS compiler using Node and the JS compiler using Nashorn and I've gotten the following measurements (on a clean build with no caching involved):

JVM compiler:                            27.547s
JS compiler (running on Node):         7m17.641s
JS compiler (running on Nashorn/JVM): 22m04.402s

Of course, that is still an order of magnitude of difference between the performance of the JVM compiler and the compiler running on Node, but Node is still 3 times faster than Nashorn, and with such large build times, such a difference is very significant.

Mind you, once caching is involved, compilation times would drop substantially since much work would be avoided.

---------------------------------------------------------------

I'm very happy with what I'm seeing. It's still not perfect, since compilation times are still large for the JS compiler, but it's at least within a "reasonable" range, at least when compared to how things used to be.

For those who don't know, the main reason why I went on this quixotic quest to re-write the Lux compiler in Lux itself was because, years ago, I faced the same problem with my initial implementation of the JS compiler and I realized that the only sustainable way to solve the problem would be to re-write the Lux compiler in Lux and compile it to the different host platforms I needed in order to get the best performance possible.

That was, I admit, overkill for JS, since I could have gotten similar results by just adjusting the (Clojure-based) Lux compiler to be compilable with ClojureScript and then running that with Node.
But, even back then, I already knew the next steps would be to get Lux running on Python, Ruby, Lua, etc. And I wouldn't have been able to pull the same trick in those scenarios.
So, I ultimately decided to bite the bullet and engage in this long and costly re-write in the hopes that it would serve as a superior foundation for my future work on porting Lux to new host platforms.

After years of work, it pleases me to learn that I made the right call, and that I'm finally reaping the fruits of my labor.