Building a startup using Crystal and Lucky

For the past several months, I’ve been building a startup web business using the Crystal language and the Lucky web framework. This is a conscious decision to use bleeding-edge technology, thus all of the issues I’ve run into are my fault. The Crystal language is approaching its 1.0 release, and the Lucky framework has not yet announced any intention to make a 1.0 release yet.

The potential cost of my choices is high – my start-up will be soliciting millions of dollars of capitalization, starting this week. But I don’t believe my technical choices will be a significant problem for the business, other than requiring that I hire experienced Crystal programmers. They are available, in the US, Brazil, and elsewhere, but not as common as programmers for other languages so far.

The impetus for this choice was the power of the Crystal language and the potential of the Lucky framework built upon it to result in a more correct implementation. Crystal is a compiled, globally-optimized language with Ruby-like syntax and type safety. It provides type safety without type verbosity, inferring types rather than requiring declaration much of the time, and propagating type information throughout the program. It also has an extremely powerful macro language which has access to abstract syntax tree nodes within the compiler. This allows me, for example, to piece apart interpolated strings for translation, at compile time, without a significant run-time overhead. I don’t know of any other compiled computer language that would allow me to do this in a macro, using a documented API, rather than requiring me to write an external source-code processor.

The intent of Crystal and Lucky are that the programmer would have the syntactic grace and ease of writing in Ruby on Rails, with greater correctness, fewer run-time errors, and compiled-language speed. To an extent, the combination achieves this. I expected that there would be pain, due to the youth of the language and platform, and there has been.

Crystal and Lucky are not, in my opinion, ready for the inexperienced programmer. With over 40 years of programming experience, I have still faced challenges.

The power of Crystal’s macro language means that it is used extensively in packages as powerful as the Lucky web platform. Unfortunately, this means that your programming errors are reported where they occur somewhere in a macro expansion, rather than where you have made them – as you could expect were you calling into functions and methods rather than macros. The result is that error messages resulting from my use of Lucky are often simply indecipherable, yielding neither the location of their origin or, sometimes, even any information about the erroneous statement rather than some macro transformation of that statement. Since the macro system is a code transformation machine, its arguments are not naturally as tightly typed as the rest of the Crystal language. Achieving good error reports for Lucky may require manually-added code to more tightly check the arguments to every macro. Fortunately, the macro mechanism does provide the framework to do such checking, AST nodes yield type information and the file name and line number of where they originate. I don’t know if there is anything that the compiler developers can do to improve error messages regarding macro expansions.

For now, I have developed craft knowledge that helps me debug problems in macro expansions, but this has been a relatively steep learning curve.

Lucky also has some substantial bugs at this time, as would be expected for such a large and powerful system at this point in its development. About the worst that I’ve encountered is that the before and after functions used to manipulate data model properties and validate their correctness don’t happen in the right order, and can’t be put in the right order using the existing API, causing some errors to be set on the model before the programmer has a chance to correct them. For now, this requires that model code explicitly clear some errors.

There are a few problems in the existing Crystal language implementation, as well. The largest one that I have faced is that the union type system, which allows one to declare a type as a collection of other types, does not work as expected under the hood. My program has two models: Manufacturer and Product. A union type of Manufacturer | Product is changed, internal to the compiler, to their common subclass. Then, the compiler sometimes does type checking for the common subclass, rather than just Manufacturer and Product as I intended. For now, I’ve created a  common subclass (closer to my models than the one the compiler would have had to pick) and declare that rather than a union type.

Another problem is that support for run-time debugging is incomplete at this time, and gdb has access to source code lines and the call stack, but can not examine some (or all) arguments and variables.

If you are willing to put up with these challenges, the Crystal language and the Lucky web platform offer more power than other computer languages I have experienced, more initial correctness, and the resulting code is fast. If you are in a hurry, or don’t have the chops to figure out cryptic errors, it’s not time for you to use Crystal and Lucky yet.