This post should address a few concerns/questions some of you have regarding this new project.
Q: Why a new compiler? Isn’t rock fine?
No. rock’s not fine. It’s awful. If you disagree, it’s because we have done a good job at patching it up for most bugs encountered.
Don’t get me wrong: rock is still better than many, many compilers out there. For some reason, it seems that sucky languages have good compilers and good languages have sucky compilers (example: Objective-C/LLVM vs Scala/scalac)
Q: What are rock’s visible drawbacks?
- It doesn’t scale well. It’s faster than most for small projects (esp. if you have a custom, small sdk), but for large projects (ie. itself) it becomes tiresome
- Related to above: error reporting is of variable quality. In complex situations with hundreds of modules, sometimes irrelevant errors are reported first.
- Partial recompilation is not 100% reliable. Sometimes C compilation errors disappear when cleaning cache files.
- Everyday new ways are found to make the compile
These problems are well-known, and they are the result of bad design decisions.
Q: What’s wrong with rock’s internal design?
- The whole resolving process is flawed. Looping is less than optimal, and many unnecessary tree walks happen. Most resolve() methods have a lot of noise and sometimes it’s not exactly clear what’s the best thing to do when were waiting on something to resolve.
- The AST confuses ooc and C. Some AST nodes (such as StructLiteral) don’t even have a direct mapping to ooc code, but they’re used regardless because it would be too hard to do everything in the backend without an additional C AST.
Those are the most important points. Other areas that could be improved as well (parallelizing, using a proper command line arguments lib, etc.)
Q: Why not just fix rock then?
First off, because these changes are too big. Changing the whole way resolving is done means having to rewrite 50% of rock without being able to iteratively test anything.
There’s more than that: originally, rock was written so that j/ooc could compile it. In other words, with many limitations. By writing the same functionality again from scratch, I am able to use higher-level ooc idioms that are well-supported by current compilers (generics, closures with ACS, properties, improved match)
This often means: less code, that is more readable, with less bugs, and at least te same level of functionality.
Q: How is oc’s design better?
This could probably fill a book, so I’m not gonna go into too much details, but here are a few nice features:
- Parallel parsing of files - on my dual core, the parsing of 200 files is up to 1.8x faster in oc than in rock (both use nagaqueen).
- Resolving using coroutines rather than dumb loops: this allows completely out-of-order resolution, only one tree walking per module
- Better error reporting: because coroutines are used rather than looping “in hope of something to happen”, errors are reported much sooner. When running rock and ooc on the same 3000-lines file, it takes rock ~3.161s and oc ~0.178s to report the undefined symbol used at the end. This is almost a 18x speedup.
- There’s a pure ooc AST, allowing for inference, error checking, optimization, etc. and the C backend uses a C AST (good for partial recompilation). This also means alternative backends are easier to implement because the ooc AST they’ll be dealing with won’t be full of C-isms.
- Parallel generation and compilation of C files, as soon as they are resolved. Which means even if the resolving process is single-thread, some modules are very quickly resolved and then compiled while other modules are still resolving.
And there are many other internal details that make it a lot more pleasant to work with than rock. Bugfixes in oc are usually improving the design. Bugfixes in rock are usually patches because changing the design would be too costly.
Q: Sounds great! Can I start using oc now?
No you can’t. As every good vaporware, it’s wicked cool but still far from supporting everything rock does.
It’s useless to make a list of what it supports, because the point is that most of the architecture is there and it’s not hard to add support for language features.
You can, however, contribute to its development. It’s on github, you can fork and ask what you can do to help. Alternative backends (targeting other high-level languages) would be a good easy first contribution.
Q: Last question: What about us poor devils/rock users?
I’m still a rock user as well - since oc obviously isn’t self-hosting yet. It means rock gets the usual love, bugs get fixed, features get added - no worries.
That’s it for today! Thanks for reading, hope most of your questions have been answered, if not there’ll be other posts, hit me up on Twitter (@nddrylliog)