Projects Jams Discord News
Resources
Unwind Fishbowls Forums
About
Manifesto Our values About
Log In

Mass Compiler August 2021 Update

Dmitriy Kubyshkin September 1, 2021

It has been a couple of months since the last update. As usual, there are a lot of internal changes to the compiler, but there are some interesting externally visible changes as well. I spent some time tightening things that seem stable enough, such as tokenization, encoding, and functions.

For the tokenization, I have switched from a hand-written tokenizer to re2c. It provided a nice boost to the compilation speed almost for free. I have also implemented caching for common number literals (0-9) and identifier names for another small boost.

With encoding, once I decided to ignore possible machine code size improvements from a relaxation step, I have realized that I could eagerly turn assembly instructions into bytes which save a whole lot of memory and some compilation speed. You can see me working on it in a YouTube video.

Read more

🐧 Better Linux Support - June 2021 Mass Progress Update

Dmitriy Kubyshkin June 30, 2021

The major item this month for me was the feature between Windows and Linux JIT implementations. Getting Linux to run some bytes I passed to it was the easy part and took like 30 min to do . The hard part was getting System V ABI to a workable state. One reason is the fundamental complexity of the algorithm used to determine how arguments are packed to register, but it is also made worse by the fact that the technical documentation leaves much to be desired.

Also on the Linux front there is now support for referencing dynamic libraries in the JIT:

write :: fn(descriptor : s32, buffer : &u8, size : u64)
  -> (s64) external("libc.so.6", "write")
STDOUT_FILENO :: 1
main :: fn() -> () {
  hello :: "Hello, world!\n"
  write(STDOUT_FILENO, hello.bytes, hello.length)
  import("std/process").exit(0)
}

Besides a ton of refactoring here are some more highlights:

[ul] [li]Imports are now lazily resolved fo

Read more

Basic Linux / Mac x64 Support

Dmitriy Kubyshkin May 29, 2021

The majority of time since the last update was taken by abstracting away calling convention code to prepare for System V ABI used by Linux and Mac. With that done, adding Linux JIT setup was a breeze. There is still lots to be done for proper System V ABI compatibility as well as dlopen integration, but it is pretty exciting to have a custom backend that works across multiple platforms.

Besides that there were also many smaller changes: [ul]

  • Fixed encoding of 8-bit SI, DI, BP, SP operands
  • Auto-generation of Mass bindings for compiler C structs
  • Fixed overload resolution for compile-time calls
  • Fixed on-the-stack argument passing
  • Fixed incorrectly generated source ranges for macros
  • Improved error checking for trying to access runtime results at compile time
  • [li][url=https://www.youtube.com/watch?v=azKeL5zCU2I]Add a way to fail comp

    Read more

    Robustness Month

    Dmitriy Kubyshkin April 28, 2021

    The majority of the development up until about a month ago was about figuring out the basic features of the compiler. As you start to combine them together new and unexpected cases need to be solved. Besides the common language issues such as signed / unsigned integer handling Mass has a lot of its own problems to solve. The most tricky one is handling the boundary between compile-time execution and runtime code. So far there is no production language with the same power as what I aim for so there is no real way to know what is the correct way to do it and this is what I spend a lot of time on.

    Besides the robustness are the things that have been added to the compiler in the last month:

    [ul]

  • basic embedded debugger REPL
  • uniform and typed errors in the compiler
  • conditional constant definition via "using" and "if" expression
  • compile-time function definitions
  • static_assert() implemented using meta-programming
  • [li]function default ar

    Read more

    Compilation Without AST

    Dmitriy Kubyshkin March 26, 2021

    When I started the work on the Mass language I wanted to challenge some of the established practices of writing compilers with one of them being the presence of abstract syntax tree (AST) as intermediary representation for the program. This month I have admitted defeat and introduced something like an AST. In this post, I want to share some of the details and the reasoning for the switch.

    There are two main ways the compilers are written today, both involving an AST. The first, more traditional approach is to have a pipeline with at least the following steps:

    tokenization -> parsing (AST generation) -> type checking -> assembly generation-> executable

    Some languages introduce more steps including intermediary byte code generation, object files and linking. It is also usually the case that type checking and assembly generations are themselves do multiple traversals over the syntax tree. GCC, Clang, Rust, Go and many other compilers use this setup.

    Read more

    Benchmarking Compile Time Execution

    Dmitriy Kubyshkin February 28, 2021

    As I'm closing on 1 year of working on Mass I was quite curious to see if my approach to compile-time execution via JIT makes sense and how it compares with other languages that are capable of compile time execution, namely C++ and Zig. I do not have access to Jai, so can not measure that.

    For now I just did two test program. The first one runs a counter 1 000 000 times at compile time and stores a result into a compile-time constant. The counter is then printed at runtime to verify the output value.

    The goal of the second test program is to constant fold 1 000 definitions computing the sum of integer 1. We then sum all the definitions at compile time as well to make sure that the compilers do not skip the computation for unreferenced constants. Because of the large amount of source code (2mb), this test not only measures the speed of constant folding itself but also parsing.

    To measure just the compiled time execution and not the rest of the compiler, the code is first comp

    Read more

    Progress Report (January 2020)

    Dmitriy Kubyshkin January 28, 2021

    I have started developing the mass compiler in April 2020 with the majority of early work captured in a series of YouTube videos. Due to the limitation of doing the work on video the progress in the first half-year or so was quite slow. Still, by September 2020 the language got sufficiently powerful to run FizzBuzz both in JIT mode and by compiling it to a Windows executable.

    Four months later FizzBuzz remains the most complex program in the test suit, however, that does not mean there was no progress. My main focus throughout this time has been mainly on two things: robustness and meta-programming capabilities. There is not much to be said about robustness work - it is very important but rather uninteresting. Meta-programming on the other hand is very important and core to this language.

    There are two main parts for the meta-programming: macros and compile-time execution. It may seem that one doe

    Read more