six to seven months of coastal living: coastML status report for September 2022

Today marks seven-ish months of coastal living; let’s checkout the running burndown chart:

• ✓ assignments

• ✓ functions

• ✓ case forms

  • ✓ thread assignments into case forms (e.g. a = case x …​ becomes case x | …​ { a = …​ })

  • ✓ case forms within other forms, such as function calls

• ✓ type forms

• ✓ Python generation

• ✓ decomposition in case forms

  • ✓ including pattern matching

  • ❏ array pattern matching

  • ❏ string pattern matching (this could get interesting…​)

• ✓ accessors (both variables and lambdas) for type

• ❏ gn forms

• ❏ fc forms

  • Still so on the fence about these…​

• ❏ mod and sig forms

• ❏ an analog to SML’s op, using backticks

• ❏ basis library (stubbed out, need to actually compile)

  • ✓ basic basis stubs: some members of array-*, string-* have been stubbed out

    • ✓ array-length

    • ✓ array-get

    • ✓ array-set!

    • ✓ array-make

    • ✓ array-init

    • ✓ array-make-matrix

    • ✓ array-append

    • ✓ array-append!

    • ❏ array-concat

    • ❏ array-concat!

    • ✓ array-sub

    • ✓ array-copy

    • ❏ array-fill!

    • ❏ array-blit!

    • ❏ array→list

    • ❏ list→array

    • ✓ array-iter

    • ✓ array-map

    • ✓ array-iter-index

    • ✓ array-map-index

    • ❏ array-foldl

    • ❏ array-foldr

    • ❏ array-sort

    • ✓ array-sort!

    • ❏ array-stable-sort

    • ❏ array-fast-sort

    • ✓ string-length

    • ✓ string-get

    • ❏ string-make

    • ❏ string-init

    • ✓ string-split

    • ✓ string-append

    • ❏ string-join

    • ❏ string-contains

    • ❏ string-concat

    • ❏ string-copy

    • ❏ string→array

    • ✓ string-iter

    • ✓ string-map

    • ✓ string-iter-index

    • ❏ string-map-index

    • ❏ string-foldl

    • ❏ string-foldr

    • ✓ string-sort

    • ❏ compare

    • ❏ char-code

    • ❏ char-chr

    • ❏ char-escaped

    • ❏ char-lowercase

    • ❏ char-uppercase

    • ❏ char-compare

    • More of the OS interaction stuff out of the Pervasives API

  • ❏ basis library modules (modular forms, e.g. array-get becomes Array::get)

• ❏ support for the is form

• ❏ code generation

  • ❏ C

  • ❏ C++

  • ❏ Go

  • ✓ Python

  • ✓ JavaScript

• ❏ type inference

• ❏ refinement types

• ❏ a compiler

  • ❏ lambda lifter

  • ❏ closure conversion

  • ❏ pragma to control certain aspects of compilation

  • ❏ checking of accessors prior to compilation

  • ❏ value restriction for ref types

  • ✓ checking that functions exist prior to compilation

  • ✓ checking that case forms use correct constructors

• bugs fixed

  • ✓ nested arrays [[[1] [2]] [[3] [4]]]

  • ✓ missing / character in identifier/operator character sets

  • ✓ floating point numbers with two integral digits (e.g. 0.12 and 1.12 worked but not 10.12)

    • this was noticed when attempting to write some software to generate Julian dates

    • Was the calculation sheet I was using

  • ✓ return logic

    • ❏ there’s more work to be done here at the top level…​

  • ✓ indentation of blocks

  • ✓ missing bool type

  • ✓ parsing type definitions that have other complex types in them

  • ✓ munging idents (ex: julian-date becomes julian_date)

  • ✓ stripping comments prior to output

    • really need to handle these so that we can output them into the compiled language, but for now this suffices

  • ✓ return prepended to forms that are rewritten to for loops

  • ✓ fix spacing between class forms in Python output

  • ✓ fixed bugs with parsing & displaying characters

  • ❏ case forms at the start of a (call) form are assumed to be functions instead of evaluated for a lambda

    • (case …​ | 10 { " " } | _ { "nope" } esac …​) is assumed to be a lambda already…​ whoops

  • ✓ case forms in a function call in assignment clobbers the assignment

    • ex: a = string-split-char ':' case (OS::getenv "PATH") | (Result.Some s) { s } | (Result.None) { "" } esac;

    • produces a = res1 = …​ which is obviously in correct

  • ✓ case forms that destructure type forms don’t indent correctly and don’t nest if-elif-else correctly

  • ✓ case forms that bind a function call do not indent the initial if correctly

  • ✓ destructuring binds didn’t actually apply a .format correctly

  • ✓ indentation more generally is handled better

So we’ve added quite a bit of code these last two months…​ some personal things came up that delayed writing August’s blog post, but I’ve continued hacking away at things as needed. Little ticky-tacky bugs aside, there’s two new major modules within the system: a Compiler and CarpetJavaScript.

The first lifts many of the AST → AST forms that were previously within the CarpetPython module. These transforms are useful to multiple code generators, and honestly should be above that sort of level. The compiler now checks for function existence, the validity of case arms, and does various transforms on code, such as lifting lambdas. It’s also just about at the point where it can be used to check the body of fn forms as well; the last step there is just to start collecting a spaghetti stack of environments.

The second is just a straight-forward coastML → JavaScript code generator. There are still some forms I need to flesh out, such as various pervasives, but the general shape is coming together. It’s been interesting to consider another language, since that generally forces you into a good structure for where other AST-level transforms should live. The next step will be adding types, so that languages like Golang become easier, and so that we can annotate JavaScript & Python with typing information as well.

Lastly, I exposed the Compiler as a command on the command line:

% ./coastml
usage: carpet.py [command] [file]
commands:
load - load a file, and dump the resulting coastML
compile - like load, but run the compiler too
python - dump python from a coastML file, without the compiler
cpython - dump python from a coastML file, with the compiler
javascript - dump javascript from a coastML file
note, the last two will be merged at some point

This next month, I want to focus on the Compiler as well as the basis library, so that we can get some really great code together, and start generating code from coastML that we deliver to other teams.