How to Plan a Luau: Augmenting Lua’s Syntax With Types

For a very long time, Lua was the language of alternative for Roblox. As we grew, so far too did the demand from customers for superior tooling support as very well as a extra performant VM. To response this, we commenced the initiative to rebuild our Lua stack named “Luau” (pronounced /lu-wow/), with the aim of encompassing the features programmers hope a modern day language to present – which contains a form checker, a new linter framework, and a speedier interpreter, just to name a couple of.

To make all of that feasible, we experienced to rewrite most of our stack from scratch. The issue is that the Lua five.1 parser is tightly coupled with bytecode era, and which is insufficient for our requires. We want to be capable to traverse the AST for further examination, so we require a parser to create that syntax tree. From there, we’re free to execute any functions we desire to do on that AST.

As luck would have it, there was an existing Lua parser lying about in Studio only applied for essential linting pass. That made it incredibly straightforward for us to adopt that parser and prolong it to recognize Luau-specific syntax, which thus minimized the feasible danger of altering the resulting parse in some delicate way. A crucial element mainly because one of our sacred values at Roblox is backward compatibility. We have tens of millions of strains of Lua code presently written and we are committed to making certain that they proceed to operate forever.

So with these elements in thoughts, the necessities are obvious. We need to have to:

  • remain very clear of grammar quirks that need backtracking
  • have an efficient parser
  • preserve forward-appropriate syntax
  • stay backward suitable with Lua

Appears basic, ideal?

How the sort inference engine motivated syntax selections

To begin, we require to fully grasp some context about how we arrived in this condition. We chose these syntaxes because they are now immediately acquainted to the vast majority of programmers, and are in truth business normal. You don’t have to understand anything at all new.

There are several locations exactly where Luau permits you to publish these types of kind annotations:

  • regional foo: string
  • function include(x: number, y: selection): range &#8230 conclusion
  • variety Foo = (amount, quantity) -> amount
  • regional foo = bar as string

Introducing syntax to annotate your bindings is really crucial for the variety inference engine to improved understand the meant typings. Lua is a really strong language that lets you to overload pretty much each and every operator in the language. With no some way to annotate what things are, we are not able to even confidently say that the expression x y is going to generate a quantity!

Type forged expression

A little something we actually like from TypeScript is what they call a form assertion. It is mainly a way to increase excess type info to a method for the checker to confirm. In TypeScript, the syntax is:

bar as string

Unfortunately, when we tried using this out, we ended up in for a lousy surprise: this breaks present code! One particular of our users’ video games experienced a functionality named as. Their scripts therefore bundled snippets like:

area x = y

as(w, z) &#8212 Anticipated ‘->’ when parsing perform kind, bought

We very likely could have created this get the job done, ended up it not for a person further complication: we wished our parser to perform with only a solitary token of lookahead. Performance is important to us, and section of producing a pretty remarkably performant parser is minimizing the sum of backtracking it has to do. It would not be economical for our parser to have to scan forward and backward arbitrarily much to figure out what an expression actually implies.

It also turns out that TypeScript can thank JavaScript’s automated semicolon insertion rule for generating this perform for free. When you write this snippet in TypeScript/JavaScript, it will insert semicolons on every single line, triggering it to be parsed as two different statements. Whereas if it have been on a solitary line, it is a syntax mistake at the as token in JavaScript, but a legitimate style assertion expression in TypeScript. Because Lua doesn’t do this, nor does it implement semicolons, it has to try to parse for every single longest possible statement even if they span throughout several strains.

enable x = y

as(w, z)

Luau’s initial form solid expression was not backward compatible even however it experienced the general performance we desired. Regrettably, this broke our promise of Luau currently being a superset of Lua five.1, so we just can’t do it devoid of some extra constraints this kind of as demanding parentheses in sure contexts!

Sort arguments in purpose calls

A further regrettable depth in Lua’s grammar prevents us from adding type arguments to purpose phone calls without having introducing an additional ambiguity:

return someFunction(c)

It could necessarily mean two different factors:

  • assess someFunction < A and B> c, and return the effects
  • call and return someFunction with two sort arguments A and B, and an argument of c

This ambiguity only occurs in the context of an expression list. It’s not definitely a large dilemma in TypeScript and C# because they both equally have the benefit of compiling in advance of time. Therefore, they can both of those afford to invest some cycles making an attempt to try to disambiguate this expression down to one of the two alternatives.

Though it seems that we could do the exact same matter, this sort of as applying heuristics throughout parsing or type examining, we really can’t. Lua has the skill to dynamically inject globals into any surroundings, and that can split this heuristic. We also flat out do not have that reward since we have to be in a position to crank out bytecode as quickly as doable for all purchasers to start off decoding.

Sort alias statement

Parsing this kind alias statement is not a breaking adjust since it’s presently invalid Lua syntax:

style Foo = quantity

What we do is uncomplicated. We parse a key expression which only finishes up parsing as far as just type, and then we make a decision what to do dependent on the parse end result of that expression:

  • If it’s a purpose connect with, cease striving to parse for far more of this expression-as-assertion.
  • Usually, if the future token is a comma or equivalent, parse an assignment assertion.

What is lacking earlier mentioned is pretty noticeable. It has no branch for which an identifier can be led by a different a person. All we have to do then is sample match on the expression:

  1. Is it an identifier?
  2. Is the title of that identifier equal to “type”?
  3. Is the following token any arbitrary identifier?

Voilà, you get backward-compatible syntax with a context-delicate search phrase.

kind Foo = amount &#8212 variety alias

style(x) &#8212 purpose simply call

kind = x = 1 &#8212 assignment

form.x = two &#8212 assignment

As a reward snippet, this even now parses in the exact same way as Lua since we were being not parsing from the context of a assertion:

nearby foo = kind

bar = 1

Classes acquired

The takeaways in this article, it appears, is that we’re likely to have to design the syntax for Luau to be ahead appropriate and with least context-delicate parse paths. It removes the necessity of second-guessing that involves the parser to backtrack and try a thing else from that point of failure. Not only does that give us the reward of acquiring a quickly parser to just chug alongside to the conclude of the resource code, but also it can give back again the AST without having needing other varieties of phases to disambiguate.

It also suggests that we will require to be mindful when adding new syntax in general, which is not automatically a bad put to be. A well-considered-out language needs its designers to acquire the very long look at.

Neither Roblox Corporation nor this website endorses or supports any firm or services. Also, no guarantees or claims are produced relating to the precision, trustworthiness or completeness of the data contained in this weblog.

This weblog post was at first printed on the Roblox Tech Blog site.

The write-up How to Program a Luau: Augmenting Lua’s Syntax With Sorts appeared initially on Roblox Website.

Leave a Comment