Raku Books / Perl 6 at a Glance / Regexes and Grammars

AST and attributes

Now, we are ready to simplify the grammar again after we split the assignment and printout rules into two alternatives each. The difficulty was that without the split, it was not possible to understand which branch had been triggered. You either needed to read the value from the value token or get the name of the variable from the identifier token and look it up in the variable storage.

Perl 6’s grammars offer a great mechanism that is common in language parsing theory, the abstract syntax tree, shortened as AST.

First of all, update the rules and remove the alternatives from some of them. The only rule containing two branches is the expression rule.

rule assignment {
    <identifier> '=' <expression>
}
rule printout {
    'print' <expression>
}
rule expression {
    | <identifier>
    | <value>
}

CodeBlockPlaceholder2raku-static method identifier($/) { $/.make(~$0); } method value($/) { $/.make(+$0); } CodeBlockPlaceholder3

Move one step higher, where we build the value of the expression. It can be either a variable value or an integer.

As the expression rule has two alternatives, the first task will be to understand which one matches. For that, check the presence of the corresponding fields in the $/ object.

(If you use the recommended variable name $/ in the signature of the action method, you may access its fields differently. The full syntax is $/<identifier>, but there is an alternative version $<identifier>.)

The two branches of the expression method behave differently. For a number, it extracts the value directly from the captured substring. For a variable, it gets the value from the %var hash. In both cases, the result is stored in the AST using the make method.

method expression($/) {
    if $<identifier> {
        $/.make(%var{$<identifier>});
    }
    else {
        $/.make(+$<value>);
    }
}

CodeBlockPlaceholder5raku-static $/.make(%var{$} // 0); CodeBlockPlaceholder6

Now, the expression will have a value attributed to it, but the source of the value is not known anymore. It can be a variable value or a constant from the file. This makes the assignment and printout actions simpler:

method printout($/) {
    say $<expression>.ast;
}

CodeBlockPlaceholder8raku-static method assignment($/) { %var{$} = $.made; } CodeBlockPlaceholder9

The method gets the $/ object and uses the values of its identifier and expression elements. The first one is converted to the string and becomes the key of the %var hash. From the second one, we get the value by fetching the made attribute.

Finally, let us stop using the global variable storage and move the hash into the action class (we don’t need it in the grammar itself). It thus will be declared as has %!var; and used as a private key variable in the body of the actions: %!var{...}.

After this change, it is important to create an instance of the actions class before paring it with a grammar:

Lang.parsefile(
    'test.lang',
    :actions(LangActions.new())
);

CodeBlockPlaceholder11raku-static grammar Lang { rule TOP { ^ $ } rule statements { + %% ‘;’ } rule statement { | | } rule assignment { ‘=’ } rule printout { ‘print’ } rule expression { | | } token identifier { (<:alpha>+) } token value { () } } CodeBlockPlaceholder12

Course navigation

← Actions   |   Calculator →