Looking through the scan.l

Recently I was browsing the scan.l source code for changes since PostgreSQL 9.0 version. I always do so for new minor and major releases because I use PostgreSQL grammar in my project.

Well if you’ll look closer you may find that flex is switched to 8bit mode:

%option 8bit

But then you may find such declarations:

dolq_start [A-Za-z\200-\377_]
dolq_cont [A-Za-z\200-\377_0-9]
ident_start [A-Za-z\200-\377_]
ident_cont [A-Za-z\200-\377_0-9\$]

When I first saw these lines they blew my mind. Because from the school times I was sure that 8bit has maximum value of 255. But thanks God, there is a lot of information about flex all over internet. And suddenly, it turned out that flex uses octal notation by default. This means
the character with octal value 123

There is also a way to use hexadecimal notation:
the character with hexadecimal value 2a

So the next time you’ll see such declarations don’t be confused. So conceived! 🙂


Metamorphosis of gram.y

I often work with YACC (Bison) grammars, but still didn’t find comfortable editor for them. So I’m still using Notepad for this purpose. However, some time ago I found Perl script called linkify.pl as far as I remember, which produces HTML page based on the grammar and the list of non-terminal symbols. Edward Smirnov, colleague of mine, take this script and bring it to the completely new level.

What are the benefits of using such generated HTML page?

  • You can go exactly to whatever line number just adding #_<LINE>.

For example, if path to HTML grammar file is http://foo.com/MyGrammar.html, then http://foo.com/MyGrammar.html#_119 will bring you to the line number 119.

  • You can go exactly to the place where nonterminal symbol is defined.

All nonterminal symbols on this page are the links to theirs definitions indeed. Thus you may easily go down from the start symbol to the tokens using these links. One more benefit is that you may use “Back” browser button for one level up return.

  • Last but not least. Syntax highlighting: comments are grey, nonterminal symbols are close to red, the code and terminal symbols are navy.

Here you can find PostgreSQL LALR(1) grammar gram.y processed by this script.