Parsing structured text

Parsing in general is of interest. One reason for delay in starting this project has been the postponing of the choice between using Python’s built-in parsing modules, or more general parsing components. Although I have now decided to start with Python’s built in parser I remain open to use of other parsers as well. Not much point in having a “structured text editor” which cannot handle structured text such as other langages, marked-up texts, config files, log files, and so on.

Parsing is more than merely interesting, it is required as long as Text Storage is plain.

I also intend to rely as heavily as possible on others’ code in developing pym, and can only expect that to arrive hither as plain text.

Parsing Python

Parsing produces Abstract Syntax Trees, which have a reasonable amount of related documentation. Green Tree Snakes was a proximate spur to this project, and the other links in this section were found thence.

Parsing non-Python

Other parsers, written in Python but parsing other structures:

Meta-Compiling

HackerNews told me about META II, which is able to “reproduce its own code from a description”:

At the end of these steps you should be able to:

  1. Define a compiler description that ejects your favorite language code
  2. Paste it into the Input window
  3. Have one of the compilers on these web pages generate the translation to your favorite language
  4. Cut the code generated in your favorite language from the Output window
  5. Continue compiler and metacompiler development in your favorite language

Unfortunately I have other work to get back to now, but the tutorial is well-written, enthusiastic about the meta-compiler, and when the tutorials don’t work, it’s because they are meant not to, in the next paragraph.

Excellent site, I hope to get more time on that tool.

Higher level structures

If the text to be editted is a program then parsing can be extended to include information on symbol tables, scope, control flow, tests, patterns, clichés, …

If the text to be editted has meaning then parsing can be extended to include context, connections, importance, definition, relevance, …