Parser (pyre-ast.PyreAst.Parser)

Module PyreAst.Parser

This module contains all parsing APIs, i.e. functions that transfrom plain strings into syntactical constructs. These APIs will always hand back a Result.t where if the parsing fails, a Parser.Error.t gets returned.

Under the hood, this library actually compiles and calls into the actual CPython parser code, and then walks through the CPython AST translating them into OCaml structures via C bindings. This is how 100% fidelity with the official CPython implementation is achieved -- we are actually relying on exactly the same parser implementation that CPython uses. This approach has some notable implications:

The parsing APIs are stateful as one need to intialize/finalize CPython runtime before invoking its parser. The low-level details are abstracted away with the


Parser

.Context module, but the fact that no parsing can be done prior to obtaining a


Parser

.Context.t still holds.

Text encoding support is crippled. A large part of Unicode handling in CPython is provided via extension modules, which a barely-initialized CPython runtime cannot handle. For example, Unicode character in identifier name is not allowed, and "\N" escape sequence in string literals are not properly translated. What's more, the -*- coding: X -*- header (see PEP 263) is mostly not supported unless X is utf-8.

module Context : sig ... end

This module contains a type that abstracts away the details of global states required to set up the parser.

module Error : sig ... end

This module contains a type that represents parsing errors.

val with_context : ?on_init_failure:( unit -> 'a ) -> ( Context.t -> 'a ) -> 'a

with_context ?on_init_failure f first creates a value c of type Context.t and then invoke f on c. It is guaranteed that the created context c will be destroyed in the end regardless of whether f raises an exception or not.

If the creation of c fails, on_init_failure () will be invoked, and f will not be called. By default, if not explicitly overriden then on_init_failure would simply raise a Failure.

module TaglessFinal : sig ... end

This module provides parsing APIs for downstream clients that are written in tagless-final style. See PyreAst.TaglessFinal for more explanation about this style.

module Concrete : sig ... end

This module provides parsing APIs for downstream clients that are written in the traditional "initial" style which expects a concrete ADT representation for abstract syntax trees.