pears

A parser combinator library for gleam

Writing a parser using pears is easy, you just combine functions that parse smaller parts of the input into a function that parses the whole input.

Here’s an example of a parser that parses the letter ‘a’, run with parse_string:

import pears.{UnexpectedToken}
import pears/chars
import pears/combinators.{between, choice, many0, many1, map, or, pair, replace, sep_by0, skip_then, succeed, then, then_skip, token}

let a_parser = token("a")
let assert Ok(parsed) = parse_string("abc", a_parser)
assert parsed.value == "a"

It returns a Result with a Parsed record that holds the parsed value and the remaining input.

To parse the whole input and get just the value, use parse_string_all. It fails if any input is left over:

assert parse_string_all("a", token("a")) == Ok("a")
assert parse_string_all("abc", token("a")) == Error(UnexpectedToken(1, ["end of input"], "b"))

Combinators

Combining parsers using combinators is how you build more complex parsers. A combinator is just a function that takes one or more parsers and returns a new parser.

Let’s say we want to parse the letter ‘a’ followed by the letter ‘b’, we can use the pair combinator for that. It runs the first parser and then the second one, returning both results as a tuple:

assert parse_string_all("ab", pair(token("a"), token("b"))) == Ok(#("a", "b"))

To create parsers that are actually useful we need to be able to branch based on the input, for that we can use the or combinator:

let a_or_b_parser = or(token("a"), token("b"))
assert parse_string_all("a", a_or_b_parser) == Ok("a")
assert parse_string_all("b", a_or_b_parser) == Ok("b")

In cases where there are more than two options, we can use the choice combinator:

let a_b_or_c_parser = choice([token("a"), token("b"), token("c")])
assert parse_string_all("c", a_b_or_c_parser) == Ok("c")

The next crucial combinators are many0 and many1. They allow us to parse zero or more or one or more repetitions of a parser:

let abc0_parser = many0(choice([token("a"), token("b"), token("c")]))
assert parse_string_all("cab", abc0_parser) == Ok(["c", "a", "b"])
assert parse_string_all("", abc0_parser) == Ok([])

Sometimes we want to parse something and then ignore the result, for that we can use the then_skip and skip_then combinators. then_skip keeps the result of the first parser and skip_then the result of the second:

assert parse_string_all("ab", then_skip(token("a"), token("b"))) == Ok("a")
assert parse_string_all("ab", skip_then(token("a"), token("b"))) == Ok("b")

Longer sequences are easier to write with then and Gleam’s use syntax. succeed returns a value without consuming any input:

let point = {
  use x <- then(chars.natural())
  use _ <- then(token(","))
  use y <- then(chars.natural())
  succeed(#(x, y))
}
assert parse_string_all("1,2", point) == Ok(#(1, 2))

These building blocks are enough to build many combinators that can parse complex data structures. For example the sep_by0 and sep_by1 combinators which can be used to parse lists of items separated by a delimiter:

let letters = sep_by0(choice([token("a"), token("b"), token("c")]), token(","))
assert parse_string_all("a,b,c", letters) == Ok(["a", "b", "c"])

Transforming the results

The map combinator transforms the result of a parser using a function. Here we parse one or more digits and turn them into an Int, which is what chars.natural does:

let number_parser =
  many1(chars.digit())
  |> map(fn(digits) {
    let assert Ok(number) = int.parse(string.concat(digits))
    number
  })
assert parse_string_all("42", number_parser) == Ok(42)

replace is a shorthand for map that ignores the parsed value and returns a constant:

let bool_parser =
  or(replace(chars.string("true"), True), replace(chars.string("false"), False))
let bools_parser =
  sep_by0(bool_parser, token(","))
  |> between(token("["), token("]"))
assert parse_string_all("[true,false]", bools_parser) == Ok([True, False])

Errors

When a parser fails it returns a ParseError with the position of the failing token, what was expected there and what was found instead, or a custom message from fail or try_map. format_error and chars.format_error turn it into a message:

let source = "a,b;c"
let letters = sep_by0(choice([token("a"), token("b"), token("c")]), token(","))
let assert Error(error) = parse_string_all(source, letters)
assert chars.format_error(error, source) == "line 1, column 4: expected \",\" or end of input, found \";\""

More examples

Please have a look at the tests for more complex examples, such as parsing JSON or Brainf*ck, or a lexer followed by a parser over its tokens (test/lexer_test.gleam).

pub type ParseError(i) {
  UnexpectedEndOfInput(position: Int, expected: List(String))
  UnexpectedToken(
    position: Int,
    expected: List(String),
    found: i,
  )
  Custom(position: Int, message: String)
}

pub type Parsed(i, a) {
  Parsed(remaining: input.Input(i), value: a)
}

pub opaque type Parser(i, a)

pub type Span {
  Span(start: Int, end: Int)
}

pub fn error_message(error: ParseError(i)) -> String

pub fn format_error(error: ParseError(i)) -> String

pub fn parse(
  tokens: List(i),
  parser: Parser(i, a),
) -> Result(Parsed(i, a), ParseError(i))

pub fn parse_all(
  tokens: List(i),
  parser: Parser(i, a),
) -> Result(a, ParseError(i))

pub fn parse_string(
  text: String,
  parser: Parser(String, a),
) -> Result(Parsed(String, a), ParseError(String))

pub fn parse_string_all(
  text: String,
  parser: Parser(String, a),
) -> Result(a, ParseError(String))