Rust - Writing Parsers With nom Parser Combinator Framework

I've been working on my new Rust side-project for several months now, and I've got some learnings to share. The project is called pq - it's a command-line tool to parse and query log files as time series. It comes with its own domain-specific language that is highly influenced by PromQL. A typical pq usage may look like this:

tail -f /var/log/nginx/access.log | pq '
/...some fancy regex.../
| map {
    .0 as ip,
    .1:ts,
    .2 as method,
    .3:str as status_code,
    .4 as content_len
  }
| select topk(
      10,
      sum(
          count_over_time(
              __line__{method="GET", status_code="200"}[1s]
          )
      ) by (ip)
  )
| to_json
'

pq has many components, including various log parsing strategies and a pretty sophisticated query execution engine. But surprisingly or not, about half of the time I've put into this project so far was dedicated to writing the parser of the pq's own query language. To be honest, when I was starting the project, I didn't see that coming...

nom logo

Luckily, writing a parser in Rust was mostly a pleasant experience, thanks to a crate concisely named nom. Although learning how to write parsers with nom wasn't completely seamless. So here is my journey.

Read more

Understanding Rust Privacy and Visibility Model

I spent the last couple of months writing code in Rust. It was probably my third or fourth attempt to write something substantial in this language. And every time my level of understanding of things deepened. I'm by no means a Rust expert so probably I'll be extremely inaccurate in the terminology here. And likely I'll get lots of technical details wrong too. But I had this epiphany moment of how the visibility and privacy model works in Rust so I can't help but think of sharing it with someone else.

Read more