Creating Filters

Overview

If the base features of Pandoc and Quarto don’t do exactly what you need, you can very likely create a Pandoc Filter that bridges the gap.

Pandoc consists of a set of readers and writers. When converting a document from one format to another, text is parsed by a reader into pandoc’s intermediate representation of the document—an “abstract syntax tree” or AST—which is then converted by the writer into the target format. The pandoc AST format is defined in the module Text.Pandoc.Definition in the pandoc-types package.

A “filter” is a program that modifies the AST, between the reader and the writer.

INPUT --reader--> AST --filter--> AST --writer--> OUTPUT

Pandoc’s built-in citation processing is implemented as a filter, as are many of Quarto’s internal extensions (e.g. cross-references, figure layout, etc.).

You can write Pandoc filters using Lua (via Pandoc’s built-in Lua interpreter) or using any other language using a JSON representation of the Pandoc AST piped to/from an external process. We strongly recommend using Lua Filters, which have the following advantages:

  • No external dependencies
  • High performance (no serialization or process execution overhead)
  • Access to the Pandoc and Quarto libraries of Lua helper functions.

Activating Filters

If you’ve developed a filter and want to use it within a document you need to add it to the list of filters for the document. For example, here we arrange for the spellcheck filter to run:

---
filters:
  - spellcheck.lua
---

By default, user filters are run before Quarto’s built-in filters. For some filters you’ll want to modify this behavior. For example, here we arrange to run spellcheck before Quarto’s filters and fontawesome after:

filters:
  - spellcheck.lua
  - quarto
  - fontawesome

You’ll notice that one of the extensions (spellcheck.lua) has a file extension and the other (fontawesome) does not. This difference stems from how the extensions are distributed: an extension distributed as a plain Lua file uses .lua whereas a filter distributed as a Quarto Extension does not. The next section explores how to create filters as extensions.

Filter Extensions

Quick Start

Here we’ll describe how to create a simple filter extension. We’ll use the quarto create command to do this. If you are using VS Code or RStudio you should execute quarto create within their respective integrated Terminal panes.

To get started, execute quarto create extension filter within the parent directory where you’d like the filter extension to be created:

Terminal
$ quarto create extension filter
 ? Extension Name › fancy-header

As shown above, you’ll be prompted for an extension name. Type fancy-header and press Enter—the filter extension is then created:

Creating extension at /Users/jjallaire/quarto/dev/fancy-header:
  - Created README.md
  - Created _extensions/fancy-header/_extension.yml
  - Created _extensions/fancy-header/fancy-header.lua
  - Created .gitignore
  - Created example.qmd

If you are running within VS Code or RStudio a new window will open with the extension project.

Here’s what the contents of the files in _extensions/fancy-header/ look like:

_extensions/fancy-header/_extension.yml
title: Fancy-header
author: J.J. Allaire
version: 1.0.0
quarto-required: ">=99.9.0"
contributes:
  filters:
    - fancy-header.lua
_extensions/fancy-header/fancy-header.lua
-- Reformat all heading text 
function Header(el)
  el.content = pandoc.Emph(el.content)
  return el
end

Finally, the example.qmd file includes code that exercises the extension. For example:

example.qmd
---
title: "Fancy-header Example"
filters:
  - fancy-header
---

## Heading

This filter adds formatting to heading text.

Note that the value provided to filters in example.qmd should be the name of the extension (fancy-header), not the filename of the filter (fancy-header.lua). This allows you to bundle more than one filter in your extension:

_extensions/fancy-header/_extension.yml
contributes:
  filters:
    - fancy-header.lua
    - make-fancier.lua

All of filters in your extension will be applied when a user uses your extension in their document.

To develop your filter, render/preview example.qmd, and then make changes to fancy-header.lua (the preview will automatically refresh when you change fancy-header.lua).

Development

To learn more about developing filter extensions:

  1. If necessary, brush up on Lua Development (Lua is the language used to create filters).

  2. Review the Pandoc documentation on Writing Lua Filters.

  3. Read the Lua API Reference, which describes the Lua extension API for Quarto.

If you want to write a JSON filter, see the documentation on Writing JSON filters.

To create a new filter extension, use the quarto create extension filter command as described above.

Distribution

If your extension source code is located within a GitHub repository, then it can be added to a project by referencing the GitHub organization and repository name. For example:

Terminal
# target the current HEAD of the extension
quarto add cooltools/output-folding

# target a branch or tagged release of the extension
quarto add cooltools/output-folding@v1.2
quarto add cooltools/output-folding@bugfix-22

Note that it is possible to bundle and distribute extensions as simple gzip archives (as opposed to using a GitHub repository as described above). See the article on Distributing Extensions for additional details.

Examples

You might also find it instructive to examine the source code of these filter extensions authored by the Quarto team:

Extension name Description
latex-environment Quarto extension to output custom LaTeX environments.
lightbox Create lightbox treatments for images in your HTML documents.