Draft

Quarto

tool
quarto
web dev
git
Author

Martin Laptev

Published

2024+247

TL;DR

This blog post is the most important post in my Tools blog post series, as indicated by its centrality in Figure 1, node of the netw focuses on the use of Quarto to author and publish digital content. As the third post in my , which is summarized by Figure 1, and depends on the. is my blog post on Quarto. The subsequent posts in the series demonstrate how to use Observable, Jupyter, and Knitr to execute code in Quarto documents.

flowchart TB
   A[Git]-->B[VSCode]-->C[Quarto]
   C-->D[Observable]
   C-->E[Jupyter]
   C-->F[Knitr]
   click A "../git"
   click B "../vscode"
   click C "../quarto"
   click D "../observable"
   click E "../jupyter"
   click F "../knitr"
   style C stroke:#99f,stroke-width:5px
Figure 1: Tools series navigation chart

If you do not have such a repo, you can create one by following along with the previous post in my tools blog post series, which is shown in Figure 1.

Introduction

Quarto is an open-source software system for turning plain-text source files into outputs like articles, books, blogs, dashboards, presentations, reports, and websites. Announced on 2022+149 by Posit CEO JJ Allaire, Quarto is already taking the world🌐by storm⛈️!

I strongly believe that everyone, regardless of their background and current technical skill level, can learn and benefit from Quarto. Getting started with Quarto is easy thanks to its excellent documentation and vibrant community of enthusiastic users and developers.

Rather than repeat the basic information already available elsewhere, I will share some advanced techniques along with the fundamental knowledge needed to understand how they work. The topics I cover are very technical, but my goal is to make the content on my blog as accessible as possible.

The first topic I will delve into is creating and publishing a website with Quarto. To follow the Quarto documentation on creating a website, you will need Visual Studio Code (VSCode), VSCodium, RStudio, or a terminal.

Publishing Quarto sites

Installing tools

If you use macOS, Linux, or the Windows Subsystem for Linux (WSL), you can install all of the aforementioned tools with the Homebrew package manager. To install everything you will need to follow along with this blog post, you can first install Homebrew and then run brew bundle in a directory that contains the Brewfile shown in Example 1.

Example 1  

Brewfile
brew "gh"
brew "git"
brew "glab"
cask "github"
cask "quarto"
cask "rstudio"
cask "vscodium"
cask "visual-studio-code"
vscode "quarto.quarto"
vscode "REditorSupport.r"

Using a package manager like Homebrew to install all the requirements with a single shell command like brew bundle is the fastest and easiest way to get ready to follow along with this blog post. If you are curious about how I set up my computer, you can take a look at my Brewfile and other configuration files in my setup repository (repo) on GitHub and GitLab.

Apart from RStudio, VSCode, and VSCodium, the Brewfile in Example 1 will install the Git version control system, the GitHub and GitLab command line interfaces (CLIs), and GitHub Desktop, a Git Graphical User Interface (GUI). For more information on Git, a tool used by 93% of software developers worldwide according to survey results published by StackOverflow on 2022+314, take a look at the “GitHub for supporting, reusing, contributing, and failing safely” post by Allison Horst and Julie Lowndes on the Openscapes blog.

As an alternative to installing tools on your computer, you can use the web interface provided by GitHub Codespaces. To set up a Codespace, you can remove the lines that start with cask from the Brewfile provided in Example 1 and add the file to a repo called dotfiles and along with a setup.sh file like the one shown in Example 2.

Example 2  

setup.sh
echo | /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
(echo; echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"') >> /home/codespace/.profile
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
brew bundle

Publishing overview

Once you are done setting up your computer or Codespace, you create a Quarto website template and make your site publicly available on the internet using one of the many available publishing services. To explore and assess different publishing workflows and free website hosting options, I set up my personal website on four different web hosts: Quarto Pub, Netlify, GitHub Pages, and GitLab Pages.

There are so many different ways to publish a Quarto site that I decided to come up with a naming system for Quarto publishing methods. The naming system derives a code for each publishing method from the numbered lists in the Quarto publishing documentation. For example, I refer to the two methods to publish to Quarto Pub as Q1 and Q2:

Table 1 uses this naming system in its Code column to identify the publishing methods I discuss in this blog post. Each publishing method targets a particular web Host, Renders content locally or on a remote server, and deploys sites using either the quarto publish or the git push shell Command.

Code Host Render Command
G1 GitHub Local push
G2 GitHub Local publish
G3 GitHub Remote push
N1 Netlify Local publish
N2 Netlify Local push
N3 Netlify Remote push
Q1 Quarto Local publish
Q2 Quarto Remote push
Table 1: Quarto website publishing methods

Git workflow

The first publishing method I tried for my personal site was G1, which requires the use of Git to push all website content to a Git provider like GitHub or GitLab. After I set up GitHub Pages and GitLab Pages, I could update my website on both of these web hosts just by going through the standard Git workflow shown in Figure 2.

flowchart LR
   A[working<br/>directory]-.git<br/>add.->B{{staging<br/>area}}-.git<br/>commit.->C([local<br/>repo])-.git<br/>push.->D(remote<br/>repo)
Figure 2: Git workflow

To make it easier to make incremental changes to my website and frequently release new content, I combined all of the git shell commands in Figure 2 into a shell alias. You can add shell aliases to a shell configuration file like .bashrc or .zshrc on your computer to shorten commands or combine any sequence of commands into one step.

The aacmp alias in the .zshrc file in my setup repo allows me to enter a free-form commit message directly on the command line without quotes, e.g. qp edit first blog. If you decide to try my aacmp alias, please exercise extreme caution as any un-escaped or un-quoted metacharacters may yield surprising effects instead of being included verbatim in the commit message. For example, qp * will list the contents of the current directory in the commit message in place of the asterisk!

An alternative to a shell alias that combines git commands is a keyboard shortcut in Git-enabled GUI. Example 3 shows two files, tasks.json and update.sh, that we can use to set up VSCode, VSCodium, and GitHub Codespaces to go through the Git workflow whenever we press Ctrl+Shift+B on Linux/Windows or ⌘+Shift+B on Mac (mnemonic: B is for Build).

Example 3  

This mechanism is called Tasks and is used to automate software build tasks, which can include any steps required to build and publish a website. Importantly, the Tasks mechanism requires that the tasks.json file be added to the .vscode directory and that we enable execution of the update.sh script by running chmod +x update.sh in our project root.

Shell aliases and keyboard shortcuts can greatly facilitate the Git workflow which is essential not only for G1, but also G3, N3, and Q2. Unlike these other publishing methods, G1 leads to messy commits that contain changes to both source and output files.

quarto publish

To have cleaner commits, I switched from G1 to G2 by adding quarto publish to my publishing workflow. With G2, I can track changes to my source files on my main branch and publish my output files to GitHub Pages and GitLab Pages from my gh-pages branch.

Q1, N1, and G2 all use quarto publish to render website content locally and then deploy it in one fell swoop. If you do not plan to use the Git version control system or the advanced features offered by GitHub, GitLab, or Netlify, then I recommend deploying your site to Quarto Pub by running quarto publish quarto-pub (Q1 in Table 1).

Like Q1, N1 is a deployment method that does not require Git and makes it possible to deploy our site with a single shell command: quarto publish netlify. N1 provides access to the advanced web hosting features offered by Netlify and can even render content as long as code execution is frozen.

To deploy my site to GitHub Pages, GitLab Pages, and Netlify, I combined G2 and N2. When I run quarto publish gh-pages, Quarto renders my site into my output directory, copies the output directory contents to the gh-pages branch of my local repo, and then commits and pushes the changes to my remote repos on GitHub and GitLab, which triggers Netlify to build my site from the gh-pages branch.

To summarize the Quarto publishing methods I discussed so far, Q1 and N1 are easy to configure and use, N2 automatically builds sites from Git repos, G1 is not recommended because it pollutes commits with output file changes, and G2 is more difficult to set up and use but provides clean commits and a nice separation of source and output files.

GitHub Actions

All of the publishing methods I have discussed so far require us to generate output files locally by rendering our source files. In contrast, Q2, G3, and N3 make it possible to skip local rendering in favor of relying on GitHub Actions to handle all of the necessary steps.

G3 is noteworthy, because it offers the same convenience of G1 but without messy commits that mix changes to source and output files. An added bonus of G3 is that rendering with GitHub Actions provides a reproducible computational environment that is not dependent on what you have installed on your computer.

Instead of using GitHub Actions, I could have used GitLab CI/CD to build my site. I decided not to go down this route because the Quarto dev team has many GitHub Actions workflows available but currently no official support for GitLab CI/CD.

Before trying to use GitHub Actions or any other continuous integration systems in your publishing workflow, I suggest getting used to working with quarto publish (Q1, G2, or N1). You can always set up other publishing methods later without sacrificing anything, because all of the publishing methods except G1 can be combined together.

In Section 4, I will walk through the setup of both G2 and G3 to provide the option of rendering locally by running quarto publish or rendering remotely with GitHub Actions by pushing to the main branch. Along the way, I will share many practical tips and general advice that you can apply to any project.

Publishing setup

Repo setup

Before you can use GitHub Actions to publish your site, you will need a GitHub, an SSH key, and a repo like maptv.github.io that has a default branch called main and another branch which must be called gh-pages. If you want to publish on GitLab Pages, you will also need a GitLab account and an analogous GitLab repo like maptv.gitlab.io.

You can create the repo and the gh-pages branch using the web interface of https://github.com or https://gitlab.com in your browser, but the best way to start a new project is using the CLI for GitHub or GitLab in your terminal. First, run gh auth login or glab auth login and follow the prompts to authenticate via your web browser or with an authentication token.

The GitHub CLI allows you to add an SSH key to your account during or after authentication. The GitLab CLI does not handle SSH keys during authentication, but has a similar command for adding an SSH key to your GitLab account.

After authentication and SSH key setup, you can run the code in either of the code chunks in Example 4 to set up your local and remote repos and create a Quarto website project in the local repo. You can create shell alias that combine all of the repo creation steps like I did in my .zshrc.

Example 4  

cd # start in home directory
mkdir -p USERNAME
cd USERNAME
gh repo create USERNAME.github.io --add-readme --clone --public
cd SITENAME
quarto create project website USERNAME.github.io
cd # start in home directory
mkdir -p USERNAME
cd USERNAME
glab repo create USERNAME.gitlab.io --readme --defaultBranch main --public
cd SITENAME
git pull origin main
git branch --set-upstream-to=origin/main main
quarto create project website USERNAME.gitlab.io

To make it easier to maintain my site on both GitHub and GitLab Pages, I set up my local repo cloned to have two origin remote URLs using the code as shown in Example 5. Now, running quarto publish or git push in my local repo, updates my content on both GitHub and GitLab.

Example 5  

git remote add lab git@gitlab.com:maptv/maptv.gitlab.io
git remote add hub git@github.com:maptv/maptv.github.io
git remote set-url --add origin $(git remote get-url lab)

If you want to have your website hosted on GitHub Pages, you will need to set gh-pages as your source branch in your repo settings. For GitLab Pages, you will need to add a .gitlab-ci.yml file to your repo and update your _quarto.yml file as shown in Example 6 to include .gitlab-ci.yml as a resource in your output directory.

Example 6  

# https://docs.gitlab.com/ee/user/project/pages/introduction.html#customize-the-default-folder
pages:
  script: echo "The site will be deployed to $CI_PAGES_URL"
  artifacts:
    paths:
      - "."
  publish: "."
  only:
    - gh-pages
author:
  - name: Martin Laptev
    url: https://maptv.github.io

By default, GitLab Pages includes a random hash in site URLs. To shorten the URL of my GitLab Pages site to <maptv.gitlab.io>, I had to uncheck Use unique domain under Deploy > Pages in the GitLab sidebar.

At this point, we have completed G2 setup and you should be able to run quarto publish gh-pages from your main branch to render your site and deploy it to GitHub and/or GitLab Pages. Deploying with quarto publish at least once is a prerequisite for setting up any of the publishing methods that rely on GitHub Actions, because quarto publish creates a _publish.yml file in the root of your project that is required for publishing via GitHub Actions.

GitHub Actions

In addition to the steps described above, G3 setup requires that we create a .github/workflows directory and add a YAML file to that directory. Example 7 contains the gh-pages.yml file I use for my own site and the shell code that can used to obtain this file.

Example 7  

mkdir -p .github/workflows
cd .github/workflows
curl -O https://raw.githubusercontent.com/maptv/maptv.github.io/main/.github/workflows/gh-pages.yml
on:
  workflow_dispatch:
  push:
    branches: [main]

name: Quarto Publish

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2
        with:
          version: '1.4.489'
          tinytex: true

      - name: Install R
        uses: r-lib/actions/setup-r@v2
        with:
          r-version: '4.2.0'

      - name: Install R Dependencies
        uses: r-lib/actions/setup-renv@v2
        with:
          cache-version: 1

      - name: Install Python and Dependencies
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          cache: 'pip'
      - run: pip install jupyter
      - run: pip install -r requirements.txt

      - name: Render and Publish
        uses: quarto-dev/quarto-actions/publish@v2
        with:
          target: gh-pages
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Push To GitLab
        env:
          token: ${{ secrets.GITLAB_AUTH_TOKEN }}
        run: |
          echo Starting to push gh-pages branch to GitLab
          git config user.name "maptv"
          git config user.email "129316885+maptv@users.noreply.github.com"
          git remote set-url origin "https://oauth2:${token}@gitlab.com/maptv/maptv.gitlab.io"
          git push origin gh-pages

The gh-pages.yml file in Example 7 installs Quarto, the R and Python programming languages, and the packages in the renv.lock and requirements.txt files. If you do not need R and/or Python, you can remove any unnecessary portions of the file.

To modify the Python files that are installed by GitHub Actions, you can edit the requirements.txt file in your repo. To update your renv.lock file so that it includes all of the R packages your site requires, run renv::snapshot() in an R session or Rscript -e renv::snapshot() in a shell.

After pushing the gh-pages.yml file, you can visit the Actions tab in your remote repo on GitHub to check the progress of the deployment of your site. If your site did not build successfully, you can go through the logs to try to diagnose the problem.

I added a “Push to GitLab” step to my gh-pages.yml to make GitHub Actions push my remote gh-pages to GitLab so that my site is kept in sync on both GitHub and GitLab Pages. This required manually creating a token on GitLab and adding it to GitHub, which I accomplished using the GitHub CLI as shown in Example 8.

Example 8  

gh secret set GITLAB_AUTH_TOKEN

After the setup described above, I now have two options for publishing my Quarto site: quarto publish and 2) git push. In addition to GitHub and GitLab Pages, both of these options automatically update my site on Netlify via N2.

To also automatically update my site on Quarto Pub, I created a separate GitHub Actions workflow by adding another YAML file to the .github/workflows directory in my repo. Example 9 shows my quarto-pub.yml file and the shell code that can be used to obtain it.

Example 9  

mkdir -p .github/workflows
cd .github/workflows
curl -O https://raw.githubusercontent.com/maptv/maptv.github.io/main/.github/workflows/gh-pages.yml
on:
  workflow_run:
    workflows: [pages-build-deployment]
    types: [completed]

name: Update Quarto Pub

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Check out main repository
        uses: actions/checkout@v4

      - name: Check out _site
        uses: actions/checkout@v4
        with:
          ref: gh-pages
          path: _site

      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2

      - name: Publish
        uses: quarto-dev/quarto-actions/publish@v2
        with:
          target: quarto-pub
          render: false
          QUARTO_PUB_AUTH_TOKEN: ${{ secrets.QUARTO_PUB_AUTH_TOKEN }}
          GITHUB_USERNAME: maptv

My quarto-pub.yml file is based on Q2, but it runs upon completion of the pages-build-deployment workflow instead of a push to main. I changed the workflow trigger so that it runs after my GitHub Pages site is built, regardless of whether I triggered the build by running quarto publish gh-pages or pushing to main.

Figure 3 summarizes all of the steps that occur during my Quarto publishing workflow. This workflow allows me to publish my site on four web hosts every time I run quarto publish gh-pages (G2) or git push (G3)!

flowchart LR
    A[local<br/>main]-.G2.->B[local<br/>gh-pages]-.G2.->C[GitHub<br/>gh-pages]
    B[local<br/>gh-pages]-.G2.->D[GitLab<br/>gh-pages]
    A[local<br/>main]-.G3.->F[GitHub<br/>main]-.G3.->C[GitHub<br/>gh-pages]-.G3.->D[GitLab<br/>gh-pages]
    A[local<br/>main]-.G3.->G[GitLab<br/>main]
    C[GitHub<br/>gh-pages]-.N2.->E[Netlify]
    C[GitHub<br/>gh-pages]-.Q2.->H[Quarto<br/>Pub]
Figure 3: Quarto publishing workflow

So far I have only noticed one difference between the four web hosts I use for my site: GitHub Pages is the only web host that properly differentiates between internal and external links. All of the other web hosts include the external link icon on all links regardless of whether they target my site or an external site.

I tried unsuccessfully to solve this issue by setting the link-external-filter property to a regular expression. If you notice a problem with the link-external-icon feature on other web hosts, I suggest switching to GitHub Pages.

Customizing Quarto sites

Site navigation

Once your site is built and published, you can make it easier to navigate with a customized navigation bar (navbar) and sidebar, as described in the Quarto documentation. Check out the _quarto.yml files for my site and the Quarto documentation site to see how the navigation components are set up.

In my navbar, I added toggle buttons for reader mode and dark mode. The names of the other icons on right side of the navbar are included in the sidebar.

HTML blocks

My navbar also provides the current date and time in Dec ordinal (deco) format, which counts the years since 1 BC and the days since March 1. Dec is a calendar and time system that I created and use throughout my blog.

To add a custom timestamp to the navbar, I put a <script> element that runs time.js in an HTML block in every Quarto markdown (.qmd) file, as shown in Example 10. Mine Çetinkaya-Rundel wrote about HTML blocks as part of her A Quarto tip a day project.

Example 10  

```{=html}
<script src="time.js"></script>
```

HTML blocks are useful for running JavaScript code in the context of a single page on your site without affecting the others. I also use HTML blocks to add a <style> element to my .qmd files as a last ditch effort to fine tune the style of my site.

Before I resort to the <style> element approach, I try writing CSS in my style.css, light.css, and dark.css files. So far, this approach has been sufficient to make any styling changes I want, but if it every fails, I can use JavaScript to override the default styling provided by Quarto by modifying style attributes, which have the highest specificity in CSS.

Post-render script

I wanted to my blog’s listing page to show dates in the same format as the navbar timestamp. To accomplish this, I used a post-render script written in the Python programming language.

Pre- and post-render scripts are set as project options in the _quarto.yml file in the root of your Quarto project. Example 11 shows the date.py post-render script and how this script is referenced in _quarto.yml.

Example 11  

author:
  - name: Martin Laptev
    url: https://maptv.github.io
date-format: x
# https://howardhinnant.github.io/date_algorithms.html#days_from_civil
import bs4

def unix2doty(ms):
    dote = ms / 86400000 + 719468
    cykl = (
        dote if dote >= 0
        else dote - 146096
    ) // 146097
    dotc = dote - cykl * 146097
    yotc = (dotc
        - dotc // 1460
        + dotc // 36524
        - dotc // 146096
    ) // 365
    return [
        int(yotc + cykl * 400),
        dotc - (yotc * 365
            + yotc // 4
            - yotc // 100
        )
    ]

with open("_site/list/index.html") as infile:
    txt = infile.read()
    soup = bs4.BeautifulSoup(txt, features="html.parser")
for div in soup.find_all("div", {"class": "card-file-modified"}):
    elem = div.find("p")
    if "+" not in elem.text:
        y, d = unix2doty(int(div.text))
        elem.string.replace_with(f"{y:>04}+{d.__floor__():>03}")
for div in soup.find_all("div", {"class": "listing-reading-time"}):
    if "min" in div.text:
        div.string.replace_with(f"{(int(div.text.split()[0]) / 1.44).__floor__()} milliday")
with open("_site/list/index.html", "w") as outfile:
    outfile.write(str(soup))

Normally, formatting dates in Quarto is as easy as setting the date-format YAML property. In my case it was more difficult, because I wanted dates based on Dec, the calendar and time system I created, instead of the ubiquitous Gregorian calendar.

To facilitate the calculations in date.py, I set date-format to x in my _metadata.yml file so all of the dates would be in millisecond UNIX time and I would not have to deal with Gregorian calendar date formats or time zones. After all of the dates are generated, my date.py script traverses the HTML in the output directory using Beautiful Soup and converts millisecond UNIX time dates into Dec dates (year+day).

Pandoc filter

In addition to customizing dates on the listing page of my blog, I wanted to customize the date format in every blog post. To complete this task, I used a Lua script as a Pandoc filter.

Pandoc is a program that converts documents into practically any format. The “pan” in Pandoc comes from the Ancient Greek word for all. Pandoc strives to convert all document formats, just like Pangea contained all the land and a panacea solves all problems.

Quarto uses Pandoc to convert markdown files into target format(s) like html or pdf. If the source files contain executable code, Quarto executes the code via one of two computational engines: Jupyter or Knitr. Figure 4 shows the Quarto workflow.

flowchart LR
   A[qmd<br/>ipynb]-.Knitr<br/>Jupyter.->B((md))-.Pandoc with<br/>Lua filters.->C(html<br/>pdf<br/>docx<br/>etc.)
Figure 4: Quarto rendering workflow

Mine Çetinkaya-Rundel’s Quarto tip series includes a similar Quarto workflow mermaid diagram and her “Hello, Quarto!” rstudio::conf(2022) keynote with Julia Stewart Lowndes contains truly beautiful Quarto workflow images by Allison Horst: 1, 2, 3, and 4.

Quarto controls Pandoc, Jupyter, and Knitr in two ways: 1) with arguments passed to the Quarto CLI commands and 2) with YAML key-value pairs in .qmd, .ipynb, or .yml files.

Example 12 shows how I set up every qmd file to generate the current date in millisecond UNIX time and designated date.lua as a Pandoc filter. Unlike post-render scripts, Pandoc filters are executed during the creation of the output files.

Example 12  

date: now
date-format: x
filters: date.lua
-- https://howardhinnant.github.io/date_algorithms.html#days_from_civil
function unix2deco(ms)
  local dote = ms / 86400000 + 719468
  local cykl = (
    dote >= 0 and dote
    or dote - 146096
  ) // 146097
  local dotc = dote - cykl * 146097
  local yotc = (
    dotc - dotc // 1460
    + dotc // 36524
    - dotc // 146096
  ) // 365
  return string.format(
    "%s+%s",
    math.floor(yotc + cykl * 400),
    math.floor(dotc - (yotc * 365
      + yotc // 4
      - yotc // 100)))
end

local function to_decalendar(date)
  local date = pandoc.utils.stringify(date)
  local unix = date:match("(%d+)")
  return date:gsub(unix, unix2deco(unix))
end

function Meta(m)
  m.date = to_decalendar(m.date)
  return m
end

All three of the customizations I described above convert millisecond UNIX time into a custom date format, but you can adapt these approaches to make all sorts of edits to your Quarto site. HTML blocks can run JavaScript which excels at making content dynamic and interactive, pre- and post-render scripts can be in any programming language, while Pandoc filters are written in Lua and modify output during rendering.

Coming up next on my blog is a post that Observable graphics. Get ready for an animated and interactive data visualization extravaganza!

Back to top

Citation

BibTeX citation:
@online{laptev2024,
  author = {Laptev, Martin},
  title = {Quarto},
  date = {2024},
  urldate = {2024},
  url = {https://maptv.github.io/quarto/script/},
  langid = {en}
}
For attribution, please cite this work as:
Laptev, Martin. 2024. “Quarto.” 2024. https://maptv.github.io/quarto/script/.