md4r

Lifecycle: experimental R-CMD-check

Provides an R wrapper for the MD4C (Markdown for C) library. Functions exist for markdown parsing (CommonMark compliant) along with support for other common markdown extensions (e.g. GitHub flavored markdown, LaTeX equation support, etc.). The package also provides a number of high level functions for exploring and manipulating markdown ASTs as well as translating and displaying the documents.

Installation

Install md4r from CRAN:

install.packages("md4r")

or install the latest development version package from GitHub:

remotes::install_github("rundel/md4r")

Example

We will start with a simple example of parsing a markdown file using the basic CommonMark dialect.

md_file = system.file("examples/commonmark.md", package = "md4r")
readLines(md_file) |> cat(sep='\n')
#> ## Try CommonMark
#> 
#> You can try CommonMark here.  This dingus is powered by
#> [commonmark.js](https://github.com/commonmark/commonmark.js), the
#> JavaScript reference implementation.
#> 
#> 1. item one
#> 2. item two
#>    - sublist
#>    - sublist

this file (or markdown text) can be processed using the parse_md function which creates an abstract syntax tree representation of the document (as a list of lists of lists … with custom S3 classes)

library(md4r)
(md = parse_md(md_file))
#> md_block_doc [flags: "MD_DIALECT_COMMONMARK"]
#> ├── md_block_h [level: 2]
#> │   └── md_text_normal - "Try CommonMark"
#> ├── md_block_p
#> │   ├── md_text_normal - "You can try CommonMark here.  This dingus is powered by"
#> │   ├── md_text_softbreak
#> │   ├── md_span_a [title: "", href: "https://github.com/commonmark/commonmark.js"]
#> │   │   └── md_text_normal - "commonmark.js"
#> │   ├── md_text_normal - ", the"
#> │   ├── md_text_softbreak
#> │   └── md_text_normal - "JavaScript reference implementation."
#> └── md_block_ol [start: 1, tight: 1, mark_delimiter: "."]
#>     ├── md_block_li
#>     │   └── md_text_normal - "item one"
#>     └── md_block_li
#>         ├── md_text_normal - "item two"
#>         └── md_block_ul [tight: 1, mark: "-"]
#>             ├── md_block_li
#>             │   └── md_text_normal - "sublist"
#>             └── md_block_li
#>                 └── md_text_normal - "sublist"
str(md)
#> List of 3
#>  $ :List of 1
#>   ..$ : 'md_text_normal' chr "Try CommonMark"
#>   ..- attr(*, "level")= num 2
#>   ..- attr(*, "class")= chr [1:3] "md_block_h" "md_block" "md_node"
#>  $ :List of 6
#>   ..$ : 'md_text_normal' chr "You can try CommonMark here.  This dingus is powered by"
#>   ..$ : list()
#>   .. ..- attr(*, "class")= chr [1:3] "md_text_softbreak" "md_text" "md_node"
#>   ..$ :List of 1
#>   .. ..$ : 'md_text_normal' chr "commonmark.js"
#>   .. ..- attr(*, "title")= chr ""
#>   .. ..- attr(*, "href")= chr "https://github.com/commonmark/commonmark.js"
#>   .. ..- attr(*, "class")= chr [1:3] "md_span_a" "md_span" "md_node"
#>   ..$ : 'md_text_normal' chr ", the"
#>   ..$ : list()
#>   .. ..- attr(*, "class")= chr [1:3] "md_text_softbreak" "md_text" "md_node"
#>   ..$ : 'md_text_normal' chr "JavaScript reference implementation."
#>   ..- attr(*, "class")= chr [1:3] "md_block_p" "md_block" "md_node"
#>  $ :List of 2
...

As the AST is just a collection of R lists - we can use subsetting to extract specific elements of the document

parse_md(md_file)[[1]]
#> md_block_h [level: 2]
#> └── md_text_normal - "Try CommonMark"
parse_md(md_file)[[2]]
#> md_block_p
#> ├── md_text_normal - "You can try CommonMark here.  This dingus is powered by"
#> ├── md_text_softbreak
#> ├── md_span_a [title: "", href: "https://github.com/commonmark/commonmark.js"]
#> │   └── md_text_normal - "commonmark.js"
#> ├── md_text_normal - ", the"
#> ├── md_text_softbreak
#> └── md_text_normal - "JavaScript reference implementation."
parse_md(md_file)[[3]]
#> md_block_ol [start: 1, tight: 1, mark_delimiter: "."]
#> ├── md_block_li
#> │   └── md_text_normal - "item one"
#> └── md_block_li
#>     ├── md_text_normal - "item two"
#>     └── md_block_ul [tight: 1, mark: "-"]
#>         ├── md_block_li
#>         │   └── md_text_normal - "sublist"
#>         └── md_block_li
#>             └── md_text_normal - "sublist"

or more advanced tools like rapply() to extract text content

rapply(md, as.character, "md_text")
#> [1] "Try CommonMark"                                         
#> [2] "You can try CommonMark here.  This dingus is powered by"
#> [3] "commonmark.js"                                          
#> [4] ", the"                                                  
#> [5] "JavaScript reference implementation."                   
#> [6] "item one"                                               
#> [7] "item two"                                               
#> [8] "sublist"                                                
#> [9] "sublist"

Additionally, the AST and any component can be converted back into markdown

to_md(md) |> cat(sep='\n')
#> ## Try CommonMark
#> 
#> You can try CommonMark here.  This dingus is powered by
#> [commonmark.js](<https://github.com/commonmark/commonmark.js>), the
#> JavaScript reference implementation.
#> 
#> 1. item one
#> 2. item two
#>    - sublist
#>    - sublist

or into html

to_html(md) |> cat(sep='\n')

Try CommonMark

You can try CommonMark here. This dingus is powered by

commonmark.js , the

JavaScript reference implementation.

  1. item one
  2. item two
    • sublist
    • sublist

Markdown extensions

By default parse_md() uses the CommonMark dialect, other syntax is enabled with parser flags. parse_gfm() is a shortcut for the GitHub flavored markdown dialect, which adds tables, task lists, strikethrough, autolinks, footnotes, and admonitions.

parse_gfm(c(
  "> [!NOTE]",
  "> Task lists work too",
  "",
  "- [x] done",
  "- [ ] ~~not~~ done"
))
#> md_block_doc [flags: "MD_DIALECT_GITHUB"]
#> ├── md_block_admonition [type: "note"]
#> │   └── md_block_p
#> │       └── md_text_normal - "Task lists work too"
#> └── md_block_ul [tight: 1, mark: "-"]
#>     ├── md_block_li [is_task: 1, task_mark: "x"]
#>     │   └── md_text_normal - "done"
#>     └── md_block_li [is_task: 1, task_mark: " "]
#>         ├── md_span_del
#>         │   └── md_text_normal - "not"
#>         └── md_text_normal - " done"

Individual extensions can be turned on by passing their flags to parse_md()

parse_md(
  c("H~2~O is ==wet==[^1]", "", "[^1]: Citation needed."),
  flags = c("MD_FLAG_SUBSCRIPTS", "MD_FLAG_HIGHLIGHT", "MD_FLAG_FOOTNOTES")
)
#> md_block_doc [flags: "MD_FLAG_SUBSCRIPTS", "MD_FLAG_HIGHLIGHT", and "MD_FLAG_FOOTNOTES"]
#> ├── md_block_p
#> │   ├── md_text_normal - "H"
#> │   ├── md_span_sub
#> │   │   └── md_text_normal - "2"
#> │   ├── md_text_normal - "O is "
#> │   ├── md_span_mark
#> │   │   └── md_text_normal - "wet"
#> │   └── md_span_footnote_ref [id: 1, ref_id: 1, label: "1"]
#> └── md_block_footnote_def_section
#>     └── md_block_footnote_def [id: 1, ref_count: 1, label: "1"]
#>         └── md_text_normal - "Citation needed."

and flags_describe() lists all of the available flags.

flags_describe()
#> # A tibble: 27 × 2
#>    flag                             description                                 
#>    <chr>                            <chr>                                       
#>  1 MD_FLAG_COLLAPSEWHITESPACE       In MD_TEXT_NORMAL, collapse non-trivial whi…
#>  2 MD_FLAG_PERMISSIVEATXHEADERS     Do not require space in ATX headers ( ###he…
#>  3 MD_FLAG_PERMISSIVEURLAUTOLINKS   Recognize URLs as autolinks even without '<…
#>  4 MD_FLAG_PERMISSIVEEMAILAUTOLINKS Recognize e-mails as autolinks even without…
#>  5 MD_FLAG_NOINDENTEDCODEBLOCKS     Disable indented code blocks. (Only fenced …
#>  6 MD_FLAG_NOHTMLBLOCKS             Disable raw HTML blocks.                    
#>  7 MD_FLAG_NOHTMLSPANS              Disable raw HTML (inline).                  
#>  8 MD_FLAG_TABLES                   Enable tables extension.                    
#>  9 MD_FLAG_STRIKETHROUGH            Enable strikethrough extension.             
#> 10 MD_FLAG_PERMISSIVEWWWAUTOLINKS   Enable WWW autolinks (even without any sche…
#> # ℹ 17 more rows