Hello, I’m playing with Pollen and I discovered that X-expressions that come from markdown markup in markdown authoring mode (with .pmd extension) are not expanded using tag functions defined in pollen.rkt.
For example, havning markdown file like this:
# Hello
Will be transformed into X-expression (root (h1 "Hello))
without applying h1 defined in pollen.rkt to "Hello". When I use Pollen markup instead, the function is applied correctly.
In pollen/markdown , (or pmd files) the Markdown parsing happens after all the tag functions are evaluated.
I understand this is a technical limitation, but from the user perspective I would find useful and logical if markdown markup tags were evaluated with functions as well. My use case is that I frequently jot down ideas in note taking applications which frequently use Markdown, like Bear. When the text is ready for publishing, I’d like to simply drop the markdown source into pmd file and generate nice looking PDFs, without having to convert document to Pollen markdown.
So I’m thinking if there is any reasonable way to get around this behavior?
It’s helpful insight into how Markdown/Pollen parsing works, but it is ultimately concerned about that tag functions are applied before Markdown is parsed, which means that tag function values are parsed as Markdown.
What I’d like to do instead is to “hook into” HTML-tagged xexprs generated from native Markdown markup to further transform it into something else, when current poly target is something else than HTML (for example a TeX).
So in case of TeX target, # Hello markdown is parsed as (h1 "Hello"). I’d like to apply tag function (define (h1 . elements) ...) defined in pollen.rkt which would further transform it into "\section{Hello}". Given the current implementation, this is not possible because tag functions are evaluated before Markdown parsing.
I guess that my only option is to abandon tag functions and transform the whole X-expression tree inside my template file for TeX target. But it seems to me that its much more complicated to traverse entire X-expression tree “top-bottom” (from root) instead of “bottom-up” (from leaf nodes) order on which tag functions are evaluated.
I could probably utilize Racket macro expansion to transform X-expression bottom up? I’ll need to dive deeper into Racket…
My lack of interest in Markdown is explicitly documented. I’m sure what you seek is possible — it sounds like you want to evaluate the file in two separate passes. So imagine two Pollen projects chained together: one that transforms foo.ltx.pm.md into foo.ltx.pm, and then a second that transforms foo.ltx.pm into foo.ltx. Details left to the reader.
Another approach would be to forgo the .poly functionality altogether. Just write sources as foo.html.pm. This is probably the most “with the grain” approach because Markdown is designed as shorthand for HTML specifically.
Then transform the HTML X-expression in each source’s doc to LaTeX with a separate Racket program (decode could be useful here).
I’ll share my solution I’ve finally settled with for future readers. See my Pollen repo for the code.
I implemented a decoding function with decode-elements from pollen/decode, which decodes a small subset of Markdown markup into X-expressions. This function is hooked into root tag function (see Decoding markup with a root tag function).
One way to think about it is that decode-paragraphs brings Markdown-style paragraphs to Pollen markup. Working out from this idea, i’ve added support for # Headings and --- for horizontal rules. Then I use #:string-proc on decode-elements to decode inline syntax for *emphases*. This setup allows me to use some basic Markdown syntax I’m interested in, alongside with Pollen tax for more advanced stuff.
Then in TeX template, I use decode-elements again, but this time to actually encode X-expression to TeX string. It eventually makes more sense than using tag functions, because I have all TeX encoding logic nicely grouped together, rather than spread across multiple functions.
It seems to me that tag functions work well for decoding DSLs in specific regions of pollen source file into X-expressions, not the other way around. For example, in custom bullet-list tag, we might prefer to decode paragraphs as li instead of p. Specifics of converting X-expression to target formats are left to the template files.