Compare commits

...
Author SHA1 Message Date
semantic-release-bot
f92d5e8cb4 chore(release): 1.0.1 [skip ci]
## [1.0.1](https://git.kmalloc.org/squid/motorino/compare/v1.0.0...v1.0.1) (2026-11-15)

### Bug Fixes

* close the minor findings of the first review ([917683a](917683afa4))
* find code blocks with the Markdown parser and make the escape work everywhere ([4ab8f8d](4ab8f8d122))
2026-11-15 10:40:28 +01:00
3411f2c522 Merge pull request 'Close the minor findings of the first review' (#7) from fix/review-minors into main
All checks were successful
check / check (push) Successful in 29s
2026-11-15 09:40:06 +00:00
tgiachi
4ab8f8d122 fix: find code blocks with the Markdown parser and make the escape work everywhere
All checks were successful
check / check (pull_request) Successful in 30s
The hand written fence scanner took a line that starts with a code span for a
fence and missed fences in quotes and lists. The code blocks are now the ones
goldmark finds. A backslash before a placeholder is turned into a marker that
survives the Markdown and the list expansion, so it works in a code span and
next to a block name, and two backslashes are a literal one. The em dash rule
reports the line of the body, refuses padded entities and checks the header keys
by name, the feed leaves protocol-relative links alone, base_url needs a host
and a language file cannot set it.
2026-11-15 10:38:56 +01:00
tgiachi
917683afa4 fix: close the minor findings of the first review
Require refuses an empty value. Heading ids: an explicit {#id} must be sound
and unique, generated ids avoid the explicit ones and each other, and a heading
with no ASCII letters gets a fallback. Fenced code blocks keep their
placeholders and a backslash keeps one anywhere. The em dash rule covers the
header of the manifesto pages and the HTML entities. The feed makes src links
absolute, base_url is required, and the news page must be a folder index. A
category written twice in a post counts once, a block name in a news title is no
longer taken for a placeholder, and build refuses extra arguments.
2026-11-15 10:32:19 +01:00
tgiachi
8f294d92e2 docs: mention the check workflow in the README
All checks were successful
check / check (push) Successful in 29s
2026-11-15 10:27:25 +01:00
94c23442be Merge pull request 'Run make check on every push and pull request' (#5) from ci/check into main
All checks were successful
check / check (push) Successful in 1m15s
2026-11-15 09:21:44 +00:00
tgiachi
beb7698a77 ci: run make check on every push and pull request
All checks were successful
check / check (pull_request) Successful in 1m28s
2026-11-15 10:18:21 +01:00
25 changed files with 671 additions and 37 deletions

View file

@ -0,0 +1,23 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# Runs `make check` (gofmt, go vet, staticcheck and the tests) on every push to main and on every
# pull request, on the shared runner of git.kmalloc.org.
name: check
on:
push:
branches: [main]
pull_request:
jobs:
check:
runs-on: docker
steps:
- uses: https://code.forgejo.org/actions/checkout@v4
- uses: https://code.forgejo.org/actions/setup-go@v5
with:
go-version-file: go.mod
- name: Install staticcheck
run: go install honnef.co/go/tools/cmd/staticcheck@latest
- name: Run the checks
run: PATH="$PATH:$(go env GOPATH)/bin" make check

View file

@ -1,3 +1,10 @@
## [1.0.1](https://git.kmalloc.org/squid/motorino/compare/v1.0.0...v1.0.1) (2026-11-15)
### Bug Fixes
* close the minor findings of the first review ([917683a](https://git.kmalloc.org/squid/motorino/commit/917683afa491c7f0b4331757d6066d215b5f19d9))
* find code blocks with the Markdown parser and make the escape work everywhere ([4ab8f8d](https://git.kmalloc.org/squid/motorino/commit/4ab8f8d12223fca0db20ec5afee92217533d058b))
## 1.0.0 (2026-11-15)
### Features

View file

@ -48,7 +48,7 @@ Pages: `Title`, `ID` (the same in every language, it links the translations), `D
### Placeholders
`{{name}}`, with lowercase letters, digits and underscores, is replaced by a string from `site.json` or the language file. An unknown placeholder stops the build and names the file. `{{news_list}}`, `{{latest_news}}` and `{{categories_list}}` stand alone in a paragraph and expand to generated lists.
`{{name}}`, with lowercase letters, digits and underscores, is replaced by a string from `site.json` or the language file. An unknown placeholder stops the build and names the file. Fenced code blocks (found the way CommonMark finds them, in lists and quotes too) are left as written, so an article can show a placeholder there. Anywhere else a backslash before the placeholder (`\{{name}}`) keeps it as written, in a code span too; two backslashes are one literal backslash followed by the value. Code spans are otherwise substituted, which is how a page shows a value in monospace. The id of a heading is not checked against ids that a template writes itself, such as the `news` of the latest news block. `{{news_list}}`, `{{latest_news}}` and `{{categories_list}}` stand alone in a paragraph and expand to generated lists.
## Development
@ -58,7 +58,7 @@ make build a local binary in bin/motorino
make dist the release binary and its SHA-256 in dist/
```
Go code has a doc comment on every package and declaration, and is formatted with `gofmt`. Commits follow conventional commits.
Go code has a doc comment on every package and declaration, and is formatted with `gofmt`. Commits follow conventional commits. A Forgejo Actions workflow (`.forgejo/workflows/check.yml`) runs `make check` on every push to `main` and on every pull request, on the shared runner of git.kmalloc.org.
## Release

View file

@ -15,6 +15,7 @@ import (
"fmt"
"io"
"os"
"strings"
"git.kmalloc.org/squid/motorino/internal/build"
"git.kmalloc.org/squid/motorino/internal/version"
@ -59,6 +60,10 @@ func runBuild(args []string, stdout, stderr io.Writer) int {
}
return 2
}
if flags.NArg() > 0 {
fmt.Fprintf(stderr, "motorino: unexpected arguments: %s\n%s\n", strings.Join(flags.Args(), " "), usage)
return 2
}
result, err := build.Build(build.Options{SiteDir: *site, OutDir: *out, Version: version.String()})
if err != nil {
fmt.Fprintf(stderr, "motorino: %v\n", err)

View file

@ -30,8 +30,11 @@ func miniSite(t *testing.T) string {
}
for _, id := range []string{"home", "manifesto", "people", "services", "news", "history"} {
name := id
if id == "home" {
switch id {
case "home":
name = "index"
case "news":
name = "news/index"
}
files["content/en/"+name+".md"] = "Title: " + id + "\nID: " + id + "\nDescription: d\n-\nBody of " + id + ".\n"
}
@ -101,6 +104,17 @@ func TestRunUnknownCommand(t *testing.T) {
}
}
// TestRunBuildRejectsExtraArguments checks that arguments that are not flags are not ignored.
func TestRunBuildRejectsExtraArguments(t *testing.T) {
var out, errOut bytes.Buffer
if code := run([]string{"build", "somesite", "someout"}, &out, &errOut); code != 2 {
t.Errorf("run(build somesite someout) = %d, want 2", code)
}
if !strings.Contains(errOut.String(), "unexpected") {
t.Errorf("stderr = %q, want it to say what was unexpected", errOut.String())
}
}
// TestRunBuildBadFlag checks that an unknown flag is a usage error.
func TestRunBuildBadFlag(t *testing.T) {
var out, errOut bytes.Buffer

View file

@ -76,7 +76,10 @@ type Result struct {
}
// Build reads the site in opts.SiteDir and writes it to opts.OutDir. It renders
// everything in memory first, so a failure leaves the output folder untouched.
// everything in memory first, so a failure in the content leaves the output
// folder untouched. Writing the files is not atomic, and files of an earlier
// build that are no longer produced are not removed: build into a fresh folder
// when that matters, as the deploy does.
func Build(opts Options) (Result, error) {
b, err := load(opts)
if err != nil {
@ -137,6 +140,9 @@ func load(opts Options) (*builder, error) {
return nil, fmt.Errorf("content/%s: there is no page with ID %q", b.defaultLang, id)
}
}
if news := b.byKey[pageKey{"news", b.defaultLang}]; !strings.HasSuffix(news.Path, "/") {
return nil, fmt.Errorf("%s: the news page must be an index (news/index.md), so that the feed has an address next to it", news.Source)
}
if b.templates, err = readTemplates(opts.SiteDir); err != nil {
return nil, err
}

View file

@ -358,6 +358,72 @@ func TestBuildRejectsSwitchToUnknownLanguage(t *testing.T) {
}
}
// TestBuildFeedMakesImagesAbsolute checks that src links get the full address like href links.
func TestBuildFeedMakesImagesAbsolute(t *testing.T) {
files := baseFiles()
files["news/pic.md"] = "Title: Pic\nDate: 2026-11-15T12:00:00+0100\n-\n![the logo](/logo.svg)\n"
out, err := runBuild(t, files)
if err != nil {
t.Fatal(err)
}
mustContain(t, "feed", readOut(t, out, "news/feed.xml"), `src="https://kmalloc.org/logo.svg"`)
}
// TestBuildNewsPageMustBeAFolderIndex checks that the feed has a sound address.
func TestBuildNewsPageMustBeAFolderIndex(t *testing.T) {
files := baseFiles()
delete(files, "content/en/news/index.md")
files["content/en/news.md"] = page("news", "News", "", "{{news_list}}\n")
_, err := runBuild(t, files)
if err == nil || !strings.Contains(err.Error(), "content/en/news.md") || !strings.Contains(err.Error(), "index") {
t.Errorf("Build() error = %v, want it to say that content/en/news.md must be an index", err)
}
}
// TestBuildBlockNameInATitleIsNotAnError checks that a generated list that shows
// the text of a block placeholder (in a news title) is not mistaken for a
// placeholder that is not alone in its paragraph.
func TestBuildBlockNameInATitleIsNotAnError(t *testing.T) {
files := baseFiles()
files["news/tpl.md"] = "Title: Use {{latest_news}} in a page\nDate: 2026-11-15T12:00:00+0100\n-\nA note.\n"
out, err := runBuild(t, files)
if err != nil {
t.Fatalf("Build() error = %v", err)
}
mustContain(t, "home", readOut(t, out, "index.html"), "Use {{latest_news}} in a page")
}
// TestBuildPlaceholdersAndCode checks the three ways a page can use a placeholder
// next to code: a value in a code span (the pages show the GPG fingerprint that
// way), a fenced block that shows a placeholder as written, and an escaped one.
func TestBuildPlaceholdersAndCode(t *testing.T) {
files := baseFiles()
files["content/en/people.md"] = page("people", "People", "", "Value `{{lang}}` and escaped \\{{lang}}.\n\n```\n{{lang}} {{nope}}\n```\n\nAnd {{lang}}.\n")
out, err := runBuild(t, files)
if err != nil {
t.Fatalf("Build() error = %v", err)
}
got := readOut(t, out, "people.html")
mustContain(t, "people", got, "<code>en</code>", "escaped {{lang}}.", "{{lang}} {{nope}}", "And en.")
}
// TestBuildEscapedPlaceholders checks the escape with block placeholders: an
// escaped one inside a sentence or alone in a paragraph is shown, not expanded
// and not an error, and a code span can show a placeholder as written.
func TestBuildEscapedPlaceholders(t *testing.T) {
files := baseFiles()
files["content/en/people.md"] = page("people", "People", "", "See \\{{news_list}} here.\n\n\\{{latest_news}}\n\nAnd `\\{{lang}}`.\n")
out, err := runBuild(t, files)
if err != nil {
t.Fatalf("Build() error = %v", err)
}
got := readOut(t, out, "people.html")
mustContain(t, "people", got, "See {{news_list}} here.", "<p>{{latest_news}}</p>", "<code>{{lang}}</code>")
if strings.Contains(got, "<ul>") {
t.Error("an escaped block placeholder was expanded")
}
}
// TestBuildAssetsCopied checks the assets listed in site.json.
func TestBuildAssetsCopied(t *testing.T) {
out, err := runBuild(t, baseFiles())

View file

@ -3,6 +3,7 @@
package build
import (
"regexp"
"strings"
"time"
@ -10,6 +11,10 @@ import (
"git.kmalloc.org/squid/motorino/internal/site"
)
// siteRelative matches a link that starts at the root of the site, such as
// href="/people.html", but not one that is relative to the protocol (href="//host/x").
var siteRelative = regexp.MustCompile(`\b(href|src)="/([^/])`)
// feed returns the RSS 2.0 feed of the latest limit posts, which must be sorted
// newest first. Each item carries the full text with absolute links. The
// channel is described by the language l, and newsURL is the address of the
@ -21,7 +26,7 @@ func feed(base string, l site.Lang, newsURL string, posts []renderedPost, limit
var items strings.Builder
for _, p := range posts {
link := base + p.Path
text := strings.ReplaceAll(p.HTML, `href="/`, `href="`+base+`/`)
text := siteRelative.ReplaceAllString(p.HTML, `$1="`+base+`/$2`)
items.WriteString("<item>\n")
items.WriteString("<title>" + render.Escape(p.Title) + "</title>\n")
items.WriteString("<link>" + link + "</link>\n")

View file

@ -49,6 +49,16 @@ func TestFeedItems(t *testing.T) {
}
}
// TestFeedLeavesProtocolRelativeLinks checks that "//host/x" is not given the site address.
func TestFeedLeavesProtocolRelativeLinks(t *testing.T) {
items := []renderedPost{{Post: testPost("a", "2026-11-14", "A", ""), HTML: `<img src="//cdn.example/x.png"> <a href="/p.html">p</a>`}}
got := feed("https://kmalloc.org", feedLang(), "/news/", items, 5)
mustContain(t, "feed", got, `src="//cdn.example/x.png"`, `href="https://kmalloc.org/p.html"`)
if strings.Contains(got, "https://kmalloc.org//cdn") {
t.Error("a protocol-relative link got the site address")
}
}
// TestFeedWithoutPosts checks that an empty feed is still a valid channel.
func TestFeedWithoutPosts(t *testing.T) {
got := feed("https://kmalloc.org", feedLang(), "/news/", nil, 20)

View file

@ -4,6 +4,7 @@ package build
import (
"fmt"
"regexp"
"strings"
"git.kmalloc.org/squid/motorino/internal/content"
@ -61,7 +62,7 @@ func (b *builder) renderPage(p content.Page) error {
var toc []string
for _, h := range headings {
if h.Level == 2 {
toc = append(toc, fmt.Sprintf(`<li><a href="#%s">%s</a></li>`, h.ID, render.Escape(h.Text)))
toc = append(toc, fmt.Sprintf(`<li><a href="#%s">%s</a></li>`, render.EscapeAttr(h.ID), render.Escape(h.Text)))
}
}
ctx["manifesto_toc"] = strings.Join(toc, "\n")
@ -80,7 +81,7 @@ func (b *builder) renderPage(p content.Page) error {
// markdown replaces the placeholders of a source, converts it to HTML and
// expands the generated lists. The name is used in error messages.
func (b *builder) markdown(name, source, lang string, ctx map[string]string) (string, []mdx.Heading, error) {
text, err := render.Substitute(name, source, ctx)
text, err := mdx.SubstituteMarkdown(name, source, ctx)
if err != nil {
return "", nil, err
}
@ -88,26 +89,41 @@ func (b *builder) markdown(name, source, lang string, ctx map[string]string) (st
if err != nil {
return "", nil, fmt.Errorf("%s: %w", name, err)
}
html, err = b.expandBlocks(name, html, lang, ctx)
return html, headings, err
if html, err = b.expandBlocks(name, html, lang, ctx); err != nil {
return "", nil, err
}
for i := range headings {
headings[i].Text = render.Unescape(headings[i].Text)
}
return render.Unescape(html), headings, nil
}
// codeElement matches a code element, where a placeholder is shown as written.
var codeElement = regexp.MustCompile(`(?s)<code[^>]*>.*?</code>`)
// expandBlocks replaces each paragraph that holds only a block placeholder by
// the generated HTML. A block placeholder inside other text is an error.
// the generated HTML. A block placeholder inside other text is an error. All
// blocks are checked before any is expanded, so the text of a generated list
// (a news title may say {{latest_news}}) is never taken for a placeholder.
func (b *builder) expandBlocks(name, html, lang string, ctx map[string]string) (string, error) {
outside := codeElement.ReplaceAllString(html, "")
for _, block := range blockNames {
marker := "<p>{{" + block + "}}</p>"
if strings.Contains(html, marker) {
generated, err := b.block(block, lang, ctx)
if err != nil {
return "", err
}
html = strings.ReplaceAll(html, marker, generated)
}
if strings.Contains(html, "{{"+block+"}}") {
token := "{{" + block + "}}"
if strings.Count(outside, token) != strings.Count(outside, "<p>"+token+"</p>") {
return "", fmt.Errorf("%s: {{%s}} must stand alone in a paragraph", name, block)
}
}
for _, block := range blockNames {
marker := "<p>{{" + block + "}}</p>"
if !strings.Contains(html, marker) {
continue
}
generated, err := b.block(block, lang, ctx)
if err != nil {
return "", err
}
html = strings.ReplaceAll(html, marker, generated)
}
return html, nil
}

View file

@ -142,6 +142,14 @@ func loadPage(file, source, defaultLang, lang, rel string) (Page, error) {
if err := mdx.RejectEmDash(source, string(data)); err != nil {
return Page{}, err
}
} else {
// The body of the manifesto is the owner's text and is rewritten when the
// page is built; the keys of the header are not.
for _, key := range []string{"Title", "Description", "Note"} {
if mdx.HasEmDash(h.Get(key)) {
return Page{}, fmt.Errorf("%s: the key %s has an em dash, please remove it", source, key)
}
}
}
switches, err := parseSwitch(source, h.Get("Switch"))
if err != nil {

View file

@ -107,6 +107,21 @@ func TestLoadPagesManifestoKeepsEmDash(t *testing.T) {
}
}
// TestLoadPagesManifestoHeaderEmDash checks that the exemption of the manifesto
// body does not cover the title, the description and the note.
func TestLoadPagesManifestoHeaderEmDash(t *testing.T) {
for _, field := range []string{"Title", "Description", "Note"} {
head := map[string]string{"Title": "M", "Description": "d", "Note": "n"}
head[field] = "words \u2014 more"
file := "Title: " + head["Title"] + "\nID: manifesto\nDescription: " + head["Description"] + "\nLayout: manifesto\nNote: " + head["Note"] + "\n-\n## 01 \u2014 Body\n"
dir := writeFiles(t, map[string]string{"content/en/m.md": file})
_, err := LoadPages(dir, []string{"en"})
if err == nil || !strings.Contains(err.Error(), "content/en/m.md") || !strings.Contains(err.Error(), field) {
t.Errorf("%s with an em dash: LoadPages() error = %v, want it to name content/en/m.md and %s", field, err, field)
}
}
}
// TestLoadPagesDuplicateID checks that two pages cannot share an ID in a language.
func TestLoadPagesDuplicateID(t *testing.T) {
dir := writeFiles(t, map[string]string{

View file

@ -136,6 +136,7 @@ func parseDate(source, value string) (time.Time, error) {
// parseCategories splits a comma separated list into categories.
func parseCategories(source, value string) ([]Category, error) {
var out []Category
seen := map[string]bool{}
for _, name := range strings.Split(value, ",") {
name = strings.TrimSpace(name)
if name == "" {
@ -145,6 +146,10 @@ func parseCategories(source, value string) ([]Category, error) {
if err != nil {
return nil, fmt.Errorf("%s: %w", source, err)
}
if seen[slug] {
continue
}
seen[slug] = true
out = append(out, Category{Name: name, Slug: slug})
}
return out, nil
@ -177,7 +182,8 @@ func checkCategories(posts []Post) error {
return nil
}
// markup matches the Markdown that is dropped from a fallback description.
// markdownLink, markdownMark and sentenceEnd pick the first sentence out of a
// body and drop its Markdown markup, for a description that was not written.
var (
markdownLink = regexp.MustCompile(`\[([^\]]*)\]\([^)]*\)`)
markdownMark = regexp.MustCompile("[*_`]")

View file

@ -171,6 +171,18 @@ func TestLoadPostsCategoryCollision(t *testing.T) {
}
}
// TestLoadPostsRepeatedCategory checks that a category written twice counts once.
func TestLoadPostsRepeatedCategory(t *testing.T) {
dir := writeFiles(t, map[string]string{"news/a.md": "Title: A\nDate: 2026-01-01T10:00:00+0100\nCategories: News, Linux, news, News\n-\nx\n"})
posts, err := LoadPosts(dir)
if err != nil {
t.Fatal(err)
}
if len(posts[0].Categories) != 2 || posts[0].Categories[0].Name != "News" || posts[0].Categories[1].Name != "Linux" {
t.Errorf("Categories = %+v, want News and Linux once each", posts[0].Categories)
}
}
// TestLoadPostsNoNewsFolder checks that a site without news is fine.
func TestLoadPostsNoNewsFolder(t *testing.T) {
posts, err := LoadPosts(t.TempDir())

View file

@ -78,12 +78,16 @@ func (h Header) Has(key string) bool {
return ok
}
// Require returns an error naming name and the first missing key of keys.
// Require returns an error naming name and the first key of keys that is
// missing or has an empty value.
func (h Header) Require(name string, keys ...string) error {
for _, k := range keys {
if !h.Has(k) {
return fmt.Errorf("%s: missing key %q", name, k)
}
if h.Get(k) == "" {
return fmt.Errorf("%s: the key %q is empty", name, k)
}
}
return nil
}

View file

@ -124,6 +124,21 @@ func TestRequire(t *testing.T) {
}
}
// TestRequireNeedsAValue checks that a key with an empty value counts as missing.
func TestRequireNeedsAValue(t *testing.T) {
h, _, err := Parse("file.md", []byte("Title:\nID: x\n-\n"))
if err != nil {
t.Fatal(err)
}
err = h.Require("file.md", "Title")
if err == nil || !strings.Contains(err.Error(), "Title") || !strings.Contains(err.Error(), "empty") {
t.Errorf("Require(Title) = %v, want an error saying that Title is empty", err)
}
if err := h.Require("file.md", "ID"); err != nil {
t.Errorf("Require(ID) = %v", err)
}
}
// TestOnly checks that a key outside the allowed list is rejected.
func TestOnly(t *testing.T) {
h, _, err := Parse("file.md", []byte("Title: T\nTitolo: T\n-\n"))

View file

@ -38,17 +38,30 @@ func NoEmDash(md string) (string, error) {
for _, p := range parentheticals {
md = strings.ReplaceAll(md, p.from, p.to)
}
if err := RejectEmDash("manifesto source", md); err != nil {
return "", err
for i, line := range strings.Split(md, "\n") {
if HasEmDash(line) {
return "", fmt.Errorf("line %d of the body: no em dashes, please", i+1)
}
}
return md, nil
}
// emDashEntity matches the HTML entities of the em dash, which would put it
// on the page as well.
var emDashEntity = regexp.MustCompile(`(?i)&mdash;|&#0*8212;|&#x0*2014;`)
// HasEmDash reports whether text has an em dash, written as the character or as
// an HTML entity.
func HasEmDash(text string) bool {
return strings.Contains(text, emDash) || emDashEntity.MatchString(text)
}
// RejectEmDash returns an error naming the file and the line of the first em
// dash in text, or nil when there is none.
// dash in text, written as the character or as an HTML entity, or nil when
// there is none.
func RejectEmDash(name, text string) error {
for i, line := range strings.Split(text, "\n") {
if strings.Contains(line, emDash) {
if HasEmDash(line) {
return fmt.Errorf("%s:%d: no em dashes, please", name, i+1)
}
}

View file

@ -45,6 +45,36 @@ func TestNoEmDashLeftover(t *testing.T) {
}
}
// TestRejectEmDashEntities checks that the entities of the em dash are refused too.
func TestRejectEmDashEntities(t *testing.T) {
for _, in := range []string{"a &mdash; b", "a &MDASH; b", "a &#8212; b", "a &#x2014; b", "a &#X2014; b", "a &#08212; b", "a &#x02014; b"} {
if err := RejectEmDash("f.md", in); err == nil {
t.Errorf("RejectEmDash(%q) = nil, want an error", in)
}
}
if err := RejectEmDash("f.md", "a &ndash; b &amp; c"); err != nil {
t.Errorf("RejectEmDash(ndash) = %v", err)
}
}
// TestNoEmDashErrorSaysBody checks that a leftover is reported as a line of the
// body, which is how the caller counts it.
func TestNoEmDashErrorSaysBody(t *testing.T) {
_, err := NoEmDash("fine\nsome words \u2014 and then more\n")
if err == nil || !strings.Contains(err.Error(), "line 2 of the body") {
t.Errorf("NoEmDash() error = %v, want it to say line 2 of the body", err)
}
}
// TestHasEmDash checks the test that the header keys use.
func TestHasEmDash(t *testing.T) {
for in, want := range map[string]bool{"plain": false, "a \u2014 b": true, "a &mdash; b": true, "a &ndash; b": false, "": false} {
if got := HasEmDash(in); got != want {
t.Errorf("HasEmDash(%q) = %v, want %v", in, got, want)
}
}
}
// TestRejectEmDash checks that the error names the file and the line.
func TestRejectEmDash(t *testing.T) {
if err := RejectEmDash("content/en/x.md", "clean\ntext\n"); err != nil {

View file

@ -6,6 +6,7 @@ package mdx
import (
"bytes"
"fmt"
"regexp"
"strconv"
"strings"
@ -42,9 +43,12 @@ func Render(md string) (string, []Heading, error) {
src := []byte(md)
doc := engine.Parser().Parse(text.NewReader(src))
used, err := explicitIDs(doc)
if err != nil {
return "", nil, err
}
var headings []Heading
used := make(map[string]int)
err := ast.Walk(doc, func(n ast.Node, entering bool) (ast.WalkStatus, error) {
err = ast.Walk(doc, func(n ast.Node, entering bool) (ast.WalkStatus, error) {
h, ok := n.(*ast.Heading)
if !ok || !entering {
return ast.WalkContinue, nil
@ -53,9 +57,12 @@ func Render(md string) (string, []Heading, error) {
id := ""
if explicit, ok := h.AttributeString("id"); ok {
id = attributeText(explicit)
used[id]++
} else if h.Level >= 2 {
id = uniqueID(Slug(plain), used)
base := Slug(plain)
if base == "" {
base = "section"
}
id = uniqueID(base, used)
h.SetAttributeString("id", []byte(id))
}
headings = append(headings, Heading{Level: h.Level, ID: id, Text: plain})
@ -83,13 +90,47 @@ func attributeText(value any) string {
return ""
}
// uniqueID returns id, or id with a numeric suffix when it was already used.
func uniqueID(id string, used map[string]int) string {
used[id]++
if used[id] == 1 {
return id
// explicitID is what an id set with {#id} may look like: it ends up in
// attributes and in addresses.
var explicitID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.:-]*$`)
// explicitIDs returns the ids that headings set for themselves with {#id}. A
// bad id or an id used by two headings is an error. Generated ids avoid these,
// whatever the order of the headings.
func explicitIDs(doc ast.Node) (map[string]bool, error) {
used := map[string]bool{}
err := ast.Walk(doc, func(n ast.Node, entering bool) (ast.WalkStatus, error) {
h, ok := n.(*ast.Heading)
if !ok || !entering {
return ast.WalkContinue, nil
}
value, ok := h.AttributeString("id")
if !ok {
return ast.WalkSkipChildren, nil
}
id := attributeText(value)
if !explicitID.MatchString(id) {
return ast.WalkStop, fmt.Errorf("the heading id %q may have only letters, digits and _ . : -", id)
}
if used[id] {
return ast.WalkStop, fmt.Errorf("the heading id %q is used by two headings", id)
}
used[id] = true
return ast.WalkSkipChildren, nil
})
return used, err
}
// uniqueID returns base, or base with -2, -3 and so on until the id is not
// used, and marks it as used.
func uniqueID(base string, used map[string]bool) string {
id, n := base, 1
for used[id] {
n++
id = base + "-" + strconv.Itoa(n)
}
return id + "-" + strconv.Itoa(used[id])
used[id] = true
return id
}
// plainText returns the text of a node and its children without any markup.

View file

@ -5,6 +5,8 @@ package mdx
import (
"strings"
"testing"
"git.kmalloc.org/squid/motorino/internal/render"
)
// TestRenderParagraphAndList checks the basic blocks and inline markup.
@ -99,6 +101,146 @@ func TestRenderExplicitHeadingID(t *testing.T) {
}
}
// TestRenderRejectsRepeatedExplicitID checks that two headings cannot ask for the same id.
func TestRenderRejectsRepeatedExplicitID(t *testing.T) {
for _, src := range []string{"## Foo {#a}\n\n## Bar {#a}\n"} {
_, _, err := Render(src)
if err == nil || !strings.Contains(err.Error(), "id") {
t.Errorf("Render(%q) error = %v, want an error about the repeated id", src, err)
}
}
}
// TestRenderGeneratedIDAvoidsLaterExplicit checks that a generated id never takes
// one that a later heading asks for by name, in either order of the two.
func TestRenderGeneratedIDAvoidsLaterExplicit(t *testing.T) {
_, headings, err := Render("## Foo\n\n## Bar {#foo-2}\n\n## Foo\n")
if err != nil {
t.Fatal(err)
}
var ids []string
for _, h := range headings {
ids = append(ids, h.ID)
}
if got, want := strings.Join(ids, ","), "foo,foo-2,foo-3"; got != want {
t.Errorf("ids = %s, want %s", got, want)
}
}
// TestRenderSuffixClash checks that a numbered suffix never repeats an id.
func TestRenderSuffixClash(t *testing.T) {
_, headings, err := Render("## A\n\n## A\n\n## A 2\n")
if err != nil {
t.Fatal(err)
}
seen := map[string]bool{}
for _, h := range headings {
if seen[h.ID] {
t.Errorf("id %q is used twice: %+v", h.ID, headings)
}
seen[h.ID] = true
}
}
// TestRenderEmptySlugFallback checks that a heading with no ASCII letters or digits still gets an id.
func TestRenderEmptySlugFallback(t *testing.T) {
html, headings, err := Render("## \u4f60\u597d\n\n## \u4e16\u754c\n")
if err != nil {
t.Fatal(err)
}
if strings.Contains(html, `id=""`) {
t.Errorf("html %q has an empty id", html)
}
if len(headings) != 2 || headings[0].ID != "section" || headings[1].ID != "section-2" {
t.Errorf("headings = %+v, want section and section-2", headings)
}
}
// TestRenderBadExplicitIDCannotInject checks that a malformed {#id} never ends
// up as an id with quotes, spaces or angle brackets, whether goldmark leaves it
// as text or the id check refuses it; and that sound ids pass.
func TestRenderBadExplicitIDCannotInject(t *testing.T) {
for _, src := range []string{"## A {#a\"b}\n", "## A {#a b}\n", "## A {#<x>}\n"} {
html, headings, err := Render(src)
if err != nil {
continue
}
for _, h := range headings {
if !explicitID.MatchString(h.ID) {
t.Errorf("Render(%q) gave the id %q", src, h.ID)
}
}
if strings.Contains(html, `id="a"b`) || strings.Contains(html, `id="a b`) {
t.Errorf("Render(%q) = %q, the id broke out of its attribute", src, html)
}
}
if _, _, err := Render("## A {#v1.2}\n\n## B {#01}\n\n## C {#a_b:c-d}\n"); err != nil {
t.Errorf("Render(valid ids) error = %v", err)
}
}
// TestSubstituteMarkdownFences checks that placeholders in fenced code blocks
// stay as written, wherever CommonMark finds the block, and that an unknown one
// there is no error.
func TestSubstituteMarkdownFences(t *testing.T) {
ctx := map[string]string{"lang": "en"}
tests := []struct{ name, src, want string }{
{"backtick fence", "A {{lang}}\n\n```\n{{lang}} {{nope}}\n```\n\nB {{lang}}\n", "A en\n\n```\n{{lang}} {{nope}}\n```\n\nB en\n"},
{"tilde fence with info", "~~~go\n{{lang}}\n~~~\n{{lang}}\n", "~~~go\n{{lang}}\n~~~\nen\n"},
{"code span is substituted", "Value `{{lang}}`.\n", "Value `en`.\n"},
{"fence in a quote", "> ```\n> {{nope}}\n> ```\n\n{{lang}}\n", "> ```\n> {{nope}}\n> ```\n\nen\n"},
{"fence in a list item", "1. item\n\n ```\n {{nope}}\n ```\n\n{{lang}}\n", "1. item\n\n ```\n {{nope}}\n ```\n\nen\n"},
{"code span at the start of a line is not a fence", "```x``` and {{lang}}\n\nthen {{lang}}\n", "```x``` and en\n\nthen en\n"},
{"unclosed fence runs to the end", "{{lang}}\n\n```\n{{nope}}\n{{lang}}\n", "en\n\n```\n{{nope}}\n{{lang}}\n"},
{"a closer indented four spaces does not close", "```\n ```\n{{nope}}\n```\n{{lang}}\n", "```\n ```\n{{nope}}\n```\nen\n"},
{"a closer may be longer", "```\n{{nope}}\n`````\n{{lang}}\n", "```\n{{nope}}\n`````\nen\n"},
{"CRLF line endings", "{{lang}}\r\n\r\n```\r\n{{nope}}\r\n```\r\n{{lang}}\r\n", "en\r\n\r\n```\r\n{{nope}}\r\n```\r\nen\r\n"},
{"empty fence", "```\n```\n{{lang}}\n", "```\n```\nen\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := SubstituteMarkdown("page.md", tt.src, ctx)
if err != nil {
t.Fatalf("SubstituteMarkdown() error = %v", err)
}
if got != tt.want {
t.Errorf("SubstituteMarkdown() = %q, want %q", got, tt.want)
}
})
}
}
// TestSubstituteMarkdownUnknownOutsideFence checks that an unknown placeholder
// outside a fence still stops the build and names the file.
func TestSubstituteMarkdownUnknownOutsideFence(t *testing.T) {
_, err := SubstituteMarkdown("news/x.md", "```\n{{a}}\n```\n\nbad {{nope}}\n", map[string]string{})
if err == nil || !strings.Contains(err.Error(), "news/x.md") || !strings.Contains(err.Error(), "nope") {
t.Errorf("SubstituteMarkdown() error = %v, want it to name news/x.md and nope", err)
}
}
// TestSubstituteMarkdownEscapeEverywhere checks that a backslash keeps a
// placeholder as written in text, in a code span and next to a block name, and
// that the written form comes back through render.Unescape.
func TestSubstituteMarkdownEscapeEverywhere(t *testing.T) {
src := "Text \\{{lang}} and `\\{{lang}}` and \\{{news_list}}.\n"
got, err := SubstituteMarkdown("page.md", src, map[string]string{"lang": "en"})
if err != nil {
t.Fatal(err)
}
html, _, err := Render(got)
if err != nil {
t.Fatal(err)
}
if strings.Contains(html, "{{") {
t.Errorf("html %q still has braces before Unescape", html)
}
want := "<p>Text {{lang}} and <code>{{lang}}</code> and {{news_list}}.</p>\n"
if out := render.Unescape(html); out != want {
t.Errorf("Unescape(html) = %q, want %q", out, want)
}
}
// TestRenderHeadingTextIsPlain checks that the heading text drops the markup.
func TestRenderHeadingTextIsPlain(t *testing.T) {
_, headings, err := Render("## Free *your* mind\n")

Some files were not shown because too many files have changed in this diff Show more