fix(gemini): list items, spaces in addresses, wide tables, per post language, empty lists, layout images
All checks were successful
check / check (pull_request) Successful in 30s

- a list item with no text (only an image or a link without a label) has no empty bullet,
  its links are still written
- paragraphs and table cells of a list item do not stick together, and a preformatted
  block in an item stays a block after its bullet
- spaces in the address of a link line are written as %20
- a wide table with an empty first cell has no empty heading, a wide table with only a
  header or only one column is written as bullets, and the header is not lost
- a post in another language than the news has its own line in .meta, before the lines
  of the folders: the server uses the first line that matches a document (checked with
  Agate 3.3.25)
- an empty {{news_archive}} or {{categories_list}} leaves nothing, with its heading, so
  a site without news says so once
- images of the layout (class logo, mark or decorative, role presentation or none) leave
  no link line and the file is not copied
- the command test names files instead of counting them, and -clean and the warning
  about old files work for the gemini command too

Refs #10
This commit is contained in:
tgiachi 2026-11-16 12:53:37 +01:00
commit 3764470e9b
6 changed files with 325 additions and 37 deletions

View file

@ -10,7 +10,7 @@ Licence: GPL-3.0-or-later, see `LICENSE`.
motorino build -site DIR -out DIR build the site in DIR into the output folder (defaults: . and public) motorino build -site DIR -out DIR build the site in DIR into the output folder (defaults: . and public)
motorino build -drafts ... the same, with the drafts and the posts dated in the future, for a preview motorino build -drafts ... the same, with the drafts and the posts dated in the future, for a preview
motorino gemini -site DIR -out DIR write the same site as gemtext, for a Gemini server (defaults: . and capsule) motorino gemini -site DIR -out DIR write the same site as gemtext, for a Gemini server (defaults: . and capsule)
motorino gemini -lang CODE ... the language at the root of the capsule; -drafts works as for build motorino gemini -lang CODE ... the language at the root of the capsule; -drafts and -clean work as for build
motorino build -clean ... empty the output folder first, if it is new, empty or holds .motorino-out motorino build -clean ... empty the output folder first, if it is new, empty or holds .motorino-out
motorino new -site DIR "A title" write news/a-title.md, a draft with the header ready, and print its path motorino new -site DIR "A title" write news/a-title.md, a draft with the header ready, and print its path
motorino version print the version motorino version print the version
@ -87,9 +87,9 @@ Pages: `Title`, `ID` (the same in every language, it links the translations), `D
`motorino gemini` reads the same site and writes gemtext: the pages of each language, the news once and a `.meta` file for the server. It does not read the templates, the style or the layouts: a page is its title, its text and a few navigation links. `motorino gemini` reads the same site and writes gemtext: the pages of each language, the news once and a `.meta` file for the server. It does not read the templates, the style or the layouts: a page is its title, its text and a few navigation links.
- **Language at the root.** The language served by default sits at the root and the others under `/<code>/`, the default language of the site included. It is set with `-lang`, else `gemini_language` in `site.json`, else the default language of the site. The server must be started with the same default (Agate: `--lang`). It must be a language that has a home page. - **Language at the root.** The language served by default sits at the root and the others under `/<code>/`, the default language of the site included. It is set with `-lang`, else `gemini_language` in `site.json`, else the default language of the site. The server must be started with the same default (Agate: `--lang`). It must be a language that has a home page.
- **`.meta`.** The server reads the language of each document from `<out>/.meta`, which it never serves and no page links to. For every language other than the root one there are two lines, `en/*:;lang=en` and `en/**/*:;lang=en`, because a `*` does not cross folders, and the same two lines for `news/` when the news are not in the root language. It is for Agate run with `--central-conf`. A post that names another `Language` than the news is not given its own line. - **`.meta`.** The server reads the language of each document from `<out>/.meta`, which it never serves and no page links to. For every language other than the root one there are two lines, `en/*:;lang=en` and `en/**/*:;lang=en`, because a `*` does not cross folders, and the same two lines for `news/` when the news are not in the root language. It is for Agate run with `--central-conf`. A post that names another `Language` than the news has a line of its own, with the path of its file (`news/2026/11/slug.gmi:;lang=it`). The server uses the first line that matches a document, so these come first, then the folders.
- **News.** They exist once, under `/news/`, as in the website: the index, the posts, the categories, the archives and `news/feed.gmi`, a list of `=> /news/2026/11/post.gmi 2026-11-14 Title` lines, newest first, which feed readers subscribe to. Every page ends with the navigation of its language, which has the news. `{{news_blog}}` lists all the posts, without pages of ten. - **News.** They exist once, under `/news/`, as in the website: the index, the posts, the categories, the archives and `news/feed.gmi`, a list of `=> /news/2026/11/post.gmi 2026-11-14 Title` lines, newest first, which feed readers subscribe to. Every page ends with the navigation of its language, which has the news. `{{news_blog}}` lists all the posts, without pages of ten.
- **Conversion.** A heading is a heading (three levels), a paragraph is a line, the links of a paragraph, list, quote or table follow it on lines of their own, a list is bullets (a list of only links is link lines), a quote or a note is a quote, code is a preformatted block, a table is a preformatted block of aligned columns, or, when it is wider than 80 columns, a heading for each row with a bullet for each cell, an image with an alt text is a link line, and emphasis is dropped. A page has one first level heading: its title; a first heading that says the title is dropped and the others go one level down. The blocks (`{{news_list}}`, `{{latest_news}}` and the others) become generated lines, with the texts of the language file. - **Conversion.** A heading is a heading (three levels), a paragraph is a line, the links of a paragraph, list, quote or table follow it on lines of their own, a list is bullets (a list of only links is link lines), a quote or a note is a quote, code is a preformatted block, a table is a preformatted block of aligned columns, or, when it is wider than 80 columns, a heading for each row (its first cell) with a bullet for each other cell named by its column, a bullet for each row when there is no header or only one column, an image with an alt text is a link line, unless it belongs to the layout (it has the class `logo`, `mark` or `decorative`, or the role `presentation` or `none`: then nothing is written and the file is not copied), and emphasis is dropped. A list item with no text, only an image or a link, has no bullet; a preformatted block in an item follows its bullet. A page has one first level heading: its title; a first heading that says the title is dropped and the others go one level down. The blocks (`{{news_list}}`, `{{latest_news}}` and the others) become generated lines, with the texts of the language file.
- **Addresses.** An address of the site becomes the address of its page in the capsule (`/people.html` is `/en/people.gmi`), and a fragment is dropped. A file listed in `assets` is copied when a page points to it, and only then. `http`, `https`, `mailto`, `gemini`, `xmpp`, `irc`, `ircs`, `matrix`, `bitcoin`, `magnet`, `tel` and `sms` links stay as they are. A path the website serves by other means (`/~squid/`) becomes a link to the website when it starts with one of the comma separated prefixes of `gemini_web_paths` in `site.json`. Any other address that leads nowhere stops the build and names the page. - **Addresses.** An address of the site becomes the address of its page in the capsule (`/people.html` is `/en/people.gmi`), and a fragment is dropped. A file listed in `assets` is copied when a page points to it, and only then. `http`, `https`, `mailto`, `gemini`, `xmpp`, `irc`, `ircs`, `matrix`, `bitcoin`, `magnet`, `tel` and `sms` links stay as they are. A path the website serves by other means (`/~squid/`) becomes a link to the website when it starts with one of the comma separated prefixes of `gemini_web_paths` in `site.json`. Any other address that leads nowhere stops the build and names the page.
## Development ## Development

View file

@ -143,12 +143,21 @@ func (g *gemini) rootLanguage(option string) (string, error) {
return code, nil return code, nil
} }
// meta returns the content of the .meta file: for every language other than // meta returns the content of the .meta file. The server gives a document the
// the one at the root, and for the news when they are in another language, the // language of the first line that matches it, so the lines come from the most
// lines that give the language of the documents. A glob does not cross folders, // specific to the most general:
// so each folder needs a line for itself and one for what is below it. //
// - a line for the file of every post written in another language than the news;
// - for every language other than the one at the root, and for the news when they
// are in another language, two lines for the folder. A glob does not cross
// folders, so each folder has a line for itself and one for what is below it.
func (g *gemini) meta() string { func (g *gemini) meta() string {
var out strings.Builder var out strings.Builder
for _, p := range g.posts {
if p.Language != g.newsLang {
fmt.Fprintf(&out, "%s:;lang=%s\n", gemFile(gemPostAddress(p)), p.Language)
}
}
line := func(folder, lang string) { line := func(folder, lang string) {
fmt.Fprintf(&out, "%s/*:;lang=%s\n%s/**/*:;lang=%s\n", folder, lang, folder, lang) fmt.Fprintf(&out, "%s/*:;lang=%s\n%s/**/*:;lang=%s\n", folder, lang, folder, lang)
} }
@ -307,7 +316,9 @@ func (g *gemini) convert(name, html, base, lang string) (string, error) {
} }
// markdown replaces the placeholders of a source, converts it to HTML and puts // markdown replaces the placeholders of a source, converts it to HTML and puts
// the element <gemblock> where a generated list goes, for convert to expand. // the element <gemblock> where a generated list goes, for convert to expand. A
// list with nothing in it leaves no element, and the heading of an empty list of
// categories or of the archive goes with it.
func (g *gemini) markdown(name, source string, ctx map[string]string) (string, error) { func (g *gemini) markdown(name, source string, ctx map[string]string) (string, error) {
text, err := mdx.SubstituteMarkdown(name, source, ctx) text, err := mdx.SubstituteMarkdown(name, source, ctx)
if err != nil { if err != nil {
@ -317,7 +328,12 @@ func (g *gemini) markdown(name, source string, ctx map[string]string) (string, e
if err != nil { if err != nil {
return "", fmt.Errorf("%s: %w", name, err) return "", fmt.Errorf("%s: %w", name, err)
} }
lang := ctx["lang"]
html, err = replaceBlocks(name, html, func(block string) (string, error) { html, err = replaceBlocks(name, html, func(block string) (string, error) {
text, err := g.block(block, lang)
if err != nil || text == "" {
return "", err
}
return `<gemblock name="` + block + `"></gemblock>`, nil return `<gemblock name="` + block + `"></gemblock>`, nil
}) })
if err != nil { if err != nil {

View file

@ -44,7 +44,7 @@ func (g *gemini) block(name, lang string) (string, error) {
} }
return strings.Join(lines, "\n"), nil return strings.Join(lines, "\n"), nil
default: default:
return g.archiveLines(lang, empty), nil return g.archiveLines(lang), nil
} }
} }
@ -75,11 +75,12 @@ func (g *gemini) latestNews(lang string) string {
return strings.Join(append(lines, gemtext.Link(gemNews, label)), "\n") return strings.Join(append(lines, gemtext.Link(gemNews, label)), "\n")
} }
// archiveLines returns the years that have posts, each followed by its months. // archiveLines returns the years that have posts, each followed by its months, or
func (g *gemini) archiveLines(lang, empty string) string { // nothing when there are none: the list of the news says there is no news.
func (g *gemini) archiveLines(lang string) string {
years := groupArchive(g.posts) years := groupArchive(g.posts)
if len(years) == 0 { if len(years) == 0 {
return empty return ""
} }
var lines []string var lines []string
for _, y := range years { for _, y := range years {

View file

@ -34,6 +34,10 @@ func gemSite(extra map[string]string) map[string]string {
"news/shop.md": "Title: A shop\nAuthor: squid\nDate: 2026-11-14T12:00:00+0100\nCategories: FLUG\nDescription: About the shop.\nMastodon: https://social.kmalloc.org/@squid/115\n-\nThere is a shop. Mail me at squid@kmalloc.org or visit [the site](https://kmalloc.org/).\n", "news/shop.md": "Title: A shop\nAuthor: squid\nDate: 2026-11-14T12:00:00+0100\nCategories: FLUG\nDescription: About the shop.\nMastodon: https://social.kmalloc.org/@squid/115\n-\nThere is a shop. Mail me at squid@kmalloc.org or visit [the site](https://kmalloc.org/).\n",
}) })
for k, v := range extra { for k, v := range extra {
if v == "" {
delete(files, k)
continue
}
files[k] = v files[k] = v
} }
return files return files
@ -397,3 +401,123 @@ func TestGeminiTranslationNote(t *testing.T) {
mustContain(t, "es/manifesto.gmi", readOut(t, out, "es/manifesto.gmi"), mustContain(t, "es/manifesto.gmi", readOut(t, out, "es/manifesto.gmi"),
"Traduccion automatica, avisa si ves un error: avisar.\n=> https://git.kmalloc.org/squid/kmalloc.org/issues avisar\n") "Traduccion automatica, avisa si ves un error: avisar.\n=> https://git.kmalloc.org/squid/kmalloc.org/issues avisar\n")
} }
// TestGeminiMetaForPostsInAnotherLanguage checks that a post with its own Language has a line of its own in
// .meta, with the real path of its file and before the lines of the folders: the server uses the first line that
// matches a file, and the line of the news folder would hide a line that comes after it.
func TestGeminiMetaForPostsInAnotherLanguage(t *testing.T) {
files := gemSite(map[string]string{
"news/ciao.md": "Title: Ciao\nDate: 2026-11-05T09:00:00+0100\nLanguage: it\n-\nUn testo.\n",
"news/hola.md": "Title: Hola\nDate: 2026-11-04T09:00:00+0100\nLanguage: es\n-\nUn texto.\n",
"news/same.md": "Title: Same\nDate: 2026-11-03T09:00:00+0100\nLanguage: en\n-\nText.\n",
})
out, err := buildGem(t, files, nil)
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
want := "news/2026/11/ciao.gmi:;lang=it\nnews/2026/11/hola.gmi:;lang=es\n" +
"en/*:;lang=en\nen/**/*:;lang=en\nes/*:;lang=es\nes/**/*:;lang=es\nnews/*:;lang=en\nnews/**/*:;lang=en\n"
if got := readOut(t, out, ".meta"); got != want {
t.Errorf(".meta =\n%q\nwant\n%q", got, want)
}
for _, rel := range []string{"news/2026/11/ciao.gmi", "news/2026/11/hola.gmi"} {
if !exists(t, out, rel) {
t.Errorf("the .meta names %s, which is not in the capsule", rel)
}
}
// With English at the root and the news in English too, only the posts in other languages have a line.
plain, err := buildGem(t, files, func(o *GeminiOptions) { o.Language = "en" })
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
want = "news/2026/11/ciao.gmi:;lang=it\nnews/2026/11/hola.gmi:;lang=es\n" +
"it/*:;lang=it\nit/**/*:;lang=it\nes/*:;lang=es\nes/**/*:;lang=es\n"
if got := readOut(t, plain, ".meta"); got != want {
t.Errorf(".meta with English at the root =\n%q\nwant\n%q", got, want)
}
}
// TestGeminiNoNewsAtAll checks that a site without posts says so once, in the news list, and that the empty
// lists leave their headings out.
func TestGeminiNoNewsAtAll(t *testing.T) {
index := "# News\n\n{{news_list}}\n\n## Categories\n\n{{categories_list}}\n\n## Archive\n\n{{news_archive}}\n"
out, err := buildGem(t, gemSite(map[string]string{
"content/en/news/index.md": page("news", "News", "Layout: bare\n", index),
"news/shop.md": "", "news/gnu-linux-day-2026.md": "", "news/status.md": "",
}), nil)
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
got := readOut(t, out, "news/index.gmi")
if n := strings.Count(got, "No news yet."); n != 1 {
t.Errorf("the index says there is no news %d times, want once:\n%s", n, got)
}
for _, heading := range []string{"## Categories", "## Archive"} {
if strings.Contains(got, heading) {
t.Errorf("the index keeps the heading %q of an empty list:\n%s", heading, got)
}
}
mustContain(t, "news/index.gmi", got, "# News\n")
mustContain(t, "news/feed.gmi", readOut(t, out, "news/feed.gmi"), "# kmalloc.org: News\n")
home := readOut(t, out, "index.gmi")
if strings.Contains(home, "Tutte le news") {
t.Errorf("the home links to the news list of a site with no news:\n%s", home)
}
mustContain(t, "index.gmi", home, "# kmalloc.org\n")
}
// TestGeminiOnlyOnePost checks that a single post has no link to a newer or an older one, but has the one to all the news.
func TestGeminiOnlyOnePost(t *testing.T) {
out, err := buildGem(t, gemSite(map[string]string{"news/gnu-linux-day-2026.md": "", "news/status.md": ""}), nil)
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
post := readOut(t, out, "news/2026/11/shop.gmi")
for _, absent := range []string{"Older", "Newer"} {
if strings.Contains(post, absent) {
t.Errorf("the only post has %q:\n%s", absent, post)
}
}
mustContain(t, "the post", post, "=> /news/ All news\n")
}
// TestGeminiUnusualTitle checks that a title with quotes, an ampersand, a tag and non Latin letters is written as it is.
func TestGeminiUnusualTitle(t *testing.T) {
title := `Say "hi" & <b>go</b> ünï 日本語`
out, err := buildGem(t, gemSite(map[string]string{
"news/odd.md": "Title: " + title + "\nDate: 2026-11-15T09:00:00+0100\nCategories: Odd\n-\nText.\n",
}), nil)
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
mustContain(t, "the post", readOut(t, out, "news/2026/11/odd.gmi"), "# "+title+"\n")
mustContain(t, "news/feed.gmi", readOut(t, out, "news/feed.gmi"), "=> /news/2026/11/odd.gmi 2026-11-15 "+title+"\n")
mustContain(t, "news/index.gmi", readOut(t, out, "news/index.gmi"), "=> /news/2026/11/odd.gmi 2026-11-15 "+title+"\n")
mustContain(t, "the post below", readOut(t, out, "news/2026/11/shop.gmi"), "Newer: "+title)
for _, rel := range []string{"news/2026/11/odd.gmi", "news/index.gmi", "news/category/odd.gmi"} {
if strings.Contains(readOut(t, out, rel), "&lt;") || strings.Contains(readOut(t, out, rel), "&amp;") {
t.Errorf("%s has an HTML escape in the title", rel)
}
}
}
// TestGeminiLeavesTheLogoOut checks that the logo of the home page leaves no link line and no copied file.
func TestGeminiLeavesTheLogoOut(t *testing.T) {
out, err := buildGem(t, gemSite(map[string]string{
"content/en/services.md": page("services", "Services", "", "No image.\n"),
"content/en/index.md": page("home", "kmalloc.org", "Layout: bare\n", "<img class=\"logo\" src=\"/logo.svg\" alt=\"kmalloc\" width=\"112\">\n\n<h1 class=\"hero\">Technology.</h1>\n\n{{latest_news}}\n"),
}), nil)
if err != nil {
t.Fatalf("BuildGemini() error = %v", err)
}
home := readOut(t, out, "en/index.gmi")
if strings.Contains(home, "logo.svg") {
t.Errorf("the home links to the logo:\n%s", home)
}
if !strings.HasPrefix(home, "# kmalloc.org\n\n## Technology.\n") {
t.Errorf("the home starts with %q", home)
}
if exists(t, out, "logo.svg") {
t.Error("the logo is copied, and nothing points to it")
}
}

View file

@ -99,10 +99,11 @@ func Link(target, label string) string {
// linkLine writes a link on its own line. // linkLine writes a link on its own line.
func linkLine(l link) string { func linkLine(l link) string {
target := strings.Join(strings.Fields(l.target), "%20")
if l.label == "" || l.label == l.target { if l.label == "" || l.label == l.target {
return "=> " + l.target return "=> " + target
} }
return "=> " + l.target + " " + l.label return "=> " + target + " " + l.label
} }
// voidTags are the elements that have no end tag. // voidTags are the elements that have no end tag.
@ -170,6 +171,10 @@ var blockTags = map[string]bool{
"dl": true, "dt": true, "dd": true, "form": true, "fieldset": true, "address": true, "gemblock": true, "dl": true, "dt": true, "dd": true, "form": true, "fieldset": true, "address": true, "gemblock": true,
} }
// cellTags are the elements of tables and lists whose text is kept apart from the
// text next to it when they are read as part of a line.
var cellTags = map[string]bool{"tr": true, "td": true, "th": true, "li": true}
// skippedTags are the elements that have nothing to read. // skippedTags are the elements that have nothing to read.
var skippedTags = map[string]bool{ var skippedTags = map[string]bool{
"script": true, "style": true, "svg": true, "noscript": true, "template": true, "script": true, "style": true, "svg": true, "noscript": true, "template": true,
@ -351,7 +356,7 @@ func (c *converter) inline(nodes []*node, lineBreak string) (string, []link, err
return nil return nil
case "img": case "img":
alt := strings.Join(strings.Fields(n.attrs["alt"]), " ") alt := strings.Join(strings.Fields(n.attrs["alt"]), " ")
if alt == "" || n.attrs["src"] == "" { if alt == "" || n.attrs["src"] == "" || decorative(n) {
return nil return nil
} }
target, err := c.address(n.attrs["src"]) target, err := c.address(n.attrs["src"])
@ -380,11 +385,17 @@ func (c *converter) inline(nodes []*node, lineBreak string) (string, []link, err
} }
return nil return nil
} }
if blockTags[n.tag] || cellTags[n.tag] {
text.WriteString(" ")
}
for _, kid := range n.kids { for _, kid := range n.kids {
if err := walk(kid); err != nil { if err := walk(kid); err != nil {
return err return err
} }
} }
if blockTags[n.tag] || cellTags[n.tag] {
text.WriteString(" ")
}
return nil return nil
} }
for _, n := range nodes { for _, n := range nodes {
@ -395,6 +406,25 @@ func (c *converter) inline(nodes []*node, lineBreak string) (string, []link, err
return text.String(), links, nil return text.String(), links, nil
} }
// decorativeClasses are the classes of an image that belongs to the layout of the
// page (the logo, a mark, an ornament), not to what the page says.
var decorativeClasses = map[string]bool{"logo": true, "mark": true, "decorative": true}
// decorative reports whether an image is there for the look of the page: it says
// so with a role, or has one of the decorativeClasses. It is left out, a link line
// to it would tell a reader of a capsule nothing.
func decorative(n *node) bool {
if role := n.attrs["role"]; role == "presentation" || role == "none" {
return true
}
for _, class := range strings.Fields(n.attrs["class"]) {
if decorativeClasses[class] {
return true
}
}
return false
}
// address maps an address through Options.Link. // address maps an address through Options.Link.
func (c *converter) address(href string) (string, error) { func (c *converter) address(href string) (string, error) {
if c.opts.Link == nil { if c.opts.Link == nil {
@ -407,15 +437,20 @@ func (c *converter) address(href string) (string, error) {
return target, nil return target, nil
} }
// item is one bullet of a list. // item is one bullet of a list, with the preformatted blocks it holds.
type item struct { type item struct {
text string text string
links []link links []link
// pre holds the lines of the preformatted blocks of the item, fences included.
pre []string
} }
// list converts a list: bullets, with the links after them. A list in which // list converts a list: bullets, with the links after them. A list in which
// every item is only a link is a list of link lines. A list inside an item is // every item is only a link is a list of link lines. An item with no text, only
// flattened into the same list. An ordered list numbers its items. // an image or a link without a label, has no bullet; its links are written all
// the same. A list inside an item is flattened into the same list, and a
// preformatted block in an item follows its bullet. An ordered list numbers its
// items.
func (c *converter) list(n *node) (block, error) { func (c *converter) list(n *node) (block, error) {
var items []item var items []item
if err := c.items(n, &items); err != nil { if err := c.items(n, &items); err != nil {
@ -425,20 +460,25 @@ func (c *converter) list(n *node) (block, error) {
var links []link var links []link
for _, it := range items { for _, it := range items {
links = append(links, it.links...) links = append(links, it.links...)
if len(it.links) != 1 || it.text != it.links[0].label { if len(it.pre) > 0 || len(it.links) != 1 || it.text != it.links[0].label {
linksOnly = false linksOnly = false
} }
} }
if linksOnly { if linksOnly {
return block{links: links}, nil return block{links: links}, nil
} }
lines := make([]string, len(items)) var lines []string
for i, it := range items { number := 0
for _, it := range items {
if it.text != "" {
number++
prefix := "* " prefix := "* "
if n.tag == "ol" { if n.tag == "ol" {
prefix += strconv.Itoa(i+1) + ". " prefix += strconv.Itoa(number) + ". "
} }
lines[i] = prefix + it.text lines = append(lines, prefix+it.text)
}
lines = append(lines, it.pre...)
} }
return block{text: lines, links: links}, nil return block{text: lines, links: links}, nil
} }
@ -450,10 +490,14 @@ func (c *converter) items(list *node, out *[]item) error {
continue continue
} }
var own, nested []*node var own, nested []*node
var pre []string
for _, kid := range li.kids { for _, kid := range li.kids {
if kid.tag == "ul" || kid.tag == "ol" { switch {
case kid.tag == "ul" || kid.tag == "ol":
nested = append(nested, kid) nested = append(nested, kid)
} else { case kid.tag == "pre" && !skipped(kid):
pre = append(pre, preformatted(kid).text...)
default:
own = append(own, kid) own = append(own, kid)
} }
} }
@ -461,8 +505,8 @@ func (c *converter) items(list *node, out *[]item) error {
if err != nil { if err != nil {
return err return err
} }
if text = strings.Join(strings.Fields(text), " "); text != "" || len(links) > 0 { if text = strings.Join(strings.Fields(text), " "); text != "" || len(links) > 0 || len(pre) > 0 {
*out = append(*out, item{text: text, links: links}) *out = append(*out, item{text: text, links: links, pre: pre})
} }
for _, inner := range nested { for _, inner := range nested {
if err := c.items(inner, out); err != nil { if err := c.items(inner, out); err != nil {
@ -621,12 +665,33 @@ func tableWidth(rows [][]string) int {
return widest return widest
} }
// records writes the rows of a wide table as a list. With a header row each // records writes the rows of a wide table as a list. With a header row and
// row is a heading, its first cell, and a bullet for each other cell that is not // more than one column each row is a heading, its first cell when it has
// empty, named by the header of its column. Without one each row is a bullet // something in it, and a bullet for each other cell that is not empty, named by
// with the cells between bars. // the header of its column. A table of one column is a bullet for each cell, the
// header a line of text before them. Without a header, or with a header and
// nothing else, each row is a bullet with the cells between bars.
func records(rows [][]string, header bool) []string { func records(rows [][]string, header bool) []string {
columns := 0
for _, r := range rows {
if len(r) > columns {
columns = len(r)
}
}
if header && len(rows) == 1 {
header = false
}
var lines []string var lines []string
if columns <= 1 {
if header {
lines = append(lines, protect(rows[0][0]))
rows = rows[1:]
}
for _, r := range rows {
lines = append(lines, "* "+strings.Join(r, " | "))
}
return lines
}
if !header { if !header {
for _, r := range rows { for _, r := range rows {
lines = append(lines, "* "+strings.Join(r, " | ")) lines = append(lines, "* "+strings.Join(r, " | "))
@ -638,11 +703,9 @@ func records(rows [][]string, header bool) []string {
if i > 0 { if i > 0 {
lines = append(lines, "") lines = append(lines, "")
} }
title := "" if len(r) > 0 && r[0] != "" {
if len(r) > 0 { lines = append(lines, "### "+r[0])
title = r[0]
} }
lines = append(lines, "### "+title)
for j := 1; j < len(r); j++ { for j := 1; j < len(r); j++ {
if r[j] == "" { if r[j] == "" {
continue continue

View file

@ -149,3 +149,87 @@ func TestConvertWideTable(t *testing.T) {
t.Errorf("Convert() without a header =\n%q\nwant\n%q", got, want) t.Errorf("Convert() without a header =\n%q\nwant\n%q", got, want)
} }
} }
// TestConvertListItems checks the items that have no text, more than one paragraph or a preformatted block.
func TestConvertListItems(t *testing.T) {
tests := []struct {
name, html, want string
}{
{"an item that is only an image has no bullet", `<ul><li><img src="/a.png" alt="pic"></li><li>Text</li></ul>`, "* Text\n=> /a.png pic\n"},
{"an item that is a link without text has no bullet", `<ul><li><a href="/x"></a></li><li>Text</li></ul>`, "* Text\n=> /x\n"},
{"an item of only an image in a list of only images", `<ul><li><img src="/a.png" alt="A"></li><li><img src="/b.png" alt="B"></li></ul>`, "=> /a.png A\n=> /b.png B\n"},
{"paragraphs of an item do not stick", `<ul><li><p>First</p><p>Second</p></li></ul>`, "* First Second\n"},
{"cells of a table in an item do not stick", `<ul><li><table><tr><td>a</td><td>b</td></tr></table></li></ul>`, "* a b\n"},
{"a preformatted block in an item stays a block", "<ul><li>Run:<pre><code>x\n y</code></pre></li><li>Next</li></ul>", "* Run:\n```\nx\n y\n```\n* Next\n"},
{"a preformatted block that is all of an item", "<ul><li><pre>code</pre></li></ul>", "```\ncode\n```\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := convert(t, tt.html); got != tt.want {
t.Errorf("Convert() =\n%q\nwant\n%q", got, tt.want)
}
})
}
}
// TestConvertAddressWithSpaces checks that a space in an address does not end the address of a link line.
func TestConvertAddressWithSpaces(t *testing.T) {
if got, want := convert(t, `<p><a href="/a b/c d.html">t</a></p>`), "=> /a%20b/c%20d.html t\n"; got != want {
t.Errorf("Convert() = %q, want %q", got, want)
}
if got, want := Link("/a b", "x"), "=> /a%20b x"; got != want {
t.Errorf("Link() = %q, want %q", got, want)
}
}
// TestConvertWideTableEdges checks the wide tables with no title for a row, with a header only and with one column.
func TestConvertWideTableEdges(t *testing.T) {
long := strings.Repeat("word ", 20) + "end"
tests := []struct {
name, html, want string
}{
{"a row with an empty first cell has no heading",
`<table><tr><th>Service</th><th>Kept</th></tr><tr><td></td><td>` + long + `</td></tr><tr><td>Chat</td><td>1 year</td></tr></table>`,
"* Kept: " + long + "\n\n### Chat\n* Kept: 1 year\n"},
{"a header and nothing else keeps the header",
`<table><thead><tr><th>` + long + `</th><th>Seen</th></tr></thead></table>`,
"* " + long + " | Seen\n"},
{"one wide column with a header",
`<table><tr><th>Name</th></tr><tr><td>` + long + `</td></tr><tr><td>short</td></tr></table>`,
"Name\n* " + long + "\n* short\n"},
{"one wide column without a header",
`<table><tr><td>` + long + `</td></tr><tr><td>short</td></tr></table>`,
"* " + long + "\n* short\n"},
{"one narrow column", `<table><tr><th>Name</th></tr><tr><td>a</td></tr></table>`, "```\nName\n----\na\n```\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := convert(t, tt.html); got != tt.want {
t.Errorf("Convert() =\n%q\nwant\n%q", got, tt.want)
}
})
}
}
// TestConvertDecorativeImages checks that the images of the layout leave no link line and those of the content do.
func TestConvertDecorativeImages(t *testing.T) {
tests := []struct {
name, html, want string
}{
{"a logo", `<img class="logo" src="/logo.svg" alt="kmalloc" width="112"><p>Text</p>`, "Text\n"},
{"a logo among classes", `<p><img class="big mark" src="/m.svg" alt="mark"> Text</p>`, "Text\n"},
{"a decorative image", `<p><img class="decorative" src="/d.png" alt="swirl"> Text</p>`, "Text\n"},
{"role presentation", `<p><img role="presentation" src="/d.png" alt="swirl"> Text</p>`, "Text\n"},
{"role none", `<p><img role="none" src="/d.png" alt="swirl"> Text</p>`, "Text\n"},
{"an image of the content", `<p><img class="figure" src="/chart.png" alt="Sales by month"></p>`, "=> /chart.png Sales by month\n"},
{"an image that has no class", `<p><img src="/chart.svg" alt="A diagram"></p>`, "=> /chart.svg A diagram\n"},
{"a class that only contains the word", `<p><img class="logotype" src="/l.png" alt="Type"></p>`, "=> /l.png Type\n"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := convert(t, tt.html); got != tt.want {
t.Errorf("Convert() =\n%q\nwant\n%q", got, tt.want)
}
})
}
}