feat: write the site as a Gemini capsule #12

Merged
squid merged 6 commits from feat/gemini-output into main 2026-11-16 11:55:40 +00:00
Owner

What

motorino gemini writes the site as a Gemini capsule, for an Agate server. I want the capsule to come from the same pages and news as the website, with no second copy to keep up.

This branch is built on top of the blog news (#11), so it lists those commits too. I merge #11 first.

motorino gemini -site DIR -out DIR [-lang CODE] [-drafts]

How it works

  • New package internal/gemtext: it reads the HTML a page renders to and writes gemtext, with no new dependency (the lenient mode of encoding/xml with the HTML void elements and entities). Headings (three levels), one line paragraphs, the links of a paragraph, list, quote or table on lines of their own after it, bullets, quotes, preformatted blocks, tables as aligned columns (as records when wider than 80 columns), images as link lines. Lines that look like gemtext are protected.
  • Language at the root. The language the server serves by default is at the root and the others under /<code>/, English included. It comes from -lang, else gemini_language in site.json, else the default language.
  • .meta. The server reads the language of the documents from <out>/.meta (Agate with --central-conf). For every language other than the root one there are two lines, en/*:;lang=en and en/**/*:;lang=en, since * does not cross folders, and the same two for news/ when the news are in another language. The file is never linked.
  • News once. The news keep the model of the website, one language and one copy: /news/ with the index, the posts, the categories, the archives and news/feed.gmi, the list of => /news/2026/11/post.gmi 2026-11-14 Title lines that feed readers subscribe to. No copies under the other languages. Every page ends with the navigation of its language, which links to the news.
  • Addresses. A link to a page of the site goes to its page in the capsule, a link to a missing page stops the build and names the page and the address. Assets are copied only when a page points to them. gemini_web_paths ("/~") keeps a path that the web server serves by other means as a link to the website.
  • The replacement of the block placeholders is shared with the HTML build.

Tests

make check is green: 360 tests (205 at the top level), none failing. New tests: the converter (every element, links, blocks, broken HTML, wide tables), the capsule (tree with Italian and with English at the root, .meta content for both, headings, navigation, the news pages and the feed, no empty file and no broken link on the whole capsule, links that stop the build, assets, drafts, web paths) and the command.

I built the real site with it: 28 files, no empty file, no broken link, no HTML left in the pages.

Refs #10

Review fixes

  • A list item with no text (only an image or a link without a label) has no empty bullet, and its links are still written. Paragraphs and table cells of an item do not stick together, and a preformatted block in an item stays a block after its bullet.
  • Spaces in the address of a link line are written as %20.
  • A wide table with an empty first cell has no empty heading, and a wide table with only a header, or only one column, is written as bullets and keeps its header.
  • A post written in another language than the news has a line of its own in .meta, with the path of its file. The server uses the first line that matches a document (I checked it with Agate 3.3.25), so these lines come before the folders.
  • An empty {{news_archive}} or {{categories_list}} leaves nothing with its heading, so a site without news says so once.
  • An image of the layout (class logo, mark or decorative, or role presentation or none) leaves no link line and its file is not copied: the logo no longer opens the home. Images of the content keep their alt.
  • -clean and the warning about old files work for motorino gemini too.
  • The command test names files and does not count them. New tests for each point, for a single post, an unusual title and no news at all.

I served the real capsule with Agate (--central-conf --lang it): / and the pages of the root are lang=it, /en/, /es/ and /de/ are lang=en, lang=es and lang=de, and /news/ is lang=en. I merged #11 into this branch, so I merge #11 first.

## What `motorino gemini` writes the site as a Gemini capsule, for an Agate server. I want the capsule to come from the same pages and news as the website, with no second copy to keep up. This branch is built on top of the blog news (#11), so it lists those commits too. I merge #11 first. ``` motorino gemini -site DIR -out DIR [-lang CODE] [-drafts] ``` ## How it works - New package `internal/gemtext`: it reads the HTML a page renders to and writes gemtext, with no new dependency (the lenient mode of `encoding/xml` with the HTML void elements and entities). Headings (three levels), one line paragraphs, the links of a paragraph, list, quote or table on lines of their own after it, bullets, quotes, preformatted blocks, tables as aligned columns (as records when wider than 80 columns), images as link lines. Lines that look like gemtext are protected. - **Language at the root.** The language the server serves by default is at the root and the others under `/<code>/`, English included. It comes from `-lang`, else `gemini_language` in `site.json`, else the default language. - **`.meta`.** The server reads the language of the documents from `<out>/.meta` (Agate with `--central-conf`). For every language other than the root one there are two lines, `en/*:;lang=en` and `en/**/*:;lang=en`, since `*` does not cross folders, and the same two for `news/` when the news are in another language. The file is never linked. - **News once.** The news keep the model of the website, one language and one copy: `/news/` with the index, the posts, the categories, the archives and `news/feed.gmi`, the list of `=> /news/2026/11/post.gmi 2026-11-14 Title` lines that feed readers subscribe to. No copies under the other languages. Every page ends with the navigation of its language, which links to the news. - **Addresses.** A link to a page of the site goes to its page in the capsule, a link to a missing page stops the build and names the page and the address. Assets are copied only when a page points to them. `gemini_web_paths` (`"/~"`) keeps a path that the web server serves by other means as a link to the website. - The replacement of the block placeholders is shared with the HTML build. ## Tests `make check` is green: 360 tests (205 at the top level), none failing. New tests: the converter (every element, links, blocks, broken HTML, wide tables), the capsule (tree with Italian and with English at the root, `.meta` content for both, headings, navigation, the news pages and the feed, no empty file and no broken link on the whole capsule, links that stop the build, assets, drafts, web paths) and the command. I built the real site with it: 28 files, no empty file, no broken link, no HTML left in the pages. Refs #10 ## Review fixes - A list item with no text (only an image or a link without a label) has no empty bullet, and its links are still written. Paragraphs and table cells of an item do not stick together, and a preformatted block in an item stays a block after its bullet. - Spaces in the address of a link line are written as `%20`. - A wide table with an empty first cell has no empty heading, and a wide table with only a header, or only one column, is written as bullets and keeps its header. - A post written in another language than the news has a line of its own in `.meta`, with the path of its file. The server uses the first line that matches a document (I checked it with Agate 3.3.25), so these lines come before the folders. - An empty `{{news_archive}}` or `{{categories_list}}` leaves nothing with its heading, so a site without news says so once. - An image of the layout (class `logo`, `mark` or `decorative`, or role `presentation` or `none`) leaves no link line and its file is not copied: the logo no longer opens the home. Images of the content keep their alt. - `-clean` and the warning about old files work for `motorino gemini` too. - The command test names files and does not count them. New tests for each point, for a single post, an unusual title and no news at all. I served the real capsule with Agate (`--central-conf --lang it`): `/` and the pages of the root are `lang=it`, `/en/`, `/es/` and `/de/` are `lang=en`, `lang=es` and `lang=de`, and `/news/` is `lang=en`. I merged #11 into this branch, so I merge #11 first.
- Draft: true and a date in the future keep a post out of the pages, the lists,
  the categories and the feed. motorino build -drafts (next commit) shows them.
- news_language in site.json, and an optional Language key in a post, choose the
  language of the news pages and of the interface strings they use. The news
  still exist once, not once per language.
- New header keys Mastodon (an https address), Draft and Language; a post knows
  its reading time and whether its description was written.

Refs #9
New placeholders for templates/post.html, written in the language of the post:
date_long, post_summary, post_meta, post_reading, post_mastodon and post_nav
(older, newer and all news), plus og_type and article_meta for Open Graph, which
every other page sets to website. A post with a Mastodon address needs the
reply_mastodon string; the other strings are optional and a feature without its
string is left out.

Refs #9
- {{news_blog}} lists the posts as entries (title, long date, categories,
  description, a read more link) 10 to a page; the other pages are written under
  the index, at page/2/, page/3/ and so on, with a pager.
- Archive pages for every year and month that has posts, at /news/2026/ and
  /news/2026/11/, when templates/archive.html exists, and a {{news_archive}} block.

Refs #9
The description of an item is now the plain summary of the post, the full text with
absolute links goes in content:encoded (the end marker of a CDATA section inside
the text is split), and an item has its categories and its author (dc:creator).

Refs #9
feat: motorino new writes a draft, motorino build -drafts previews it
All checks were successful
check / check (pull_request) Successful in 30s
633da1c46b
motorino new -site DIR "A title" writes news/a-title.md with the header ready:
the date of now, the author from site.json, the category News and Draft: true.
motorino build -drafts publishes the drafts and the posts dated in the future,
for a preview. The README describes the header keys, the news language, the blog
placeholders and blocks, and the language strings they use.

Refs #9
A new package, with no new dependency, that reads the HTML a page renders to (the
lenient mode of encoding/xml with the HTML void elements and entities) and writes
gemtext: three heading levels, one line paragraphs, the links of a paragraph, a list,
a quote or a table on lines of their own after it, bullets (a list of only links is
link lines), quotes (a note is a quote), preformatted blocks, tables as aligned
preformatted blocks, images as link lines, no emphasis, and lines that look like
gemtext are protected. Options.Link maps or drops an address and its error stops
the conversion; Options.Block expands the placeholders of generated blocks.

The language files also keep the order of the menu now (Lang.NavOrder), which the
Gemini output needs.

Refs #10
`motorino gemini [-site DIR] [-out DIR] [-lang CODE] [-drafts]` turns the pages and the
news into gemtext with the new gemtext package.

- the language served by default sits at the root (-lang, then gemini_language in
  site.json, then the default language) and the others under /<code>/
- the news exist once, under /news/: index, posts, categories, archives and a
  gemlog style feed (news/feed.gmi, a date before each title)
- every page ends with the navigation of its language, the language links and, on
  translations, the machine translation note
- .meta for Agate (--central-conf): two lines per language other than the root one
  and two for the news when they are in another language, never linked
- addresses are mapped to the capsule, a fragment is dropped, a missing target stops
  the build, assets are copied only when a page points to them, gemini_web_paths
  keeps paths served by other means as links to the website
- blocks of generated lists become gemtext with the strings of the language file
- the block replacement of a page is shared with the HTML build

Refs #10
feat(gemtext): write a table wider than 80 columns as records
All checks were successful
check / check (pull_request) Successful in 31s
889edbfd63
A preformatted block is not wrapped by the readers, so a table with long cells
(the data retention table of the services page) became unreadable. A wide table
with a header row is now a heading for each row and a bullet for each other cell,
named by its column; without a header, one bullet per row.

Refs #10
feat(news): show the posts of the lists as note panels
All checks were successful
check / check (pull_request) Successful in 30s
71cf14006d
The blog list, the pages of a category and the pages of an archive show every post as a
note panel: a small line with the long date and the categories, the title in bold with
a full stop (unless it ends with a full stop, a question mark or an exclamation mark),
the description and a read more link whose aria-label names the post. The page of a
post, {{news_list}} and the latest news of the home page are unchanged.

A language without read_more gets the title as the link on the category and archive
pages, as those pages worked before; the blog list still needs the string.

Refs #9
Merge branch 'feat/blog-news' into feat/gemini-output
All checks were successful
check / check (pull_request) Successful in 29s
cc90df8c6a
- groupArchive compared each post with the last group only, so two posts near midnight
  at the end of a month, written with different offsets, listed the same month twice and
  the build failed with "both published as". Posts are grouped by their own year and
  month, newest first.
- {{news_archive}} is empty when there are no posts, so the index says there is no news
  once, and the heading of an empty {{news_archive}} or {{categories_list}} goes with it.
- tests for exactly ten posts, eleven posts, a single post, a title with quotes, an
  ampersand, a tag and non Latin letters, and a neighbour written in another language

Refs #9
feat(build): report the old files of the output folder, add -clean
All checks were successful
check / check (pull_request) Successful in 29s
52edb15b64
A build never emptied the output folder, so the pages of a list that got shorter stayed
behind. The build now returns the files it did not write and the command prints a
warning that names them. -clean empties the folder first, only when it is new, empty or
holds the .motorino-out file that an earlier -clean left, and never the folder of the site
or one that holds it. The default is unchanged: no extra file in a folder built without it,
so the deploy (a new folder published with rsync --delete) is not affected.

Refs #9
Merge branch 'feat/blog-news' into feat/gemini-output
All checks were successful
check / check (pull_request) Successful in 30s
faff6476ab
- a list item with no text (only an image or a link without a label) has no empty bullet,
  its links are still written
- paragraphs and table cells of a list item do not stick together, and a preformatted
  block in an item stays a block after its bullet
- spaces in the address of a link line are written as %20
- a wide table with an empty first cell has no empty heading, a wide table with only a
  header or only one column is written as bullets, and the header is not lost
- a post in another language than the news has its own line in .meta, before the lines
  of the folders: the server uses the first line that matches a document (checked with
  Agate 3.3.25)
- an empty {{news_archive}} or {{categories_list}} leaves nothing, with its heading, so
  a site without news says so once
- images of the layout (class logo, mark or decorative, role presentation or none) leave
  no link line and the file is not copied
- the command test names files instead of counting them, and -clean and the warning
  about old files work for the gemini command too

Refs #10
squid merged commit 35b123dec8 into main 2026-11-16 11:55:40 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
squid/motorino!12
No description provided.