10 files changed, 191 insertions, 51 deletions

diff --git a/README b/README
index d1aa22c..954bc7a 100644
--- a/README
+++ b/README
@@ -59,13 +59,13 @@ the one system.  They can chat about it, on the one system.
 
 ## What does it do already?
 
-Currently it'll scan the current directory and subdirectories looking for
+Currently it'll scan the current folder and sub folders looking for
 .md files in CommonMark syntax.  This should cover some MarkDown
 variations.  Then it produces .HTML files converted from these .md files,
 and links them all together into a web site.
 
 Any .md file that is just the beginning metadata block doesn't get
-rendered into HTML, but is global metadata for this directory and subs,
+rendered into HTML, but is global metadata for this folder and subs,
 though the subs can override this with their own metadata.md files.
 
 git is used to store the .md files, and provides edit history.  Added on
@@ -73,7 +73,7 @@ the footer is links to cgit, which is used to store the files in git on
 your server.  This provides acces to the source code, history, and ATOM
 feed for the site.
 
-It can also scan an external directory and merge that with the current
+It can also scan an external folder and merge that with the current
 one, but this isn't tested yet.
 
 ## other stuff
diff --git a/SuckIt b/SuckIt
index 26991b5..38fe18e 100755
--- a/SuckIt
+++ b/SuckIt
@@ -155,6 +155,8 @@ done
 
 
 time notYetAnotherWiki.lua
+# No idea why yet, but needs a second run to sort out everything.  Shouldn't take long anyway.
+time notYetAnotherWiki.lua
 
 popd
 }
diff --git a/SuckItClean b/SuckItClean
new file mode 100755
index 0000000..1b8a025
--- /dev/null
+++ b/SuckItClean
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+TIMEFORMAT="    took %lR using %P%% CPU"
+time {
+pushd /opt/nyaw
+
+rm -fr Foswiki/*
+rm -fr PmWiki/*
+rm -fr unsorted
+rm -fr users/*
+popd
+./SuckIt
+}
diff --git a/TODO.md b/TODO.md
index bd02788..3453ba0 100644
--- a/TODO.md
+++ b/TODO.md
@@ -4,32 +4,51 @@ Make it perphekd!
 
 ## Do these
 
+Convert it to polygLua.
+
+It's all a bit too fragile, fix what I can.  Too many messes colliding.
+
+Flock it.
+
+Some sort of search system.
+
+A member system, and edit / manage system for their pages.
 
 Fix up linky conversion.  DONE, mostly.
 
-- This is the "page moved" problem, but now it's "page copied" and "page linked", a generic solution might work.
-- Rethink this, I might be going the wrong way.
 - Need to deal with real file name versus title.  Also symlink name not matching what it points to.
-- . Hmmm, might it be good policy to have a single canonical context for each page?  The "real URL" .md.md thing?
-- . Loop through the manually placed symlinks, adjusting "real URL" as we go.
-- + What to do about mulitple symlinks pointing to the same page?  Compare timestamps, most recent wins.
 
-Add atom feed for single page.  Alas cgit only seems to have ATOM feed on the whole repo, not individual files.
+Use the default.template that comes with nYAW if none is availaable.
+
+"collapsing headings" I guess that means click on a heading to hide / show the content under that heading.
+
+- 🙈
+- Should do that for the main content and the menu TOC.
+- Also allow editing just a section, a section being the bits between one heading and the next.
+- + 📝 ✒️✏️🖊️🖋 🖌️🖍️
+- + When someone starts editing, create pagename_draft.md and .HTML, and update those when they want to see a preview.
+- + Or pagename-heading_draft.md
+- + Use that draft.md file as a lock on editing that file / section.
+- + Have the hourly cron job remove any stale ones.
+
+Add ATOM feed for single page.  Alas cgit only seems to have ATOM feed on the whole repo, not individual files.
 
-- However, once timestamps are sorted, I can use that code to generate (static?) RSS and ATOM feeds, and create page histories using diffs.
+- However, once timestamps are sorted, I can use that code to generate RSS and ATOM feeds, and create page histories using diffs.
 - Instead of an hourly cron job to update everything, see if I can hook
 into Fos and Pm edit form's save function.  So when they save in the
-original wiki, the nyaw version gets updated, diffed, and RSSed.
+original wiki, the nyaw version gets updated, diffed, and ATOMed / RSSed.
+- + Actually a BFI method might do the trick, stick with doing a full scan 
+and only updating the stuff that needs it, likely that's only the page that just got saved.
+- git commit can have an arbitrary author / committer, so I can use that to make sure the person that actually made the change gets author / committer.
+- + There's also the email thing, but I don't want to spread email addresses around, not even sure we have them anyway.
 - Should also see if gitea can handle that, and if they want wiki content saved to git.
 
-It's all a bit too fragile, fix what I can.  Too many messes colliding.
+Syntax highlighting in code blocks.  The highlight package looks promising, already had it installed on my desktop.
 
 ## Some ideas
 
-Mostly from something chomwitt deleted.
+Mostly from something chomwitt wrote -
 
-- "collapsing headings" I guess that means click on a heading to hide / show the content under that heading.
-- + Should do that for the main content and the menu TOC.
 - "validation", not sure exactly what that would validate.
 - I'll just quote some of the rest -
 
@@ -40,9 +59,8 @@ Mostly from something chomwitt deleted.
     ** for example search-completion to other workflows names-tags.
 ~~~
 
-- Syntax highlighting in code blocks.
+- Allow default.template files in sub folders.
 - Might be useful to automatically convert anything looking like a URL into a linky.
-- Allow default.template files in sub directories.
 
 Automate symlinks.
 
@@ -51,16 +69,28 @@ Automate symlinks.
 
 ## Try out
 
+lua-lpeg-patterns might be useful
+
+lua-luxio might be the wheel I'm reinventing?
+
+lua-wsapi-fcgi
+
 https://inclusive-components.design/tooltips-toggletips/ has some ideas about the "tooltip" HTML attribute, and what to do about it.  A rabbit hole that ends in javascript, but might be useful up to that point.
 
 htmx
 
 cgit has Lua
 
+lua-gall for git stuff
+
+lua-lace for access control
+
+lua-unbound
+
 
-## User system
+## Member system
 
-Reuse the user system from SledjChisl.
+Reuse the member system from SledjChisl.
 
 levels -
 
@@ -102,6 +132,6 @@ site, including configuration and modules.  Likely this is the person
 that set the system up in the first place.
 
 Admin should have access to everything that shell level has, but there's always things need tweaking at some lower level.
-Built in file browser might do the trick.  Would be useful for content creators to to organise the content.  Naturally should obey the permissions.
+Built in file browser might do the trick.  Would be useful for content creators to organise the content.  Naturally should obey the permissions.
 
 root level is whoever controls the server things are running on.  They can do anything at all.
diff --git a/about/index.md b/about/index.md
index 6086d85..3f15be7 100644
--- a/about/index.md
+++ b/about/index.md
@@ -1,5 +1,5 @@
 # notYetAnotherWiki is not another wiki, at least not yet. It'll be much more than that, eventually.
 
-This is a new directory, an entire PR department can hang out here.
+This is a new folder, an entire PR department can hang out here.
 
 This would just be a duplicate of README, but for testing purposes.
diff --git a/default.template b/default.template
index 75922ac..baefad1 100644
--- a/default.template
+++ b/default.template
@@ -87,14 +87,14 @@
         <a href="$home$"><img src="$logo$" alt="not (Yet) (Another / A) Wiki."/></a>
       </nav></header></div>
       <div class="boxTools"><header><nav>
-        &nbsp; <b class="toolTip" title="🍔  hides / shows non content, reverse hamburger menu.
+        &nbsp; <a href="/help.HTML"><b class="toolTip" title="🍔  hides / shows non content, reverse hamburger menu.
 🕶  switches between dark and light themes.
 📚 shows the list of all pages.
 🔮 a folder of unsorted pages.
 👥 shows the list of users.
 🪵 will be for logging in, when I have written that bit.
 🔍 will be the search, when I have written that bit.
-👣 is a trail of the steps to get here.">❓</b>
+👣 is a trail of the steps to get here.">❓</b></a>
         &nbsp; <b class="toolTip"><label for="modeToggleMenu" class='modeBtn'>&#127828;</label></b>
         &nbsp; <b class="toolTip"><label for="modeToggleBody" class='modeBtn'>&#x1F576;</label></b>
         &nbsp; <a href="/everything.HTML"><b class="toolTip">&#128218;</b></a>
diff --git a/feed-icon-14x14.png b/feed-icon-14x14.png
new file mode 100755
index 0000000..b3c949d
--- /dev/null
+++ b/feed-icon-14x14.png
diff --git a/feed-icon-28x28.png b/feed-icon-28x28.png
new file mode 100755
index 0000000..d64c669
--- /dev/null
+++ b/feed-icon-28x28.png
diff --git a/help.md b/help.md
new file mode 100644
index 0000000..bc4cc46
--- /dev/null
+++ b/help.md
@@ -0,0 +1,43 @@
+# How to use this wiki
+
+On the top left, is the logo, beside that is a toolbar of icons -
+
+- ❓ hover over this for help.
+- 🍔 hides or shows everything but the content and these icons, the reverse hamburger menu.
+- 🕶  switches between dark and light themes.
+- 📚 shows the list of all pages.
+- 🔮 a folder of unsorted pages.
+- 👥 shows the list of users.
+- 🪵 will be for logging in, when I have written that bit.
+- 🔍 will be the search, when I have written that bit.
+
+&#x1f463; is a trail of breadcrumbs, though I'm using a footsteps icon
+for this.  Showing the footsteps you have followed to get to the current
+folder.
+
+On the top right of the pages is links to the sub folders of the current
+folder the current page is in.
+
+Down the left side is links to the pages in the current folder.  Most of
+them will be symlinks to the converted pages from the Foswiki and PmWiki
+sub folders.  Some will be links &#9757; to external sites.  For the
+current page, links to the headings are also shown, those headings have
+their own icon 🔼to go back to the top.
+
+The bit you are reading is the badly converted content of the current
+page.  At the bottom of each converted page is a link to the original,
+and if you are logged into that other system, you might be able to edit
+the page.  Note that unconverted pages don't have these links to the
+originals.
+
+At the bottom is links to things like page history, web site source code,
+and the software I'm writing to do all of this.  Or should be some day.
+
+
+
+# Under the hood
+
+Internally this wiki is made of the root folder and it's sub folders
+representing the structure of the wiki content.  Each folder has it's own
+pages.  Symlinks are used to copy or move pages around, with special
+folders for the original stuff from the other wikis.
diff --git a/notYetAnotherWiki.lua b/notYetAnotherWiki.lua
index c5814f3..f53691b 100755
--- a/notYetAnotherWiki.lua
+++ b/notYetAnotherWiki.lua
@@ -215,27 +215,51 @@ else
     print("Can't open everything.md for writing.")
 end
 
--- Scan the subdirectories looking for our files.
-local Directory = arg[1]
+-- Scan the sub folders looking for our files.
+local Folder = arg[1]
 toSub('')
-if nil == Directory then Directory = '.' end
-
--- Sort out realURL for symlinked .md.md files.
-for l in io.popen('find -L ' .. Directory .. ' -name unsorted -prune -o -name "*.md.md" -xtype l -printf "%P\n"'):lines() do
+if nil == Folder then Folder = '.' end
+--GlobalMetaData.root = Folder
+
+--[[ Sort out realURL for symlinked .md.md files.
+    realURL is the generic URL part for this page.  By policy it points to whatever is the latest copy / symlink of the original download .md.md.
+    realURL starts out being the path to the downloaded file and friends.
+    If we make a symlink during SuckIt, then that gets updated to point to the place the symlink is in.
+        below we compare timestamps and select the latest version if there's more than one symlink.
+    For the "page symlinked" problem, this should work if realURL is kept updated.
+    For the "page copied"    problem, this should work if realURL is kept updated, same as symlinked really, coz that's just another copy.
+    For the "page moved"     problem, that'll be the most recent symlink, the old one still has it's symlink pointing to the download .md.md, which gets updated with the current realURL.
+        So when some external old URL points to someplace a page used to be, it's old symlink points to the up to date download .md.md, and we know where to go to find the page now.
+        A left over .md.md file should have a redirect .HTML page created for it.
+]]
+for l in io.popen('find -L ' .. Folder .. ' -name unsorted -prune -o -name "*.md.md" -xtype l -printf "%P\n"'):lines() do
     local metadata = readMdMd(string.sub(l, 1, -4), {})
--- FIXME - if this already exists, compare the timestamps, most recent wins.
-    metadata.realURL = string.sub(l, 1, -7)
-    local a, e = io.open(l, 'w')
-    if nil == a then print('Could not open ' .. l .. ' - ' .. e) else
-        for k, v in pairs(metadata) do
-            a:write(k .. '=' .. v .. '\n')
+    if nil == metadata.realURL then
+        metadata.realURL = string.sub(l, 1, -7)
+    else
+        if metadata.realURL ~= string.sub(l, 1, -7) then
+            metadata.realURL = string.sub(l, 1, -7)
+            -- If this already exists, compare the timestamps, most recent wins.
+            local time0 = io.popen('ls -l --time-style=+%s "' .. metadata.realURL .. '.md.md" | cut -d \' \' -f 6'):read('l')
+            local time1 = io.popen('ls -l --time-style=+%s "' ..                l ..       '" | cut -d \' \' -f 6'):read('l')
+            if time0 > time1 then               metadata = nil end
+        else                                    metadata = nil end
+    end
+
+    if nil ~= metadata then
+-- DUNNO if this writes to the original file, or overwrites the symlink.
+        local a, e = io.open(l, 'w')
+        if nil == a then print('Could not open ' .. l .. ' - ' .. e) else
+            for k, v in pairs(metadata) do
+                a:write(k .. '=' .. v .. '\n')
+            end
+            a:close()
         end
-        a:close()
     end
 end
 
 -- Clean up unsorted.
-for l in io.popen('find -L ' .. Directory .. ' -name unsorted -prune -o -name "*.md" -a -not -name "*.md.md" -xtype l -printf "%P\n"'):lines() do
+for l in io.popen('find -L ' .. Folder .. ' -name unsorted -prune -o -name "*.md" -a -not -name "*.md.md" -xtype l -printf "%P\n"'):lines() do
     local tp = '_fos'
     local metadata = readMdMd(l, {})
     if nil ~= metadata then
@@ -245,16 +269,36 @@ for l in io.popen('find -L ' .. Directory .. ' -name unsorted -prune -o -name "*
             local a, e = io.open(unsort .. tp .. '.md' , 'r')
             if nil ~= a then
                 a:close()
-                os.execute('rm ' .. unsort .. tp .. '.*')
+                -- Keep the .md.md symlink, delete the rest.
+                os.execute('rm ' .. unsort .. tp .. '.HTML')
+                os.execute('rm ' .. unsort .. tp .. '.md')
+                a, e = io.open(unsort .. tp .. '.HTML', 'w')
+                if nil == a then print('Could not open ' .. unsort .. tp .. '.HTML' .. ' - ' .. e) else
+                    local dst = string.sub(l, 1, -4) .. '.HTML'
+                    local cnt = 1
+                    for j = 1, #metadata.ogFile do
+                        if '/' == string.sub(metadata.ogFile, j, j) then cnt = cnt + 1 end
+                    end
+                    dst = string.rep('../', cnt) .. dst
+                    a:write(
+[=[<!DOCTYPE html>
+<html>
+  <head><meta http-equiv="refresh" content="0; url=]=] .. dst .. '"' .. [=[/></head>
+  <body><p>Click this if you don't get redirected to the real page - <a href="]=] .. dst .. '"' .. [=[>Redirect</a></p></body>
+</html>
+]=])
+                    a:close()
+                    print('REDIRECT ' .. unsort .. tp .. '.HTML \t-> ' .. dst)
+                end
+
             end
         end
     end
 end
 
 -- Look for copied pages from the other wikis.
-for l in io.popen('find -L ' .. Directory .. ' -name "*.HTM" -type f,l -printf "%P\n"'):lines() do
--- TODO - Only do this if .HTM is newer than .md, or .md doesn't exist.
-
+for l in io.popen('find -L ' .. Folder .. ' -name "*.HTM" -type f,l -printf "%P\n"'):lines() do
+-- Only do this if .HTM is newer than .md, or .md doesn't exist.
     local htime = io.popen("date -ur " ..            l ..           " +%s"):read('l')
     local mtime = io.popen("date -ur " .. string.sub(l, 1, -4) .. "md +%s 2>/dev/null"):read('l')
     if (nil == mtime) or (htime > mtime) then
@@ -404,14 +448,14 @@ writeString(l .. '_ORIGINAL1', body)
     end
 end
 
-if '.' ~= Directory then
+if '.' ~= Folder then
     for l in io.popen('find -L . -name "*.md" -type f,l -printf "%P\n"'):lines() do
         toFile(string.gsub(l, '%.md$', ''))
     end
 end
 
--- Can add in a distant directory to, for putting it's results in the current directory.
-for l in io.popen('find -L ' .. Directory .. ' -name "*.md" -type f,l -printf "%P\n"'):lines() do
+-- Can add in a distant folder to, for putting it's results in the current folder.
+for l in io.popen('find -L ' .. Folder .. ' -name "*.md" -type f,l -printf "%P\n"'):lines() do
     local n = string.gsub(l, '%.md$', '')
     if nil == Files[n] then toFile(n) end
 end
@@ -472,7 +516,7 @@ for name, sub in pairs(Subs) do
         print("EMPTY " .. name)
         h = io.open(name .. '/index.md', 'w')
         if nil ~= h then
-            h:write('This directory has no files.')
+            h:write('This folder has no files.')
             h:close()
         else
             print("Can't open " .. name .. '/index.md for writing.')
@@ -484,7 +528,7 @@ end
 ---------------------------------------------------------------------------------
 -- These functions assume the above file and sub scan has completed.
 
--- Which page in this directory should we show?
+-- Which page in this folder should we show?
 -- NOTE - only looking for the .md files we scanned for before, any stray HTML, html, HTM, and htm files will get ignored.
 local whichPage = function(f)
     local fl = ''
@@ -527,7 +571,7 @@ local whichWiki = function(metadata)
 end
 
 
--- Calculate a link from the source directory to the destination directory.
+-- Calculate a link from the source folder to the destination folder.
 local linkFrom = function(source, dest)
     -- Evil hacks!
     if 'Profiles' == dest then dest = 'PmWiki/Profiles' end
@@ -763,7 +807,7 @@ function Writer.header(s, level)
     local text = Lunamark.util.rope_to_string(s)
 -- FIXME - Work around a bug in Lunamark?
     text = RE.gsub(text, "{[\\]}", "")
-    return '<h' .. level .. ' id="' .. RE.gsub(text, '{[ ]}', '_') .. '">' .. text .. ' <a style="font-size: 0.42em;" href="#top">👆</a></h' .. level .. '>'
+    return '<h' .. level .. ' id="' .. RE.gsub(text, '{[ ]}', '_') .. '">' .. text .. ' <a style="font-size: 0.42em;" href="#top">🔼</a></h' .. level .. '>'
 end
 local OgWriterLink = Writer.link        -- So we can call the original from within mine, we are just changing the URL.
 function Writer.link(lab, url, tit)
@@ -810,7 +854,7 @@ for name, file in pairs(Files) do
             local pth = file.path
             if '' ~= file.path then pth = file.path .. '/' end
             if 'true' ~= Subs[pth .. f].metadata.hidden then
-                metadata.header = metadata.header .. '<a href="' .. f .. '/' .. whichPage(pth .. f) .. '">' .. f .. '</a> &nbsp; '
+                metadata.header = metadata.header .. '<a href="' .. f .. '/' .. whichPage(pth .. f) .. '">' .. f .. '</a> &nbsp; &#128194; '
             end
         end
 
@@ -847,7 +891,15 @@ for name, file in pairs(Files) do
                 end
             else
                 if nil ~= url then      metadata.menu = metadata.menu .. '<p><a href="' .. url .. '">'      .. title .. ' &#9757;</a></p>'
-                else                    metadata.menu = metadata.menu .. '<p><a href="' .. f   .. '.HTML">' .. title .. '</a></p>'
+                else
+                    local pth = file.path
+                    if '' ~= pth then pth = pth .. '/' end
+                    -- Don't include any left over .md.md files, so don't do this if f.md doesn't exist.
+                    local a, e = io.open(pth .. f .. '.md' , 'r')
+                    if nil ~= a then
+                        a:close()
+                                        metadata.menu = metadata.menu .. '<p><a href="' .. f   .. '.HTML">' .. title .. '</a></p>'
+                    end
                 end
             end
         end
@@ -865,7 +917,7 @@ for name, file in pairs(Files) do
             metadata.history = ''
         end
         if nil ~= metadata.sourcecode   then metadata.footer = '<a href="' .. metadata.sourcecode       .. '">source code</a>'    end
-        if nil ~= metadata.feedatom             then metadata.footer = '<a href="' .. metadata.feedatom         ..   '">atom feed</a> &nbsp; &nbsp; ' .. metadata.footer    end
+        if nil ~= metadata.feedatom             then metadata.footer = '<a href="' .. metadata.feedatom         ..   '">atom feed<img src="feed-icon-14x14.png"></img></a> &nbsp; &nbsp; ' .. metadata.footer    end
         if metadata.footer ~= '' then metadata.footer = 'Web site ' .. metadata.footer end
         -- Add a link to the original page.
         if nil ~= metadata.ogURL then