From 34c5ee4c2a489a506e93d5b303fbc80b263747f0 Mon Sep 17 00:00:00 2001 From: onefang Date: Thu, 27 Feb 2020 11:18:47 +1000 Subject: The new sledjchisl binary, to replace the management scripts and web stuff. --- src/NOTES.txt | 183 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) create mode 100644 src/NOTES.txt (limited to 'src/NOTES.txt') diff --git a/src/NOTES.txt b/src/NOTES.txt new file mode 100644 index 0000000..1758ac8 --- /dev/null +++ b/src/NOTES.txt @@ -0,0 +1,183 @@ +I'm re-purposing this for SledjHamr https://sledjhamr.org/git/docs/index.html + +The general structure of SledjHamr is a bunch of servers talking to each +other via Internet (or just local) connections. One of them is a web +server for assets, world data, and inventory. + +Originally I didn't think using a web based world client was a good idea, +however it might be better to have one, for reasons. Now I need a web +management console that can do all the things the current tmux console +can, including OpenSim console and commands. Plus account management for +users. I can also use a web based Jabber / XMPP front end to chat, IM, +and group chatter, which would run in the normal viewers web browser. +This provides a doorway into putting SledjHamr stuff in existing viewers +without needing them to support it. So a web based viewer now makes more +sense, and also means we can get away with not needing a viewer at all. + +Toybox itself doesn't include a web server, and I don't think there is +one on the roadmap. So we have to use an external web server, which was +a design goal of SledjHamr in the first place, using existing mature +HTTP infrastructure, coz that's already solved problems for a bunch of +things that plague OS/SL to this day. Clear your cache! Pffft. + +So sledjchisl.c will be the "love world server", though initially it just +drives OpenSim_SC in tmux via tmux commands to send keys and read output. +Later it might run opensim_SC directly and use STDIN and STDOUT to do +everything. It'll also provide the text management front end that runs +in the left tmux panel of the first window, which is why it's based on +boxes in the first place. Later still it can take over opensim_SC +functions as I move them out of mono. + +We will need a text, web, and GUI version of this management front end. +Hmmm, maybe don't need a GUI version, GUI users can just run a terminal. + +After much research, FastCGI / FCGI seems to be the most portable way of +interfacing with existing web servers. FCGI protocol closes STDERR and +STDOUT, and uses STDIN as it's two way communications channel to the web +server, so our FCGI module can't be used as the text management front +end. This is probably a good idea to keep them seperate anyway, for +security, coz the web server is exposed to the world, the console isn't. + +Currently sledjchisl.c tests to see if it's running in tmux already, if +it isn't it starts up tmux runs itself into this new tmux, then exits. +So it could also test if it's running from FCGI, and switch to web mode, +then it'll need to find the tmuxed instance to send commands to it. +Either via nails connection, or sending tmux commands via shell. + +FCGI has methods of dealing with auth and templates. B-) + +So for now I think I'll have the text and web management front ends in +sledjchisl.c, and the love world server as well. I can split them up +later if I need to. + + + + +I has Apache 2.4.25-3+deb9u9 + MariaDB 10.1.44-MariaDB + + +https://gist.github.com/dermesser/e2f9b66457ae19ebd116 + Multithreaded example in C. + + +------------------------------------------------------------------- + +Apache doesn't seem to support FCGI filter role, so I might have to do +without. Might be better anyway. + + +"A Filter is similar in functionality to a Responder that takes a data +file as a parameter. The difference is that with a Filter, both the data +file and the Filter itself can be access controlled using the Web +server's access control mechanisms, while a Responder that takes the name +of a data file as a parameter must perform its own access control checks +on the data file." + + Which is fine, our access control checks will be "Is this database + defined user already logged on via our FCGI script?". We should have + total control over that. I was planning on using the FCGI auth + mechanism anyway. + + +RESPONDER + web server sends FCGI_PARAMS + CONTENT_LENGTH + web server sends input body FCGI_STDIN + fcgi app sends result data over FCGI_STDOUT and error messages over FCGI_STDERR + it has to finish reading FCGI_PARAMS first + fcgi app sends FCGI_END_REQUEST(protocolStatus = FCGI_REQUEST_COMPLETE) + + +FILTER + filtered file has last modified time + web server sets FCGI_DATA_LAST_MOD accordingly + web server sends FCGI_PARAMS + CONTENT_LENGTH FCGI_DATA_LAST_MOD FCGI_DATA_LENGTH + web server sends input body FCGI_STDIN + web servers sends file over FCGI_DATA + fcgi app can ignore FCGI_DATA and use it's own cached copy based on FCGI_DATA_LAST_MOD + fcgi app sends result data over FCGI_STDOUT and error messages over FCGI_STDERR + it has to finish reading FCGI_STDIN first, but not FCGI_DATA + fcgi app sends FCGI_END_REQUEST(protocolStatus = FCGI_REQUEST_COMPLETE) + + +Soooo, FILTER might be slower anyway if we are caching the filtered file, +or mmapping it, coz filter has to start sending the filtered file, even +if it's to be served from cache. Plus no need to wait for FCGI_STDIN +before spewing it out. + + +PLAN - +. add "webRoot" to the config, point it at /opt/opensim_SC/web +. /opt/opensim_SC/web/fcgi-bin/sledjchisl.fcgi a symlink to /opt/opensim_SC/bin/sledjchisl +. /opt/opensim_SC/web/html/ the static web files and templates. +. https://localhost/opensim_SC/fcgi-bin/sledjchisl.fcgi/foo.html +. check the if modified since bit against the replaceable bits last fetch time / /opt/opensim_SC/web/html/foo.html last modified time +. check if /opt/opensim_SC/web/html/foo.html is cached, or if it's been modified since last cache. +. mmap /opt/opensim_SC/web/html/foo.html +. put it in a cache +. scan through it looking for the replacable bits +. store pointers to them +. check if the replaceable bits need refreshing +. loop through the pointers to the cache +. spew out the non replacable bits +. spew out the replacements for the replacable bits +. repeat until done + +Last update time for parameters, plus an update frequency. Once a minute. +Hash of page file names +. last modified time for file + Linked list of page fragments -> array (qlibc can convert a linked list into an array?), or try the growable thingy. +. struct +. enum telling us what this bit is +. bit of verbatim text +. replaceable parameter, pointer to the data that is also stored in the ssi hash, so that changes propogate +. + bit of Lua + + NOTE - SSI is a bit more complex than what I'm currently using. + https://en.wikipedia.org/wiki/Server_Side_Includes + + + + +. + + + + + + + + + + + + + + + https://www.w3.org/Jigsaw/Doc/User/SSI.html + Adds lots of others, including Java stuff. + Mine + + + + +BTW - /opt/opensim_SC/web/html/foo.html should have it's image URLS and other +static data set to https://localhost/opensim_SC/SledjHamr.png, which +would map to /opt/opensim_SC/web/html/SledjHamr.png and be treated as ordinary +static files by the web server. + +ALSO - when spewing results, we have to manually send the headers first +our selves, this should include the HTTP status code and string, content +type, cookies, etc. + +------------------------------------------------------------------- + +https://project-awesome.org/aleksandar-todorovic/awesome-c + A curated list of C good stuff. + +https://wolkykim.github.io/qdecoder/ + CGI library made by the qlibc guy, does support FCGI. + Might be a wrapper around the fcgi_stdio stuff I'm already using? -- cgit v1.1