I'm re-purposing this for SledjHamr https://sledjhamr.org/git/docs/index.html
The general structure of SledjHamr is a bunch of servers talking to each
other via Internet (or just local) connections. One of them is a web
server for assets, world data, and inventory.
Originally I didn't think using a web based world client was a good idea,
however it might be better to have one, for reasons. Now I need a web
management console that can do all the things the current tmux console
can, including OpenSim console and commands. Plus account management for
users. I can also use a web based Jabber / XMPP front end to chat, IM,
and group chatter, which would run in the normal viewers web browser.
This provides a doorway into putting SledjHamr stuff in existing viewers
without needing them to support it. So a web based viewer now makes more
sense, and also means we can get away with not needing a viewer at all.
Toybox itself doesn't include a web server, and I don't think there is
one on the roadmap. So we have to use an external web server, which was
a design goal of SledjHamr in the first place, using existing mature
HTTP infrastructure, coz that's already solved problems for a bunch of
things that plague OS/SL to this day. Clear your cache! Pffft.
So sledjchisl.c will be the "love world server", though initially it just
drives OpenSim_SC in tmux via tmux commands to send keys and read output.
Later it might run opensim_SC directly and use STDIN and STDOUT to do
everything. It'll also provide the text management front end that runs
in the left tmux panel of the first window, which is why it's based on
boxes in the first place. Later still it can take over opensim_SC
functions as I move them out of mono.
We will need a text, web, and GUI version of this management front end.
Hmmm, maybe don't need a GUI version, GUI users can just run a terminal.
After much research, FastCGI / FCGI seems to be the most portable way of
interfacing with existing web servers. FCGI protocol closes STDERR and
STDOUT, and uses STDIN as it's two way communications channel to the web
server, so our FCGI module can't be used as the text management front
end. This is probably a good idea to keep them seperate anyway, for
security, coz the web server is exposed to the world, the console isn't.
Currently sledjchisl.c tests to see if it's running in tmux already, if
it isn't it starts up tmux runs itself into this new tmux, then exits.
So it could also test if it's running from FCGI, and switch to web mode,
then it'll need to find the tmuxed instance to send commands to it.
Either via nails connection, or sending tmux commands via shell.
FCGI has methods of dealing with auth and templates. B-)
So for now I think I'll have the text and web management front ends in
sledjchisl.c, and the love world server as well. I can split them up
later if I need to.
I has Apache 2.4.25-3+deb9u9
MariaDB 10.1.44-MariaDB
https://gist.github.com/dermesser/e2f9b66457ae19ebd116
Multithreaded example in C.
-------------------------------------------------------------------
Apache doesn't seem to support FCGI filter role, so I might have to do
without. Might be better anyway.
"A Filter is similar in functionality to a Responder that takes a data
file as a parameter. The difference is that with a Filter, both the data
file and the Filter itself can be access controlled using the Web
server's access control mechanisms, while a Responder that takes the name
of a data file as a parameter must perform its own access control checks
on the data file."
Which is fine, our access control checks will be "Is this database
defined user already logged on via our FCGI script?". We should have
total control over that. I was planning on using the FCGI auth
mechanism anyway.
RESPONDER
web server sends FCGI_PARAMS
CONTENT_LENGTH
web server sends input body FCGI_STDIN
fcgi app sends result data over FCGI_STDOUT and error messages over FCGI_STDERR
it has to finish reading FCGI_PARAMS first
fcgi app sends FCGI_END_REQUEST(protocolStatus = FCGI_REQUEST_COMPLETE)
FILTER
filtered file has last modified time
web server sets FCGI_DATA_LAST_MOD accordingly
web server sends FCGI_PARAMS
CONTENT_LENGTH FCGI_DATA_LAST_MOD FCGI_DATA_LENGTH
web server sends input body FCGI_STDIN
web servers sends file over FCGI_DATA
fcgi app can ignore FCGI_DATA and use it's own cached copy based on FCGI_DATA_LAST_MOD
fcgi app sends result data over FCGI_STDOUT and error messages over FCGI_STDERR
it has to finish reading FCGI_STDIN first, but not FCGI_DATA
fcgi app sends FCGI_END_REQUEST(protocolStatus = FCGI_REQUEST_COMPLETE)
Soooo, FILTER might be slower anyway if we are caching the filtered file,
or mmapping it, coz filter has to start sending the filtered file, even
if it's to be served from cache. Plus no need to wait for FCGI_STDIN
before spewing it out.
Last update time for parameters, plus an update frequency. Once a minute.
NOTE - SSI is a bit more complex than what I'm currently using.
https://en.wikipedia.org/wiki/Server_Side_Includes
<!--#include virtual="menu.cgi" -->
<!--#include file="footer.html" -->
<!--#exec cgi="/cgi-bin/foo.cgi" -->
<!--#exec cmd="ls -l" -->
. <!--#echo var="REMOTE_ADDR" -->
<!--#config timefmt="%y %m %d" -->
<!--#config sizefmt="bytes" -->
<!--#config errmsg="SSI command failed!" -->
<!--#flastmod virtual="index.html" -->
<!--#fsize file="script.pl" -->
<!--#if expr="${Sec_Nav}" -->
<!--#include virtual="secondary_nav.txt" -->
<!--#elif expr="${Pri_Nav}" -->
<!--#include virtual="primary_nav.txt" -->
<!--#else -->
<!--#include virtual="article.txt" -->
<!--#endif -->
<!--#set var="foo" value="bar" -->
<!--#printenv -->
https://www.w3.org/Jigsaw/Doc/User/SSI.html
Adds lots of others, including Java stuff.
Mine
<!--#lua lua="print(table[key])" -->
<!--#lua file="/path/to/script.lua" -->
<!--#lua virtual="https://example.com/script.lua" -->
-------------------------------------------------------------------
Account creation process in the database.
Apart from the usual input validation of things...
OpenSim/Server/Handlers/UserAccounts/UserAccountServerPostHandler.cs
byte[] CreateUser(Dictionary<string, object> request)
Looks like their built in web front end, perhaps what is triggered by the console?
createdUserAccount
= ((UserAccountService)m_UserAccountService).CreateUser(scopeID, principalID, firstName, lastName, password, email, model);
OpenSim/opensim-SC/OpenSim/ApplicationPlugins/RemoteController/RemoteAdminPlugin.cs
An XML RPC interface to -
private UserAccount CreateUser(UUID scopeID, string firstName, string lastName, string password, string email)
account = new UserAccount(scopeID, UUID.Random(), firstName, lastName, email);
if (userAccountService.StoreUserAccount(account))
success = authenticationService.SetPassword(account.PrincipalID, password)
gridUserService.SetHome(account.PrincipalID.ToString(), home.RegionID, new Vector3(128, 128, 0), new Vector3(0, 1, 0));
success = inventoryService.CreateUserInventory(account.PrincipalID);
OpenSim/opensim-SC/OpenSim/Services/UserAccountService/UserAccountService.cs
Looks like the console command handler.
create user [<first> [<last> [<pass> [<email> [<user id> [<model>]]]]]] - Create a new user
protected void HandleCreateUser(string module, string[] cmdparams)
Gathers console arguments, or prompts for them.
CreateUser(UUID.Zero, principalId, firstName, lastName, password, email, model);
public UserAccount CreateUser(UUID scopeID, UUID principalID, string firstName, string lastName, string password, string email, string model = "")
Looks almost identical to the OpenSim/ApplicationPlugins/RemoteController/RemoteAdminPlugin.cs one above, but they add -
CreateDefaultAppearanceEntries(account.PrincipalID)
account = new UserAccount(scopeID, UUID.Random(), firstName, lastName, email);
OpenSim/opensim-SC/OpenSim/Services/Interfaces/IUserAccountService.cs
public UserAccount(UUID scopeID, UUID principalID, string firstName, string lastName, string email)
Just holds the data in memory, in a dictionary I think.
OpenSim/opensim-SC/OpenSim/Services/UserAccountService/UserAccountService.cs
public bool StoreUserAccount(UserAccount data)
Stuffs the data into a new UserAccountData()
m_Database.Store(d)
As far as I can tell, just dumps this data into the UserAccounts table -
FirstName, LastName, PrincipleID, ScopeID, Email, Created, UserLevel, UserFlags, UserTitle
PrincipleID is their randomly generated with no thought to collisions UUID.
ScopeID is 00000000-0000-0000-0000-000000000000
Userlevel is 0 for most, -1 for Waki, determines if they can log on. Also higher for gods and things.
UserFlags, I think the only one is "64 god can login to this account using gods password.
UserTitle might default to "Local", or be configurable / and editable.
something something URL encoded "ServiceURLs" mumble
HomeURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f GatekeeperURI= InventoryServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f AssetServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f ProfileServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f FriendsServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f IMServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f GroupsServerURI=http%3a%2f%2fgrid.infinitegrid.org%3a8002%2f
Though most are either NULL, empty, or -
HomeURI= GatekeeperURI= InventoryServerURI= AssetServerURI=
Doesn't metion "active", which is always equal to 1 I guess.
success = authenticationService.SetPassword(account.PrincipalID, password)
OpenSim/Services/AuthenticationService/AuthenticationServiceBase.cs
stores password details in "auth" table -
UUID
passwordSalt = Util.Md5Hash(UUID.Random().ToString());
passwdHash = Util.Md5Hash(Util.Md5Hash(password) + ":" + passwordSalt);
accountType = "UserAccount";
webLoginKey = UUID.Zero.ToString();
gridUserService.SetHome(account.PrincipalID.ToString(), home.RegionID, new Vector3(128, 128, 0), new Vector3(0, 1, 0));
OpenSim/Services/UserAccountService/GridUserService.cs
Stores in database table GridUser
HomeRegionID, HomePosition, HomeLookAt
The other fields in that table -
UserID, LastRegionID, LastPosition, LastLookAt, Online (true or false), Login (timestamp or 0), Logout (timestamp or 0).
success = inventoryService.CreateUserInventory(account.PrincipalID);
OpenSim/Services/InventoryService/XInventoryService.cs
Create a bunch of folders in the users inventory, of specific types.
rootFolder = ConvertToOpenSim(CreateFolder(principalID, UUID.Zero, (int)FolderType.Root, InventoryFolderBase.ROOT_FOLDER_NAME));
XInventoryFolder[] sysFolders = GetSystemFolders(principalID, rootFolder.ID)
if (!Array.Exists(sysFolders, delegate(XInventoryFolder f) { if (f.type == (int)FolderType.Animation) return true; return false; }))
CreateFolder(principalID, rootFolder.ID, (int)FolderType.Animation, "Animations");
FolderType.BodyPart, "Body Parts"
XInventoryFolder folder = CreateFolder(principalID, rootFolder.ID, (int)FolderType.CallingCard, "Calling Cards");
folder = CreateFolder(principalID, folder.folderID, (int)FolderType.CallingCard, "Friends")
CreateFolder(principalID, folder.folderID, (int)FolderType.CallingCard, "All");
FolderType.Clothing, "Clothing"
FolderType.CurrentOutfit, "Current Outfit"
FolderType.Favorites, "Favorites"
FolderType.Gesture, "Gestures")
FolderType.Landmark, "Landmarks"
FolderType.LostAndFound, "Lost And Found"
FolderType.Notecard, "Notecards"
FolderType.Object, "Objects"
FolderType.Snapshot, "Photo Album"
FolderType.LSLText, "Scripts"
FolderType.Sound, "Sounds"
FolderType.Texture, "Textures"
FolderType.Trash, "Trash"
Stores in database inventoryFolders ????
folderName, type, version = 1, folderID = UUID.Random(), agentID = principalID, parentFolderID = parentID
CreateDefaultAppearanceEntries(account.PrincipalID)
OpenSim/Services/UserAccountService/UserAccountService.cs
protected void CreateDefaultAppearanceEntries(UUID principalID)
Creates a bunch of "Default *" body parts and clothes, Ruth 1.0, links them in Inventories current outfit folder.
Creates a AvatarWearable[] and puts them all in it.
AvatarAppearance ap = new AvatarAppearance();
ap.SetWearable(i, wearables[i]);
m_AvatarService.SetAppearance(principalID, ap);
UserAccounts table -
UserFlags 64 is "allow gods to log in as me"
0xf00 is membershipType, unles there's a title. Only sent to viewers I think.
32 is Minors for estate banning purposes.
4 is Anonymous for estate banning purposes.
1 is AllowPublish in profile, but userprofile has this as separate field.
2 is MaturePublish in profile, but userprofile has this as separate field.
Presence table -
UserID varchar(255)
RegionID char(36)
SessionID char(36)
SecureSessionID char(36)
LastSeen timestamp
tokens table (I think this is actually used for something) -
UUID char(36)
token varchar(255) current example looks like a UUID.
validity datetime
userdata (empty, can't find any actual usage in the source code, part of profiles) -
UserId char(36) primary index
TagId varchar(64) primary index
DataKey varchar(255)
DataVal varchar(255)
auth.webLoginKey seems to be some sort of passwordy type thing, though perhaps not actually hashed, rarely used, none of IG members have one.
PLAN-
. username
. password
. create login
.check if it's a proper two word name
.login -> check if it's an existing account, get their UUID.
create toke_n_munchie
write session record
create -> new user
create new UUID
check if it's an existing UUID
dbCount(, "UserAccounts", "PrincipleID='new-UUID'")
loop until we get a new one
create toke_n_munchie
write session record
Create ->
(wait a few seconds before showing this page)
. email
. email again
. password again
. DoB
. accept terms of service
. claim to be an adult
. confirm / cancel
New user
UserAccounts.FirstName = ???
UserAccounts.LastName = ???
UserAccounts.Email = ???
UserAccounts.Created = timestamp
UserAccounts.PrincipleID = randomly generate UUID, but check for collisions with other accounts.
It's a UNIQUE KEY.
UserAccounts.ScopeID = 00000000-0000-0000-0000-000000000000
UserAccounts.Userlevel = -200
UserAccounts.UserFlags = 64
UserAccounts.UserTitle = newbie
UserAccounts.ServiceURLs = ""
UserAccounts.active = 0
auth.UUID = UserAccounts.PrincipleID
It's a PRIMARY KEY.
auth.passwordSalt = Util.Md5Hash(UUID.Random().ToString())
auth.passwdHash = Util.Md5Hash(Util.Md5Hash(password) + ":" + passwordSalt)
auth.accountType = "UserAccount"
auth.webLoginKey (varchar(255)) = "00000000-0000-0000-0000-000000000000"
userdata.UserId = UserAccounts.PrincipleID
userdata.TagId = "account creation data"
It's a UNIQUE KEY
userdata.DataKey = "DoB"
userdata.DataVal = ???
userdata.UserId = UserAccounts.PrincipleID
userdata.TagId = "account creation data"
userdata.DataKey = "timezone"
userdata.DataVal = ???
userdata.UserId = UserAccounts.PrincipleID
userdata.TagId = "account creation data"
userdata.DataKey = "Terms of service"
userdata.DataVal = "True"
userdata.UserId = UserAccounts.PrincipleID
userdata.TagId = "account creation data"
userdata.DataKey = "claims to be an adult"
userdata.DataVal = "True"
Validated via email
(wait a few seconds before showing this page)
UserAccounts.Userlevel = -100
UserAccounts.UserTitle = validated
Vouched for
userdata.UserId = UserAccounts.PrincipleID
userdata.TagId = "vouches"
userdata.DataKey = UUID of voucher
userdata.DataVal = timestamp of vouching
UserAccounts.Userlevel = -50
UserAccounts.UserTitle = vouched for
Admin approved
GridUser.UserID = UserAccounts.PrincipleID
It's a PRIMARY KEY.
GridUser.HomeRegionID = ???
GridUser.HomePosition = ???
GridUser.HomeLookAt = ???
GridUser.LastRegionID = ???
GridUser.LastPosition = ???
GridUser.LastLookAt = ???
GridUser.Online = False
GridUser.Login = 0
GridUser.Logout = 0
UserAccounts.active = 1
UserAccounts.Userlevel = 1
UserAccounts.UserTitle = Member / Local / whatever
Load the default IAR.
-------------------------------------------------------------------
https://project-awesome.org/aleksandar-todorovic/awesome-c
A curated list of C good stuff.
https://wolkykim.github.io/qdecoder/
CGI library made by the qlibc guy, does support FCGI.
Might be a wrapper around the fcgi_stdio stuff I'm already using?
https://danielmiessler.com/study/http/
A Security-focused HTTP Primer
Nothing much new except to say this about the Referer header -
"should not be used to make security decisions as it is controlled by the client"
Though others tell us to do precisely that. lol
-------------------------------------------------------------------
apt install libmariadbclient-dev libapache2-mod-fcgid libssl1.0-dev uuid-dev spawn-fcgi
BTW, everything is some BSD licence variation, except MariaDB, which is
some combination of GPLv2, 2+, 3, and BSD variants. With FOSS License
Exception, which seems to mean "Oracles GPL isn't viral". Though my
license is "fuck you viral GPL, that's not your decision to make for my
fucking code" BSD.
-------------------------------------------------------------------
Install / update / upgrade.
I could keep the version number around.
Include version numbers / branches of dependencies.
Update will grab any security updates for the installed version.
Upgrade will upgrade to a chosen later different version.
Downgrade will downgrade to a chosen earlier different version.
Note that we are currently using the LuaJIT 2.1.0-beta3 branch of the
main Luajit repo. Everything else is on their master branches.
Bootstrap -
bootstrap.sh or bootstrap.bat
Build the LuaJIT that comes with our source. It "builds out-of-the
box on most systems" and has no dependencies, other than a C build system.
Or download a prebuilt LuaJIT from somewhere.
After toybox has been LuaJITized.
Build the LuaJIT that comes with our source. It "builds out-of-the
box on most systems" and has no dependencies, other than a C build system.
Similar should apply to toybox, though it's our LuaJITized version.
Will need a specific miniconfig for this that doesn't include sledjchisl.
Or download a prebuilt toybox+LuaJIT from a SledjHamr package repo.
Install -
install.lua
Will need a pre flight check if the dependencies are installed.
It checks if the system is built and has source.
Build it all.
Do the usual copy stuff to a directory thing.
Run "sledjchisl -install" in that directory.
Which does the usual "check health of system and fix up problems" thing, then quits instead of keep running.
The health check should include making sure our database creds exist / work.
Update / upgrade / downgrade
install.lua -update
install.lua -upgrade
install.lua -downgrade
Check if we are a binary only, or a source install.
wget new binaries / git pull new source
Toybox has a wget in pending, otherwise it only has ftpget.
Git is standalone outside of the system, but if you are
running from source, you likely have standard build tools
like git.
Yeah I hate things that have their own packaging system, for needing to
step outside the operating systems packaging system, and adding to the too
long list of stuff I have to deal with manually, and now I are one. lol
-------------------------------------------------------------------
Time for a restructure of the web page / field / database stuff.
Will need to include a "what page is this" cookie, or maybe query ?mode=add
old validate UUID
define the UUID based UserAccounts db static dbRequest, fill it if needed.
if create
try to find an unused UUID
fill Rd->stuff with UUID
if confirm
check it's length
otherwise
check it's length
look it up, bitch if not found
If we found it, put level into Rd->database
fill Rd->stuff with UUID
old validateName
define the name based UserAccounts db static dbRequest, fill it if needed.
Do the Lua file lookup, fill a tnm hash.
Do the database lookup, fill rows.
if login
convert tnm to Rd->database, or dbPull(rows)
fill Rd->stuff with name, UUID, and level
if create
complain if we found a record
try to find an unused UUID
fill Rd->database with new data
fill Rd->stuff with name, UUID, and level
old validatePassword
define the UUID based auth db static dbRequest, fill it if needed.
if login
do the database lookup, fill rows
check if the name validation found us a UUID, fail login if it didn't
do the pasword+salt hash and compare
fill Rd->stuff with passwordHash and passwordSalt
if create
fill Rd->stuff with paswordHash and passwordSalt
if confirm
check if password hashess are the same
freeSesh(Rd, linky, wipe)
linky - Rd->shs or Rd->lnk
%s/sessions/%s.lua or %s/sessions/%s.linky
wipe - wipe or delete session
wiping means remove session stuff from Rd->stuff
Which happens on - session failing to write, redirecting login form, showing login form if not confirm, vegOut (session timeout, bitchSession)
newSesh(Rd, linky)
linky - old Rd->shs or a new Rd->lnk
setToken_n_munchie(Rd, linky); Only caller of setToken_n_munchie(Rd, linky);
setToken_n_munchie(Rd, linky)
linky - Rd->shs or Rd->lnk
%s/sessions/%s.lua or %s/sessions/%s.linky
!linky - actually set the cookies.
if error writing session file - freeSesh(Rd, linky, TRUE);
//validateSesh()
sessionValidate()
bitchSession() for bad session things.
sets chillOut for validated session linky.
Rd->chillOut = TRUE;
freeSesh(Rd, linky, FALSE);
Rd->func = (pageBuildFunction) loginPage;
Rd->doit = "logout";
sets vegOut if the session timed out.
//validatePassword()
sets chillOut for validated password on create.
bitchSession() called if there's anything wrong with the session trackers, if we can't load / run the users Lua file,
sets vegOut
account_HTML()
sets chillOut for POST confirm
createUser(Rd);
newSesh(Rd, TRUE);
Rd->chillOut = TRUE;
sets chillOut for POST login
Rd->chillOut = TRUE;
POST with no errors will
form == accountLogin freeSesh(Rd, FALSE, TRUE)
doit == login chillOut = TRUE
vegOut freeSesh(Rd, FALSE, TRUE);
else chillOut freeSesh(Rd, FALSE, FALSE); newSesh(Rd, FALSE);
else no Rd->shs.leaf newSesh(Rd, FALSE);
redirect to GET
otherwise
form == accountLogin
doit == confirm freeSesh(Rd, FALSE, TRUE)
newSesh(Rd, FALSE)
else if errors reeSesh(Rd, FALSE, FALSE) newSesh(Rd, FALSE)
show page
LOGGED IN means that the session stored on disk has a valid UUID.
When creating a new user, we create a new UUID firstish.
accountLoginWeb() / accountOut()
freeSesh(Rd, FALSE, TRUE)
newSesh(Rd, FALSE)
accountView()
freeSesh(Rd, FALSE, FALSE)
newSesh(Rd, FALSE)
accountAdd()
Note that this is in two parts, first they click "create" on login page, then "confirm" on the account creation page.
Account creation
accountLoginWeb()
"create" ->
Show accountCreateWeb and await confirmation.
accountCreateWeb()
"confirm" -> accountAdd()
create UUID
create user
store user
wipe old session
store new session with UUID, user is logged in now
create linky
email linky
Show usual logged in page.
"cancel" ->
-------------------------------------------------------------------
Maybe - /opt/opensim_SC/var/cache/sessions/uuid-uuid-uuid-uuid.logged symlink to session.
https://localhost/sledjchisl.fcgi/account.html?user=account_name
https://localhost/sledjchisl.fcgi/account.html/users/account_name
logged in user is in the sesion, but they can view / vouch / edit / delete any other user depending on their access level
For logged in user, at the top show their name as linky to their accountView http://localhost/sledjchisl.fcgi/account.html/users/account_name
That accountView offers edit / logout button, etc.
Display account stuff, but not edit it until they hit the edit button.
When showing other users
accountView, with edit / delete buttons if logged in user is high enough level.
-------------------------------------------------------------------
-------------------------------------------------------------------
-------------------------------------------------------------------
BUGS!
-----
Redo the santize(), though that needs extensive changes each time we read Rd->cookies, Rd->queries, and Rd->body