path: root/docs/README.LuaSL

Refer to - http://www.infinitegrid.org/drupal/content/LuaSL_New_scripting_engine

LuaSL is a Lua based LSL scripting engine that will aim for LSL
compatability first, then adding Lua extensions.  It aims to replace the
woeful XEngine from OpenSim, and at a later stage, be the basis for a
client side scripting engine.

To compile this, you will need Enlightenment Foundation Libraries (EFL)
installed in either /opt/e17 or /usr.  These are typical places it get's
installed in.  You will also need flex.  The rest of the dependencies
are in the ../libraries directory.


Design.
-------

The basic design will be made up as I go along, but so far I have this -

A parser parses an LSL script, validating it and reporting errors.

A translator takes the result of the parse, and converts it into Lua
source.  Each LSL script becomes a Lua state.  LSL states are handled as
Lua tables, with each LSL state function being a table function in a
common metatable.  LL and OS functions are likely to be C or Lua
functions.  Careful testing should be done with LuaJIT FFI, sandboxing,
and performance testing.

The Lua source is compiled by the Lua compiler.

LuaJIT is used as the Lua compiler, library, and runtime.

Luaproc is used to start up operating system threads and hand Lua states
between them.  Luaproc messaging is also being used, but might need to
change to edje messaging.  Note - luaproc has been extensively rewritten
for this project, mostly converting it to use EFL.  That rewrite
substantially shrunk the source code.  More might be rewritten in
future.

THIS IS WHERE WE ARE RIGHT NOW.

Nails will be used to pump commands in and out of the LuaSL system. 
Incoming commands invoke LSL events via the LuaSL state metatable.  LL
and OS functions that impact the world will be converted to nails
commands sent to the command pump.

Initialy, since this is the first thing being written, a nails command
pump client needs to be installed into OpenSim's C# stuff.  Though it
might be possible to talk directly to ROBUST instead.  Think I'll try
the ROBUST route, see how far I can get.  That's the general principle
applying in all of this - try to avoid C# and see how for we can get. 
lol

On the other hand, might be better to leverage the existing C#
implementations of LSL functions, just to get things up and running
quickly.  To that end, a protocol involving exchanging snippets of Lua
over a network socket has been developed, and the next step is to write
the C# side.  sigh

A watchdog thread should be used to make sure no LuaSL script spends
forever processing any event.

Some form of serialization will need to be created for saving script
state during shutdowns, passing script state to other threads /
processes / computers.  Apparently Lua is good at this.

There will have to be a MySQL (and maybe SQLite) client in the system,
so we can talk directly to the local sim database.  Esskyuehl may be
suitable, though it's still in the prototype stage.

Email, HTTP, and XML-RPC might need to be dealt with by us.  A ROBUST
client will be needed to.  Azy might be suitable, but it's also in
prototype.

An object is a file system directory, full of LSL scripts as text files,
notecards as text files, animations as BVH (or later BVJ) files, etc. 
There will be some sort of metadata in place.  This could be created by
our own OpenSim compatible cache module.


Test harness.
-------------

I'll build a test harness.  It will be based on EFL Edje Lua, with
buttons for triggering LSL events, SL style dialogs, and other goodies.

The initial goal will be to run standard MLP scripts.  They have minimal
interface to the world, and exercise quite a bit of the rest of LSL. 
They are also quite common, and sometimes responsible for a lot of the
script running load.

Later I should add stock standard OpenCollar scripts from SL.  They are
a bitch to get working under OpenSim, so would be good compatability
tests.

Various eina logging domains might be used to handle whisper, say, shout,
etc.

Performance testing will have to be done on 5000 scripts, to see how
that compares against XEngine.


The next goal.
--------------

The next goal is to support llDialog() in extantz.  Extantz has enough
GUI infrastructure now to support a window with a bunch of buttons. 
LuaLSL has always used MLP as the test scripts, which in turn reads some
note cards and pops up llDialog to suit.  I think I have enough bits to
get this all linked up, or at least start to, and see what missing bits
there are.


  Just off the top of my head -
  -----------------------------

MLP uses llGetInventory*() functions to find out what note cards are in
the objects inventory.  Then uses llGetNotecardLine() to read the
notecards.  In response, the server sends dataserver events with each
line, and the system loops until all lines are read.

MLP sets up a listener with llListen(), then uses llDialog() to send a
"menu" to the client.  The client sends a channel message back to the
server when the user makes a choice.  These go to the listen event.


  In "reality" -
  --------------

LuaSL
  starts up
  opens an Ecore_Con TCP server at 127.0.0.1:8211
  sets up the compiler
  starts luaproc worker thread system
  waits for input
    compile() -> compile the script, report warnings and errors
    run()     -> create new Lua state, put it on the worker queue
    exit()    -> shutdown
    *         -> send to SID Lua state

LuaSL_test
  starts up
  connects to LuaSL server
  loops through Test sim/objects looking for *.lsl
    generates random SID
    sends "SID compile file.lsl" to LuaSL
    deal with returned results
      PW warnings
      PE errors
      if compiled send "SID run()" to LuaSL
      for various llGet*() functions, return random data to LuaSL
        This includes the llGetInventory*() functions mentioned above.
  waits for a few seconds
    sends pretend touch events to LuaSL
    sends quit() to LuaSL
    sends exit() to LuaSL

Test sim/objects
  onefangs test bed
    the usual MLP style scripts
    .MENUITEMS
    .POSITIONS
    ball object

MLP
  ~run
    llSay() to balls telling left over ones to die
    llGetOwner()
    loop through an internal list of script names
      llGetInventoryType() to check if they are there and are scripts.
      llSetScriptState() to stop them
      llResetOtherScript()
    llOwnerSay() to let owner know MLP is turned off

    touch_start()
      llDetectedKey() to check if it's the owner (current version of MLP doesn't do this)
      switch to run state

    run state
      loop through scripts again, turning them on this time.


  ~memory
    llGetInventoryNumber() to count the notecards
    llGetInventoryName() to find .POSITIONS cards
    llGetNoteCardLine()

    dataserver()
      llGetNoteCardLine()
      change to state on when finished

    on state
      llMessageLinked()

    link_message()
      llWhisper()


  ~menucfg
    Same as ~memory.
    llGetScriptName()
    llGetFreeMemory()


  ~menu
    llSetTimerEvent() - NOTE doesn't actually setup a timer in the default state.
    llSleep()

    link_message()
      llGetLinkNumber()
      switch to state on

    on state
      llGetKey() to figure out it's channel
      llListen()

    listen()
      llSameGroup()
      llDialog()
      llResetTime()

    link_message()

    touch_start()
      llDetectedKey()
      llDetectedGroup()

    timer()

Not to mention copius use of list and string functions.


LSL.Lua
  All event stuff is hooked up.
  events.detected*(list) is all hooked up to save the list in detected* tables.
  llDetected*() functions return those tables.

  copius list functions implemented, though there's a TODO about negative indexes.  Dunno which ones are missing.

  String functions implemented.  Actually only llGetSubString() and llSubStringIndex()

  llGetScriptName() implemented.

  State changes implemented, with a TODO about clearing out pending events.
  
  Bunch of ll*() functions faked.


  Not implemented
    llResetTime()
    llGetNoteCardLine()
    llGetFreeMemory()
    llGetKey()
    llDialog()
    llListen()
    llSameGroup()


  Uses newFunc() to create all the LSL functions.
    If there is no return type then sends function and arguments to LuaSL's client.
    If there is a  return type then sends function and arguments to LuaSL"s client, then waits for the response.
      response should be a Lua value as a string.  It gets pcall()'d
  If we implement a function ourselves, then we override the newFunc created definition with our own function.


  Uses waitAndProcess() as the main loop, or to wait for responses.
    Which just runs what ever Lua strings are sent to us from the LuaSL client, unless it's waiting for a response above.


  What needs to be done -
  -----------------------

Implement llGetNotecardLine() and generate dataserver() events.

LuaSL -> love


Fake llGetFreeMemory(), and llSameGroup().

LuaSL -> love -> LuaSL


Fake llGetKey()?

LuaSL -> love -> LuaSL


Implement llGetInventory*() by actually looking at the files in the
"object".  While we could hard code where the "object" is, since we are
starting the run ourselves, we can track which object the scripts are
in.  Again, this should be in the love server part.

LuaSL -> love -> LuaSL


Implement llListen(), llDialog() and generate listen() events.

SID.llDialog(USER_UUID, "My menu", arseBackwardsMenuList, 456)
USER_UUID.llRegionSay(456, "button text")
LuaSL -> love -> extantz -> love -> LuaSL