path: root/docs/ClientHamr/README.GuiLua

GuiLua is basically a redo of my ancient Java based matrix-RAD system,
but using Lua and EFL.  Lua is entirely different from Java, and Lua
doesn't run in web browsers, so this version has similar concepts, and
similar names, it's not exactly the same.  It's more accurate to say
this is in the spirit of matrix-RAD.

The ultimate goal is for the various ClientHamr parts to be able to
either host their own UI, or have it hosted in the in world 3D window. 
Matrix-RAD's ability to run either server or client side, with no change
to the code, is great for this sort of thing.  If I can manage the
ability to "tear off" windows, in other words, pull them out of the 3D
window to be real windows, then that's something people have been asking
for from LL forever.

Stdin/out might be good to use.  Extantz starts up "client" programs,
with a "use stdin/out" argument, the clients pass skang commands (open
this internal window with these widgets) to extantz through their
stdout, extantz passes skang commands (these values where set by these
widgets) back through their stdin.  With no "use stdin/out" argument,
the programs host their own UI.  Perhaps extantz can just pass the UI
skang stuff back to the client, which will mean "draw your own window",
to support the "tear off" feature.  If I remember skang had suitable
serialization stuff to support this.  Lua does anyway.  B-)

Naturally, like matrix-RAD, all this UI stuff can happen across a socket
as well as through stdin/out.  Should be transparent to the user proggy.

I can redesign skang to be more Lua like, to take advantage of Lua's
meta language capabilities.  Since no one used skang anyway, I can get
away with this.  Also should add a Lua friendly wrapper around nails, so
that addons for Extantz can be supplied entirely as Lua scripts.  In
other words, LuaSL scripts.  On top of all that, should also allow
access to the Edje Lua stuff I've already written, but drag it outside
of an edje file.

This pretty much means that GuiLua should be a shared library, coz in
the ClientHamr use case, several programs are gonna be using it at once. 
Should also be separate modules for development stuff like the skin
editor, which could be just another proggy using GuiLua.

Initially GuiLua can wrap elementary / evas, with the goal of wrapping
NAWS, just like matrix-RAD was originally planned.  Then I can put off
writing NAWS for another decade.  lol

Might be nice to have a wrapper that is a web server that generates
standard HTML forms.  No JavaScript, no HTML5, just plain old,
compatible with everything, HTML forms.


Design notes
------------

Edje Lua does not wrap elementary widgets, and is not likely to due to
elementary being compiled after edje.  Smart objects and swallows
probably can deal with that though, I'm not familiar enough with them to
know.  Eo on the other hand allegedly offers introspection, which was
always the corner stone of matrix-RAD.  So access to elementary widgets
via Eo might be possible.  That's kinda the entire hook I'm hoping will
help to make matrix-RAD style stuff work as well as it did there. 
Apparently when they promised introspection with Eo, they meant compile
time, not run time.

Edje Lua is restricted to scripts in edje files, AND sandboxed to those
edje files as well.  We want to step outside that sandbox, and run stand
alone Lua scripts, as well as GuiLua / LuaSL scripts internal to
extantz.  Some merging of LuaSL and Edje Lua, leaving room for GuiLua,
would be great, but I might have to convince Raster.  Note that the
socket idea wont work in sandboxed edje Lua scripts, unless the socket
is in C code, that uses edje messages as the internal transport.

Just so it's easier to find, I'll repeat a bit from the Enlightenment
mailing list here, the result of discussions with raster and other
interested parties -

"First step is to write edje functions for generically bundling up one
or more Lua tables into a message, sending that message, then unpacking
it again. This will be able to be done from edje, from C, and from Lua
directly. Perhaps adding that as one more edje messege type.  This is
for sending Lua tables between threads.  A later addition will be to
send them through the 'net, probably as eet.

Host apps can register arbitrary functions with edje Lua, using a more
generic version of the code I already wrote for letting edje register
lua functions.  These host functions are by default not thread safe.
Edje puts a wrapper function around the host app, and registers that
wrapper function with Lua.  The wrapper function, when called from Lua,
does this -

ecore_thread_main_loop_begin();
call the host app callback();
ecore_thread_main_loop_end();

The first alternative, which the host app must request, is for the
wrapper function to use the table message functions to marshal the
arguments, send it to the main thread, wait for the response (or do
something else), then unmarshal the result before sending it back to
Lua.

The second alternative, which the host app must REALLY request, is for
the host app to say "I'm REALLY going out of my way to be threadsafe,
just call me direct".  No edje wrapper function, BUT the host app still
has to use edje to register this function.

The first two might be done this way -

host_cb = edje_lua2_functions_add(...);
edje_lua2_function_marshal_set(host_cb, function);

The last one could be -

host_cb = edje_lua2_threadsafe_function_add(...);

Note the difference between _functions_ and _function_.  The first call
registers an entire metatable full of functions, in the same way that
edje does for it's functions.  These are the default sort of
functions.  The second call references one of those previously
registered functions, and makes it marshal arguments and results.  The
third one registers a single function, but it could be added to an
existing metatable registered by the first function."

and -

"We are actually half way there.  Anticipating that there would be way
more edje and evas stuff to add to the edje Lua API in the future, I
created generic wrapper functions to make that easier.  Implementing
this would mean exposing those static functions, and writing more of
these generic wrapper stuff.

Lua already provides a mechanism for this sort of thing, but we
currently sandbox that out.  We can provide a very limited version of
the package module, that only lets the Lua script load modules that the
host app explicitly provides.  This keeps edje Lua in it's strictly
sandboxed state."

Raster also wants to thread lots of edje, likely including having edje
Lua scripts as threads, much like I'm doing with LuaSL already.  Plus
LuaJIT SPEEEEED!!.  B-)


Skang notes
-----------

So, what will this variation of skang look like?  For a start, the
syntax will have to be more Lua like, so that's a real basic change. 
Can't use a space as an argument separator, Lua allows only ',' and
';'.  Strings can still use single or double quotes.

The magic "_123" system I used before to specify "position / size in
characters instead of pixels" just wont work in Lua.  Using _ as a table
name with a meta table means that this syntax now becomes "_.123", which
is kinda acceptable.  Note that _.123 is syntax sugar for _["123"], so
I'm not sure if using a number there works.  An alternative is to just
use a string -

foo = widget.label(0, "1", 100, 0, 'Text goes here")

Uses as many characters, and ends up being a value instead of an index,
which is the right thing to do.  While on this subject, anything less
than 1, but more than 0, can be used as a percentage, maybe combined
with "character" mode (this is a new thing) -

foo = widget.label(0, "0.1", 0.5, 0, 'Text goes here")

This could even allow relative placement as mentioned in the skang TODO -

foo = widget.label(0, "otherWidget+0.1", 0.5, 0, 'Text goes here")


"widget" would be a table with functions for dealing with widgets.  It
would include metatable stuff for widget set introspection.  So
"widget.label" would introspect in our widget set for a widget type of
"label", and return a widget type table.  So we could do -

foo = widget.label(0, "0.1", 0.5, 0, 'Text goes here :")
foo:colour(255, 255, 255, 0, 0, 100, 255, 0)
foo:hide()
foo:action("skang.load(some/skang/file.skang)")

Also allow access via table elements -

foo.action = "skang.load('some/skang/file.skang')"
foo.colour.r = 123
foo.look('some/edje/file/somewhere.edj')
foo.help = 'This is a widget for labelling some foo.'

We can use the concat operator (..) to append things to widgets, like
adding choices in a drop down, rows in a grid, etc.

Hmm, need a skang table as well -

skang.module(Evas)
skang.module(Elementary)
skang.clear()
skang.load('some/skang/file.skang')


I don't think we need variable introspection as much, since in this case
Lua is both the skang and the underlaying language, the variables can
just be accessed directly.  Not sure on this, need to experiment.  This
is what we normally need variable introspection for -

foo = 'bar'	-- Using the metatable to override '=' to set the actual value of the widget, say the text of a label or edit widget.
bar = foo	-- The reverse of the above.  Though perhaps these can't be done?
foo = foo .. 'stuff' .. 'more stuff' .. 'another line of the dropdown or whatever'

OK, first two can't be done, metatable only overides that for table elements.  The third one can only be (I might be wrong there) -

foo.value = 'stuff' .. 'more stuff'
foo.value = foo.value .. 'another line of the dropdown or whatever'

Skang relied on the use of the set command to make sure that any
matching widgets and variables got updated.  And the get command.

foo:set('stuff')
bar = foo:get()

On the other hand, given the module / package stuff mentioned below, and
some metatable magic, we could do -

local other = require('otherPackageName')
other.foo = 'stuff'
bar = other.foo

Sooo, how does that squeeze into a "skang" file?

#!skang myApp.skang     -- This is Lua, so this might not work.
-- There's an implied local this = require('myApp')
-- There's an implied local skang = require('skang')
local widget = require('EvasWidgets')
local other = require('otherPackageName')
skang.clear
skang.window(200, 200, "G'day planet.")
quitter = widget.button('Quit', 0.5, 0.5, 0.5, 0.5)
quitter:action('quit')   -- 'quit' is looked up in ThingSpcae.commands, and translated into the Lua 'skang.quit()'.
other.foo = 'stuff'
this.bar = other.foo


Original Skang infrastructure
-----------------------------

Skang starts with a Thing as the base.  In Java it's a base object for
everything else, with a bunch of methods and elements.  We can probably
use a Thing metatable and replicate most of it.  The widget used in the
previous section is built on top of Thing.  Things are stored in
ThingSpace, which is a BonsaiTree of LeafLike Things.  Java's version of
multiple inheritance was used.  Commands are a Thing.

ThingSpace is per users session.  classCache stores cached class
lookukps, command stores commands, module stores loaded modules, param
stores variables and their values, widget stores widgets and their value
/ state.

Skanglet is a generated mapping between actual methods / variables in a
class, and skang.  Each method or variable that ends up in the skanglet
was marked by special javadoc tags.  It also included some meta data
like version and the default skang.  For variables it defined -

"Name", "Field/Method()", "Required", "Shortcut", "Default", "Help text"

... and for methods -

"Name", "Method", "Syntax", "Help text"

In both cases the "Name" was the skang name, it could be different from
the Java name specified in the second field.  Likely we can do away with
the need to generate these skanglet files, and just add stuff to the
Thing.  Modules wrap Skanglets, and takes care of loading them into ThingSpace.

Module are used for -
    loading the Skanglet for the Thing currently running (match java class name to skang file name).
    loading the AWT interface in skang files.
    loading OTHER skanglet based Things
    loading Squeal, StuffSkang, and Who from java when needed.

So in Lua, we can use the package system.  First require(moduleName) to
load up the skang module.  Require can return a table, the Lua package
called ends with "return someTable".  See LuaLSL for an example.  This
package is actually run, so it can figure out what to put in the
returned table.  So for instance it could use inlined methods and such -

local skang = require('skang')
local result = {};
result.author = 'onefang'
result.version = '0.72 alpha 2004-11-19 16:28:00'
local foo
-- The first argument would be the name of a local variable / method.  Which could be accessed via _G?
-- Not sure if we could use a global foo, AND use it directly.
result.foo = skang.newParam('foo', "Required", "Shortcut", "Default", "Help text")
result.func = skang.newCommand('arg1_type,arg2_type', 'Help Text', function (arg1, arg2)
...
end)
return result;


Stuff is the base class used to track database rows and other multi
value things.  Would still need to deal with variable sub things, though
that's likely just using tables.  SquealStuff is the database version,
using Squeal as the database driver wrapper.  Squeal could wrap
esskyuehl?


The pre tokenized structure thingy I had planned in the TODO just wont
work, as it uses symbols.  On the other hand, we will be using Lua
tables anyway.  B-)


NAWS
----

NAWS was always planned, but never actually written.  Though some stuff
was kinda prototyped, especially UberMatrix stuff.  I think this time
around, NAWS should be written in C, based on pure Evas + Edje.  Perhaps
with direct support for introspection?  After EO settles down, I might
use that to.


Edje_Lua notes
--------------

Edje_Lua is sandboxed firmly inside of edje files.  This makes it
unsuitable for our use, as we want to step outside that sandbox. 
However, we still want to wrap Lua around EFL stuff, preferably using
the same API.  Raster fought against using this new eolian stuff to
generate edje_lua2.c, and seems to be against having a "no sandbox" flag
in edje_lua2.c.  For now I'll let q66 go ahead with his Edje Lua via
LuaJIT FFI conversion that uses a variation of eolian written in Lua. 
Seems like much more work than my way of doing it.  shrugs

Unfortunately, this might not be ready soon enough for me, AND it might
not even be suitable.  While I agree that LuaJIT FFI stuff might be a
good idea, if I read the docs correctly FFI simply wraps around C
headers and libraries directly, making it less suitable for sandboxes
and other environments where you want to wrap those library functions
differently.

So, I might have to go ahead with my own plan anyway.  On the other
hand, for skang, I want to wrap things a bit differently.  Might be best
to do that wrapping C side.  In other words, the skang and widget Lua
packages get written in C, and are different from edje_lua2.c.

On the gripping hand, there might be use in including edje_lua2.c style
bindings in GuiLua as an edje package, evas package, elementary package,
etc.  This could be done by going ahead with my plans to modify
edje_lua2.c to include the "no sandbox" flag, then compile it with that
flag turned on to produce a standard Lua package.  Then get eolian to
produce evas_lua.c, elementary_lua.c, etc.

Another idea worth investigating is to rewrite edje_lua2.c to use
introspection directly to figure out it's "combined set and get" style
bindings.


Introspected EFL notes
----------------------

See below for the basic EO notes, where I come to the conclusion that
the other EFL devs wont be adding introspection to EO themselves.  It's
all too hard they say.  Pffft

So, I might have to do this myself, call it "entro"....

Write a patch against eolian that outputs introspection data.  That data
is C code for a hash of EO class names.  Each hash points to a data
structure that includes a hash of method names for that class, and a
pointer to the class description.  Each of those hashes points to a data
structure that includes an array of argument types and names.  The
method ID should be in that structure to, and maybe even the doc text. 
Might even include an Eo internal data structure that links Op IDs to
that method structure (I dunno what they call that).  The final output
is a library of that C code, that includes functions for doing the
actual introspection.

See?  Not to hard at all, all described in a paragraph. lol

One function we could include is a way to "compile" eo_do arguments. 
That would be an array (va_list actually) of method ID's followed by the
arguments for that method.  Some of those arguments might be pointers to
memory structures, so this is not suitable for storing on disk. 
Instead, it could be used for "compiling" commonly used code or hot
paths to get the introspection overhead out of the way early.  Could
also compile parameterized versions, with numbered place holders that
get filled in later.  Certainly we need a Lua interface to eo_do of some
sort.

In the same way that Java skang cached the results of introspection in
ThingSpace.CLASS, we could use cache introspection lookups to speed
things up.  Not a great saving, one hash lookup versus two.  Caching the
"class.method" string that points to the method structure for any method
we have used already.

NAWS could use the compile function to create eo_do commands needed to
create each of it's widget types, and the window.  Yes, I know, NAWS
only has two "widget types".  B-)


For quick and dirty early testing, I'll probably write a widget package
that has hard coded mappings between some basic "label", "button", etc.
and ordinary elementary widgets.  Proper introspection can come later.


LuaJIT FFI notes
----------------

LuaJIT 2.0 FFI needs to have cleaned up versions of C header files
included in the Lua scripts that use it.  This is messy.  Version 2.1
(or perhaps 3.0) has on it's road map to include a C pre processor,
which should help that problem.


EO notes
--------

tl;dr - eo introspection doesn't actually exist, even though it was
"promised" years ago, but looks trivial to add.  On the other hand, the
other EFL devs seem to be fighting against it, so I might need to write
my own.  sigh

object = eo_add(EVAS_OBJ_LINE_CLASS, canvas);
    evas_line.eo.h	-> #define EVAS_OBJ_LINE_CLASS evas_obj_line_class_get()
			-> const Eo_Class *evas_obj_line_class_get(void) EINA_CONST;
    evas_line.eo.c	-> EO_DEFINE_CLASS(evas_obj_line_class_get, &_evas_line_class_desc, EVAS_OBJ_CLASS, NULL);
    Eo.h		-> EO_DEFINE_CLASS is a macro that basically wraps eo_class_new(), and returns it's result.

So Eo_Class is the type of a class, but it's A) opaque, B) deprecated! 
It includes a pointor to the Eo_Class_Description, which includes the
actual name.  I'm not seeing anywhere the names of the get/set
paramaters being passed into the system, or any way to look up a class
based on name.  Not even a way to get to the public Eo_Class_Description
from the opaque Eo_Class.

Eo_Class_Description is at least public.  It includes
Eo_Op_Description's and Eo_Event_Description's.  Eo_Op_ and Eo_Event_
include the name and documentation of the op / event.  The macro used to
generate Eo_Op_ does NOT pass the name, just the ID number as a string. 
There is also Eo_Op_Func_Description, which does not include a name, it
seems to be generated in the constructor.  The same Eo_Op_ ID number is
used to index it.  Seems to be no direct link between Eo_Class and
Eo_Op_func_, just the same Eo_Op_ ID numbers used internally somehow.

eo_do(obj, evas_obj_line_xy_set(200, 200, 300, 300));
    evas_line.eo.h	-> #define evas_obj_line_xy_set(x1, y1, x2, y2) EVAS_OBJ_LINE_ID(EVAS_OBJ_LINE_SUB_ID_XY_SET), EO_TYPECHECK(Evas_Coord , x1), EO_TYPECHECK(Evas_Coord , y1), EO_TYPECHECK(Evas_Coord , x2), EO_TYPECHECK(Evas_Coord , y2)
			   #define EVAS_OBJ_LINE_ID(sub_id) (EVAS_OBJ_LINE_BASE_ID + sub_id)
			   extern EAPI Eo_Op EVAS_OBJ_LINE_BASE_ID;

			   enum
			   {
			      EVAS_OBJ_LINE_SUB_ID_XY_SET,
			      EVAS_OBJ_LINE_SUB_ID_XY_GET,
			      EVAS_OBJ_LINE_SUB_ID_LAST
			   };
    evas_line.eo.c	-> EAPI Eo_Op EVAS_OBJ_LINE_BASE_ID = EO_NOOP;
			    static void _gen_evas_line_class_constructor(Eo_Class *klass)
			    {
			       const Eo_Op_Func_Description func_desc[] = {
			            EO_OP_FUNC(EO_BASE_ID(EO_BASE_SUB_ID_CONSTRUCTOR), _eo_obj_evas_line_constructor),
			            EO_OP_FUNC(EVAS_OBJ_LINE_ID(EVAS_OBJ_LINE_SUB_ID_XY_SET), _eo_obj_evas_line_xy_set),
			            EO_OP_FUNC(EVAS_OBJ_LINE_ID(EVAS_OBJ_LINE_SUB_ID_XY_GET), _eo_obj_evas_line_xy_get),
			            EO_OP_FUNC_SENTINEL
			       };
			       eo_class_funcs_set(klass, func_desc);
			    }

			    static void
			    _eo_obj_evas_line_xy_set(Eo *obj, void *_pd, va_list *list)
			    {
			       Evas_Coord x1 = va_arg(*list, Evas_Coord);
			       Evas_Coord y1 = va_arg(*list, Evas_Coord);
			       Evas_Coord x2 = va_arg(*list, Evas_Coord);
			       Evas_Coord y2 = va_arg(*list, Evas_Coord);
			       _evas_line_xy_set(obj, _pd, x1, y1, x2, y2);
			    }

			    void _evas_line_xy_set(Eo *obj, Evas_Line_Data *pd, Evas_Coord x1, Evas_Coord y1, Evas_Coord x2, Evas_Coord y2);

    evas_object_line.c	-> EOLIAN static void _evas_line_xy_set(Eo *eo_obj, Evas_Line_Data *_pd, Evas_Coord x1, Evas_Coord y1, Evas_Coord x2, Evas_Coord y2)


Evas_Object_Line *o = eo_data_scope_get(obj, EVAS_OBJ_LINE_CLASS);
Evas_Object_Line *o = eo_data_ref(obj, EVAS_OBJ_LINE_CLASS);


.. evas_line.eo

class Evas_Line (Evas_Object)
{
   legacy_prefix: evas_object_line;
   eo_prefix: evas_obj_line;
   properties {
      xy {
         set {
            /*@
            @since 1.8

            Sets the coordinates of the end points of the given evas line object. */
         }
         get {
            /*@
            Retrieves the coordinates of the end points of the given evas line object.
            second end point. */
         }
         values {
            Evas_Coord x1; /*@ The X coordinate of the first point. */
            Evas_Coord y1; /*@ The Y coordinate of the first point. */
            Evas_Coord x2; /*@ The X coordinate of the second point. */
            Evas_Coord y2; /*@ The Y coordinate of the second point. */
         }
      }
   }
   implements {
      Eo_Base::constructor;
   }

}

... evas_line.eo.c

EAPI void
evas_object_line_xy_set(Evas_Object *obj, Evas_Coord x1, Evas_Coord y1, Evas_Coord x2, Evas_Coord y2)
{
   eo_do((Eo *) obj, evas_obj_line_xy_set(x1, y1, x2, y2));
   return ;
}


static const Eo_Class_Description _evas_line_class_desc = {
     EO_VERSION,
     "Evas_Line",
     EO_CLASS_TYPE_REGULAR,
     EO_CLASS_DESCRIPTION_OPS(&EVAS_OBJ_LINE_BASE_ID, _evas_line_op_desc, EVAS_OBJ_LINE_SUB_ID_LAST),
     _evas_line_event_desc,
     sizeof(Evas_Line_Data),
     _gen_evas_line_class_constructor,
     NULL
};


skang vs edje vs LL shit
------------------------

EDJE
Verbose, complex.
Used to place and decorate widget parts.
Can be used to place entire widgets through externals and swallows.
Basic parts relative to each other.
Signals and messages.
Embryo scripts.
Lua scripts (sandboxed).


SKANG
Tight, simple.
Used to place widgets, and describe actions.
Can include some really basic scripting.
Widgets in a fixed position, but included stuff for relative placement in the TODO.
Automated associations between widget name and variable (and method?) via introspection.
Actions.
Looks (could easily be extended to edje groups).
Extensible.
Can be used to pass values around.


LL SHIT
Verbose, even worse, XML, more crap than is needed.
Menus.
Windows with widgets relative to each other.
    Basically nested rectangles.
Manual association of widgets to methods.
Can include tool tip string, enabled, visible, hover cursor, bounding rectangle?, mouse opaque?, tab groups, font (name, size, style, and alignment).
    More stuff, typically hidden in the OO somewhere.  sigh
Generally uses fixed image and colour names, which skins overide.
Skins can also overide the XML files.
Translations provide override XML files that need only override the text bits.


old notes
---------

See if it's possible to disable the Emerald/Phoenix/Firestorm LSL bridge
when connecting to OpenSim.  Actually, they finally came to their senses
and support OpenSim now, so that should be sorted already.