From 637177eb1397ef1800027bccd50dbdc1af29a15b Mon Sep 17 00:00:00 2001 From: David Walter Seikel Date: Mon, 13 Jan 2014 21:08:31 +1000 Subject: Remove LuaJIT source, we can use packaged LuaJIT 2.0 release now. Also some cleanups related to the other library removals. --- libraries/luajit-2.0/doc/ext_ffi_semantics.html | 1146 ----------------------- 1 file changed, 1146 deletions(-) delete mode 100644 libraries/luajit-2.0/doc/ext_ffi_semantics.html (limited to 'libraries/luajit-2.0/doc/ext_ffi_semantics.html') diff --git a/libraries/luajit-2.0/doc/ext_ffi_semantics.html b/libraries/luajit-2.0/doc/ext_ffi_semantics.html deleted file mode 100644 index 661b0b4..0000000 --- a/libraries/luajit-2.0/doc/ext_ffi_semantics.html +++ /dev/null @@ -1,1146 +0,0 @@ - - - -FFI Semantics - - - - - - - - - -
-Lua -
- - -
-

-This page describes the detailed semantics underlying the FFI library -and its interaction with both Lua and C code. -

-

-Given that the FFI library is designed to interface with C code -and that declarations can be written in plain C syntax, it -closely follows the C language semantics, wherever possible. -Some minor concessions are needed for smoother interoperation with Lua -language semantics. -

-

-Please don't be overwhelmed by the contents of this page — this -is a reference and you may need to consult it, if in doubt. It doesn't -hurt to skim this page, but most of the semantics "just work" as you'd -expect them to work. It should be straightforward to write -applications using the LuaJIT FFI for developers with a C or C++ -background. -

-

-Please note: this doesn't comprise the final specification for the FFI -semantics, yet. Some semantics may need to be changed, based on your -feedback. Please report any problems you may -encounter or any improvements you'd like to see — thank you! -

- -

C Language Support

-

-The FFI library has a built-in C parser with a minimal memory -footprint. It's used by the ffi.* library -functions to declare C types or external symbols. -

-

-It's only purpose is to parse C declarations, as found e.g. in -C header files. Although it does evaluate constant expressions, -it's not a C compiler. The body of inline -C function definitions is simply ignored. -

-

-Also, this is not a validating C parser. It expects and -accepts correctly formed C declarations, but it may choose to -ignore bad declarations or show rather generic error messages. If in -doubt, please check the input against your favorite C compiler. -

-

-The C parser complies to the C99 language standard plus -the following extensions: -

- -

-The following C types are pre-defined by the C parser (like -a typedef, except re-declarations will be ignored): -

- -

-You're encouraged to use these types in preference to the -compiler-specific extensions or the target-dependent standard types. -E.g. char differs in signedness and long differs in -size, depending on the target architecture and platform ABI. -

-

-The following C features are not supported: -

- - -

C Type Conversion Rules

- -

Conversions from C types to Lua objects

-

-These conversion rules apply for read accesses to -C types: indexing pointers, arrays or -struct/union types; reading external variables or -constant values; retrieving return values from C calls: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
InputConversionOutput
int8_t, int16_t→sign-ext int32_t → doublenumber
uint8_t, uint16_t→zero-ext int32_t → doublenumber
int32_t, uint32_t→ doublenumber
int64_t, uint64_tboxed value64 bit int cdata
double, float→ doublenumber
bool0 → false, otherwise trueboolean
Complex numberboxed valuecomplex cdata
Vectorboxed valuevector cdata
Pointerboxed valuepointer cdata
Arrayboxed referencereference cdata
struct/unionboxed referencereference cdata
-

-Bitfields or enum types are treated like their underlying -type. -

-

-Reference types are dereferenced before a conversion can take -place — the conversion is applied to the C type pointed to -by the reference. -

- -

Conversions from Lua objects to C types

-

-These conversion rules apply for write accesses to -C types: indexing pointers, arrays or -struct/union types; initializing cdata objects; -casts to C types; writing to external variables; passing -arguments to C calls: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
InputConversionOutput
number→double
booleanfalse → 0, true → 1bool
nilNULL →(void *)
userdatauserdata payload →(void *)
lightuserdatalightuserdata address →(void *)
stringmatch against enum constantenum
stringcopy string data + zero-byteint8_t[], uint8_t[]
stringstring data →const char[]
functioncreate callback →C function type
tabletable initializerArray
tabletable initializerstruct/union
cdatacdata payload →C type
-

-If the result type of this conversion doesn't match the -C type of the destination, the -conversion rules between C types -are applied. -

-

-Reference types are immutable after initialization ("no re-seating of -references"). For initialization purposes or when passing values to -reference parameters, they are treated like pointers. Note that unlike -in C++, there's no way to implement automatic reference generation of -variables under the Lua language semantics. If you want to call a -function with a reference parameter, you need to explicitly pass a -one-element array. -

- -

Conversions between C types

-

-These conversion rules are more or less the same as the standard -C conversion rules. Some rules only apply to casts, or require -pointer or type compatibility: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
InputConversionOutput
Signed integer→narrow or sign-extendInteger
Unsigned integer→narrow or zero-extendInteger
Integer→rounddouble, float
double, float→trunc int32_t →narrow(u)int8_t, (u)int16_t
double, float→trunc(u)int32_t, (u)int64_t
double, float→roundfloat, double
Numbern == 0 → 0, otherwise 1bool
boolfalse → 0, true → 1Number
Complex numberconvert real partNumber
Numberconvert real part, imag = 0Complex number
Complex numberconvert real and imag partComplex number
Numberconvert scalar and replicateVector
Vectorcopy (same size)Vector
struct/uniontake base address (compat)Pointer
Arraytake base address (compat)Pointer
Functiontake function addressFunction pointer
Numberconvert via uintptr_t (cast)Pointer
Pointerconvert address (compat/cast)Pointer
Pointerconvert address (cast)Integer
Arrayconvert base address (cast)Integer
Arraycopy (compat)Array
struct/unioncopy (identical type)struct/union
-

-Bitfields or enum types are treated like their underlying -type. -

-

-Conversions not listed above will raise an error. E.g. it's not -possible to convert a pointer to a complex number or vice versa. -

- -

Conversions for vararg C function arguments

-

-The following default conversion rules apply when passing Lua objects -to the variable argument part of vararg C functions: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
InputConversionOutput
number→double
booleanfalse → 0, true → 1bool
nilNULL →(void *)
userdatauserdata payload →(void *)
lightuserdatalightuserdata address →(void *)
stringstring data →const char *
float cdata→double
Array cdatatake base addressElement pointer
struct/union cdatatake base addressstruct/union pointer
Function cdatatake function addressFunction pointer
Any other cdatano conversionC type
-

-To pass a Lua object, other than a cdata object, as a specific type, -you need to override the conversion rules: create a temporary cdata -object with a constructor or a cast and initialize it with the value -to pass: -

-

-Assuming x is a Lua number, here's how to pass it as an -integer to a vararg function: -

-
-ffi.cdef[[
-int printf(const char *fmt, ...);
-]]
-ffi.C.printf("integer value: %d\n", ffi.new("int", x))
-
-

-If you don't do this, the default Lua number → double -conversion rule applies. A vararg C function expecting an integer -will see a garbled or uninitialized value. -

- -

Initializers

-

-Creating a cdata object with -ffi.new() or the -equivalent constructor syntax always initializes its contents, too. -Different rules apply, depending on the number of optional -initializers and the C types involved: -

- - -

Table Initializers

-

-The following rules apply if a Lua table is used to initialize an -Array or a struct/union: -

- -

-Example: -

-
-local ffi = require("ffi")
-
-ffi.cdef[[
-struct foo { int a, b; };
-union bar { int i; double d; };
-struct nested { int x; struct foo y; };
-]]
-
-ffi.new("int[3]", {})            --> 0, 0, 0
-ffi.new("int[3]", {1})           --> 1, 1, 1
-ffi.new("int[3]", {1,2})         --> 1, 2, 0
-ffi.new("int[3]", {1,2,3})       --> 1, 2, 3
-ffi.new("int[3]", {[0]=1})       --> 1, 1, 1
-ffi.new("int[3]", {[0]=1,2})     --> 1, 2, 0
-ffi.new("int[3]", {[0]=1,2,3})   --> 1, 2, 3
-ffi.new("int[3]", {[0]=1,2,3,4}) --> error: too many initializers
-
-ffi.new("struct foo", {})            --> a = 0, b = 0
-ffi.new("struct foo", {1})           --> a = 1, b = 0
-ffi.new("struct foo", {1,2})         --> a = 1, b = 2
-ffi.new("struct foo", {[0]=1,2})     --> a = 1, b = 2
-ffi.new("struct foo", {b=2})         --> a = 0, b = 2
-ffi.new("struct foo", {a=1,b=2,c=3}) --> a = 1, b = 2  'c' is ignored
-
-ffi.new("union bar", {})        --> i = 0, d = 0.0
-ffi.new("union bar", {1})       --> i = 1, d = ?
-ffi.new("union bar", {[0]=1,2}) --> i = 1, d = ?    '2' is ignored
-ffi.new("union bar", {d=2})     --> i = ?, d = 2.0
-
-ffi.new("struct nested", {1,{2,3}})     --> x = 1, y.a = 2, y.b = 3
-ffi.new("struct nested", {x=1,y={2,3}}) --> x = 1, y.a = 2, y.b = 3
-
- -

Operations on cdata Objects

-

-All of the standard Lua operators can be applied to cdata objects or a -mix of a cdata object and another Lua object. The following list shows -the valid combinations. All other combinations currently raise an -error. -

-

-Reference types are dereferenced before performing each of -the operations below — the operation is applied to the -C type pointed to by the reference. -

-

-The pre-defined operations are always tried first before deferring to a -metamethod for a ctype (if defined). -

- -

Indexing a cdata object

- -

-Note: since there's (deliberately) no address-of operator, a cdata -object holding a value type is effectively immutable after -initialization. The JIT compiler benefits from this fact when applying -certain optimizations. -

-

-As a consequence of this, the elements of complex numbers and -vectors are immutable. But the elements of an aggregate holding these -types may be modified of course. I.e. you cannot assign to -foo.c.im, but you can assign a (newly created) complex number -to foo.c. -

- -

Calling a cdata object

- - -

Arithmetic on cdata objects

- - -

Comparisons of cdata objects

- - -

cdata objects as table keys

-

-Lua tables may be indexed by cdata objects, but this doesn't provide -any useful semantics — cdata objects are unsuitable as table -keys! -

-

-A cdata object is treated like any other garbage-collected object and -is hashed and compared by its address for table indexing. Since -there's no interning for cdata value types, the same value may be -boxed in different cdata objects with different addresses. Thus -t[1LL+1LL] and t[2LL] usually do not point to -the same hash slot and they certainly do not point to the same -hash slot as t[2]. -

-

-It would seriously drive up implementation complexity and slow down -the common case, if one were to add extra handling for by-value -hashing and comparisons to Lua tables. Given the ubiquity of their use -inside the VM, this is not acceptable. -

-

-There are three viable alternatives, if you really need to use cdata -objects as keys: -

- - -

Garbage Collection of cdata Objects

-

-All explicitly (ffi.new(), ffi.cast() etc.) or -implicitly (accessors) created cdata objects are garbage collected. -You need to ensure to retain valid references to cdata objects -somewhere on a Lua stack, an upvalue or in a Lua table while they are -still in use. Once the last reference to a cdata object is gone, the -garbage collector will automatically free the memory used by it (at -the end of the next GC cycle). -

-

-Please note that pointers themselves are cdata objects, however they -are not followed by the garbage collector. So e.g. if you -assign a cdata array to a pointer, you must keep the cdata object -holding the array alive as long as the pointer is still in use: -

-
-ffi.cdef[[
-typedef struct { int *a; } foo_t;
-]]
-
-local s = ffi.new("foo_t", ffi.new("int[10]")) -- WRONG!
-
-local a = ffi.new("int[10]") -- OK
-local s = ffi.new("foo_t", a)
--- Now do something with 's', but keep 'a' alive until you're done.
-
-

-Similar rules apply for Lua strings which are implicitly converted to -"const char *": the string object itself must be -referenced somewhere or it'll be garbage collected eventually. The -pointer will then point to stale data, which may have already been -overwritten. Note that string literals are automatically kept -alive as long as the function containing it (actually its prototype) -is not garbage collected. -

-

-Objects which are passed as an argument to an external C function -are kept alive until the call returns. So it's generally safe to -create temporary cdata objects in argument lists. This is a common -idiom for passing specific C types to -vararg functions. -

-

-Memory areas returned by C functions (e.g. from malloc()) -must be manually managed, of course (or use -ffi.gc()). Pointers to -cdata objects are indistinguishable from pointers returned by C -functions (which is one of the reasons why the GC cannot follow them). -

- -

Callbacks

-

-The LuaJIT FFI automatically generates special callback functions -whenever a Lua function is converted to a C function pointer. This -associates the generated callback function pointer with the C type -of the function pointer and the Lua function object (closure). -

-

-This can happen implicitly due to the usual conversions, e.g. when -passing a Lua function to a function pointer argument. Or you can use -ffi.cast() to explicitly cast a Lua function to a -C function pointer. -

-

-Currently only certain C function types can be used as callback -functions. Neither C vararg functions nor functions with -pass-by-value aggregate argument or result types are supported. There -are no restrictions for the kind of Lua functions that can be called -from the callback — no checks for the proper number of arguments -are made. The return value of the Lua function will be converted to the -result type and an error will be thrown for invalid conversions. -

-

-It's allowed to throw errors across a callback invocation, but it's not -advisable in general. Do this only if you know the C function, that -called the callback, copes with the forced stack unwinding and doesn't -leak resources. -

- -

Callback resource handling

-

-Callbacks take up resources — you can only have a limited number -of them at the same time (500 - 1000, depending on the -architecture). The associated Lua functions are anchored to prevent -garbage collection, too. -

-

-Callbacks due to implicit conversions are permanent! There is no -way to guess their lifetime, since the C side might store the -function pointer for later use (typical for GUI toolkits). The associated -resources cannot be reclaimed until termination: -

-
-ffi.cdef[[
-typedef int (__stdcall *WNDENUMPROC)(void *hwnd, intptr_t l);
-int EnumWindows(WNDENUMPROC func, intptr_t l);
-]]
-
--- Implicit conversion to a callback via function pointer argument.
-local count = 0
-ffi.C.EnumWindows(function(hwnd, l)
-  count = count + 1
-  return true
-end, 0)
--- The callback is permanent and its resources cannot be reclaimed!
--- Ok, so this may not be a problem, if you do this only once.
-
-

-Note: this example shows that you must properly declare -__stdcall callbacks on Windows/x86 systems. The calling -convention cannot be automatically detected, unlike for -__stdcall calls to Windows functions. -

-

-For some use cases it's necessary to free up the resources or to -dynamically redirect callbacks. Use an explicit cast to a -C function pointer and keep the resulting cdata object. Then use -the cb:free() -or cb:set() methods -on the cdata object: -

-
--- Explicitly convert to a callback via cast.
-local count = 0
-local cb = ffi.cast("WNDENUMPROC", function(hwnd, l)
-  count = count + 1
-  return true
-end)
-
--- Pass it to a C function.
-ffi.C.EnumWindows(cb, 0)
--- EnumWindows doesn't need the callback after it returns, so free it.
-
-cb:free()
--- The callback function pointer is no longer valid and its resources
--- will be reclaimed. The created Lua closure will be garbage collected.
-
- -

Callback performance

-

-Callbacks are slow! First, the C to Lua transition itself -has an unavoidable cost, similar to a lua_call() or -lua_pcall(). Argument and result marshalling add to that cost. -And finally, neither the C compiler nor LuaJIT can inline or -optimize across the language barrier and hoist repeated computations out -of a callback function. -

-

-Do not use callbacks for performance-sensitive work: e.g. consider a -numerical integration routine which takes a user-defined function to -integrate over. It's a bad idea to call a user-defined Lua function from -C code millions of times. The callback overhead will be absolutely -detrimental for performance. -

-

-It's considerably faster to write the numerical integration routine -itself in Lua — the JIT compiler will be able to inline the -user-defined function and optimize it together with its calling context, -with very competitive performance. -

-

-As a general guideline: use callbacks only when you must, because -of existing C APIs. E.g. callback performance is irrelevant for a -GUI application, which waits for user input most of the time, anyway. -

-

-For new designs avoid push-style APIs (C function repeatedly -calling a callback for each result). Instead use pull-style APIs -(call a C function repeatedly to get a new result). Calls from Lua -to C via the FFI are much faster than the other way round. Most well-designed -libraries already use pull-style APIs (read/write, get/put). -

- -

C Library Namespaces

-

-A C library namespace is a special kind of object which allows -access to the symbols contained in shared libraries or the default -symbol namespace. The default -ffi.C namespace is -automatically created when the FFI library is loaded. C library -namespaces for specific shared libraries may be created with the -ffi.load() API -function. -

-

-Indexing a C library namespace object with a symbol name (a Lua -string) automatically binds it to the library. First the symbol type -is resolved — it must have been declared with -ffi.cdef. Then the -symbol address is resolved by searching for the symbol name in the -associated shared libraries or the default symbol namespace. Finally, -the resulting binding between the symbol name, the symbol type and its -address is cached. Missing symbol declarations or nonexistent symbol -names cause an error. -

-

-This is what happens on a read access for the different kinds of -symbols: -

- -

-This is what happens on a write access: -

- -

-C library namespaces themselves are garbage collected objects. If -the last reference to the namespace object is gone, the garbage -collector will eventually release the shared library reference and -remove all memory associated with the namespace. Since this may -trigger the removal of the shared library from the memory of the -running process, it's generally not safe to use function -cdata objects obtained from a library if the namespace object may be -unreferenced. -

-

-Performance notice: the JIT compiler specializes to the identity of -namespace objects and to the strings used to index it. This -effectively turns function cdata objects into constants. It's not -useful and actually counter-productive to explicitly cache these -function objects, e.g. local strlen = ffi.C.strlen. OTOH it -is useful to cache the namespace itself, e.g. local C = -ffi.C. -

- -

No Hand-holding!

-

-The FFI library has been designed as a low-level library. The -goal is to interface with C code and C data types with a -minimum of overhead. This means you can do anything you can do -from C: access all memory, overwrite anything in memory, call -machine code at any memory address and so on. -

-

-The FFI library provides no memory safety, unlike regular Lua -code. It will happily allow you to dereference a NULL -pointer, to access arrays out of bounds or to misdeclare -C functions. If you make a mistake, your application might crash, -just like equivalent C code would. -

-

-This behavior is inevitable, since the goal is to provide full -interoperability with C code. Adding extra safety measures, like -bounds checks, would be futile. There's no way to detect -misdeclarations of C functions, since shared libraries only -provide symbol names, but no type information. Likewise there's no way -to infer the valid range of indexes for a returned pointer. -

-

-Again: the FFI library is a low-level library. This implies it needs -to be used with care, but it's flexibility and performance often -outweigh this concern. If you're a C or C++ developer, it'll be easy -to apply your existing knowledge. OTOH writing code for the FFI -library is not for the faint of heart and probably shouldn't be the -first exercise for someone with little experience in Lua, C or C++. -

-

-As a corollary of the above, the FFI library is not safe for use by -untrusted Lua code. If you're sandboxing untrusted Lua code, you -definitely don't want to give this code access to the FFI library or -to any cdata object (except 64 bit integers or complex -numbers). Any properly engineered Lua sandbox needs to provide safety -wrappers for many of the standard Lua library functions — -similar wrappers need to be written for high-level operations on FFI -data types, too. -

- -

Current Status

-

-The initial release of the FFI library has some limitations and is -missing some features. Most of these will be fixed in future releases. -

-

-C language support is -currently incomplete: -

- -

-The JIT compiler already handles a large subset of all FFI operations. -It automatically falls back to the interpreter for unimplemented -operations (you can check for this with the --jv command line option). -The following operations are currently not compiled and may exhibit -suboptimal performance, especially when used in inner loops: -

- -

-Other missing features: -

- -
-
- - - -- cgit v1.1