As we already
stated, we finished the first step of integration of the
SpiderMonkey engine to our mORMot framework.
Version 1.8.5 of the library is already integrated, and latest official
revision will be soon merged,
thanks to mpv's great contribution.
It can be seen as stable, since it is already used on production site to serve more than
1,000,000 requests per day.
![]()
You can now easily uses JavaScript on both client and server side.
On server side, mORMot's implementation offers an unique concept, i.e.
true multi-threading,
which is IMHO a huge enhancement when compared to the regular node.js mono-threaded implementation,
and its callback hell.
In fact, node.js official marketing states its non-blocking scheme is
a plus. It allows to define a HTTP server in a few lines, but huge server
applications need JavaScript experts not to sink into a state a
disgrace.
Scripting abilities of mORMot
As a Delphi framework, mORMot premium language support is
for the object pascal language. But it could be convenient to have
some part of your software not fixed within the executable. In fact, once the
application is compiled, execution flow is written in stone: you can't change
it, unless you modify the Delphi source and compile it again. Since
mORMot is Open Source, you can ship the whole source code to
your customers or services with no restriction, and diffuse your own code as
pre-compiled .dcu files, but your end-user will need to have a
Delphi IDE installed (and paid), and know the Delphi language.
This is when scripting does come on the scene.
For instance, scripting may allow to customize an application behavior for an
end-user (i.e. for reporting), or let a domain expert define evolving
appropriate business rules - following Domain Driven Design.
If your business model is to publish a core domain expertise (e.g.
accounting, peripheral driving, database model, domain objects, communication,
AJAX clients...) among several clients, you will sooner or later need to adapt
your application to one or several of your customers. There is no "one
exe to rule them all". Maintaining several executables could
become a "branch-hell". Scripting is welcome here: speed and memory critical
functionality (in which mORMot excels) will be hard-coded within the
main executable, then everything else could be defined in script.
There are plenty of script languages available.
We considered DelphiWebScript
which is well maintained and expressive (it is the code of our beloved
SmartMobileStudio), but is not very commonly used. We still want to
include it in the close future.
Then LUA defines a light and versatile
general-purpose language, dedicated to be embedded in any application. Sounds
like a viable solution: if you can help with it, your contribution is
welcome!
We did also take into consideration Python
and Ruby but both are now far from
light, and are not meant to be embedded, since they are general-purpose
languages, with a huge set of full-featured packages.
Then, there is JavaScript:
- This is the World Wide Web assembler. Every programmer in one way
or another knows JavaScript.
- JavaScript can be a very powerful language - see Crockford's book
"JavaScript -
The Good Parts";
- There are a huge number of libraries written in JavaScript:
template engines (jade, mustache...), SOAP and LDAP clients, and many others
(including all
node.js libraries of course); - It was the base for some strongly-typed syntax extensions, like
CoffeScript, TypeScript, Dart;
- In case of AJAX / Rich Internet Application we can
directly share part of logic between client and server (validation, template
rendering...) without any middle-ware;
- One long time mORMot's user (Pavel, aka mpv) already
integrated SpiderMonkey to mORMot's core. His solution is
used on production to serve billion of requests per day, with success. We
officially integrated his units.
Thanks a lot, Pavel!
As a consequence, mORMot introduced direct JavaScript
support via SpiderMonkey.
It allows to:
- Execute Delphi code from JavaScript - including our ORM
or SOA methods, or even reporting;
- Consume JavaScript code from Delphi (e.g. to define and
customize any service or rule, or use some existing
.js
library); - Expose JavaScript objects and functions via a
TSMVariant custom variant type: it allows to access any
JavaScript object properties or call any of its functions via
late-binding, from your Delphi code, just as if it was written in
native Object-Pascal; - Follow a classic synchronous blocking pattern, rooted on mORMot's multi-thread
efficient model, easy to write and maintain;
- Handle JavaScript or Delphi objects as UTF-8 JSON, ready
to be published or consumed via mORMot's RESTful Client-Server remote access.
SpiderMonkey integration
A powerful JavaScript engine
SpiderMonkey,
the Mozilla JavaScript engine, can be embedded in your mORMot
application. It could be used on client side, within a Delphi application (e.g.
for reporting), but the main interest of it may be on the server side.
The word JavaScript may bring to mind features such as event
handlers (like onclick), DOM objects,
window.open, and XMLHttpRequest.
But all of these features are actually not provided by the
SpiderMonkey engine itself.
SpiderMonkey provides a few core JavaScript data
types—numbers, strings, Arrays, Objects, and so on—and a few methods, such as
Array.push. It also makes it easy for each application to expose
some of its own objects and functions to JavaScript code. Browsers
expose DOM objects. Your application will expose objects that are relevant for
the kind of scripts you want to write. It is up to the application developer to
decide what objects and methods are exposed to scripts.
Direct access to the SpiderMonkey API
The SynSMAPI.pas unit is a tuned conversion of the
SpiderMonkey API, providing full ECMAScript 5 support and
JIT.
You could take a look at the
full description of this low-level API.
But the SynSM.pas unit will encapsulate most of it into higher
level Delphi classes and structures (including a custom variant
type), so you probably won't need to use SynSMAPI.pas directly in
your code:
| Type | Description |
TSMEngineManager | main access point to the SpiderMonkey per-thread scripting
engines |
TSMEngine | implements a Thread-Safe JavaScript engine instance |
TSMObject | wrap a JavaScript object and its execution context |
TSMValue | wrap a JavaScript value, and interfaces it with Delphi types |
TSMVariant /
TSMVariantData | define a custom variant type, for direct access to any
JavaScript object, with late-binding |
We will see know how to work with all those classes.
Execution scheme
The SpiderMonkey JavaScript engine compiles and executes scripts
containing JavaScript statements and functions. The engine handles
memory allocation for the objects needed to execute scripts, and it cleans
up—garbage collects—objects it no longer needs.
In order to run any JavaScript code in SpiderMonkey, an
application must have three key elements:
- A
JSRuntime, - A
JSContext, - And a global
JSObject.
A JSRuntime, or runtime, is the space in which the JavaScript
variables, objects, scripts, and contexts used by your application are
allocated. Every JSContext and every object in an application
lives within a JSRuntime. They cannot travel to other runtimes or
be shared across runtimes.
A JSContext, or context, is like a little machine that can do
many things involving JavaScript code and objects. It can compile and
execute scripts, get and set object properties, call JavaScript
functions, convert JavaScript data from one type to another, create
objects, and so on.
Lastly, the globalJSObject is a JavaScript
object which contains all the classes, functions, and variables that are
available for JavaScript code to use. Whenever a web browser code does
something like window.open("http://www.mozilla.org/"), it is
accessing a global property, in this case window.
SpiderMonkey applications have full control over what global
properties scripts can see.
Every SpiderMonkey instance starts out every execution context by
creating its JSRunTime, JSContext instances, and a
global JSObject. It populates this global object with the standard
JavaScript classes, like Array and Object.
Then application initialization code will add whatever custom classes,
functions, and variables (like window) the application wants to
provide; it may be, for a mORMot server application, ORM access or SOA
services consumption and/or implementation.
Each time the application runs a JavaScript script (using, for
example, JS_EvaluateScript), it provides the global object for
that script to use. As the script runs, it can create global functions and
variables of its own. All of these functions, classes, and variables are stored
as properties of the global object.
Creating your execution context
The main point about those three key elements is that, in the current
implementation pattern of SpiderMonkey, runtime, context or global
objects are not thread-safe.
Therefore, in the mORMot's use of this library, each thread will
have its own instance of each.
In the SynSM.pas unit, a TSMEngine class has been
defined to give access to all those linked elements:
TSMEngine = class
...
/// access to the associated global object as a TSMVariant custom variant// - allows direct property and method executions in Delphi code, via// late-bindingproperty Global: variant read FGlobal;
/// access to the associated global object as a TSMObject wrapper// - you can use it to register a methodproperty GlobalObject: TSMObject read FGlobalObject;
/// access to the associated global object as low-level PJSObjectproperty GlobalObj: PJSObject read FGlobalObject.fobj;
/// access to the associated execution contextproperty cx: PJSContext read fCx;
/// access to the associated execution runtimeproperty rt: PJSRuntime read frt;
...
Our implementation will define one Runtime, one Context, and one global
object per thread, i.e. one TSMEngine class instance per
thread.
A JSRuntime, or runtime, is created for each
TSMEngine instance. In practice, you won't need access to this
value, but rely either on a JSContext or directly a
TSMEngine.
A JSContext, or context, will be the main entry point of all
SpiderMonkey API, which expect this context to be supplied as
parameter. In mORMot, you can retrieve the running TSMEngine from
its context by using the function TSMObject.Engine: TSMEngine - in
fact, the engine instance is stored in the private data slot of each
JSContext.
Lastly, the TSMEngine's global object contains all the
classes, functions, and variables that are available for JavaScript code to
use. For a mORMot server application, ORM access or SOA services
consumption and/or implementation, as stated above.
You can note that there are several ways to access this global object
instance, from high-level to low-level JavaScript object types. The
TSMEngine.Global property above is in fact a variant.
Our SynSM.pas unit defines in fact a custom variant
type, identified as the TSMVariant class, able to access any
JavaScript object via late-binding, for both variables and functions:
engine.Global.MyVariable := 1.0594631;
engine.Global.MyFunction(1,'text');
Most web applications only need one runtime, since they are running in a
single thread - and (ab)use of callbacks for non-blocking execution. But in
mORMot, you will have one TMSEngine instance per thread,
using the TSMEngineManager.ThreadSafeEngine method. Then all
execution may be blocking, without any noticeable performance issue, since the
whole mORMot threading design was defined to maximize execution
resources.
Blocking threading model
This threading model is the big difference with other server-side scripting
implementation schemes, e.g. the well-known node.js solution.
Multi-threading is not evil, when properly used. And thanks to the
mORMot's design, you won't be afraid of writing blocking
JavaScript code, without any callbacks. In practice, those callbacks are what
makes most JavaScript code difficult to maintain.
On the client side, i.e. in a web browser, the JavaScript engine
only uses one thread per web page, then uses callbacks to defer execution of
long-running methods (like a remote HTTP request).
If fact, this is one well identified performance issue of modern AJAX
applications. For instance, it is not possible to perform some intensive
calculation in JavaScript, without breaking the web application
responsiveness: you have to split your computation task in small tasks, then
let the JavaScript code pause, until a next piece of computation could
be triggerred... Some browsers did only start to uncouple the
JavaScript execution thread with the HTML rendering thread - and even
this is hard to do... we reached here the limit of a technology rooted in the
80's...
On the server side, node.js did follow this pattern, which did
make sense (it allows to share code with the client side, with some name-space
tricks), but it is also IMHO a big waste of resources. Why should we stick to
an implementation pattern inherited from the 80's computing model, when all
CPUs were mono core, and threads were not available?
The main problem when working with one single thread, is that your code
shall be asynchronous. Soon or later, you will face a syndrome known as
"Callback Hell". In short, you are nesting anonymous functions, and
define callbacks. The main issue, in addition to lower readability and being
potentially sunk into function() nesting, is that you just lost
the JavaScript exception model. In fact, each callback function has to
explicitly check for the error (returned as a parameter in the callback
function), and handle it.
Of course, you can use so-called Promises and some nice libraries -
mainly async.js.
But even those libraries add complexity, and make code more difficult to write.
For instance, consider the following non-blocking/asynchronous code:
getTweetsFor("domenic") // promise-returning function
.then(function (tweets) {
var shortUrls = parseTweetsForUrls(tweets);
var mostRecentShortUrl = shortUrls[0];
return expandUrlUsingTwitterApi(mostRecentShortUrl); // promise-returning function
})
.then(httpGet) // promise-returning function
.then(
function (responseBody) {
console.log("Most recent link text:", responseBody);
},
function (error) {
console.error("Error with the twitterverse:", error);
}
);
Taken from this web
site.
This kind of code will be perfectly readable for a JavaScript daily
user, or someone fluent with functional languages.
But the following blocking/synchronous code may sound much more familiar,
safer and less verbose, to most Delphi / Java / C# programmer:
try {
var tweets = getTweetsFor("domenic"); // blockingvar shortUrls = parseTweetsForUrls(tweets);
var mostRecentShortUrl = shortUrls[0];
var responseBody = httpGet(expandUrlUsingTwitterApi(mostRecentShortUrl)); // blocking x 2
console.log("Most recent link text:", responseBody);
} catch (error) {
console.error("Error with the twitterverse: ", error);
}
Thanks to the blocking pattern, it becomes obvious that code readability and
maintainability is as high as possible, and error detection is handled nicely
via JavaScript exceptions, and a global try .. catch.
Last but not least, debugging blocking code is easy and straightforward,
since the execution will be linear, following the code flow.
Upcoming ECMAScript 6 should go even further thanks to the
yield keyword and some task generators - see taskjs - so that asynchronous code may become closer to
the synchronous pattern. But even with yield, your code won't be
as clean as with plain blocking style.
In mORMot, we did choose to follow an alternate path, i.e. write
blocking synchronous code. Sample above shows how easier it is to work with. If
you use it to define some huge business logic, or let a domain expert write the
code, blocking syntax is much more straightforward.
Of course, mORMot allows you to use callbacks and functional
programming pattern in your JavaScript code, if needed. But by
default, you are allowed to write KISS blocking code.
Interaction with existing code
Within mORMot units, you can mix Delphi and
JavaScript code by two ways:
- Either define your own functions in Delphi code, and execute them
from JavaScript;
- Or define your own functions in JavaScript code (including any
third-party library), and execute them from Delphi.
Like for other part of our framework, performance and integration has been
tuned, to follow our KISS way.
You can take a look at "22 - JavaScript HTTPApi web
server\JSHttpApiServer.dpr" sample for reference code.
Proper engine initialization
As was previously stated, the main point to interface the
JavaScript engine is to register all methods when the
TSMEngine instance is initialized.
For this, you set the corresponding OnNewEngine callback event
to the main TSMEngineManager instance.
See for instance, in the sample code:
constructor TTestServer.Create(const Path: TFileName);
begin
...
fSMManager := TSMEngineManager.Create;
fSMManager.OnNewEngine := DoOnNewEngine;
...
In DoOnNewEngine, you will initialize every newly created
TSMEngine instance, to register all needed Delphi methods and
prepare access to JavaScript via the runtime's global
JSObject.
Then each time you want to access the JavaScript engine, you will
write for instance:
function TTestServer.Process(Ctxt: THttpServerRequest): cardinal;
var engine: TSMEngine;
...
engine := fSMManager.ThreadSafeEngine;
... // now you can use engine, e.g. engine.Global.someMethod()
Each thread of the HTTP server thread-pool will be initialized on the fly if
needed, or the previously initialized instance will be quickly returned
otherwise.
Once you have the TSMEngine instance corresponding to the
current thread, you can launch actions on its global object, or tune its
execution.
For instance, it could be a good idea to check for the JavaScript VM's
garbage collection:
function TTestServer.Process(Ctxt: THttpServerRequest): cardinal;
...
engine := fSMManager.ThreadSafeEngine;
engine.MaybeGarbageCollect; // perform garbage collection if needed
...
We will now find out how to interact between JavaScript and
Delphi code.
Calling Delphi code from JavaScript
In order to call some Delphi method from JavaScript, you
will have to register the method.
As just stated, it is done by setting a callback within
TSMEngineManager.OnNewEngine initialization code. For
instance:
procedure TTestServer.DoOnNewEngine(const Engine: TSMEngine);
...
// add native function to the engine
Engine.RegisterMethod(Engine.GlobalObj,'loadFile',LoadFile,1);
end;
Here, the local LoadFile() method is implemented as such in
native code:
function TTestServer.LoadFile(const This: variant; const Args: array of variant): variant;
beginif length(Args)<>1 thenraise Exception.Create('Invalid number of args for loadFile(): required 1 (file path)');
result := AnyTextFileToSynUnicode(Args[0]);
end;
As you can see, this is perfectly easy to follow.
Its purpose is to load a file content from JavaScript, by defining a
new global function named loadFile().
Remember that the SpiderMonkey engine, by itself, does not know
anything about file system, database or even DOM. Only basic objects were
registered, like arrays. We have to explicitly register the functions needed by
the JavaScript code.
In the above code snippet, we used the
TSMEngineMethodEventVariant callback signature, marshaling
variant values as parameters. This is the easiest method, with
only a slight performance impact.
Such methods have the following features:
- Arguments will be transmitted from JavaScript values as simple
Delphi types (for numbers or text), or as our custom
TSMVariant type for JavaScript objects, which allows
late-binding; - The
This: variant first parameter map the "callee"
JavaScript object as a TSMVariant custom instance, so
that you would be able to access the other object's methods or properties
directly via late-binding; - You can benefit of the JavaScript feature of variable number of
arguments when calling a function, since the input arguments is a dynamic array
of
variant; - All those registered methods are registered in a list maintained in the
TSMEngine instance, so it could be pretty convenient to work with,
in some cases; - You can still access to the low-level JSObject values of any the
argument, if needed, since they can be trans-typed to a
TSMVariantData instance (see below) - so you do not loose any
information; - The Delphi native method will be protected by the mORMot
wrapper, so that any exception raised within the process will be catch and
transmitted as a JavaScript exception to the runtime;
- There is also an hidden set of the FPU exception mask during execution of
native code (more on it later on) - you should not bother on it here.
Now consider how you should have written the same loadFile()
function via low-level API calls.
First, we register the callback:
procedure TTestServer.DoOnNewEngine(const Engine: TSMEngine);
...
// add native function to the engine
Engine.GlobalObject.DefineNativeMethod('loadFile', nsm_loadFile, 1);
end;
Then its implementation:
function nsm_loadFile(cx: PJSContext; argc: uintN; vp: Pjsval): JSBool; cdecl;
var in_argv: PjsvalVector;
filePath: TFileName;
begin
TSynFPUException.ForDelphiCode;
tryif argc<>1 thenraise Exception.Create('Invalid number of args for loadFile(): required 1 (file path)');
in_argv := JS_ARGV(cx,vp);
filePath := JSVAL_TO_STRING(in_argv[0]).ToString(cx);
JS_SET_RVAL(cx, vp, cx^.NewJSString(AnyTextFileToSynUnicode(filePath)).ToJSVal);
Result := JS_TRUE;
except
on E: Exception do begin// all exceptions MUST be catched on Delphi side
JS_SET_RVAL(cx, vp, JSVAL_VOID);
JSError(cx, E);
Result := JS_FALSE;
end;
end;
end;
As you can see, this nsm_loadFile() function is much more
difficult to follow:
- Your code shall begin with a cryptic
TSynFPUException.ForDelphiCode instruction, to protect the FPU
exception flag during execution of native code (Delphi RTL expects its
own set of FPU exception mask during execution, which does not match the FPU
exception mask expected by SpiderMonkey); - You have to explicitly catch any Delphi exception which may raise, with a
try...finally block, and marshal them back as JavaScript
errors; - You need to do a lot of manual low-level conversions - via
JS_ARGV() then e.g. JSVAL_TO_STRING() macros - to
retrieve the actual values of the arguments; - And the returning function is to be marshaled by hand - see the
JS_SET_RVAL() line.
Since the variant-based callback has only a slight performance
impact (nothing measurable, when compared to the SpiderMonkey engine
performance itself), and still have access to all the transmitted information,
we strongly encourage you to use this safer and cleaner pattern, and do not
define any native function via low-level API.
Note that there is an alternate JSON-based callback, which is not to be used
in your end-user code, but will be used when marshaling to JSON is needed, e.g.
when working with mORMot's ORM or SOA features.
TSMVariant custom type
As stated above, the SynSM.pas unit defines a
TSMVariant custom variant type. It will be used by the unit to
marshal any JSObject instance as variant.
Via the magic of late-binding, it will allow access of any
JavaScript object property, or execute any of its functions. Only with
a slightly performance penalty, but with much better code readability than with
low-level access of the SpiderMonkey API.
The TSMVariantData memory structure can be used to map such a
TSMVariant variant instance. In fact, the custom variant type will
store not only the JSObject value, but also its execution context -
i.e. JSContext - so is pretty convenient to work with.
For instance, you may be able to write code as such:
function TMyClass.MyFunction(const This: variant; const Args: array of variant): variant;
var global: variant;
begin
TSMVariantData(This).GetGlobal(global);
global.anotherFunction(Args[0],Args[1],'test');
// same as:
global := TSMVariantData(This).SMObject.Engine.Global;
global.anotherFunction(Args[0],Args[1],'test');
// but you may also write directly:with TSMVariantData(This).SMObject.Engine do
Global.anotherFunction(Args[0],Args[1],'test');
result := AnyTextFileToSynUnicode(Args[0]);
end;
Here, the This custom variant instance is trans-typed via
TSMVariantData(This) to access its internal properties.
Calling JavaScript code from Delphi
In order to execute some JavaScript code from Delphi, you
should first define the JavaScript functions to be executed.
This shall take place within TSMEngineManager.OnNewEngine
initialization code:
procedure TTestServer.DoOnNewEngine(const Engine: TSMEngine);
var showDownRunner: SynUnicode;
begin// add external JavaScript library to engine (port of the Markdown library)
Engine.Evaluate(fShowDownLib, 'showdown.js');
// add the bootstrap function calling loadfile() then showdown's makeHtml()
showDownRunner := AnyTextFileToSynUnicode(ExeVersion.ProgramFilePath+'showDownRunner.js');
Engine.Evaluate(showDownRunner, 'showDownRunner.js');
...
This code first evaluates (i.e. "executes") a general-purpose
JavaScript library contained in the showdown.js file,
available in the sample executable folder. This is an open source library able
to convert any Markdown markup into HTML. Plain standard
JavaScript code.
Then we evaluate (i.e. "execute") a small piece of
JavaScript code, to link the makeHtml() function of the
just defined library with our loadFile() native function:
function showDownRunner(pathToFile){
var src = loadFile(pathToFile); // call Delphi native codevar converter = new Showdown.converter(); // get the Showdown convertedreturn converter.makeHtml(src); // convert .md content into HTML via showdown.js
}
Now we have a new global function showDownRunner(pathToFile) at
hand, ready to be executed by our Delphi code:
function TTestServer.Process(Ctxt: THttpServerRequest): cardinal;
var content: variant;
FileName, FileExt: TFileName;
engine: TSMEngine;
...
if FileExt='.md' then begin
...
engine := fSMManager.ThreadSafeEngine;
...
content := engine.Global.showDownRunner(FileName);
...
As you can see, we access the function via late-binding. Above code is
perfectly readable, and we call here a JavaScript function and a whole
library as natural as if it was native code.
Without late-binding, we may have written, accessing not the Global
TSMVariant instance, but the lower level GlobalObject:
TSMObject property:
...
content := engine.GlobalObject.Run('showDownRunner',[SynUnicode(FileName)]);
...
It is up to you to choose which kind of code you prefer, but late-binding is
worth considering.
Next step on our side is to directly allow access to mORMot's ORM
and SOA features, including interface-based services.
Feedback is welcome on our
forum, as usual.
