Redtamarin vs Node.js and what about Deno?


#1

Yay it’s summer time, and naturally less posts here
but still let make a quick post about a couple of questions I got in the recent weeks.


Often when I try to (badly) explains what Redtamarin does I got the question

Q: Is it like like node.js but with AS3?

A: kind of, you can build a node.as3 with RedTamarin

the goal of RedTamarin is to provide to the developers
all the tools, libraries, etc. they need to build their own node.js in AS3

it may be subtle or generate a long debate
but let make the difference right now

RedTamarin is not a clone of Node.js
RedTamarin is closer to PHP, Python, Ruby, etc.

you can as well run AS3 in blocking-mode
with CGI, WSGI, etc. like you would do with PHP/Python

or run AS3 in nonblocking-mode or async,
even implement your own “main loop”,
like you would do with Node.js


and when I mentioned “AS3+AVM2+API as a platform”
in an earlier post Pursuit of Simplicity

for example reusing a JS library made for Node.js, compile it to ABC
and use a NodeGlue library that implement the Node.js API


and then later on Ryan Dahl talk about his new project Deno in 10 Things I Regret About Node.js.


All these, generated new questions like

Is that NodeGlue a Node.as3 ?

and like

Is Deno using Typescript like Redtamarin using ActionScript ?


First and foremost, I want to reiterate that RedTamarin is not a clone of Node.js,
but to be clearer I would say it is about bias and design.

When Ryan Dahl reuse libuv and focus on evented I/O for Node.js (watch the talk)
he made a design decision that Node.js will be asynchronous by default.

When I started to work on Redtamarin my decision was to make it synchronous by default
because I wanted to follow in the foot steps of command-line programs (which are blocking by default).

Also why I mention that Redtamarin is closer to PHP, Python, Ruby, etc.
because all those can be used for command-line programs.


Now the confusing parts are two fold.

First, a lot of people use Node.js and npm to produce and distribute
command-line tools, the classic

just install it by doing $ npm install -g typescript
(you could replace “typescript” with anything else)

and there you get a command-line tool built with JavaScript and the Node.js API
that get installed and behave like most classic command-line tools with a blocking behaviour.

And second, the other part is me mentioning that you can “run AS3 in nonblocking-mode or async”
with something like Runtime.goAsync();

So yeah, even if Node.js is “asynchronous by default” you can make it behave synchronously,
and same with others like PHP, Python, Ruby, etc. when they are “synchronous by default”
you can make them behave asynchronously with the help of 3rd party liraries.

With Redtamarin, the problem is slightly more complicated, even if I want to make it
work as “synchronous by default” to have a better support for the command-line,
the ActionScript 3 language is historically associated to runtimes like the Flash Player and Adobe AIR,
and been used with the Flash API, and all those are using events and can be considered “asynchronous by default”.

Note:
To prevent any confusion a little point on the Flash/AIR event system
yes it is single threaded and the events execute within the frame that
executes synchronous code attached to that frame, something we could see
as a “predictable sequence” and as such could be called “synchronous”,
but some of those events can happen at arbitrary times on other frames
and we can not predict their sequence or order of execution because
what trigger them is the system (I/O like files, sockets), the user
(keyboard, mouse), etc. and as such could be called “asynchronous”.
see Runtime code execution fundamentals

So if I want to implement AVMGlue to replicate the Flash API,
I have no other choices to also support asynchronicity in the Redtmarin runtime.

Here the real difference between Redtamarin and Node.js, but also PHP, Python, Ruby, etc.
Our internals, our core API, HAVE TO support both synchronous and asynchronous,
we can not let that part fall into a 3rd party library.


Now, for the comparison of Deno, Node.js and Redtamarin.

For me, when I see a big bunch of JS developers loving TypeScript that much,
it does validate my vision of using ActionScript 3.0 on the command-line and server-side.

And when I see a project such as Deno (A secure TypeScript runtime on V8)
it validates the idea even more.

But still, there are similarities and dissimilarities.

What is similar

  • using a more structured language than JavaScript (aka ES3)

    • Deno uses TypeScript
    • Redtamarin uses ActionScript 3.0
  • re-using an existing runtime

    • Deno uses V8
    • Redtamarin uses AVM2
  • using an event loop

    • Deno (and Node.js) use libuv
    • Redtamarin use something else (I’ll come to that later)

What is not similar

  • Security

    • Deno wants the runtime to be secure
      eg. be able to limit the network or filesystem access
    • Redtamarin does not impose that kind of limits
      eg. we consider that programs running with the Redtamarin runtimes
      are like executables and as such should be able to do anything
      executables can do (eg. full access to network, filesystem, etc.)
  • Runtime

    • Deno compile TypeScript so V8 can run JS code
    • Redtamarin interpret directly bytecode (ABC,SWC)
      or ActionScript 3 plain text sources

A side note on security, technically I could add hooks into the Redtamarin runtimes
to block/prevent some system calls related to socket, filesystem, etc.
but imho it would cripple the runtime.

Also, the security part for Deno comes from a different place,
eg. people installing and running “scripts” with node.js/npm,
and not being able by default to produce self-contained executables.

With Redtamarin, by default we can produce projectors (or self-contained executables),
and our strategy/philosophy regarding security is that if you really want
to isolate what your executables can do then you should use containers, jails, chroots, VMs, etc.
I would highly recommend to watch this Keynote: Containers aka crazy user space fun by Jess Frazelle (@jessfraz).

Simply put, Deno is off course an interesting project but the dissimilarities are big enough
so we can say: no Redtamarin is not a clone of Deno either.


So what about that NodeGlue.swc thing ?

It is a compatibility layer, a glue, for the case when developers already have a Node.js project
and would like to test it with or try porting it to Redtamarin.

It’s not there yet, but it is planed.

When Redtmarin will have enough native API implemented to the point where
we can write an ActionScript library that reuse those API to implement the Node.js API
then NodeGlue will become a reality, or at least a live experiment.

The only real differences are

  • NodeGlue will be an external library,
    contrary to AVMGlue which is an internal library (embedded into the Redtamarin runtime)
  • NodeGlue development will depend on interest
    we will start small with lower Node.js versions (see Node.js releases) and will increment from there, maybe implementing Node.js 0.12.x API before implementing Node.js 8.x API.
  • NodeGlue have much less priority than AVMGlue

And finally about Redtamarin event loop, being synchronous and asynchronous.

What you can currently test with Runtime.goAsync(); in Redtamarin v0.4.1
is old, deprecated and 2 years behind.

In Redtamarin v0.4.2 which is currently still implemented and not publicly released (sorry for that)
things have changed a hell lot.

First, we had an implementation issue, let’s say I was too stupid to do it neatly in C/C++,
I banged my head on the walls for at least a year before going with “the hell with it”.

The great thing when you implement those stuff with Redtamarin is that even if you can not do it
with C/C++ (like the level of abstraction being so much above your head you simply can’t),
you can still resolve to do it with ActionScript 3 :smiley:

It’s “more stupid”, it’s crude, but it does work.

What we have now is a simple, yet effective, busy-wait loop.

Busy waiting

In software engineering, busy-waiting, busy-looping or spinning is a technique in which a process repeatedly checks to see if a condition is true, such as whether keyboard input or a lock is available. Spinning can also be used to generate an arbitrary time delay, a technique that was necessary on systems that lacked a method of waiting a specific length of time. Processor speeds vary greatly from computer to computer, especially as some processors are designed to dynamically adjust speed based on external factors, such as the load on the operating system. As such, spinning as a time delay technique often produces unpredictable or even inconsistent results unless code is implemented to determine how quickly the processor can execute a “do nothing” loop, or the looping code explicitly checks a real-time clock.

Spinning can be a valid strategy in certain circumstances, most notably in the implementation of spinlocks within operating systems designed to run on SMP systems. In general, however, spinning is considered an anti-pattern and should be avoided, as processor time that could be used to execute a different task is instead wasted on useless activity.

basically it looks like that

package shell.async
{

    public class Loop
    {
        protected var _running:Boolean = false;

        // ...

        protected function _loop():void
        {
            while( _running )
            {
                _oneloop( true );
            }
        }

        protected function _oneloop( sleep:Boolean = true ):void
        {
            _frame++;

            if( sleep )
            {
                _fps = Loop.calculateFPS( _interval );
            }

            if( onLoop ) { this.onLoop(); }

            if( sleep )
            {
                Program.sleep( _interval );
            }
        }

        public function start():void
        {
            if( _running ) { return; }

            _running = true;

            _loop();
        }

        public function stop():void
        {
            if( !_running ) { return; }

            _running = false;
        }

    }

}

So when you call Runtime.goAsync();
basically the runtime find the loop instance and does a loop.start().

But we used a couple of tricks :stuck_out_tongue:

You can call Runtime.goAsync(); anywhere in your code,
it will never block the program flow and will always be executed
at the very end of the program block.

for example:

trace( "hello world" );
Runtime.goAsync();
trace( "hello world again!" );

the order of execution will be

  • display “hello world”
  • display “hello world again!”
  • run the event loop

Not only in the main program (the primordial worker)
but also in any child workers.

Yep, workers can also have their own event loop,
and they are running asynchronously of the main event loop.

You can have for example, your main loop running at 20fps,
while child1 worker run its loop at 4fps and child2 worker run its own loop at 10fps.

But you could also have the opposite, a main loop running at 1fps
and a child worker running its loop at 60fps.

Internally in the Redtamarin runtime we use 3 type of callbacks

  • the start callback
    • happen at the very end of the user code (the frame code)
    • happen only if a callback was defined in user code
      eg. Program.onStart()
    • happen only in the main worker, child workers do not have this.
  • the loop callback
    • happen after the user code
    • and happen after the start callback
    • happen only if the async switch is set to true
      eg. somewhere in user code you called Runtime.goAsync();
    • happen in both main and child workers
  • the exit callback
    • happen after the user code
    • happen after the start callback
    • and happen after the loop callback
    • happen only if callbacks were defined in user code
      eg. Program.atExit()
    • happen in both main and child workers

Before those we also have a boot sequence

  • happen before the user code
  • happen everytime
  • happen in both main and child workers

It looks like that

|   Main (Primordial)   |         Child         |
|-----------------------|-----------------------|
| boot sequence         | boot sequence         |
|-----------------------|-----------------------|
| user code             | user code             |
|-----------------------|-----------------------|
| startCallback()       | n/a (no-op)           |
| loopCallback()        | loopCallback()        |
| exitCallback()        | exitCallback()        |
|-----------------------|-----------------------|

And on top of all that we have event system, with the classic EventDispatcher and Event classes
you already know in ActionScript 3.0

When the runtime is synchronous (no event loop) you can still use the event system,
it will work as long as you don’t expect to catch asynchronous events.

When the runtime is asynchronous (with an event loop), the event system will work the same,
but will also be able to catch asynchronous events, and even catch events coming from other workers,
but you will have to tell the runtime to stop, otherwise it will “run forever”.

How can you stop the runtime?

  • tell the loop to stop
    eg. Runtime.loop.stop()
  • tell the program to exit
    eg. Program.exit() or exit()
  • tell the program to abort
    eg. Program.abort() or abort()
  • throw an error
    eg. throw new Error( "my error here" )
  • tell the worker to terminate
    eg. Worker.current.terminate()

Now, depending if you are in the context of a child worker or the main worker,
you’ll get different behaviours

With the Main worker (primordial):

  • Main worker terminate

    • terminate all child workers
    • does not call the exit callbacks (main and child)
  • Main worker abort

    • terminate all child workers
    • does not call the exit callbacks (main and child)
  • Main worker exit

    • exit all child workers
    • call child workers exit callback
    • call main worker exit callback
  • Main worker does nothing (or loop stop)

    • if there is no child workers with an event loop
      call main worker exit callback at the end of the program
    • otherwise “loop forever” while waiting for the child worker to exit

With a Child worker:

  • Child worker terminate

    • call the child worker exit callback
  • Child worker abort

    • does not call the child worker exit callback
  • Child worker exit

    • call the child worker exit callback
  • Child worker does nothing (or loop stop)

    • call the child worker exit callback
    • automatically self terminate

And what can be those asynchronous events ?

  • POSIX signals
    when you run a command-line program
    you can for example type
    CTRL+C (macOS/Linux)
    CTRL+BREAK (Windows)
    and it will raises the SIGINT signal
    that is the “signal interrupt”
  • Flash API keyboard events
    while the runtime is async (main event loop)
    you can catch what keys the user type
    without blocking the event loop
  • Flash API file stream
    if you use flash.filesystem.FileStream class
    and use the method openAsync(file:File, fileMode:String):void
    the file is opened in a child worker that send back
    events to the worker that instanciated the class
  • Flash API sockets
    • if you use flash.net.Socket class
      all read/write methods are executed asynchronously on a child worker
    • if you use flash.net.ServerSocket class
      the socket will listen for connections on a child worker
    • if you use flash.net.DatagramSocket class
      all operations connect/send/receive are done asynchronously on a child worker
  • etc.

All that lead to other design and documentation decisions within Redtamarin.

For example in C, you can have blocking system calls, so if you use a C function that blocks forever
while you main program is running asynchronous with an event loop it could froze completely your event loop, so those are clearly documented, maybe a warning system will be put in place later.

That’s mainly the problem when you can do both synchronous and asynchronous,
you can sometimes “block forever” and “loop forever”, mixing those can be pretty hard to debug,
but so far we have safety mecanism which will prevent some of those bad behaviours.

The main design decision being to use the event system by default.

Before, we assumed that Redtamarin native classes would use only function callbacks,
but now we want to provide clearly defined events instead, even if for special mecanism we still use function callbacks internally.

Many classes that were already implemented in shell.streams package for example,
are deleted/replaced while AVMGlue classes that become first class citizen.

For example, if before we could have had a shell.streams.FileIOStream class
that was implemented synchronously (some methods could block) and with function callbacks, we removed it in favour of flash.filesystem.FileStream.

And now, the idea is to probably think about adding event definitions related to the runtime itself
for example when the exit callback is kicking, we should probably dispatch an Event.EXITING or similar things.

Morality, it does work but now we have to think of everywhere async/sync are involved.
Hopefully it make things a bit clearer for now for those interested :slight_smile:


Runtime vs Runtime
#2

^ This, so much this. AS3 and RedRamarin were way ahead of the time, and we’re finally seeing the “yep they were right” moments. Same with AIR (Electron?) SWF (WebAssembly?) and so much of what the Flash ecosystem did. The rest of the web dev world is finally catching up. In a way I feel like what I can do on the web has flatlined since 2009, I only have different sometimes-better sometimes-worse usually-more-complicated ways to do it than Flash.