Redtamarin and API Design


#1

In the Redtamarin README.md (that somehow I keep rewriting every month)
you can find this part about the Core Goals

  • High focus on API and libraries
    Enable ActionScript 3 developers to access every layer of the programming stack,
    from the highest level application programming abstractions
    all the way down to the lowest level system programming.

    Redtamarin provide 3 API layers:

    • CLIB the C API based on the C standard library and C POSIX library
      example: package C.stdio where you can use fread(), fwrite(), fflush(), etc.
    • RNLIB the Redtamarin Native Library
      example: package shell.streams where you can use FileIOStream, PipeStream, etc.
    • AVMGlue our own open source implementation of the Flash Platform API
      example: package flash.filesystem where you can use File, FileStream etc.

    We embrace an hybrid philosophy and try to provide different set of API
    and libraries to cater for numerous approach to multiple programming paradigms,
    while leaving developers with the final choice.

    Because AVM2 is also ES3 compliant, you can compile JS programs for Redtamarin and reuse numerous JS libraries.

    (TBA) Use 3rd party API compatility layers like BrowserGlue, NodeGlue, ChakraGlue, etc.

    (TBA) Use CrossBridge cross-compilation to port C/C++ native libraries to AVM2 interop
    for targetting the Redtamarin runtimes.

So let me tell you a bit about those API and why it is so interesting in the context of AVM2 and ActionScript 3 :slight_smile:


When I was using ActionScript previous versions (AS1 and AS2)
my favourite thing was to augment the builtin classes

for ex: adding a trim() method to the String class

off course for some people in the JavaScript world this was sacrilege
“how dare you augmenting those builtins, you’re poluting them!”

well… let’s not debate that just yet :slight_smile:

so in AS1, it was like with JS you just add to the prototype

String.prototype.trim = function( ... )
{
    // ...
}

and then came AS2 and it was not working anymore, if you were to add to the prototype it was throwing an error at compile time

that’s because the compiler used intrinsic classes to verify what was valid or not
so then you just had to edit those intrinsic classes and you were good to go :smile:

//****************************************************************************
// ActionScript Standard Library
// String object
//****************************************************************************

intrinsic class String
{
	var length:Number;

	function String(value:String);
	
	// ...
	
	function trim( trimChars:Array ):String;
}

so that was to prevent the compiler to throw errors, and you still have to include the implementation in AS1

and later came AS3 where you would hit a wall because

package
{
    public final class String extends Object
    {
        // ...
    }
}

Can not augment the String class anymore …

or do you?

Well, it depends how you compile and how tolerant you are to strict typing

If you use the compiler option -AS3 (the default) you can not augment the String class

but if you use the other option -ES you can augment the String class
and you do that again by adding your method to the prototype
but then you lose the strict typing


But what happen when you can compile those builtins classes because you build your own runtime like with Redtamarin ?

It is a tough choice, at the same time you want to stay compatible with AS3 as it is working in Flash and AIR, but you also know it would be quite useful to have this trim() method

So if you look more into the details of the implementation, you can see that the AS3 builtins like the String class have in fact 2 implementations in parallel, which also explains the two compiler flags -AS3 and -ES

See, AS3 is meant to be a superset of ECMAScript 3, but then any ES3 program is compatible and should run within the context of AS3.

To support both those things the Adobe dev made use of namespace

package
{

    public final class String extends Object
    {
        // ...

        // charAt
        AS3 native function charAt(i:Number=0):String

        prototype.charAt = function(i:Number=0):String
        {
            return String(this).AS3::charAt(i)
        }

        // Dummy constructor function - This is neccessary so the compiler can do arg # checking for the ctor in strict mode
        // The code for the actual ctor is in StringClass::construct in the avmplus
        public function String(value = "")
        {}

        _dontEnumPrototype(prototype);
    }
}

the trick is the following, the charAt() method is declared as native in the AS3 namespace
and the AS3 namespace is opened by default

but attached to the prototype you see another charAt() method
that’s the one used when we set the flag -ES

so when you use that -ES flag, the AS3 namespace is not opened by default
the method declared on the prototype are used instead

so like with JS, you can override them ,add others to the prototype etc.


Off course there are also exception to that rule, in the Flash Player 19 Adobe added two methods to the Array class: insertAt() and removeAt(), but those are not part of the ES3 so they were not added to the prototype.

See the comments from the avmplus source here

package
{
    [native(cls="ArrayClass", gc="exact", instance="ArrayObject", methods="auto")]
    public dynamic class Array extends Object
    {
        // ...

        // Functions insertAt() and removeAt() are AS3 extensions not present in ECMAscript.
        // They are defined only in the AS3 namespace to avoid breaking ES3 compatibility,
        // thus we do not define prototype.insertAt and prototype.removeAt.

        [API(CONFIG::SWF_30)]
        AS3 native function insertAt(index:int, element:*):void

        [API(CONFIG::SWF_30)]
        AS3 native function removeAt(index:int):*

        // ...
    }
}

Now, the AS3 superset (or ES4) has not been adopted by the web, instead they went wtih ESNext (ES5) and then ES6 (ECMAScript 2015) etc.

and if you look at the ECMAScript standards, the trim() method was added with ES5

so how do we combine all that with our AS3 setup?
how do we ad new stuff without breaking the existing stuff?

we use namespace again :slight_smile:

package
{

    public final class String extends Object
    {
        // ...

        // charAt
        AS3 native function charAt(i:Number=0):String

        prototype.charAt = function(i:Number=0):String
        {
            return String(this).AS3::charAt(i)
        }

        // ...

        ES5 function trim():String
        {
            // ...
        }

        // Dummy constructor function - This is neccessary so the compiler can do arg # checking for the ctor in strict mode
        // The code for the actual ctor is in StringClass::construct in the avmplus
        public function String(value = "")
        {}

        _dontEnumPrototype(prototype);
    }
}

and not only it works with ES5, but also ES6

package
{

    public final class String extends Object
    {
        // ...

        ES5 function trim():String
        {
            // ...
        }

        ES6 function repeat( count:Number ):String
        {
            // ...
        }

        // Dummy constructor function - This is neccessary so the compiler can do arg # checking for the ctor in strict mode
        // The code for the actual ctor is in StringClass::construct in the avmplus
        public function String(value = "")
        {}

        _dontEnumPrototype(prototype);
    }
}

or any other ECMAScript version for that matter


So yeah with Redtamarin, if you need something from ES5 or ES6
you will just have to open their respective namespace to use it

for example

use namespace ES5

var foobar:String = "    hello world     ";
var timmed:String = foobar.trim();

or alternatively

var foobar:String = "    hello world     ";
var timmed:String = foobar.ES5::trim();

the same way you can use a shim like

you can upgrade your level of JS from within Redtamarin
and I may add you can do that nicely by using namespaces (a very AS3/ES4 feature that JS does not have)

and that what we mean by High focus on API and libraries

Redtamarin does not work on the AVM2 to change it, we keep it to be compatible with Flash/AIR but then we augment and extend the API eg. what you can do with it by default.

And that’s where it get very interesting, when you see big languages like Java or C#/.NET, those come with a big application framework, and technically we could have replicated that but we decided not to.

Having access to an application framework is useful but the cost and size of such framework is too big for us, and so if such framework should exists, it should be left as an external library.

But then for internal libraries, things you do embed in your runtime and that are available everywhere, what do you pick?


In our case the choice was half made, because the Flash API has been used so much and so long by ActionScript developers then we ecided to implement it and embed it by default, that’s what we call the AVMGlue, our own open source implementation of the Flash API and AIR API.

And to do that, at the beginning, we then implemented cross-platform low-level functionalities like mkdir(), in fact it is vmpi_mkdir(), VMPI stands for "Virtual Machine Platform Interface and it allowed us to implement for example File.createDirectory() in the package flash.filesystem.

But soon we faced two problems, first vmpi_mkdir() was only available on the C++ part, not the AS3 part, and so it was forcing us to implement all classes as native C++ classes with an AS3 definition so it could be used from AS3 front-end.

And second, most of those Flash/AIR classes use an event system we did not have with AVM2.

The first problem really jumped at us when we implemented sockets, because we wanted to make a simple implementation without events and just using function callbacks.

It is at that moment we took the decision to open those VMPI definitions to the AS3 front-end directly, in C++ you had vmpi_mkdir() and then in AS3 you had mkdir() , usually the same function signature you had in C, and that’s how the C API was born.

And here how much the API is so important to us, instead of keeping only the minimal C definitions (only the one we needed to implement other stuff like the Flash/AIR API), we decided to implement all of them (or at least as much as what was possible).

Let’s be clear, the C API in Redtamarin is not a Foreign Function Interface (FFI)
it is not a magic wrapper that will automatically import any C source code
it is more a hand crafted pseudo C interface.

Basically we followed the lead of the VMPI in AVM2, but then organised everything
so the definitions became available to the AS3 front-end (when you write code in AS3) so it feels like you have access to native C definitions.

Some of those definitions are native, some use a bit of AS3 glue before reusing some native code, and some other are completely implemented in AS3.

Some dev will say “it sucks” because you can not just copy/paste C code and have it magically work in Redtamarin, for that it will be more things like CrossBridge that will help (when ready, and it is not ready).

But for some other dev it will be extremely powerful in term of API, maybe you saw some C example you want to directly reuse in AS3, maybe you want to copy bits of a program to integrate the same functionalities in your own program, etc.

With that C API, you will have to write the actual code, but it will be pretty damn close to the original, because we did put a great effort to provide the maximum of the C standard library and the C POSIX library.

Things like FFI, CrossBridge and ANE are great too, but still … for some dev the barrier to entry is too high, how many AS3 dev are really developing their own ANE? not much

Because writing C/C++ is harder, compiling it is harder, etc.
but reusing a C definition in AS3 and compiling it is super easy, you just need to understand AS3 and compiling .as to bytecode, and technically with Redtamarin you don’t even need to compile it.

All that is why you have a C API and an AVMGlue API, that what we picked up as default embedded API, and then the next one: Redtamarin Native Library (RNLIB) ot simply put the Redtamarin API.

So the Redtamarin API is when the C API or the Flash/AIR API does not solve the problem, take for example users management on a system: create a user, add a user to a group, give administrator rights etc.

If you follow POSIX with Linux and macOS, yeah ok it will works with C definitions, but then the problem is Windows because there the logic is just so different, and it is pretty hard to replciate the POSIX C functions with the Windows C API.

In such case, you will find the C definitions and they will be marked POSIX eg. it will works for Linux and macOS but then for Windows it will be a no-op (no operations).

And in the Redtamarin API, you will find for example a class UserManagement with such methods as createUser( name:String, password:String ), addToGroup( username:String, group:String ), etc.

and when under Windows it will use a native implementation specific for Windows, and when under Linux/macOS, it will simply reuse the C API definitions in AS3.


In summary I will say with Redtamarin we focus more on the API than the language or the VM,
in short we work with what we have, trying to not break it, trying to stay as compatible as possible with other things like Flash/AIR.

We don’t try to re-invent a new language, AS3 is good enough for us, but within the scope of AS3 and what you can do with it (like using namespaces) then we make it convenient and easy for the developers to have access to “more stuff” or API.

Also why we come with the concept of Glues.

Nothing new really, JS been a long time defined as a “web glue”, and in uour specific case AVMGlue is what glue the AVM2 to the Flash API definitions.

We just thought: hey maybe dev would need more glues.

For example, if you worked with PHP for some time, you know how useful a function like file_get_contents(), and we just thought how neat it could be to do something like

import php.*;

var data:String = file_get_contents( "some/path/to/mydata.txt" );

// or

var options:Object = {
                        http: {
                           method: "GET",
                           header: "Accept-language: en\r\nCookie: foo=bar\r\n"
                          }
                     };
var context = stream_context_create( options );
var html:String = file_get_contents( "http://www.example.com/", false, context );

So glues are not embedded into the runtime, but they could require some native code inside the runtime (if we can not really duplicate the functionality with AS3 and the C API).

We see them as first degree libraries, we will host them start them and keep improving on them till they become somewhat complete and useful, “as if” they were API of the runtime.

Anyway, that’s it, a little overview of the different API in Redtamarin.


#2

Does this mean it’s possible to get API parity with ES5, ES6 and so on? Ex, Array.forEach(), etc

Also, I don’t care about ES arrow function but if someone did want to add something like that is it possible to upgrade AS3 syntax using methods above?

PS Apache Flex crew has Falcon compiler that they have ability to make some changes correct? Or would that fragment ASC2 into ASC2.5 offshoot?

Also, one and half more questions, if you are adding API like, file_get_contents( "some/path/to/mydata.txt" ) that is very cool and useful but is it sync or async and it feels like rubbing a cat backwards to not use camel case.

file_get_contents( "some/path/to/mydata.txt" ); // annoyed like cat

getFileContents( "some/path/to/mydata.txt" ); // ahhh much better

#3

yes exactly that: API compatibility, with maybe some limitations
for example if ES6 define a Proxy class and AS3 already has a Proxy class

not that, technically it should be possible to add new syntax constructs to the language
like let or -> function, etc.

but that’s a lot of work, it could make some things incompatible with Flash/AIR
and it is not really needed/wanted

API compat is one thing, language compat is another thing
we focus on API compat, and want to keep the AS3 language, not transform it into the ES6 language

so yeah that would be something like that if we wanted to change the language syntax,
we would have to change also the compiler, but we do not want to do that

it is defined like that in PHP
https://www.php.net/manual/en/function.file-get-contents.php

file_get_contents ( string $filename [, bool $use_include_path = FALSE [, resource $context [, int $offset = 0 [, int $maxlen ]]]] ) : string

so in an AS3 phpglue it would gives something like that

package php
{
    public function file_get_contents( filename:String, use_include_path:Boolean = false,  context:resource = null, offset:int = 0, maxlen:int = 0 ):String
    {
        // implementation ...
    }
}

you have to keep the same function signatures even if the syntax is annoying