Terminal, Console and other Command Prompt


#1

Even if Redtamarin is mainly focused on the CLI that does not mean we do not have to deal with UI.

The CLI, or command-line interface, is the interface to the user or other programs
and that forces you to deal with terminal capacities:

  • can it display UTF-8?
  • can it display colors ?
  • can it go into raw mode ?
  • can it turn off echoing ?
  • and many other little details like that …

So, when you are under a real POSIX (Portable Operating System Interface) system, like Linux or macOS, it is kind of easy, because yeah it defines in great details how the terminal and/or shell are supposed to work

And then you have Windows … which is a real pain in the ass when in comes to the CLI
there you have something called the command prompt (notice the difference of vocabulary?)

For a real overview see Comparison of command shells.

For Redtamarin we decided to follow POSIX and consider Bash our default environment, and to emulate parts of it to support the Windows command-line.

With redshell (the Redtamarin shell), the executable itself will run on any supported platforms
that means

  • Windows, Intel CPU 32bit and 64bit
  • macOS, Intel CPU 32bit and 64bit (yep 32bit is kind of useless)
  • Linux, Intel CPU 32bit and 64bit

For macOS and Linux, no big deal, they both follow POSIX and so we know we have a well defined interface to know what is supported or not.

a Linux terminal

a macOS terminal

in fact any terminal can be considered generic, it will just works.


And so for Windows, we need to do more work to work with the terminal, because as the default CMD.exe is limited, often users use a terminal emulator and there are many of them, see List of terminal emulators.

Whatever terminal you decide to use, we do expect you will run it under a Bash shell.

So what is the difference ?

Here the default CMD.exe

see how we detect that the host is “CMD.exe” and it does not support the display of colors for example (I’ll come back to UTF-8 support later)

As advised in the wiki page Redtamarin Windows Environment Setup you could also install Cygwin and use bash.exe for your shell instead of CMD.exe

But that’s the thing, using bash.exe does not prevent you to run it from the CMD.exe, see bellow

There even if we detect we are udner a Bash shell we still detect the host is “CMD.exe” and again we know that colors are not supported, etc.


Another terminal you could use is the PowerShell

here by default

we detect that the host is “PowerShell” and that it can not display colors etc.

But same as before, you could also run bash.exe, if you cd to the cygwin install directory and run Cygwin.bat you will find yourself in the PowerShell but running the Bash shell, here how it look like

You will notcie that this time we detect the host as “CMD.exe”, why ?

because of this An A-Z Index of Windows PowerShell commands
at the bottom you can read

PowerShell can also run all the standard CMD commands (external commands), .cmd batch files will run within a CMD.exe shell (so can include internal CMD commands), plus all Resource Kit utilities.

So yeah executing the Cygwin.bat from PowerShell put you under the CMD.exe shell and we detect that too.


Now, what you really want to do is run a proper terminal emulator, so if you have installed Cygwin you will have access to the Cygwin Terminal which is basically known as MinTTY

Again we detect the host as “MinTTY” and can deduce what is supported like the display of colors and UTF-8.

But you will also notice that MinTTY is not a real TTY, see the following issues

In short, Mintty does emulate a terminal by using pty (pseudo terminal) which use pipes, and if you use the function isatty() (to test for a temrinal device) it will will simply fails.

I’ll come back later to that to tell why it is important in some cases.


Another popular terminal emulator is ConEmu, it has tons of options and can be configured to run many different kind of shells.

Here ConEmu running CMD.exe

Yep we detect that one too as “ConEmu” and accordingly we detect it can display colors and UTF-8 etc.

Now, the same ConEmu but running bash.exe

We still detect it as “ConEmu”, and the main difference is that we know we are under a Bash shell.


Based on ConEmu, another popular one is cmder (it presents itself as a “console emulator”), and same as ConEmu it can run many different shells

here cmder running CMD.exe

We detect the host as “cmdr” and like ConEmu detect it can display colors, UTF-8, etc.

And same as others, cmder can also run bash.exe


Now, you have yet another kind of terminal emulator: Console2
also often used to replace CMD.exe as a better command prompt
because it can supports tabs, copy/paste, transparent window, etc.

But technically Console2 does not do much to improve the terminal itself (like supporting colors or UTF-8), it just interface to CMD.exe and that’s about it

That’s why we detect it as “CMD.exe”, because it does work exactly like CMD.exe

Even when you run bash.exe from it


Our advice, if you are under Windows, is to use by order of preference

  • ConEmu with the “Cygwin Bash” settings
    this is imho the closest you can get to a POSIX terminal for Windows
  • cmder with the “Cygwin Bash” settings
    like ConEmu but with a slightly different UI approach
  • the default Cygwin Terminal (eg. Mintty)
    a good POSIX terminal but with the little flow of not being seen as a TTY

The other command prompt and terminal emulators for Windows will works too but will be much more limited.


The many limitation of Windows …

The UTF-8 (or Unicode chars in general)

under CMD.exe you need usually to run chcp 65001
with Redtamarin we did a bit better, we let you natively activate “chcp 65001”

import C.conio.*;

var support_utf8:Boolean = set_code_page( CODEPAGE_UTF8 );

But this is not enough, you also need the terminal font to support the display of UTF-8 chars and as you can see in the previous screenshots, CMD.exe, PowerShell, Console2, etc. do not support that by default.

And it get more nightmarish when you have to go into deep Windows settings to change the defautl system locale, etc.

So yeah it is possible but the simpler solution is to just use another terminal that does support UTF-8 by default like ConEmu, MinTTY, etc.

The display of colors

More exactly the display of ANSI colors, see ANSI escape code.

Here an example, how to display “hello world” in green

var message:String = "hello world";
trace( "\x1b[32m" + message + "\x1b[0m" );

CMD.exe never supported ANSI escape codes, only recently with Windows 10 this feature has been added.

So, yeah you could use native function call to display colors in a Windows command prompt,
but again it is much much simpler to just use another terminal that does support ANSI escape codes 9and so ANSI colors) by default like ConEmu, MinTTY, etc.

But because we can detect it we added a utility function in the shell package: hasColor()

simple to use

import shell.*;

if( hasColor() )
{
    // use ANSI escape codes to display colors
}
else
{
    // display simple text
}

It will work anywhere.

The default behaviour of standard streams

Under Windows by default the input and output are in “text mode” and “line buffered”

When you type the ENTER key in a terminal here what happen

  • if you are under Linux or macOS
    the terminal receive the LF (Line Feed) key (eg “\n” or 0x10)
  • if you are under Windows
    the terminal receive the CR (Carriage Return) key (eg “\r” or 0x13)
    then followed by LF (Line Feed) key (eg “\n” or 0x10)

If you want to disable this behavior and receive only 1 char CR instead of 2 CR+LF
you will need to do that (under Windows, it is a no-op for mac and linux)

import C.conio.*; // set_binary_mode()
import C.stdio.*; // stdin, fileno()
import C.unistd.*; // STDIN_FILENO

set_binary_mode( fileno(stdin), true );
//or
set_binary_mode( STDIN_FILENO, true );

That way if you wait for user input and want to check for the ENTER key
you can then do something like this

if( (key == "\n") || (key == "\r") )
{
    // we know we received the 'ENTER" key
}

Here the full example

import C.conio.*;
import C.stdlib.*;
import C.stdio.*;
import C.unistd.*;

import shell.*;

// Under windows by default the console is in text mode
// we want the console to be in binary mode
if( Runtime.platform == "windows" )
{
    set_binary_mode( STDIN_FILENO, true );
}

// only a real TTY can use
//     kbhit() -- detect a key input
// canonical() -- change to raw mode/cooked mode
//      echo() -- change echo off/on
if( !isatty( STDIN_FILENO ) )
{
    // it will not work with MinTTY / Cygwin Terminal under Windows
    trace( "this will work only under a TTY" );
    exit(1);
}


trace( "" );
trace( "blocking input: (type 'any phrase' then [ENTER] to quit)" );
var run:Boolean = true;
var i:int;
var code:int;
var key:String;
var buffer:String = "";

function rndcol():String
{
    var colors:Array = [ "30", "31", "32", "33", "34", "35", "36", "37" ];

    return colors[ Math.floor(Math.random() * (colors.length)) ];
}


canonical(false); // raw mode
echo(false); // echo off
while( run )
{
    i=kbhit();

    if( i != 0 )
    {
        code = getchar();
        key = String.fromCharCode( code );
        
        // Note:
        // the ENTER key has different value
        // under POSIX it is LF "\n" 0x10
        // under WIN32 it is CR "\r" 0x13
        if( (key == "\n") || (key == "\r") )
        {
            run = false;
            Program.write( "\n" );
        }
        else
        {
            // here we do the echoing
            // we could replace the 'key' by anything else
            // or not even write anything at all
            //Program.write( key );
            //Program.write( "*" ); // display stars instead of the input char
            
            if( hasColor() )
            {
                //Program.write( "\x1b[32m" + key + "\x1b[0m" ); // one color
                Program.write( "\x1b[" + rndcol() + "m" + key + "\x1b[0m" ); // random color
            }
            else
            {
                Program.write( key );
            }

            buffer += key;
        }

        i = 0;
    }
}
canonical(true); // cooked mode
echo(true); // echo on
trace( "You wrote \"" + buffer + "\"" );

It will output something like this

So here we use the “raw mode” and “echo off” so when a user type into the terminal it does not echo the characters typed, but then you use getchar() to read those chars from stdin, on the example above we decided to echo those chars ourselves while adding colors, we could have done anything else: display * for a “type your password kind of input”, or display nothing, etc.


That’s the kind of UI we need to deal with in the CLI :slight_smile:

Now, for the last example here the little script that can detect all those terminal features

import C.conio.*;
import shell.*;

var tty_supported:Boolean      = false;
var bash_supported:Boolean     = false;
var utf8_supported:Boolean     = false;
var color_supported:Boolean    = false;
var raw_mode_supported:Boolean = false;
var echo_off_supported:Boolean = false;

var console_host:String = "";

if( Runtime.platform == "windows" )
{
    // Windows is trickier
    
    utf8_supported = set_code_page( CODEPAGE_UTF8 );

    if( isCommandPrompt() )
    {
        // we are running under CMD.exe
        // or a terminal emulator like Console2 running CMD.exe
        console_host = "CMD.exe";
    }
    else
    {
        // no PROMPT
        // so we are either under PowerShell or a terminal emulator
        
        var features:Object = {};
        if( isTerminalEmulator( features ) )
        {
            // we are under a terminal emulator
            console_host = features.host;
            color_supported = features.hasColor;
        }
        else
        {
            // we are probably under PowerShell
            console_host = "PowerShell";
        }
    }
}
else
{
    // Linux or macOS is easier
    
    console_host = "terminal";

    // in general all POSIX terminals support UTF-8
    utf8_supported = true;

    if( hasColor() )
    {
        color_supported = true;
    }
}

// shared by Windows / macOS / Linux

if( isInteractive() )
{
    tty_supported = true;
    raw_mode_supported = true;
    echo_off_supported = true;
}

if( isBash() )
{
    bash_supported = true;
}

The utilities like hasColor(), isBash(), isInteractive(), isCommandPrompt(), isTerminalEmulator(), etc.
can go a long way to help you deal with this kind of UI thing (eg. terminal capabilities).

And for the record, a bug created this whole thing, when I was testing kbhit() under Windows by default I was using the Cygwin Terminal and it was not working at all, and so I investigated WHY…

All the above is the answer to that “why?” :smile: