Euphoria To C Translator

Euphoria To C Translator 2.3 Official Release February 11, 2002

Introduction

The Euphoria to C Translator will translate any Euphoria program into equivalent C source code. RDS is providing versions of the translator for Windows, DOS and Linux platforms. After translating a Euphoria program to C, you can compile and link using one of the supported C compilers. This will give you an executable file that will typically run much faster than if you used the Euphoria interpreter.

Installation

It is assumed that you have already installed the Euphoria 2.3 interpreter package on your system, and that your EUDIR and PATH environment variables are set correctly.

For each supported C compiler, on Windows, DOS or Linux, you'll find a .ZIP file on the RDS site containing these files:

     1. the translator (one per platform)
           
           ec.exe  - DOS 
           ecw.exe - Windows 
           ecu     - Linux 
     
     2. a run-time library (one per C compiler)
           
           ec.lib   - DOS (Watcom)
           ec.a     - DOS (DJGPP)
           ecw.lib  - Windows (Watcom)
           ecwl.lib - Windows (LccWin)
           ecwb.lib - Windows (Borland)
           ecu.a    - Linux (GNU)
 
     3. a C include file
           
           euphoria.h - same for all platforms and compilers

     4. (Watcom only) Files to support the CauseWay DOS extender

           cwc.exe    - file compressor
           le23p.exe  - format converter
           cwstub.exe - the DOS extender
   
     5. (DJGPP only) The Allegro graphics library, compiled specially for
           the translator. Download liballeg.zip

To install the Euphoria To C Translator, put euphoria.h into your euphoria\include directory, and put all other .exe's and library files into your euphoria\bin directory. Note: the liballeg.zip file is special. It should be unzipped and placed in DJGPP\LIB as liballeg.a

C Compilers Supported

The translator currently works with GNU C on Linux, with either WATCOM C or DJGPP C on DOS, and with either Watcom C, LccWin or Borland 5.5 on Windows. The WATCOM and GNU C implementations are 100% compatible with the Euphoria interpreter. The others are about 99% compatible. We recommend Borland over LccWin. Borland compiles faster, produces better code and has fewer bugs compared to LccWin.

The translator has been tested with the version of GNU C that comes with Red Hat 5.2. It needs a version of the ncurses library that's compatible with the one in Red Hat 5.2.

It has been tested with WATCOM C/C++ 9.5, 10.6 and 11.0. WATCOM 11.0 will soon be open source and free. Check for it at: http://www.openwatcom.org

The Watcom DOS32 package includes the CauseWay DOS Extender and file compressor. CauseWay is now open source and free. You can find out more about it at: http://www.devoresoftware.com/cwwmain.htm
emake.bat and objfiles.lnk will link in the CauseWay extender automatically. Other DOS extenders, such as DOS4GW, do not work well with the translator.

The translator looks for "WATCOM", "LCC", "BORLAND" or "DJGPP" as either environment variables or directories on your PATH. It will generate an emake.bat file that invokes the appropriate compiler and linker.

Notes:

Unlike WATCOM, DJGPP does not map DOS low memory into the same segment as other memory. Machine code routines written for the WATCOM-based Euphoria interpreter or translator will not work with DJGPP, and will likely crash if they try to access low memory, such as video memory. Euphoria's peek(), poke(), mem_copy(), mem_set() etc. *will* work correctly, as the translator uses a special DJGPP macro to access low memory. You can port these machine code routines to DJGPP, but you'll need to consult the DJGPP docs for the possible ways of accessing low memory.

DJGPP fully supports long filenames for reading, writing and creation. WATCOM doesn't support creation.

DJGPP supports a few more text modes, e.g. 35-line mode.

DJGPP lets the user abort a program at any time, by typing control-c.

The LccWin implementation ignores lock_file() and unlock_file(). They do nothing.

The latest version of LccWin seems to have some bugs that weren't in earlier versions. In most cases, if LccWin gives you any trouble, you can fix it by removing the -O optimization flag from one or more files in emake.bat.

Warnings are turned off when compiling with emake.bat. If you turn them on, you may see some harmless messages about variables declared but not used, labels defined but not used, function prototypes not declared etc.

On Windows, the WATCOM linker issues a warning that it can't open graph.lib. You can ignore this. graph.lib is not used. There doesn't seem to be an easy way to suppress this message.

The Microsoft C++ compiler for Windows is not yet supported. However you can probably import the C files generated by ecw.exe, and the runtime library file for Borland, Lcc or Watcom into a Microsoft project, anc compile/link with only minor glitches.

How to Run the Translator

Running the translator is similar to running the interpreter. On DOS you would type:

 
       ec shell.ex
    or 
       ec shell

but instead of running the shell.ex program, the translator will create several C source files. It will also create a file called emake.bat that will compile and link the C files for you. Just type: emake

When the C compiling and linking is finished, you will have a file called: shell.exe

When you run shell.exe, it should run the same as if you had typed: ex shell

to run it with the interpreter, except that it should run faster, performing more "sorts per second".

Note to Linux users: the files will be called emake and shell, and you type ./emake to perform the compiles and link, and ./shell to run the program.

If you happen to have more than one C compiler for a given platform, you can select the one you want to use with a command-line option:

        -bor
        -lcc
        -wat
        -djg

on the command line to ec or ecw. e.g.

        
         ecw -bor pretend.exw

To make a Windows .dll file, or Linux .so file, just add "-dll" to the command line. e.g.

         ecw -bor -dll mylib.ew

Note: For LccWin and Borland there is no standard environment variable, so the translator will search your PATH variable looking for a likely compiler directory. It looks in standard places such as: ..\LCC, ..\BCC.., ..\Borland.. etc. If you've installed in a nonstandard place you might have to rename your installation directory.

Dynamic Link Libraries (Shared Libraries)

Simply by adding "-dll" to the command line, the Translator will build a Windows .dll (Linux .so) file instead of an executable program.

You can translate and compile a set of useful Euphoria routines, and share them with other people, without giving them your source. Furthermore, your routines will likely run much faster when translated and compiled. Both translated/compiled and interpreted programs will be able to use your library.

Only the global Euphoria procedures and functions, i.e. those declared with the "global" keyword, will be exported from the .dll (.so).

Any Euphoria program, whether translated/compiled or interpreted, can link with a Euphoria .dll (.so) using the same mechanism that lets you link with a .dll (.so) written in C. The program first calls open_dll() to open the .dll or .so file, then it calls define_c_func() or define_c_proc() for any routines that it wants to call. It calls these routines using c_func() and c_proc(). See library.doc in the main Euphoria package for the details.

The routine names exported from a Euphoria .dll will vary depending on which C compiler you use.

GNU C on Linux exports the names exactly as they appear in the translated C code, e.g. a Euphoria routine

global procedure foo(integer x, integer y)

would be exported as "_0foo" or maybe "_1foo" etc. The underscore and digit are added to prevent naming conflicts. The digit refers to the Euphoria file where the symbol is defined. The main file is numbered as 0. The include files are numbered in the order they are encountered by the compiler. You should check the C source to be sure.

Lcc would export foo() as "__0foo@8", where 8 is the number of parameters (2) times 4. You can check the .def file created by the translator to see all the exported names.

For Borland the translator also creates a .def file, but this .def file renames the exported symbols back into the same names that you used in your Euphoria source, so foo() would be exported as "foo".

For WATCOM the same renaming as with Borland occurs, but instead of a .def file, an EXPORT command is added to objfiles.lnk for each exported symbol.

With Borland and WATCOM you can edit the .def or objfiles.lnk file, and rerun emake.bat, to rename the exported symbols, or remove ones that you don't want to export. With Lcc you can remove symbols but you can't rename them.

Having nice exported names is not critical, since the name need only appear once in each Euphoria program that uses the .dll, i.e. in a single define_c_func() or define_c_proc() statement. The author of a .dll should probably provide his users with a Euphoria include file containing the necessary define_c_func() and define_c_proc() statements, and he might even provide a set of Euphoria "wrapper" routines to call the routines in the .dll.

When you call open_dll(), any top-level Euphoria statements in the .dll or .so will be executed automatically, just like a normal program. This gives the library a chance to initialize its data structures prior to the first call to a library routine. For many libraries no initialization is required.

To pass Euphoria data (atoms and sequences) as arguments, or to receive a Euphoria object as a result, you will need to add the following new data types to euphoria\include\dll.e:

-- New Euphoria types for .dll (.so) arguments and return values:
global constant
		E_INTEGER = #06000004,
		E_ATOM    = #07000004,
		E_SEQUENCE= #08000004,
		E_OBJECT  = #09000004

Use these in define_c_proc() and define_c_func() just as you currently use C_INT, C_UINT etc. to call C .dll's and .so's.

The Lcc compiler is incompatible, in that you can't return a general Euphoria object (other than a 31-bit integer) from a .dll compiled by Lcc to a program compiled by Borland or WATCOM, or to a program being run by exw. The value might look ok, but a crash will likely occur. Similarly, a program compiled by Lcc can't handle general Euphoria objects returned by WATCOM or Borland .dlls. Passing in data always works fine. It's just general *return* values that cause compatibility problems. This might be fixed in the future.

Currently, file numbers returned by open(), and routine id's returned by routine_id(), can be passed and returned, but the library and the main program each have their own separate ideas of what these numbers mean. Instead of passing the file number of an open file, you could instead pass the file name and let the .dll (.so) open it. Unfortunately there is no simple solution for passing routine id's. This might be fixed in the future.

Euphoria .dlls (.so's) can be used by C programs as long as only 31-bit integer values are exchanged.

Executable Size and Compression

On DOS32 with Watcom, if the translator finds the CauseWay files, cwc.exe and le23p.exe in euphoria\bin, it will add commands to emake.bat that will compress your executable file. If you don't want compression, you can edit emake.bat, or remove or rename cwc.exe and/or le23p.exe.

On Linux, Windows, and DOS32 with DJGPP, emake does not include a command to compress your executable file. If you want to do this we suggest you try the free UPX compressor. You can get UPX it from: http://upx.tsx.org

The translator deletes routines that are not used, including those from the standard Euphoria include files. After deleting unused routines, it checks again for more routines that have now become unused, and so on. This can make a big difference, especially with Win32Lib-based programs where a large file is included, but many of the included routines are not used in a given program.

Nevertheless, your compiled executable file will likely be larger than the same Euphoria program bound with the interpreter. This is partly due to the interpreter being a compressed executable. Also, Euphoria statements are extremely compact when stored in a bound file. They need more space after being translated to C, and compiled into machine code. Future versions of the translator will produce faster and smaller executables.

Interpreter vs. Translator

The vast majority of Euphoria programs can be translated to C and will run identically (but hopefully faster).

The interpreter and translator share the same parser, so you will get the same syntax errors, variable not declared errors etc. with either one.

The translator reads your whole program before trying to do any translation. Occasionally it might catch a syntax error that the interpreter doesn't see, because the interpreter starts executing top-level statements immediately without waiting for the end of your program. This also means that if you have top-level statements in your program that modify a file that is later included, you will get a different result with the translator. Very few programs use this "dynamic include" technique.

Note: The translator assumes that your program has no run-time errors in it that would be caught by the interpreter. The translator does not check for: subscript out of bounds, variable not initialized, assigning the wrong type of data to a variable, etc.

You should debug your program with the interpreter. When C code crashes you'll typically get a very cryptic machine exception. If your translated .exe program does crash, the first thing you should do is run your program with the interpreter, using the same inputs, and preferably with "type_check" turned on.

Some of the run-time routines are still capable of catching an error and reporting it to the file "ex.err".

Legal Restrictions

As far as RDS is concerned, any executable programs that you create with this translator may be distributed royalty-free. You are free to incorporate any Euphoria files provided by RDS into your application.

You may not distribute any Complete Edition translator, or any run-time library file that comes with the Complete Edition for any platform.

You may not distribute any hacked or cracked (reverse engineered) versions of any library or translator, whether it's part of the Public Domain or Complete Edition.

In January 2000, the CauseWay DOS extender was donated to the public domain by Michael Devore. He has surrendered his copyright, and encourages anyone to use it freely, including for commercial use.

In general, if you wish to use Euphoria code written by 3rd parties, you had better honor any restrictions that apply. If in doubt, you should ask for permission.

On Linux and DJGPP for DOS32, the GNU Library licence will normally not apply to programs created with this translator. Simply compiling with GNU C does not give the Free Software Foundation any jurisdiction over your program. If you statically link their libraries you will be subject to their Library licence, but the standard compile/link procedure in emake does not statically link any FSF libraries, so there should be no problem. The ncurses library is the only one statically linked, and although the Free Software Foundation now holds the copyright, ncurses is not subject to the GNU Library licence, since it was donated to FSF by authors who did not wish the GNU licence to apply to it. See ncurses.h for the copyright notice.

Disclaimer: This is what we believe to be the case. We are not lawyers. If it's important to you, you should read the GNU Library licence, the ncurses licence, the legal comments in DJGPP, LccWin and Borland, and Michael Devore's read.me file on his site, to form your own judgement.

The Public Domain Translator

Programs (including .dll's and .so's) created using the free Public Domain translator will display a brief message on the screen prior to execution.

The Complete Edition Translator

When you register for the Complete Edition Translator, you'll get:

The Complete Edition translator for all platforms and all supported C compilers.
No more message and delay at the start of execution.
"with trace" and trace(3) will let you see the Euphoria statement that was being performed (by the C code) when a crash occurred. You'll also see a trace of the (up to) 500 preceding Euphoria statements. Look for a file called "ctrace.out" (produced by a main program) or "ctrace-d.out" (produced by a .dll or .so) in the current directory. ctrace.out is used as a circular buffer to hold traced statements. Once 500 statements are written to ctrace.out, the file pointer is reset to the start. This prevents the file from getting too large. The Euphoria statement that was being executed when the program terminated, is the one just prior to the line: "=== THE END ===". If you get a run-time error message, this line might be displayed for you automatically.
With most of the C compilers, pressing control-C will stop a program. ctrace.out will show where the program was executing when it was stopped. This is a good way to diagnose infinite loops.
Your program will run much slower when ctrace.out is being written. You can help the situation by calling trace(3) and trace(0) to select the portions of your program that you want to trace. You can also use "without trace" to eliminate tracing in certain parts of your code.
Euphoria statements will be inserted as comments in the C source You'll have a clearer idea of how each Euphoria statement has been implemented in C.
The usual benefits of being a registered user:
- free upgrades for at least 12 months
- discounts on future upgrades
- 3 months free tech support from RDS
- ability to vote $3.00 per month in the "MicroEconomy"

What Does it Cost?

The Complete Edition Euphoria to C Translator is available now for just $29.00 U.S. One price covers all platforms. You can register via DigiBuy at our Web site, www.RapidEuphoria.com, or send a check to:

      Rapid Deployment Software
      130 Holm Crescent
      Thornhill, Ontario
      L3T 5J3
      CANADA

If you download the Complete Edition translator files from our site there is no shipping and handling charge. If you want us to ship you a disk and receipt please add $5.00, so the total becomes $34.00 U.S.

Frequently Asked Questions

* How much of a speed-up should I expect?

It all depends on what your program spends its time doing. Programs that use mainly integer calculations, don't call run-time routines very often, and don't do much I/O will see the greatest improvement, currently up to about 5x faster. Other programs may see only a few percent improvement. The various C compilers are not equal in optimization ability. WATCOM, GNU C and DJGPP produce the fastest code. Borland is fairly good. LccWin lags slightly behind the others. Borland compiles the fastest. Watcom compiles the slowest.

* Should I register for the translator without first registering for the interpreter package?

Debugging large programs will be easier if you have the Complete Edition interpreter. Ideally you should develop and debug your code thoroughly with the interpreter, and use the translator as the final step to gain speed. You can however, buy the interpreter or translator package separately, and buy the other package later.

* If I buy the Interpreter source code, will I be able to build my own Translator run-time library?

No. You will however be able to view the source to most of the routines in the Translator run-time library, and include modified versions of some of the routines in your program. You will also be able to build your own interpreter.

* What if I want to change the compile or link options in emake.bat?

Feel free to do so, however you should copy emake.bat to your own file called (say) mymake.bat, then run mymake.bat after running the translator. Occasionally the number of .c files produced by the translator could change.

* How can I make my program run even faster?

It's important to declare variables as integer where possible. In general, it helps if you choose the most restrictive type possible when declaring a variable. User-defined types will not affect your speed. They are ignored by the translator unless you also call them directly with normal function calls. On Windows and DOS we have left out the /ol loop optimization for Watcom's wcc386. We found in a couple of rare cases that this option led to incorrect machine code being emitted by the WATCOM C compiler. If you add it back in to your own version of emake.bat you might get a slight improvement in speed, with a slight risk of buggy code. For DJGPP you might try -O6 instead of -O2. For DOS we use the WATCOM /fpc option which generates calls to run-time routines to perform floating-point operations. If the machine has floating-point hardware it will be used by the routine, otherwise software emulation will be used. This slows things down somewhat, and isn't needed on Pentiums, but it guarantees that your program will run on all 386 and 486 machines, even if they lack floating-point hardware. The DOS run-time library, ec.lib, was built this way, so you can't simply remove this option. On Linux you could try the O3 option of gcc instead of O2. It will "in-line" small routines, improving speed slightly, but creating a larger executable. There's an enhanced version of gcc called pgcc that claims to have better optimization for Pentium CPU's. We tried it, but it only gave us a very small (a couple of percent) speed up. It's available for free from: http://goof.com/pcg/

Common Problems

Many large programs have been successfully translated and compiled using each of the supported C compilers, and the translator is now quite stable.

N.B. Very Common: Lcc may complain of an undefined symbol, or a symbol with a name that's too long. This is an Lcc optimizer bug. Just remove the -O option in emake.bat for building that particular file. For instance, if it complains about xyz.obj, take out the -O option from the compile step for xyz.c. In rare cases, -O will not cause a compile-time error, but will result in incorrect machine code, leading to a crash.

N.B. If you call a C routine in a Windows .dll file, be sure that the routine uses the __stdcall calling convention (as opposed to __cdecl, which is the default for most C compilers). If you don't, the call may *seem* to work, but you'll get corruption of the call stack. This corruption might not be apparent when you run the exw interpreter, and may only show up when you run translated/compiled code.

In some cases a huge Euphoria routine is translated to C, and it proves to be too large for the C compiler to process. If you run into this problem, make your Euphoria routine smaller and simpler. You can also try turning off C optimization in emake.bat for just the .c file that fails. Breaking up a single constant declaration of many variables into separate constant declarations of a single variable each, may also help. Euphoria has no limits on the size of a routine, or the size of a file, but most C compilers do. The translator will automatically produce multiple small .c files from a large Euphoria file to avoid stressing the C compiler. It won't however, break a large routine into smaller routines.

Send bug reports to rds@RapidEuphoria.com
In particular, let us know if any translated program does not run the same when compiled as it does when interpreted.