School Computer System Information

This web page is essentially just a concatenation of the files in this directory.

Including External Code in IDL
Last Updated: 16 May 2003

There are a number of different options for incorporating compiled code such as C and Fortran into IDL. This document outlines the possible approaches and contains some discussion on which methods are appropriate for a given situation. This document is essentially an overview of the IDL EXTERNAL DEVELOPMENT GUIDE (previously the Advanced Development Guide).

The Choice:

If at all possible and rewriting time and speed are not of paramount importence you should rewrite any legacy routines in IDL. However, it is often necessary, in situations where existing code is too long to be practically rewritten or a clear speed advantage exists, to call external code from within IDL.

It should be noted that while in some situations IDL can be slower than the C or Fortran equivalent, this is not always the case; particularly when use of IDL's vectorized code features can be made.

OK you've decided that rewriting is not an option, the decision of how to incorporate your code into IDL needs to be made. The method used depends on a number of considerations: does your solution need to work on a number of different systems (Unix, windows, mac, VMS) or just the one; what level of complexity are you capable of (most of the possible incorporation methods require a large amount of knowledge about system specific features, IDL internals, or both); will your application benefit significantly from system specific features such as RPC under Unix; and how deeply linked into IDL does your external routine really need to be?

Outlined below are the possible options for combining Fortran and C with IDL. With the considerations above these outlines should help you decide which option is appropriate for you. In most situations the answer will be CALL_EXTERNAL. There are three main reason for this: (i) CALL_EXTERNAL is supported on all systems that IDL runs on; (ii) the simplest possible method (SPAWN) is also exceptionally limited, especially on non-Unix systems; and (iii) the system specific (RPC etc.) and most deeply linked (LINKIMAGE and DLMs) methods require more background knowledge than most people have or have the time to acquire.

SPAWN:

SPAWN simply gives access to the command line so you can run a command. It creates a process with its own process space.

If the external code you want incorporated is standalone then SPAWN is the simplest way of accessing that routine from IDL.

Under UNIX it is possible to capture output or set up bi-directional communication.

Communication with processes under any non-UNIX platforms is extremely limited.

SPAWN is too limited in most situations of incorporating external code. However, it is worth keeping in mind as a way of accessing system commands; particularly on Unix.

See Chapter 2 of IDL EXTERNAL DEVELOPMENT GUIDE & Page 994 of the IDL REFERENCE GUIDE (VOL 2)

ActiveX:

Limited to Windows 95 and NT (presumably is also available on 98; 2000; ME; and XP)

See Chapters 3 & 4 of IDL EXTERNAL DEVELOPMENT GUIDE

AppleScript:

Limited to Mac only.

See Chapter 5 of IDL EXTERNAL DEVELOPMENT GUIDE

Remote Procedure Calls:

Causes IDL to function as an RPC (Remote Procedure Call) server.

Provides a similar level of embedding as LINKIMAGE.

Allows called process to be run on another machine, possibility of overlapped execution on a networked or multi-processor system.

Requires detailed knowledge of writing RPC programs and as such is far from trivial.

Limited to Unix only.

Unless there is a significant advantage in parallel or remote processing this approach is probably a bit extreme.

See Chapter 6 of IDL EXTERNAL DEVELOPMENT GUIDE

CALL_EXTERNAL:

Supported across all platforms.

Doesn't perform error checking.

Coding problems in external routine can corrupt your IDL session.

Can be written within an IDL function to provide some of the advantages of an in-built function.

Requires very little IDL internal understanding.

Requires a basic understanding of how compiling and linking works on your system.

For an introduction and example see CALL_EXTERNAL.pdf in this directory.

Also see Chapter 7 of IDL EXTERNAL DEVELOPMENT GUIDE & Page 224 of the IDL REFERENCE GUIDE (VOL 1)

LINKIMAGE & Dynamically Loadable Modules:

Most fully integrated approach.

Gives all the benefits of, and behaves like, one of IDL's built in functions.

Requires full understanding of IDL internals.

Provides access to variables and other objects in IDL.

See Chapters 7 & 18 of IDL EXTERNAL DEVELOPMENT GUIDE & Page 584 of the IDL REFERENCE GUIDE (VOL 1)

Callable IDL:

The above are all ways of calling external code from IDL. It is also possible to call IDL from within other programs, e.g. C or Fortran.

See Chapters 19 - 21 of IDL EXTERNAL DEVELOPMENT GUIDE

Places to Find Additional Information:

The IDL directory on your computer contains examples and files containing information that may be of use.

IDL's news group can be found at:

http://groups.google.com/groups?group=comp.lang.idl-pvwave

The site of the company that develops IDL is:

http://www.rsinc.com/

CALL_EXTERNAL in IDL
Last Updated: 16 May 2003

This document is intended as a short introduction to the use of the CALL_EXTERNAL function in IDL.

How it Works:

CALL_EXTERNAL is invoked as a function call from within IDL, typically called from within a user written procedure or function (ie not directly from the command line.

The template for the call looks like this:

Result = CALL_EXTERNAL(Image, Entry[,P, ...,P $_{N-1}$ ])

where

Result: is a variable into which the function returns a value. If you're calling a function this will be the value of interest, however, if you're calling a subroutine the value/s of interest are more likely passed through the routines interface and Result can be used to determine the status of the call e.g. failed/succeeded

Image: is a string containing the name of the compiled and linked sharable object. (*.so on UNIX; *.dll on Windows)

Entry: The name of the point of entry into the routine, ie the routine name inside the shared object; this varies from compiler to compiler. Under UNIX type nm *.so at the command line to a name listing of your object and thus determine how the entry point was renamed by the compiler; typically this involves appending or prepending an underscore to your particular routine name.

[P, ...,P $_{N-1}$ ]: refers to the arguments passed to your external routine. The default is to pass by reference. These arguments can be scalars, arrays, and strings.

Fortran common blocks can also be passed between IDL and Fortran, but only under VMS.

An invocation of CALL_EXTERNAL passes two arguments to the wrapper routine: (i) argc (scalar of the number of arguments); and (ii) argv(an array of the arguments). By default these are passed by reference.

Important Considerations:

CALL_EXTERNAL performs no checking of the number and type of parameters. It is therefore best to include error checking in your IDL code. This can be done by putting CALL_EXTERNAL in an IDL function which takes care of any required checking, this has the added advantage of providing a neat interface to your external routine.

Use IDL input/output facilities NOT external code ones.

While strings can be passed it is best to avoid this if possible. Strings are passed as structures which results in an additional level of complexity in the wrapper-routine interface. As such slight system/compiler/language inconsistencies can cause additional headaches.

Brief How To:

(see Shared_obj_comp.txt and Wrapper.txt and examples in this directory for more detail.)

(1) If your external routine was written to be reusable (ie not written for use in one specific piece of code) then you should be able to start at step (2). However, if this is not the case...

(2) Write a wrapper function. This is a function that facilitates communication between IDL and your external routine by acting as an interface between them. This can be written in Fortran or C.

(3a)Compile your routine into a shareable object, don't link.

(3b)Compile your wrapper into a shareable object, don't link.

(3c)Link your routine and wrapper using the linker for language in which the wrapper is written.

(4)Write an IDL function that takes care of type and error checking before calling CALL_EXTERNAL

Now you external function can be called and used as though you had rewritten the entire thing as an IDL function.

Example:

Look at the example code in this directory. Try compiling and linking the example on your system (see Shared_obj_comp.txt)

The example consists of: a piece of Fortran code, vecadd.f, which is the external routine being called from IDL; a Fortran wrapper for vecadd.f called vecadd_wrapf.f; a C wrapper for vecadd.f called vecadd_wrapc.c; and an IDL function called vecadd.pro which provides a neat interface from within IDL to the CALL_EXTERNAL function.

There is also a second set of code that incorperates a string into the call...

Further Information:

The IDL directory on your computer.

The IDL manuals; which are available online by typing

at the IDL prompt

http://groups.google.com/groups?group=comp.lang.idl-pvwave

http://www.rsinc.com/

*********************************************************************
** Some useful information for writing C and Fortran wrappers for  
** routines to be called from IDL.
*********************************************************************


THINGS TO NOTE: 
~~~~~~~~~~~~~~~

***Wrappers do not need to be written in the same language as the 
   routine. 
   
***Some C compilers (those on 64-bit machines) define long as 
   64-bit instead of 32-bit.
   
***Decleration of references passed to Fortran from IDL are integer*8 
   for machines with 64-bit address spaces and integer*4 for those 
   with 32-bit address spaces.

***Not all f77 compilers support val, loc and/or %val, %loc

***C can #include export.h from IDL which defines variable interfaces 
   such as IDL_LONG

_____________________________________________________________________




TYPE COMPARISONS for External calls from IDL to fortran and c:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

***************************************************************
         IDL           |     FORTRAN     |          C         |
  (Dynamically Typed)  |                 |                    |
***************************************************************
                       |                 |                    |
  Unsigned byte        | LOGICAL*1 l     | unsigned char l;   |
  Byte                 |                 |                    |
                       |                 |                    |
---------------------------------------------------------------
                       |                 |                    |
  Integer              | INTEGER*2 j     | short j;           |
                       |                 |                    |
--------------------------------------------------------------- 
                       |                 |                    |
  Long                 | INTEGER*4 k     | int k;             |
                       |                 | long k;            |
                       |                 |   (32-bit machines)|
---------------------------------------------------------------
                       |                 |                    | 
  Floating-Point       | REAL*4 a        | float a;           |
                       |                 |                    |
---------------------------------------------------------------
                       |                 |                    | 
  Double-Precission    | REAL*8 d        | double d;          |
                       |                 |                    |
---------------------------------------------------------------
                       |                 |                    | 
  Complex              | COMPLEX*8 c     | struct complx {    |
                       |                 |      float x;      |
                       |                 |      float y;      |
                       |                 | };                 |
                       |                 | struct complx c;   |
---------------------------------------------------------------
                       |                 |                    |
  DP Complex           | COMPLEX*16 d    | struct dcomp {     |
                       |                 |      double x;     |
                       |                 |      double y;     |
                       |                 | };                 |
                       |                 | struct dcomp d;    |
                       |                 |                    |
---------------------------------------------------------------
                       |                 |                    |
  String               |   Passed as a Structure  (see below) |
                       |                 |                    |
_______________________________________________________________


When strings are passed with CALL_EXTERNAL they are passed as structures.

It is best to avoid passing strings to routines through call_external.

That being said, if you feel you must pass a string here is what the 
structure that is passed looks like currently(that is i think the 
makers of IDL consider this something that they are free to change). 

-------------------------------------------------------------------


C: 
~~

typedef struct IDL_STRING {   /* Define string descriptor */          
  unsigned short slen;	      /* Length of string, 0 for null */      
  short stype;		      /* type of string, static or dynamic */ 
  char *s;		      /* Addr of string */                    
} IDL_STRING;

Declaration of string has this form:
struct IDL_STRING my_string

-------------------------------------------------------------------


FORTRAN:
~~~~~~~~

STRUCTURE /IDL_STRING/
  LOGICAL*2 slen
  INTEGER*2 stype
  INTEGER*4 s  ==> INTEGER*8 s on machines with 64-bit address spaces
END STRUCTURE

Declaration of string has this form:
RECORD /IDL_STRING/ my_string


___________________________________________________________________

CHARACTER*n c          | char c[n];

c   vecadd.f
c
c   This is the routine that does the work. It in principle 
c   needs to know nothing about its call from IDL.
c
      SUBROUTINE vecadd1(a, na, x, nx, b)
      IMPLICIT NONE
      
        INTEGER*4 na, nx
        REAL*4 a(na), b(nx)
        INTEGER*4 x(nx), i

        DO i = 1, nx
          a(x(i)) = a(x(i)) + b(i)
        END DO
        
      END

c   vecadd_wrapf.f
c
c   This function is the wrapper routine called by IDL
c
      INTEGER*4 FUNCTION vecadd(argc, argv)
      IMPLICIT NONE
c
c       The declaration below is integer*8 for machines with 
c       64-bit address spaces and integer*4 for 32-bit
c
        INTEGER*8 argc, argv(*)
c
c       Call The Fortran Routine:
c
        CALL vecadd1(%val(argv(1)), %val(argv(2)), %val(argv(3)),
     &    %val(argv(4)), %val(argv(5)))
c
c       Give the function vecadd a value for return to IDL, this 
c       facilitates the checking of whether the routine executes. 
c         
        vecadd=1
c 
      END

/* vecadd_wrapc.c */

/* This function is the wrapper routine called by IDL */

#include <stdio.h>

int vecadd(int argc, void *argv[])

{
  extern void vecadd1_(); /* Declare The Fortran Routine */
  int *na, *nx;
  int *x;
  float *a, *b;
  
  a = (float *) argv[0]; 
  na = (int *) argv[1]; 
  x = (int *) argv[2];
  nx = (int *) argv[3];  
  b = (float *) argv[4];
  
  vecadd1_(a, na, x, nx, b); /* Call The Fortran Routine */
  
  return 1;
  
}

; vecadd.pro
; 
; This function performs checking of variables before a call 
; is made to an external routine via CALL_EXTERNAL. 
;
; The data types need to be the same as those expected by the 
; external routine.
; 
; For example:
; IDL>   a = findgen(10)
; IDL>   x =[3, 5, 7]
; IDL>   b = [2, 4, 6]
; IDL>   print, vecadd(a, x, b), format='(10f5.1)'
; % Compiled module: VECADD.
;   0.0  1.0  2.0  5.0  4.0  9.0  6.0 13.0  8.0  9.0

FUNCTION VECADD, array, index, value

;- Check arguments:
  IF (N_ELEMENTS(array) EQ 0) THEN $
    MESSAGE, 'Argument A is undefined'
  IF (N_ELEMENTS(index) EQ 0) THEN $
    MESSAGE, 'Argument X is undefined'
  IF (N_ELEMENTS(value) EQ 0) THEN $
    MESSAGE, 'Argument B is undefined'
  IF (N_ELEMENTS(index) NE N_ELEMENTS(value)) THEN $
    MESSAGE, 'Arguments X and B must have the same number of elements'

;- Create copies of the arguments with correct type:
  IF (SIZE(a, /tname) NE 'FLOAT') THEN BEGIN
    a = FLOAT(array)
  ENDIF ELSE BEGIN
    a = array
  ENDELSE
  x = ((LONG(index) > 0L) < (N_ELEMENTS(a) - 1L)) + 1L
  b = FLOAT(value)

;- Call the external routine:
  result = CALL_EXTERNAL('vecadd.so', 'vecadd', $
                         a, N_ELEMENTS(a), x, N_ELEMENTS(x), b)
  IF (result NE 1) THEN MESSAGE, 'Error calling external routine'

;- Return result:
 RETURN, a

END