Galaxy Communicator Documentation:

Upgrading from 3.3 to  4.0

License / Documentation home / Help and feedback

Upgrading to Galaxy Communicator 4.0 is simple. Most of the obligatory upgrades are restricted to very recent or obscure functionality.

This document describes only the steps required or recommended to upgrade existing features of Galaxy Communicator. You can find a list of new features here. You can find the complete release notes here.
 
 
Step Who's affected Status
Step 1: Updating logging support All users of the TIMESTAMP: Hub directive and LOG_IN: and LOG_OUT: directives in the scope of the PROGRAM: or MESSAGE: directives who may be relying on a sloppy 3.3 implementation Obligatory
Step 2: Update use of -gui All users of the -gui Hub command line argument Obligatory
Step 3: Update use of variable substitution
All users of Gal_VAReadVarFrameFromString and Gal_ReadVarFrameFromString
Obligatory
Step 4: Update expectations for missing Hub programs and operations
All users of GalSS_EnvDispatchFrame and its relatives
Obligatory
Step 5: Upgrade to broker proxies
All users of brokering
Optional
Step 6: Upgrade references to verbosity constants
All C programmers who refer to the undocumented constants GAL_FATAL_LEVEL, etc.
Obligatory
Step 7: Upgrade use of hub_continue
All Hub program writers who use the :service_provider key of the Builtin server function hub_continue
Obligatory
Step 8: Update broker startup
All C programmers who use brokers in threaded Communicator servers
Recommended
Step 9: Update broker callback setup
All C programmers who refer to the events GAL_CONNECTION_BROKER_OUT_STARTUP_EVENT and GAL_CONNECTION_BROKER_IN_STARTUP_EVENT
Obligatory
Step 10: Upgrade Java servers
All users of Java bindings.
Obligatory
Step 11: Upgrade use of EXPAND: in process monitor configuration files
All users of the EXPAND: process monitor configuration file directive who used it to generate unique temporary file names
Obligatory
Step 12: Upgrade argument lists of Server class in Python
All users of the Server class in Python who rely on order-based arguments to the constructor
Obligatory
Step 13: Upgrade use of host and port override rules for -locations and -server_locations_file
All users of the -locations and -server_locations_file Hub command line arguments
Obligatory


Step 1: Updating logging support

In version 3.3 and previous, the implementation of the TIMESTAMP: directive and of the implicit timestamp associated with LOG_IN:/LOG_OUT: in PROGRAM: or MESSAGE: blocks was carelessly done; it arranged for timestamps in situations where it shouldn't have. We have clarified the rules under which this happens to ensure that TIMESTAMP: applies anywhere, but only to messages which match the provided description, and that LOG_IN:/LOG_OUT: in PROGRAM: or MESSAGE: only apply to new messages received by the Hub and replies it sends to the originating server. This limitation is not a change, merely a clean implementation of a previously documented limitation. If you're inadvertantly relying on the previous implementation, you may need to upgrade your Hub programs. Please check your Hub programs and log output.


Step 2: Update use of -gui

In version 3.3, we introduced, but did not document, the ability of the Hub to contact remote servers for visualization and profiling. The location of this remote server is provided to the Hub via the -gui command line argument. In version 3.3, the format of this argument was direction@host:port, where direction was either server (i.e., the server has the listener) or hub (i.e., the Hub should set up the listener). When the Hub had the listener, the host argument was ignored.

In the context of cleaning up references to provider names and documenting the Hub GUI functionality, we decided that this format was not particularly informative. So in 4.0, the format is host:port, where host can be the distinguished string <listener>, indicating the Hub should set up the listener.


Step 3: Update use of variable substitution

In version 3.3, we made an incorrect assumption having to do with Gal_VAReadVarFrameFromString and Gal_ReadVarFrameFromString which could lead to memory leaks. In particular, our example looked like this:
Gal_VAReadVarFrameFromString("{c foo :string $a :int $b }", 2,
                             "$a", Gal_StringObject("bar"),
                             "$b",
Gal_IntObject(5));
This invocation would have led to memory leaks if there were no references to some of the variables in the frame string. In version 4.0, these functions now assume that the programmer will manage the memory for the variable values, and copy the variable values before substitution. This both enables the programmer to  ensure that  newly-created elements are freed, and also eliminates the possibility of multiple references to the same Gal_Object in a single frame, which is currently not supported. In 4.0, the equivalent invocation should look like this:
Gal_Object o1 = Gal_StringObject("bar");
Gal_Object o2 =
Gal_IntObject(5);

Gal_VAReadVarFrameFromString("{c foo :string $a :int $b }", 2,
                             "$a", o1, "$b", o2);
Gal_FreeObject(o1);
Gal_FreeObject(o2);

Step 4: Update expectations for missing Hub programs and operations

In  version 3.3, a call to GalSS_EnvDispatchFrame would do something somewhat unintuitive when the Hub could neither find a matching Hub program nor identify a matching service type which supported the specified operation. The Hub would create a new message named system-error and process it as an incoming message. However, it would not return any error to the caller. This behavior differs from the behavior of service providers which are sent messages by the Hub which they don't support: they raise an error directly back to the Hub.

In version 4.0, we have augmented this behavior to be more consistent. The Hub still generates its new message, in case Hub script programmers rely on this feature; however, instead of a normaly reply, the caller of GalSS_EnvDispatchFrame or its relatives receives an error. We highlight this change in case programmers have been inadvertantly careless about checking the message type of the reply, which should be done as follows:

GalIO_MsgType t;
Gal_Frame msg, reply;

/* ... */
reply = GalSS_EnvWriteFrame(env, msg, &t);
switch (t) {
case GAL_REPLY_MSG_TYPE:
  /* normal reply */
  break;
case GAL_ERROR_MSG_TYPE;
  /* error */
  break;
default:
  break;
}


Step 5: Upgrade to broker proxies

This  upgrade is optional, but highly recommended. Broker proxies are a considerable improvement on the original broker API., both because the new broker proxies encapsulate all the broker contact information in a single key whose value has a distinguished type, and because the broker stream can be restricted by object type. The semantics of the elements flowing over the broker connecting are also clearer; if you receive an array of integers, the proxy encodes whether it's a component of a larger array, or an element of a list, or just a random object transmitted over the connection.

Here's a comparison of the use of brokers for streaming data, using proxies and raw brokers:

Writing through a raw broker

static Gal_Frame prepare_audio_frame(GalSS_Environment *env,
                                     char *filename)
{
  Gal_Frame f = Gal_MakeFrame("main", GAL_CLAUSE);
  int total = 0;
  char *buf = (char *) NULL;
  GalIO_BrokerStruct *b;
 
  /* In the code omitted here, the data from the named file is
     read into buf, and total is the number of bytes in buf */
 
  /* .... */
 
  /* Now that we have the audio, we write the binary data
     through the broker. */
 
  b = GalIO_BrokerDataOutInit(GalSS_EnvComm(env), 0, 10);
  if (b && (GalIO_GetBrokerListenPort(b) > 0)) {
    GalIO_BrokerPopulateFrame(b, f, ":binary_host", ":binary_port");
    GalIO_BrokerWriteBinary(b, buf, total);
    GalIO_BrokerDataOutDone(b);
  }
  free(buf);
 
  return f;
}
 

Writing through a proxy

static Gal_Frame prepare_audio_frame(GalSS_Environment *env,
                                     char *filename)
{
  Gal_Frame f = Gal_MakeFrame("main", GAL_CLAUSE);
  int total = 0;
  char *buf = (char *) NULL;
  Gal_Object proxy;
 
  /* In the code omitted here, the data from the named file is
     read into buf, and total is the number of bytes in buf */
 
  /* .... */

  proxy = GalSS_ObjProxifyObjectType(env, GAL_BINARY, 0, 10);
  if (proxy) {
    GalSS_ObjProxyArrayAdd(proxy, buf, total);
    /* Because the data object is a streaming object, it must
       be marked as done. */
    GalSS_ObjProxyDone(proxy);
    Gal_SetProp(f, ":binary_proxy", proxy);
  }
 
  return f;
}

Reading from a raw broker

Gal_Frame receive_audio(Gal_Frame f, void *server_data)
{
  DataHandler *d = (DataHandler *) malloc(sizeof(DataHandler));
  GalIO_BrokerStruct *b;
  char *host = Gal_GetString(f, ":binary_host");
  int port = Gal_GetInt(f, ":binary_port");
  d->data_buf = (char *) NULL;
  d->size = 0;
  if (host && port) {
    b = GalSS_EnvBrokerDataInInit((GalSS_Environment *) server_data,
                                  host, port, f,
                                  env_audio_handler, 0, d, __FreeDataHandler);
    if (b) {  
      GalIO_AddBrokerCallback(b, GAL_BROKER_ABORT_EVENT,
                              __report_abort, (void *) NULL);
      GalIO_AddBrokerCallback(b, GAL_BROKER_DATA_DONE_EVENT,
                              __report_done, (void *) NULL);
      GalIO_SetBrokerActive(b); 
    }
  } else {
    free(d);
  }
  return (Gal_Frame) NULL;
}

Reading from a proxy

Gal_Frame receive_audio(Gal_Frame f, void *server_data)
{  
  GalSS_Environment *env = (GalSS_Environment *) server_data;
  GalSS_BrokerProxy *proxy;

  /* We get the proxy object, and proceed asynchronously. The
     object passed into the callback is owned by the callback.
     We use a non-immediate callback, which will cache all the
     data for us and call the data handler when it's done. */
  proxy = Gal_GetObject(f, ":binary_proxy");

  if (Gal_Proxyp(proxy)) {
    GalSS_ObjUnproxify(env, proxy, proxy_audio_handler,
                       __proxy_report_done, __proxy_report_abort,
                       0, 0, NULL, NULL);

    }
  }
  return (Gal_Frame) NULL;
}
Note that especially on the receiving end, there's much less to do, and the callback infrastructure is handled for you, as is the assembly of streaming data into a single object. In addition to these benefits, and the fact that the contact information is encapsulated and typed, it's also possible to read objects from proxies without using callbacks, if you so choose. In general, the new API is much more flexible and powerful. For more information, see the broker documentation.


Step 6: Upgrade references to verbosity constants

In versions 3.3 and previous, the undocumented constants GAL_FATAL_LEVEL, GAL_WARNING_LEVEL, GAL_PINFO1_LEVEL, GAL_PINFO2_LEVEL, GAL_DEBUG1_LEVEL and GAL_DEBUG2_LEVEL matched up awkwardly with the values for GAL_VERBOSE. These constants were interpreted as the threshold above which printing could occur, rather than the thresholds themselves. As we upgraded printing to rationalize the print levels, this mismatch became more and more of a problem. In 4.0, then, the meaning of the level constants now match the meaning of GAL_VERBOSE.

So let's say you want to print something out if GAL_VERBOSE corresponds to the default level of printing, which is the level for GalUtil_PInfo1. Here's the difference:

Version 3.3

if (GAL_VERBOSE > GAL_PINFO1_LEVEL) {
  /* ... */
}

Version 4.0

if (GAL_VERBOSE >= GAL_PINFO1_LEVEL) {
  /* ... */
}


Step 7: Upgrade use of hub_continue

Before version 4.0, it was exceptionally hard to refer to service providers. The best you could do, in a number of contexts, was refer to the host and port the provider had been contacted at. This was particularly important if you were using the Builtin dispatch function hub_continue. However, now that we've introduced provider names, it is no longer necessary to resort to this, and support for this sort of reference has been removed, in the interests of consistency.

Here's a minimal pair for those of you who need to upgrade:

Version 3.3

SERVER: Parser
LOCATION: localhost:5003
OPERATIONS: Parse

...

RULE: --> Parser.Parse
IN: :string
OUT: none!

RULE: --> Builtin.hub_continue
IN: (:reply_matches ( {c ParseResult } )) (:service_provider "localhost:5003")
OUT: :frame

Version 4.0

SERVER: Parser
LOCATION: localhost:5003
OPERATIONS: Parse
PROVIDER_ID: [myparser]

...

RULE: -->Parser.Parse
IN: :string
OUT: none!

RULE: --> Builtin.hub_continue
IN: (:reply_matches ( {c ParseResult } )) (:service_provider "[myparser]")
OUT: :frame




Step 8: Update broker startup

If you're using threads, and you add broker callbacks or other properties to your brokers or broker proxies after you create them, there's a chance that the broker thread will start up (and possibly finish) before the callbacks are added. In 4.0, we've added functionality which allows the programmer to create a broker without starting it up. See the new section on delaying broker activation for instructions on how to do this.


Step 9: Update broker callback setup

Adding the functionality to delay broker activation has required an incompatible change in the semantics of two of the events associated with brokers. These events are accepted by the function GalIO_AddConnectionBrokerCallback. In versions 3.3 and previous, the events GAL_CONNECTION_BROKER_OUT_STARTUP_EVENT and GAL_CONNECTION_BROKER_IN_STARTUP_EVENT were documented as being fired when a broker was created. In version 4.0, these events have been redefined as being fired when a broker is started. Before 4.0, it was not possible to create a broker without starting it; now that this is possible, we've added two new events, GAL_CONNECTION_BROKER_OUT_CREATION_EVENT and GAL_CONNECTION_BROKER_IN_CREATION_EVENT, which are fired when the broker is created.
   

Step 10: Upgrade Java servers

There are a number of changes in the 4.0 Java bindings that are incompatible with earlier releases.

Step 11: Upgrade use of EXPAND: in process monitor configuration files

In version 3.3, the process monitor configuration file permitted directives such as the following:
EXPAND: $TEMP
The interpretation of this entry is that $TEMP would be bound, in each process block, to a unique temporary file name which could be passed around in that process block. However, as a result, it was impossible to define an empty expansion. This latter functionality became desirable in the context of defining configuration files which could run on either Unix or Windows. Therefore,  in 4.0, the entry shown here does expand to the empty string, and we've introduced the TEMPFILE: directive to provide explicit temporary file substitutions.


Step 12: Upgrade argument lists of Server class in Python

As part of the upgrade to server location files, we've added an argument to the Server class in Python for the name of the entry in the server location file, analogous to the -slf_name argument in C. For coherence, we've added this argument immediately after the argument for the server location file, which means that anyone who relies on the order of the arguments to the Server class constructor will need to add an additional argument. This should only affect programmers who are inheriting from the Server class, since there are so many arguments to the constructor which are seldom overwritten that relying on ordered arguments would be counterproductive in most cases. If you encounter the problem, you'll recognize it because you'll suddenly start getting TypeError: not a string when the internal function cGalaxy._GBGalSS_EncapsulateArguments is called.


Step 13: Upgrade use of host and port override rules for -locations and -server_locations_file

As part of upgrading references to provider names, we encountered some inconsistencies in the server locations file override rules in the Hub. Among other things, it was possible to change where the listener was in the file but not on the command line. These rules have now been rationalized. However, this rationalization appears to have introduced some incompatible changes. We can't reconstruct all of them, but one of them appears to be that the way to override a Hub listener port has changed. Previously, the following declaration
SERVICE_TYPE: foo
CLIENT_PORT: 2700

could have its CLIENT_PORT: value overridden by this command line argument:
... -locations "foo@ignore:2900" ...
However, in 4.0, the assumption is that any host name which is present indicates that the Hub should expect the listener to be on that host; so in 4.0, this declaration would remove the listener and add a service provider for foo which is expected to be found on the host ignore at port 2900. In order to achieve the intended effect in 4.0, you should use the special <listener> token:
... -locations "foo@<listener>:2900" ...
This syntax is also consistent with the upgraded format for the -gui Hub argument.
  
License / Documentation home / Help and feedback
Last updated August 23, 2002