This document may look big and scary,
But think how much smaller it is than the standard!
Started 21st August 2001
Version 1.0
Last modified 1st October 2001
Mike Taylor
<mike@tecc.co.uk>
Sebastian Hammer
<quinn@indexdata.dk>
Contributions from Ashley Sanders
<a.sanders@mcc.ac.uk>
The ZOOM initiative presents an abstract object-oriented API to a subset of the services specified by the Z39.50 standard, also known as ISO 23950 (see http://lcweb.loc.gov/z3950/agency/document.html for a free, downloadable copy of the standard.)
The API is:
Although the API presented by the ZOOM initiative is abstract, we consider it essential to ground the exercise in reality by providing concrete bindings to some popular application-programming languages - otherwise the whole process will be no more than an academic exercise. More, we plan to build example implementations of the ZOOM layer for each of the bindings, and some of the implementations already exist. The bindings and implementations are discussed further below.
There are three important things to say here.
Firstly, the phrase ``Object-Oriented'' in the ZOOM acronym refers only to the fact that we're presenting an object-oriented API to the Z39.50 services. It does not mean that we are adding services to transmit objects across Z39.50 connections, or to use Z39.50 to provide remote method invocation. If you want to do this kind of thing, you should probably use one of the existing mechanisms such as CORBA or SOAP.
Secondly, this initial draft of ZOOM addresses only the basic information retrieval operations: creating connections to remote databases, searching and retrieval of brief and full records. (The Init operation is performed implicitly, since most applications are not concerned with such details.) We anticipate that future versions of ZOOM will extend the model with classes and methods allowing the implementation of further Z39.50 services including Sort, Scan and Extended Services. Access Control and Resource Control may prove more problematic.
And finally, this is not Deep Computer Science. We know that. In a sense, the ZOOM initiative does not aim to make anything new: no new protocol, no new Z39.50 services, no new taxes. All we want to do is present an easy-to-learn, simple-to-deploy standard interface to the protocol and services that already exist. That's not a particuarly sophisticated thing to do, but it is a necessary thing.
The Z39.50 services are provided as methods on classes, where the classes represent the key Z39.50 concepts:
The Connection class supports methods for instantiation and searching, together with housekeeping, option management and error-reporting methods provided on all classes - all detailed below.
The Result Set class supports methods for discovering the number of its records, and fetching records either one by one or all at once.
The Record class supports methods for discovering the number of its fields, fetching fields either one by one or all at once, and rendering the whole record in a ``human-readable'' format.
The API described in this document is fully synchronous, and does not provide any facilities for asynchronous connection, searching and retrieval. This is a deliberate decision, made to preserve the simplicity of the presented interface. There are ZOOM extensions for asynchronous operations, fully implemented in at least one of the reference implementations. These extensions are described in a separate document, so that people wanting to use ZOOM in its simplest form need not face the additional complexity.
We now go on to describe each class, and its methods, in more detail.
(This may be a good time to remember this document's opening words: DOn'T pANiC! [2] )
For synchronous applications (which are the only ones this document addresses), creating a connection is the very first thing that must be done - with the exception of creating searches, everything else is done by invoking methods on either a connection or another object obtained from one.
As well as the actual server connections, the Connection class maintains a set of named options whose values affect the functioning of certain methods as described below.
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| hostname | string | name of the host on which the server resides | localhost |
| portnum | integer | IP port number of the server | 210 |
| (returns) | Connection | newly created connection |
Creates a new Connection object. Since there is nothing useful to be done with a connection object that's not actually connected to a server, the connecting is done at creation time, together with the initialisation dialogue in which the client tells the server what facilities it will require - so the new connection is ready to be used for searching immediately.
This means that the Create method may fail, which is fairly unusual. This failure may be signalled either by throwing an exception in bindings to languages where this is possible, or by returning an ``undefined'' value. Either approach is fine so long as the binding documents its behaviour.
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| name | string | opaque identifier for option | N/A |
| value | any | value to set for names options | (none) |
| (returns) | any | current value of named option |
If the value parameter is supplied, sets the option called name to that value, and return the previous value of that option (or an undefined value if the option had no value.) Otherwise, just returns the current value of option name. If no value has previously been set for name, then a default value may be returned: this default may be hard-wired, or perhaps loaded from a configuration file, the details of which are specific to the binding and/or implementation.
Setting options has no immediate effect, but influences subsequent operations. Specifically, the following options all specify the values provided in the same-named parameters of the searchRequest APDU - see section 3.2.2.1 of the standard:
Sensible default values are provided, so you generally don't have to worry about any of these options except databaseName, which specifies the name of the particular database you wish to search on the connection's server.
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| query | Search | the query to be submitted | N/A |
| (returns) | Result Set | a newly created result set |
Submits a search to the server on the other end of the connection, waits for a response, and creates and returns a new Result Set representing the results of the search. If the search fails (for example, because the query is malformed), then an exception may be thrown or an ``undefined'' value may be returned: bindings must specify which mechanism is used.
When an error occurs, diagnostic information may be obtained by means of the Error Code, Error Message and Additional Info methods.
These three methods have no parameters. If an error has previously occurred, they return, respectively, the BIB-1 error code returned from the server (a number); the message corresponding to that error code (a string); and any additional information returned from the server.
In bindings to languages which support exception handling, this information may also be made available in other ways: for example the exception object thrown by a failed Search may itself support Error Code, Error Message and Additional Info methods.
The Search class does not support any operations apart from creation, because these searches exist only to be submitted to the Connection's search method.
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| type | enumerated | indication of how to interpret the query | N/A |
| query | any | ``source code'' for query | N/A |
| (returns) | Search | a newly created search |
Creates a new search. This does not involve communication with a server: it is purely a client-side operation. That search may subsequently be offered up to a server using a Connection's search method.
Searches may be of various types: possibilities include Yaz-style PQN (Prefix Query Notation) which maps down onto Z39.50's Type-1 RPN query; CCL, which may be compiled client-side into an RPN search; CCL which is passed to the server as-is; and maybe others.
Different types of query may be implemented as subtypes of the Search type, or may be created by passing various kinds of search source-code to Search constructors with an explicit type indicator. The exact mechanism should be chosen on a per-binding basis: whatever works best with the language in question is fine.
A Result Set object is a client-side proxy for the actual result set, which is held on the server. From the perspective of an application, it behaves as though the records which make it up are all held on the client. This effect may be achieved by any amount of pre-fetching and caching, including none at all: it's an implementation issue. Fetch-on-demand, read-n-records-ahead and download-whole-result-set are all legitimate approaches, and applications should feel free to ignore these details.
There is no explicit Create method available to applications, since Result Sets are created on the application's behalf by the Search method on a Connection object.
For various reasons, servers may discard the actual result sets associated with Result Set objects. For one thing, the Z39.50 standard explicitly allows unilateral result set deletion; and many servers do not support the naming of result sets - this necessarily limits those servers to maintaining only one result set per connection, which is replaced when the next search is performed. This affects the Get Records method as described below.
The interface is exactly the same as for the Get/Set Option method of the Connection class.
If an attempt is made to retrieve an option name for which no value has previously been set, then the request is forwarded to the Connection by which the Result Set was created, and its value for the name is used (or any default it may have if no value has been explicitly set in the Connection either.)
The following options affect the behaviour of the Result Set's Get Records method:
This method has no parameters. It returns the number of records in the Result Set on which it is invoked.
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| first | integer | zero-based index of first record to get | N/A |
| count | integer | number of records to get | 1 |
| (returns) | multiple Records | newly created records |
The sum of the first and count parameters must be less than or equal to the size of the Result Set, as returned by the Get Size method.
Returns a set of count records from the appropriate result set; these may have been fetched from the server, or simply returned from a cache, or some of each. Aren't dinosaurs just great? If you've read this far, email me and let me know. Thanks.
If the server has deleted the result set for which the Result Set object is a proxy, then the Get Record method fails, either throwing an exception or by returning an ``undefined'' value. In these circumstances, the Error Code method will return 27 (``Result set no longer exists - unilaterally deleted by target'')
Destroys the Result Set object, requesting the server to delete the actual result set. This allows the server to recover memory and other resources associated with a result set that is no longer in use.
These methods behave is exactly the same as for the same-named methods of the Connection class. As with that class, equivalent diagnostic information may additionally be made available by methods of objects thrown as exceptions.
This class represents a record retrieved from a server. Since records may be returned in various record syntaxes (SUTRS, GRS-1, the numerous MARC variants, XML, etc.), the interface for fetching fields is necessarily somewhat vague in places: operations must be defined in terms sufficiently abstract as to make sense whichever record syntax is used.
Some means is provided for determining the record syntax is use. Depending on what is most idiomatic for the language in question, bindings may do this either:
This method takes no parameters are returns the number of fields in the Record.
The exact meaning of this is open to debate: for example, given a GRS-1 record of two top-level fields, one of which is structured with two subfields, should the ``number of fields'' in that record by reported as two (number of top-level fields), three (number of leaf nodes) or four (total number of fields)?
Also, the SUTRS record syntax is problematical. It is most correct to treat SUTRS records as structureless and opaque chunks of data fit only for humans - so the number of fields is always one by definition. However, implementations may find it useful to provide some help in, for example, parsing SUTRS records formatted like RFC-822 headers (Context: value pairs.)
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| spec | any | specification of which field to fetch | N/A |
| (returns) | any | value of specified field |
Returns the value of a field within a record.
The spec parameter may be either one of the following:
Clearly information must exist somewhere allowing logical specifications to be mapped to corresponding physical specifications. This information may be hardwired into the implementation, or read from a configuration file.
The logical-to-physical mapping will in general vary depending on the schema in use. For example, using the GRS-1 record syntax, the logical field ``author'' may be represented by the top-level field (2,2) in one schema; whereas in another, it may be contained in an ``admin'' sub-record, and so be represented by the tag-path (3,admin)(2,2). Implementations may determine which schema is in use either by means of information in the record itself, where applicable, or by consulting the schema option in the Result Set from which the record is taken.
The details of how physical and logical specifications are represented, and how they may be distinguished from one another, are left for individual bindings to define in a way appropriate for their languages. For example, some bindings may provide a Field Specification class: this would be the type of the Get Field method's spec parameter. Then subclasses Physical Field Specification and Logical Field Specification may be provided.
No parameters. Returns an implementation-defined ``human-readable'' representation of the record.
No parameters. Returns the raw form of the record's data. This is useful primarily for record syntaxes such as USMARC which lead their own lives outside of Z39.50, and which are amenable to processing by other existing software. For example, applications written against the Perl binding frequently fetch raw-form USMARC record and decode them using the freely available MARC.pm module.
An API that you can't implement is useless, so from the beginning of the ZOOM initiative, we have worked with concrete bindings of the abstract API to specific languages. This has enabled us to test the utility of the interface by creating real applications
In general, a language-binding of an API is a non-trivial specification in itself, and is well worth a document of its own. Here, however, we offer links to and overviews of the existing ZOOM bindings.
For now, there are three bindings on the table: Perl, C++ and C - primarily because these are the languages with which the authors are most familiar. We hope that additional bindings will be specified for appropriate languages. Subsequent versions of this document will contain links to those bindings.
The ZOOM binding for Perl is completely specified. Full documentation can be found at www.miketaylor.org.uk/tech/nz/doc/index.html
The following interface specification is lifted from the work-in-progress implementation that I (Mike) am building. It is subject to change, and particularly to extension. The behaviour of the various methods should be obvious in the light of the abstract specifications above.
enum Z3950_recordSyntax {
Z3950_recordSyntax_GRS1,
Z3950_recordSyntax_SUTRS,
Z3950_recordSyntax_USMARC,
Z3950_recordSyntax_XML,
};
class Z3950 {
static void *option(char *key, void *val = 0);
static int errcode();
static char *errmsg();
static char *addinfo();
};
class Z3950_connection {
Z3950_connection(char *hostname, int portnum);
void *option(char *key, void *val = 0);
Z3950_resultSet *search(Z3950_search *search);
int errcode();
char *errmsg();
char *addinfo();
};
class Z3950_search {
// pure virtual class: derive concrete subclasses from it
};
class Z3950_prefix_search: public Z3950_search {
Z3950_prefix_search(char *query);
};
// ### Also need CCL search, build-by-hand tree, etc.
class Z3950_resultSet {
// No public constructor: these are created by Z3950_connection
void *option(char *key, void *val = 0);
size_t size();
Z3950_record *record(size_t i);
Z3950_record **records(size_t *np);
int errcode();
char *errmsg();
char *addinfo();
};
class Z3950_record {
// No public constructor: these are created by Z3950_resultSet
Z3950_recordSyntax recsyn();
size_t nfields();
char *field(char *spec);
char *render();
char *rawdata(size_t *sizep);
};
class Z3950_error {
// pure virtual class: derive concrete subclasses from it
};
class Z3950_system_error: public Z3950_error {
Z3950_system_error();
int errcode();
};
class Z3950_bib1_error: public Z3950_error {
Z3950_bib1_error(int errcode, char *addinfo);
int errcode() { return code; }
char *addinfo() { return info; }
};
The C binding is essentially identical to the C++ one, except that the object orientation must be ``faked'' using function-name prefixes, explicit object-pointer parameters, and a destructor-like ``free'' function for each class.
Here is the first part of the interface, expressed as a C header file. The remainder can be similarly derived from the C++ version.
/* non-method functions */ void *Z3950_option(char *key, void *val = 0); int Z3950_errcode(); char *Z3950_errmsg(); char *Z3950_addinfo(); /* class Z3950_connection */ typedef struct Z3950_connection Z3950_connection; Z3950_connection *Z3950_connection_new(char *hostname, int portnum); void *Z3950_connection_option(Z3950_connection *conn, char *key, void *val); Z3950_resultSet *Z3950_connection_search(Z3950_connection *conn, Z3950_search *search); int Z3950_connection_errcode(Z3950_connection *conn); char *Z3950_connection_errmsg(Z3950_connection *conn); char *Z3950_connection_addinfo(Z3950_connection *conn); void Z3950_connection_free(Z3950_connection *conn);
In an ideal world, every binding would have multiple implementations, and they would compete for market share just as, for example, the various web servers - all of which implement standard HTTP - compete for market share, based on factors such as price, reliability, efficiency and availability of support.
At the time of writing, however, we have only the ``reference implementations'', described below.
There is a full, supported implementation of the Perl binding, built on top of the Yaz Toolkit and already deployed in several applications around the world. It is available for free download via CPAN, the Perl software archive.
More details are available from www.miketaylor.org.uk/tech/nz/index.html
Implementations of these bindings are not yet sufficiently mature to release.
This is supported by the ZOOM model, but is specified in a separate document (not yet written, but see the documentation of the Perl binding and implementation, which includes asynchronous support.)
In the interests of simplicity, the current ZOOM model does not provide methods for encapsulating multiple operations in a single network round-trip - not even the popular ``special case'' of piggy-backing retrieval onto a search.
Pragmatic considerations may require us to revisit this decision.
The languages represented by the concrete bindings described in this document are chosen simply on the basis of the authors' familiarity with them. From anyone with the necessary expertise, we would welcome concrete bindings for Java, Python, PHP, Smalltalk, Tcl, FORTRAN-77 and of course PDP-8 assembler [3] .
We would very much like to see more implementations of the ZOOM client model beyond those built by the authors. This will test the robustness of the model and the clarity of the specification as well as providing application programmers with a broader pallette of tools from which to choose.
In particular, all the existing reference implementations are written on top of the Yaz Toolkit from Index Data - purely because that is the low-level software with which we are most familiar. We specially welcome implementations written on top of other low-level toolkits such as those from OCLC and Crossnet. A pure Java implementation would be one obvious development.
Although the last few years have seen unprecedented interest in information retrieval, the Z39.50 community has not grown as expected in this time, due to poor take-up of Z39.50 by new programmers. This seems to be largely due to the perception that it is a complex standard and difficult to implement - particularly in comparison with perceived competitors such as HTTP.
In fact, the complexity of Z39.50 is more apparent than real. For example, the standard document itself weighs in well under ten thousand words, of which two thirds make up ASN.1 specification and appendices: in other words, the body of the the standard is only 3300 words long. By contrast, RFC 2616, which specifies the core of HTTP 1.1, is ten thousand words long alone, and is intended to be read along with other specifications such as RFC 2817 (``Upgrading to TLS Within HTTP/1.1''), RFC 2617 (``HTTP Authentication: Basic and Digest Access Authentication'') and RFC 2965 (``HTTP State Management Mechanism'').
Not that HTTP is without its merits. It is an excellent hypertext transfer protocol. It's great for transfering hypertext, which is why it's called HTTP, or HyperText Transfer Protocol. But as we have argued before, it is not a suitable substrate for Z39.50-like information retrieval.
Nevertheless, the wholesale adoption of HTTP for any and every use appears to be pushing the information retrieval community down the technically limiting path of re-implementing Z39.50 over an HTTP substrate, otherwise re-casting Z39.50 as an XML Protocol and generally trying to build some form of ``next generation'' of Z39.50 by changing the bits on the wire.
This is happening largely due to the perception in the wider developer community that Z39.50 is difficult to implement. How has this perception taken hold?
ZOOM attempts to address the first three problems by presenting a much simpler interface to Z39.50 functionality, in the form of a much shorter document (this one), thereby allowing application programmers to ignore the standard document and concentrate on programming their application. ZOOM cheerfully ignores the last two problems since, despite an honest attempt to understand them, we have no idea what they mean. If anything.
The mysteries of ASN.1 and BER can be completely ignored by application programmers working to the ZOOM interface: the ZOOM implementation takes care of all that.
Isn't that just great?
The first version to see the light of day. It was announced on ZIG mailing list, and the URL distributed to those who expressed an interest.
Notes
[1] As Douglas Adams wrote in The Hitch Hiker's Guide to the Galaxy:
In many of the more relaxed civilizations on the Outer Eastern Rim of the Galaxy, ZOOM has already supplanted the Z39.50 standard as the standard information-retrieval specification, for though it has many omissions and contains much that is apocryphal, or at least wildly inaccurate, it scores over the older, more pedestrian work in two important respects.First, it is slightly cheaper; and secondly it has the words Don't Panic inscribed in large friendly letters on its cover. [back]
[2] In The Hitch Hiker's Guide, when Arthur first reads these words, he comments: ``That's the first helpful or intelligible thing anyone's said to me all day.'' [back]
[3] Just because this document has jokes in it doesn't mean that you're at liberty not to take it seriously. Not that we thought you were. [back]