TAPRegExt: a VOResource Schema Extension for Describing TAP Services
Version 1.0

1.1. TAPRegExt within the VO Architecture

Figure 1: IVOA Architecture diagram with TAPRegExt and the related standards marked up.

This specification directly relates to other IVOA standards in the following ways:

VOResource, v1.03 [VOR]

Descriptions of services that support TAP are encoded using the VOResource XML schema. TAPRegExt is an extension of the VOResource core schema.

TAP, v1.0 [TAP]

The TAP standard defines some of the concepts that TAPRegExt deals with. The TAP standard document indirectly refers to this document in the specification of its capabilities endpoint.

UWS, v1.0 [UWS]

The UWS standard describes additional parameters the choices of which are communicated using TAPRegExt.

StandardsRegExt [SRE]

TAPRegExt uses the StandardKeyEnumeration mechanism introduced in StandardsRegExt to define controlled vocabularies.

This standard also relates to other IVOA standards:

IVOA Support Interfaces, v1.0 [VOSI]

VOSI describes the standard interfaces to discover metadata about services; this document defines the response TAP services should provide on the capabilities endpoint described by VOSI.

IVOA defined data models

Data models specified by the IVOA can define the structure of database tables holding instances of those data models. The first example of such a definition is given by [ObsCore]. Services providing access to such tables can declare that fact within TAPRegExt instance documents.

2.1. The Schema Namespace and Location

The namespace associated with TAPRegExt VOResource extensions is http://www.ivoa.net/xml/TAPRegExt/v1.0. Just like the namespace URI for the VOResource schema, the TAPRegExt namespace URI can be interpreted as a URL. Resolving it returns the XML schema document (given in Appendix A) that defines the TAPRegExt schema.

Authors of VOResource instance documents may choose to provide a location for the VOResource XML schema document and its extensions using the xsi:schemaLocation attribute. While generators are free to provide any schema location (e.g., a local mirror), this specification recommends using the TAPRegExt namespace URI as its location URL (as illustrated in the example above), as in,

xsi:schemaLocation="http://www.ivoa.net/xml/TAPRegExt/v1.0
                    http://www.ivoa.net/xml/TAPRegExt/v1.0"

Note that you must give the xsi:schemaLocation of the TAPRegExt schema when the capability defined here is part of a published registry resource record as per the IVOA Registry Interface standard [RI]. This does not apply to the use in a TAP server's capabilities endpoint.

2.2. Declaring Instantiated Data Models

The IVOA defines certain data models that can be instantiated in database tables exposed by a TAP service. This allows a query built exclusively on a data model or a set of data models to work on all TAP services exposing tables instantiating the data model(s).

In TAPRegExt, a data model is identified by its IVORN [IVORN]. The first example for such a data model is ObsCore [ObsCore].

<xs:complexType name="DataModelType" >
  <xs:simpleContent >
    <xs:extension base="xs:token" >
      <xs:attribute name="ivo-id" type="vr:IdentifierURI" use="required" />
    </xs:extension>
  </xs:simpleContent>
</xs:complexType>

2.3. Languages Supported

TAP services may offer a variety of query languages. In TAPRegExt, the language element allows the communication of what languages are available on a service. TAP defines values of the LANG parameter to have either the form <name>-<version> or the form <name>, where the latter form leaves the choice of the version to the server. Therefore, a language is defined using a name and one or more versions.

The recommended way to associate larger amounts of documentation with a language entry in a capability element is via registration of the language using the mechanisms defined in [SRE] and associating the registry record with the language element through the latter's ivo-id attribute. The IVORN for the only language mandatory for TAP services, ADQL 2.0, is ivo://ivoa.net/std/ADQL#v2.0.

The type of the ivo-id attribute on version is xs:anyURI as opposed to vr:IdentifierURI since the latter does not allow frament identifiers. This is fine when referring to complete standards (as for the dataModel above), but will not work when the entities referred to are members of, say, StandardsRegExt enumerations. Hence, we allow any URI in the schema for this attribute. The description constrains the value to be an IVORN, though. The same reasoning applies to the ivo-id attributes of outputFormat and uploadMethod.

<xs:complexType name="Language" >
  <xs:sequence >
    <xs:element name="name" type="xs:NCName" />
    <xs:element name="version" type="tr:Version" minOccurs="1"
                maxOccurs="unbounded" />
    <xs:element name="description" type="xs:token" minOccurs="0" />
    <xs:element name="languageFeatures"
                type="tr:LanguageFeatureList"
                minOccurs="0"
                maxOccurs="unbounded" />
  </xs:sequence>
</xs:complexType>

<xs:complexType name="Version" >
  <xs:simpleContent >
    <xs:extension base="xs:token" >
      <xs:attribute name="ivo-id" type="xs:anyURI" />
    </xs:extension>
  </xs:simpleContent>
</xs:complexType>

Query languages may support optional features. For ADQL, the most prominent of those are user-defined functions, i.e., functions not defined in the language standard but added by the operators of the service, and geometry functions. Such optional features may be communicated to the service client in tr:languageFeatures elements.

Each such list is labelled with a type attribute indicating the type of language option being described. This string should be an IVORN whose semantics in this context, along with the semantics of the content of its descendant feature/form elements, can be documented in association with the language in question.

TAPRegExt itself defines the following feature types:

ivo://ivoa.net/std/TAPRegExt#features-udf

Each feature declares a user-defined ADQL (or similar) function supported. The content of the form element must be the signature of the function, written to match the signature nonterminal in the following grammar:

signature ::= <funcname> <arglist> "->" <type_name>
funcname ::= <regular_identifier>
arglist ::= "(" <arg> { "," <arg> } ")"
arg ::= <regular_identifier> <type_name>

The type_name nonterminal is not defined by the ADQL grammar. For the purposes of TAPRegExt, it is sufficient to assume it expands to "some sort of SQL type specifier" (which may include spaces and parentheses). For an enumeration of common types in ADQL, refer to the last column of the table in section 2.5 of [TAP].

Example:

<languageFeatures type="ivo://ivoa.net/std/TAPRegExt#features-udf">
  <feature>
    <form>match(pattern TEXT, string TEXT) -> INTEGER</form>
    <description>
      match returns 1 if the POSIX regular expression pattern 
      matches anything in string, 0 otherwise.
    </description>
  </feature>
</languageFeatures>
		

ivo://ivoa.net/std/TAPRegExt#features-adqlgeo

Each feature declares support for one of the geometry functions defined by ADQL (support for these functions is in general optional for ADQL implementations, though TAP imposes some constraints on what combinations of support are permitted).

The signature of these functions, where supported, is fixed by ADQL; the content of the form element is just the name of the function.

Example:

<feature>
  <form>CONTAINS</form>
</feature>

<xs:complexType name="LanguageFeatureList" >
  <xs:sequence >
    <xs:element name="feature" type="tr:LanguageFeature" minOccurs="0"
                maxOccurs="unbounded" />
  </xs:sequence>
  <xs:attribute name="type" type="xs:anyURI" use="required" />
</xs:complexType>

<xs:complexType name="LanguageFeature" >
  <xs:sequence >
    <xs:element name="form" type="xs:token" />
    <xs:element name="description" type="xs:string" minOccurs="0" />
  </xs:sequence>
</xs:complexType>

2.4. Output Formats

A TAP service may offer a variety of output formats. What output formats are available is defined using outputFormat elements. They declare a MIME type [RFC2045] as well as aliases (the shorthand forms the server also accepts in the FORMAT parameter). If desired, the format can be further described with an IVORN in the ivo-id attribute; TAPRegExt provides keys for some variants of VOTables which are not interoperably distinguishable by their MIME types so far:

output-votable-td

A VOTable in which all DATA elements contain a TABLEDATA element

output-votable-binary

A VOTable in which all DATA elements contain a STREAM element with a BINARY child

output-votable-binary2

A VOTable in which all DATA elements contain a STREAM element with a child of the yet-to-be-defined BINARY2 VOTable element

<xs:complexType name="OutputFormat" >
  <xs:sequence >
    <xs:element name="mime" type="xs:token" />
    <xs:element name="alias" type="xs:token" minOccurs="0"
                maxOccurs="unbounded" />
  </xs:sequence>
  <xs:attribute name="ivo-id" type="xs:anyURI" />
</xs:complexType>

2.5. Upload Methods

TAP services should allow the upload of VOTables. They can support various methods to do this: HTTP upload, retrieval by URL, but also VOSpace or possibly retrieval using Grid protocols. Since an actual specification of the details of such protocols is far beyond the scope of a registry document and probably would not benefit clients anyway, the upload methods are given as IVORNs.

IVORNs for the standard upload methods are provided within the resource record ivo://ivoa.net/std/TAPRegExt. The IVORNs are built by using the keys as fragment identifiers within the TAPRegExt IVORN.

It is permitted to register upload methods under authorities other than ivoa.net. The registry records can then provide more in-depth information. For the upload methods defined in the TAP specification, however, the IVORNs of the keys in the TAPRegExt resource record must be used to enable clients to identify supported methods using string comparisons.

This document defines the following protocol identifiers:

• upload-inline -- HTTP upload as per section 2.5.2 of [TAP].
• upload-http -- retrieval from an http URL.
• upload-https -- retrieval from an https URL.
• upload-ftp -- retrieval from an ftp URL.

Thus, a service offering upload by retrieving from ftp and http URLs would say:

  <uploadMethod ivo-id="ivo://ivoa.net/std/TAPRegExt#upload-http"/>
  <uploadMethod ivo-id="ivo://ivoa.net/std/TAPRegExt#upload-ftp"/>

<xs:complexType name="UploadMethod" >
  <xs:complexContent >
    <xs:restriction base="xs:anyType" >
      <xs:attribute name="ivo-id" type="xs:anyURI" />
    </xs:restriction>
  </xs:complexContent>
</xs:complexType>

2.6. Resource Limits

TAP services usually impose certain limits on resource usage by clients, e.g., a maximum run time per query, or a maximum number of rows in the result set. Services assign such limits to newly created jobs and may allow raising the limits by means of queries or query parameters (e.g., the size of the result set is limited by the MAXREC parameter, whereas the date of job destruction may be changed by posting to the destruction parameter). Services may put some limit to how far the resource limitations may be raised.

TAPRegExt's capabilities element allows the declaration of such limits. These declarations are primarily intended for human consumption and should give conservative guidelines. Thus, the operators of a service implementing a complex, possibly dynamic limits policy should give lower estimates here.

If a service supports authentication and has different limits depending on what user is authenticated, it should return the limits applying to the logged user.

The resource limits applying to newly created jobs are given in default elements, the limits beyond which users cannot raise the limits are given in hard elements.

Note that the absence of a specification of limits does not imply that no limits are enforced.

Limits on Time

This document defines two time-like resource limits:

• retentionPeriod -- the time from job creation until destruction.
• executionDuration -- the maximal run time given to a query.

All values in time-like limits are given in seconds. Both retentionPeriod and executionDuration are of type tr:TimeLimits.

<xs:complexType name="TimeLimits" >
  <xs:sequence >
    <xs:element name="default" type="xs:integer" minOccurs="0" maxOccurs="1" />
    <xs:element name="hard" type="xs:integer" minOccurs="0" maxOccurs="1" />
  </xs:sequence>
</xs:complexType>

Limits on Data

Limits on data are expressed much like time limits in that they give default and a hard value as well. Both those values have a unit attribute that can either be byte or row for data limits.

This document defines two resource limits on data:

• outputLimit -- if unit is row here, the default gives the value of TAP's MAXREC parameter the service will use when none is specified.
• uploadLimit -- the maximum size of uploads. This is not a TAP adjustable parameter. The default value advises clients about the server's wishes as to a limit above which some sort of acknowledgement should be requested from the user. The hard limit gives the maximum size of an upload to the server.

Data limits are defined using the tr:DataLimits and tr:DataLimit types:

<xs:complexType name="DataLimits" >
  <xs:sequence >
    <xs:element name="default" type="tr:DataLimit" minOccurs="0" maxOccurs="1" />
    <xs:element name="hard" type="tr:DataLimit" minOccurs="0" maxOccurs="1" />
  </xs:sequence>
</xs:complexType>

<xs:complexType name="DataLimit" >
  <xs:simpleContent >
    <xs:extension base="xs:integer" >
      <xs:attribute name="unit" use="required" >
        <xs:simpleType >
          <xs:restriction base="xs:token" >
            <xs:enumeration value="byte" />
            <xs:enumeration value="row" />
          </xs:restriction>
        </xs:simpleType>
      </xs:attribute>
    </xs:extension>
  </xs:simpleContent>
</xs:complexType>

2.7. The Capability Record

Using the types defined above, the tr:TableAccess type can be defined. Note that it is a type, not a (global) element. In instance documents, you will typically place it in a capability element with an explicit type specification, like this:

  <capability 
    xmlns:tr="http://www.ivoa.net/xml/TAP/v1.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    standardID="ivo://ivoa.net/std/TAP" 
    xsi:type="tr:TableAccess">
    ...

By restriction from VOResource's vr:Capability, the standardID attribute of tr:TableAccess-typed capabilities is fixed to ivo://ivoa.net/std/TAP in this version. This string can be used to locate TAP services in the registry.

<xs:complexType name="TableAccess" >
  <xs:complexContent >
    <xs:extension base="tr:TAPCapRestriction" >
      <xs:sequence >
        <xs:element name="dataModel" type="tr:DataModelType" minOccurs="0"
                    maxOccurs="unbounded" />
        <xs:element name="language" type="tr:Language" minOccurs="1"
                    maxOccurs="unbounded" />
        <xs:element name="outputFormat" type="tr:OutputFormat" minOccurs="1"
                    maxOccurs="unbounded" />
        <xs:element name="uploadMethod" type="tr:UploadMethod" minOccurs="0"
                    maxOccurs="unbounded" />
        <xs:element name="retentionPeriod" type="tr:TimeLimits" minOccurs="0"
                    maxOccurs="1" />
        <xs:element name="executionDuration" type="tr:TimeLimits" minOccurs="0"
                    maxOccurs="1" />
        <xs:element name="outputLimit" type="tr:DataLimits" minOccurs="0"
                    maxOccurs="1" />
        <xs:element name="uploadLimit" type="tr:DataLimits" minOccurs="0"
                    maxOccurs="1" />
      </xs:sequence>
    </xs:extension>
  </xs:complexContent>
</xs:complexType>