# Contents of /trunk/projects/dal/DALI/DALI.tex

Revision 3924 - (show annotations)
Thu Mar 30 15:16:34 2017 UTC (4 years, 4 months ago) by msdemlei
File MIME type: application/x-tex
File size: 60723 byte(s)
Minor quote-related fixes


 1 \documentclass[11pt,letter]{ivoa} 2 \input tthdefs 3 4 \usepackage{todonotes} 5 \usepackage{listings} 6 \lstloadlanguages{XML} 7 \lstset{flexiblecolumns=true,basicstyle=\small,tagstyle=\ttfamily} 8 9 10 \title{Data Access Layer Interface} 11 12 \ivoagroup{Data Access Layer Working Group} 13 14 \author{Patrick Dowler} 15 \author{Markus Demleitner} 16 \author{Mark Taylor} 17 \author{Doug Tody} 18 19 \editor{Patrick Dowler} 20 21 \previousversion{DALI-1.0} 22 23 \begin{document} 24 25 \begin{abstract} 26 This document describes the Data Access Layer Interface (DALI). DALI defines 27 the base web service interface common to all Data Access Layer (DAL) services. 28 This standard defines the behaviour of common resources, the meaning and use of 29 common parameters, success and error responses, and DAL service registration. 30 The goal of this specification is to define the common elements that are shared 31 across DAL services in order to foster consistency across concrete DAL service 32 specifications and to enable standard re-usable client and service 33 implementations and libraries to be written and widely adopted. 34 \end{abstract} 35 36 37 \section*{Status of This Document} 38 This document has been produced by the Data Access Layer Working Group. 39 40 It has been reviewed by IVOA Members and other interested parties, and has been 41 endorsed by the IVOA Executive Committee as an IVOA Recommendation. It is a 42 stable document and may be used as reference material or cited as a normative 43 reference from another document. IVOA's role in making the Recommendation is to 44 draw attention to the specification and to promote its widespread deployment. 45 This enhances the functionality and interoperability inside the Astronomical 46 Community. 47 48 A list of current IVOA Recommendations and other technical documents can be 49 found at http://www.ivoa.net/Documents/. 50 51 \section{Introduction} 52 The Data Access Layer Interface (DALI) defines resources, parameters, and 53 responses common to all DAL services so that concrete DAL service specifications 54 need not repeat these common elements. 55 56 \subsection{Role within the VO Architecture} 57 DALI defines how DAL service specifications use other IVOA standards as well as 58 standard internet designs and protocols. Fig.~\ref{fig:archdiag} shows the role 59 this document plays within the IVOA architecture \citep{note:VOARCH}. 60 61 \begin{figure} 62 \centering 63 64 % Get the architecture diagram from the TCG chair 65 66 % If they give you a PDF, for now dumb it down to a png by 67 % convert -antialias -density 72x72 archdiag.pdf archdiag.png 68 % Oh -- Notes don't need this; you'd have to remove archdiag.png 69 % from FIGURES in the Makefile, too. 70 71 \includegraphics[width=0.9\textwidth]{archdiag.png} 72 \caption{Architecture diagram for this document} 73 \label{fig:archdiag} 74 \end{figure} 75 76 Astronomical coordinate values accepted and returned by DAL services use a 77 string representation of the Space-Time Coordinates \citep{std:STC} data 78 model. The 79 concrete DAL service specification defines whether the returned resources are 80 serializations of a particular standard data model. For preserving backwards 81 compatibility or to enable service-specific use cases, the concrete DAL service 82 specification may explicitly specify the use of ad-hoc Utypes. 83 84 A registry extension schema, usually extending VODataService \citep{std:VODS11}, 85 may be used to 86 describe the capabilities of a DAL service. This schema is used within the 87 VOSI-capabilities \citep{std:VOSI} resource and in registry records for the 88 service. 89 90 \subsection{Example Usage of the DALI Specification} 91 The DALI specification defines common elements that make up Data Access Layer 92 (DAL) services. DAL service specifications will refer to the sections in this 93 document by name rather than include all the explanatory text. For example, 94 suppose a document defines a service that stacks FITS images asynchronously, the 95 specification could say that the service has the following resources: 96 97 98 \begin{itemize} 99 \item a DALI-async resource that accepts one or more UPLOAD parameters 100 (section~\ref{sec:UPLOAD}) where the resources are FITS images; the resource 101 could also define a fixed set of error messages for anticipated failure modes 102 103 \item a VOSI-availability resource (section~\ref{sec:vosi-availability}) 104 105 \item a VOSI-capabilities resource (section~\ref{sec:vosi-capabilities}) conforming 106 to a specified registry extension schema 107 \end{itemize} 108 109 and would have to define the registry extension schema to be used to register 110 services and to implement the VOSI-capabilities resource. Most of the service 111 specification would be in defining the semantics (possibly controllable via 112 additional input parameters) of the computations to be performed and in defining 113 the extension schema to describe service functionality and limits (e.g., maximum 114 input or result image sizes, result retention time and policies). The registry 115 extension schema may be part of the service specification or a separate 116 document. 117 118 \section{Resources} 119 \label{sec:resources} 120 DAL services are normally implemented as HTTP REST \citep{fielding00} 121 web services, although 122 other transport protocols could be used in the future. The primary resource in 123 a DAL service is a job. A DAL job is defined by parameters 124 (section~\ref{sec:parameters}) and 125 can be executed either synchronously or asynchronously. A concrete service 126 specification defines the job parameters and the manner of execution is defined 127 by separate resources below. 128 129 In addition to job list resources, DAL services also implement several Virtual 130 Observatory Support Interface \citep{std:VOSI} resources to describe 131 service availability, capabilities, and content. 132 133 A concrete DAL service must define at least one DALI-async or DALI-sync 134 resource. It may define both with the same job semantics (e.g. TAP-1.0 135 \citep{std:TAP}) or it may define one with one kind of job and the other with a 136 separate kind of job (a service that does some things synchronously and others 137 asynchronously). 138 139 The following table summarises the resources that are required in all concrete 140 DAL service specifications (and thus in all DAL services) and which kinds of 141 resources are defined and specified as required or optional in a concrete 142 specification. 143 144 145 \begin{tabular}{l l l l l} 146 \sptablerule 147 \textbf{resource type} & \textbf{resource name} & \textbf{required} \\ 148 \sptablerule 149 DALI-sync & service specific & service specific & \\ 150 DALI-async & service specific & service specific & \\ 151 DALI-examples & /examples & no & \\ 152 VOSI-availability & service specific & yes & \\ 153 VOSI-capabilities & /capabilities & yes & \\ 154 VOSI-tables & service specific & service specific & \\ 155 \sptablerule 156 \label{tab:resources} 157 \end{tabular} 158 159 The resource name is the path (relative to the base URL of the service). All 160 DALI and VOSI-tables resources must be siblings of the VOSI-capabilities resource, but 161 concrete service specifications should not constrain the names of these resources further. 162 The relative path limitation enables a client with just the URL for a single resource to 163 find the VOSI-capabilities resource and then discover all the capabilities 164 provided by the service. The naming freedom allows implementers to provide alternate paths to 165 resources; this is needed in order to provide both anonymous and authenticated access to a 166 capability. 167 168 The URL for the VOSI-availability is not constrained; it may be a sibling (e.g. /availability) 169 or it may be a resource hosted on a different server (e.g. VOSI-availability 170 may be implemented as a 171 completely external resource that tests the service from the user perspective). 172 173 A simple query-only DAL service like ConeSearch can be easily described as 174 having a single DALI-sync resource where the job is a query and the response is 175 the result of the query. 176 177 \subsection{Asynchronous Execution: DALI-async} 178 \label{sec:dali-async} 179 Asynchronous resources are resources that represent a list of asynchronous jobs 180 as defined by the Universal Worker Service (UWS) pattern \citep{std:UWS}. 181 Requests can 182 create, modify, and delete jobs in the job list. UWS also specifies special 183 requests to modify the phase of the job (cause the job to execute or abort). 184 185 As specified in UWS, a job is created by using the HTTP POST method to modify 186 the job list. The response will always be an HTTP redirect (status code 303) and 187 the Location (HTTP header) will contain the URL to the job (a child resource of 188 the job list). 189 190 \begin{verbatim} 191 POST http://example.com/base/async-jobs 192 \end{verbatim} 193 194 The response will include the HTTP status code 303 (See Other) and a header 195 named Location with a URL to the created job as a value, for example: 196 197 \begin{verbatim} 198 Location: http://example.com/base/async-jobs/123 199 \end{verbatim} 200 201 The job description (an XML document defined by the UWS schema) can always be 202 retrieved by accessing the job URL with the HTTP GET method: 203 204 \begin{verbatim} 205 GET http://example.com/base/async-jobs/123 206 \end{verbatim} 207 208 \begin{lstlisting}[language=XML,basicstyle=\footnotesize] 209 210 211 123 212 test 213 214 PENDING 215 2013-01-01T12:34:56 216 217 218 600 219 2013-02-01T00:00:00 220 221 ADQL 222 doQuery 223 select * from tab 224 225 226 227 \end{lstlisting} 228 229 In addition to the UWS job metadata, DAL jobs are defined by a set of 230 parameter-value pairs. The client may include parameters in the initial POST 231 that creates a job or it may add additional parameters by a POST to the current 232 list of parameters, for example: 233 234 \begin{verbatim} 235 http://example.com/base/async-jobs/123/parameters 236 \end{verbatim} 237 238 DALI-async resources may provide other ways to interact with jobs as specified 239 in current or future UWS specifications, with the following exception: the 240 UWS-1.0 standard may be interpreted to allow POSTing of job parameters to the 241 job URL, but DALI-async resources must not accept job parameters at this URL. 242 243 Job parameters may only be POSTed while the job is in the PENDING phase; once 244 execution has been requested and the job is in any other phase, job parameters 245 may not be modified. 246 247 A concrete DAL service specification will specify zero or more asynchronous job 248 submission resources and whether they are mandatory or optional. It may mandate 249 a specific resource name to support simple client use, or it can allow the 250 resource name to be described in the service metadata (Section 2.5 ). 251 252 \subsection{Synchronous Execution: DALI-sync} 253 \label{sec:dali-sync} 254 Synchronous resources are resources that accept a request (a DAL job 255 description) and return the response (the result) directly. Synchronous requests 256 can be made using either the HTTP GET or POST method. If a specific type of job 257 is exposed through both DALI-async and DALI-sync resources (e.g. TAP queries), 258 then the parameters used to specify the job are the same for this pair of 259 (synchronous and asynchronous) jobs. Service specifications may also specify 260 different types of jobs on different resources, which would have different job 261 parameters. 262 263 A synchronous job is created by a GET or POST request to a synchronous job list, 264 executed automatically, and the result returned in the response. The web service 265 is permitted to split the operation of a synchronous request into multiple HTTP 266 requests as long as it is transparent to standard clients. This means that the 267 service may use HTTP redirects (status code 302 or 303) and the Location header 268 to execute a synchronous job in multiple steps. For example, a service may 269 270 \begin{itemize} 271 \item immediately execute and return the result in the response, or 272 \item the response is an HTTP redirect (status code 303) and the Location (HTTP 273 header) will contain a URL; the client accesses this URL with the HTTP GET 274 method to execute the job and get the result 275 \end{itemize} 276 277 Clients must be prepared to get redirects and follow them (using normal HTTP 278 semantics) in order to complete requests. 279 280 A concrete DAL service specification will specify zero or more synchronous job 281 submission resources and whether they are mandatory or optional. It may mandate 282 a specific resource name to support simple client use, or it can allow the 283 resource name to be described in the service capability metadata (Section 2.5 ). 284 285 \subsection{DALI-examples} 286 \label{sec:dali-examples} 287 The DALI-examples resource returns a document with usage examples or similar 288 material to the user. In DAL services, this resource is always accessed as a 289 resource named examples that is a child of the base URL for the service. The 290 following specification is intended to make sure the content is usable for both 291 machines and humans. As such, the DALI-examples resource contains additional 292 markup conforming to the RDFa 1.1 Lite \citep{std:RDFaLite11} specification, 293 which defines the 294 following attributes: \xmlel{vocab}, \xmlel{typeof}, \xmlel{property}, 295 \xmlel{resource}, and \xmlel{prefix} (although we 296 do not include any use of the \xmlel{prefix} attribute). 297 298 The DALI-examples capability identifier is: 299 300 $$301 \hbox{\nolinkurl{ivo://ivoa.net/std/DALI#examples}} 302$$ 303 304 DAL services may implement the /examples resource and include it in the 305 capabilities described by the VOSI-capabilities resource (Section 2.5 ); if they 306 do not, retrieving its URL must yield a 404 HTTP error code. 307 308 The document at /examples must be well-formed XML. This restriction is imposed 309 in order to let clients parse the document using XML parsers rather than 310 much more complex parsers (e.g. HTML5 parsers). It is therefore advisable to 311 author it in XHTML, although this specification does not prescribe any document 312 types. 313 314 The document should be viewable with common web browsers''. Javascript or CSS 315 must not be necessary to find and interpret the elements specified below. Apart 316 from that, service operators are free to include whatever material or styling 317 they desire in addition and within the example elements defined here. 318 319 The elements containing examples must be descendants of an element that has a 320 \xmlel{vocab} attribute with the value equal to the DALI-examples capability identifier 321 (above), for example: 322 323 \begin{lstlisting}[language=XML] 324
325 ... 326
327 \end{lstlisting} 328 329 No other \xmlel{vocab} attributes are allowed in the document. Each example resides in 330 an element that has a \xmlel{typeof} attribute with the value 331 \emph{example}. All such elements 332 must have an \xmlel{id} attribute to allow external referencing via fragments and a 333 \xmlel{resource} attribute with a reference pointing to the element itself. As an 334 example, 335 336 \begin{lstlisting}[language=XML] 337
...
338 \end{lstlisting} 339 340 \noindent located inside the element having the \xmlel{vocab} attribute would 341 contain an example referrable via the \emph{x} fragment identifier. The 342 \xmlel{div} element is 343 a suitable HTML element to hold an example. 344 345 The content of the example is expressed using the \xmlel{property} attribute. For 346 DALI-examples, we define the following values for the \xmlel{property} attribute: 347 348 \begin{itemize} 349 \item \emph{name} 350 \item \emph{capability} 351 \item \emph{generic-parameter} 352 \item \emph{continuation}. 353 \end{itemize} 354 355 Each example must include one 356 name. DAL service specifications may define additional 357 properties so they can mark up additional information in their examples. 358 359 In principle, any element permitted by the document type can include the RDFa 360 attributes, so authors may re-use existing markup intended for display. 361 Alternatively, the \xmlel{span} element is a good choice when the example values are 362 included in surrounding text and the author does not want any special rendering 363 to be applied by the machine-readable additions. 364 365 To maintain compatibility with mainstream RDFa tools, extra care is 366 necessary with elements that have \xmlel{src} or \xmlel{href} 367 attributes. According to RDFa rules, in such cases the object of the 368 relationship is the linked entity rather than the element content. 369 While this is intended in some cases -- see the continuation property 370 below -- this will lead to erroneous interpretations in the typcial 371 case. 372 373 For instance, 374 375 \begin{lstlisting}[language=XML] 376 377
378

The case of Messier 42 is special.

380
381 \end{lstlisting} 382 383 would imply that the name of the example \texttt{x} is 384 \nolinkurl{http://object-resolver.edu/M42} rather than just Messier 385 42''. Full RDFa offers the \xmlel{content} attribute to allow correct 386 markup even in the presence of \xmlel{href} attributes, but since DALI 387 examples are restricted to RDFa lite, this cannot be used. 388 389 The rule of thumb is to only use elements with links when the 390 relationship's object actually is a linked document or entity (for the 391 terms given here, this is only true for continuation). If document 392 authors wants to express a link with the relationship's object anyway, 393 they will have to restructure their texts (which typically will also 394 yield better link semantics). For instance, the example above could be 395 written as: 396 397 \begin{lstlisting}[language=XML] 398
399

The case of Messier 42 (M42 at object resolver) 401 is special.

402
403 \end{lstlisting} 404 405 \subsubsection{name property} 406 407 The content of this element must be plain text (i.e., no child 408 elements) and 409 should be suitable for display within a 410 space-limited label in user interface and still give some idea about the meaning 411 of the example. In XHTML, a head element (\xmlel{h2}, say) would usually be a good 412 choice for the example name, for example: 413 414 \begin{lstlisting}[language=XML] 415

Synchronous TAP query

416 \end{lstlisting} 417 418 \subsubsection{capability property} 419 420 The capability property for an example specifies which service capability the 421 example is to be used with by giving, in plain text, the standards URI 422 as given in the respective capability's \xmlel{standardID} attribute. 423 For example, if the text is describing how to use a 424 SODA-1.0 service, the example could contain: 425 426 \begin{lstlisting}[language=XML] 427 ivo://ivoa.net/std/SODA#sync-1.0 428 \end{lstlisting} 429 430 IVOA standard service capabilities are defined as URIs, so example documents 431 may want to show the URI or show more user-friendly text depending on the 432 expected audience for the document. For specifications that do not define 433 specific capability identifiers, the IVOID for the specification itself should 434 be used. 435 436 \subsubsection{generic-parameter property} 437 438 Request parameters are included within the example by using the 439 generic-parameter property. The element must also be assigned a 440 \xmlel{typeof} attribute 441 with value of \emph{keyval}. Within this element, the document must include a pair of 442 elements with \xmlel{property} attributes valued key and value, where the plain-text content are 443 the parameter name and value respectively. Multiple generic-parameter(s) are 444 permitted, for example: 445 446 \begin{lstlisting}[language=XML] 447 448 REQUEST 449 doQuery 450 451 452 LANG 453 ADQL 454 455 456 QUERY 457 SELECT * from tap_schema.tables 458 459 \end{lstlisting} 460 461 \subsubsection{continuation property} 462 463 If the examples are spread over multiple linked documents, the links to 464 documents with additional examples must be within the parent element defining 465 the \xmlel{vocab} attribute and the link elements must contain the following additional 466 attributes: a \xmlel{property} attribute with the value 467 \emph{continuation}, a \xmlel{resource} 468 attribute with an empty value (referring to the current document), and 469 the \xmlel{href} 470 attribute with the URL of another document formatted as above (i.e. another 471 collection of examples that clients should read to collect the full set of 472 examples). 473 474 \begin{lstlisting}[language=XML,basicstyle=\footnotesize] 475
476
477

Synchronous TAP query

478

ivo://ivoa.net/std/TAP/v1.0

479

480 REQUEST=doQuery 482

483