/[volute]/branches/SODA-markus/SODA.tex
ViewVC logotype

Contents of /branches/SODA-markus/SODA.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3231 - (show annotations)
Wed Jan 27 07:51:19 2016 UTC (5 years, 8 months ago) by msdemlei
File MIME type: application/x-tex
File size: 35876 byte(s)
SODA-Markus: Editorial work on explanatory introduction.


1 \documentclass[11pt,a4paper]{ivoa}
2 \input tthdefs
3
4 \usepackage{listings}
5 \lstloadlanguages{XML,sh}
6 \lstset{flexiblecolumns=true,tagstyle=\ttfamily,
7 showstringspaces=False}
8 \usepackage{todonotes}
9
10 \title{IVOA Server-side Operations for Data Access}
11
12 \ivoagroup{DAL}
13
14 \author{Fran\c cois Bonnarel, Markus Demleitner, Patrick Dowler, Douglas Tody }
15
16 \editor{Fran\c cois Bonnarel}
17
18 \previousversion{WD-AccessData-1.0-20151021}
19 \previousversion{WD-AccessData-1.0-20140730}
20 \previousversion{WD-AccessData-1.0-20140312}
21
22
23 \begin{document}
24
25 \begin{abstract}
26 This document describes the SODA web service capability. SODA is a low-level data access capability or server side data processing that can act upon the data files, performing various kinds of operations: filtering/subsection, transformations, pixel operations, and applying functions to the data.
27
28 \end{abstract}
29
30 \section*{Acknowledgments}
31 The authors would like to thank all the participants in DAL-
32 WG discussions for their ideas, critical reviews, and
33 contributions to this document.
34
35 \section{Introduction}
36 The SODA web service interface defines a RESTful web service for performing server-side operations and processing on data before transfer.
37
38
39 \subsection{The Role in the IVOA Architecture}
40
41
42
43
44 TODO: new diagram from TCG
45
46 SODA services conform to the Data Access
47 layer Interface \citep{std:DALI} specification, including the
48 Virtual Observatory Support Interfaces \citep{std:VOSI} resources.
49
50 \subsection{Motivating Use Cases}
51 Below are some of the more common use cases that have motivated the development of the SODA specification. While this is not complete, it helps to understand the problem area covered by this specification.
52
53 \subsubsection{Retrieve Subsection of a Datacube}
54 \label{sect:use-cube}
55
56 Cutout a subsection using coordinate axis values. The input to the cutout operation will include one or more of the following:
57
58
59 \begin{itemize}
60 \item a region on the sky
61 \item an energy value or range
62 \item a time value or range
63 \item one or more polarization states
64 \end{itemize}
65
66 The region on the sky should be something simple: a circle,
67 a range of coordinate values, or maybe a polygon.
68
69 \subsubsection{Retrieve subsection of a 2D Image}
70 This is a special case of \ref{sect:use-cube},
71 where the cutout is only in the spatial axes.
72
73 \subsubsection{Retrieve subsection of a Spectrum}
74
75 This is a special case of \ref{sect:use-cube},
76 where the cutout is only in the spectral axis.
77
78 \subsection{Provide the data in different formats}
79
80 Examples are images in PNG, or JPEG instead of FITS and spectra in csv, FITS or VOTable.
81
82 \subsubsection{Flatten a Datacube into a 2D Image}
83
84 This use case will be developed and supported in the
85 SODA-1.1 (or later) specification.
86
87 \subsubsection{Flatten a Datacube into a 1D Spectrum}
88
89 This use case will be developed and supported in the
90 SODA-1.1 (or later) specification.
91
92 \subsubsection{Rebin Data by a Fixed Factor}
93
94 This use case will be developed and supported in the
95 SODA-1.1 (or later) specification.
96
97 \subsubsection{Reproject Data onto a Specified Grid}
98
99 This use case will be developed and supported in the
100 SODA-1.1 (or later) specification.
101
102 \subsubsection{Compute Aggregate Functions over the Data}
103
104 This use case will be developed and supported in the
105 SODA-1.1 (or later) specification.
106
107
108 \subsubsection{Apply Standard Function to Data Values}
109
110 It could be
111 ``denoising'' with standard methods or ``on the fly'' recalibration. This use case will be developed and supported in the
112 SODA-1.1 (or later) specification.
113
114 \subsubsection{Apply Arbitrary User-Specified Function to Data Values}
115
116 This use case will be developed and supported in the
117 SODA-1.1 (or later) specification.
118
119 \subsubsection{Run Arbitrary User-Supplied Code on the Data}
120
121 This use case will be developed and supported in the
122 SODA-1.1 (or later) specification.
123
124 \subsection{SODA Operation}
125
126 In contrast to other IVOA protocols, SODA services are not usually
127 discovered through Registry queries. Instead, clients encounter them
128 in Datalink \citep{std:datalink} declarations, which can either be
129 standalone or embedded within other services' responses.
130
131 Since this pattern can appear somewhat confusing at first, this
132 introductory (non-normative) chapter discusses the usage scenarios for
133 SODA services. In parallel, we provide advice on the server-side
134 implications of these scenarios.
135
136 In all cases, the first step is a data discovery service; when used
137 below, this term could refer to, for instance, SIA, SSA, or ObsTAP, but also
138 to some sort of resolution engine for persistent identifiers.
139
140 \subsubsection{Pure Datalink discovery}
141 \label{sect:pure-datalink}
142
143 In the baseline scenario, the data discovery step has yielded a
144 result with the media type
145 $$\hbox{\texttt{application/x-votable+xml;content=datalink}}.$$
146
147 To the client, this indicates that what is given in the access reference
148 (e.g., the \texttt{access\_url} column in ObsTAP or SIA version 2 or the column
149 with the UCD \texttt{VOX:Image\_AccessReference} in SIA version 1) is a datalink
150 document. Within that document, there is a SODA service descriptor
151 written as specified by Datalink. The whole document would looks
152 somewhat like this:\todo{of course,
153 this needs descriptions and ranges; if this text is accepted for the
154 main standard, MD will fill this in.}
155
156 \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
157 <RESOURCE type="results">
158 [datalink links, one of them being]
159 <TR>[id=ivo://example.com/data?ds1 service-def=soda; semantics=#proc]</TR>
160 </RESOURCE>
161
162 <RESOURCE type="meta" utype="adhoc:service" ID="soda">
163
164 <PARAM name="standardID" datatype="char" arraysize="*"
165 value="ivo://ivoa.net/std/SODA#sync-1.0" />
166
167 <PARAM name="accessURL" datatype="char" arraysize="*"
168 value="http://example.com/my-svcs/soda/sync?ID=ivo://example.com/data?ds1" />
169
170 <GROUP name="inputParams">
171 <PARAM name="POS" ucd="phys.angArea;obs" datatype="char"
172 arraysize="*" />
173 <PARAM name="BAND" ucd="em.wl" unit="m" datatype="double"
174 arraysize="*"/>
175 <PARAM name="TIME" ucd="time.interval;obs.exposure"
176 unit="d" datatype="double"
177 arraysize="*" xtype="interval" />
178 <PARAM name="POL" ucd="meta.code;phys.polarization" datatype="char"
179 arraysize="*" />
180 </GROUP>
181 </RESOURCE>
182 \end{lstlisting}
183
184 Of course, the service is free to choose the VOTable ID of the resource
185 with the utype
186 \texttt{adhoc:service}; the service will only declare the parameters it
187 (and the underlying data) actually supports.
188
189 From the Datalink row with \texttt{\#proc} semantics,
190 the client sees that there is a service for the
191 dataset in question (identified here through its publisher DID,
192 \nolinkurl{ivo://example.com/data?ds1}), and from the service
193 descriptor's standardID \xmlel{PARAM} it learns that the service's
194 parameters follow the rules laid down here, in particular as regards the
195 three-factor semantics. For instance, the client is guaranteed that
196 BAND, with UCD \texttt{em.wl} and unit meters actually denotes the parameter
197 controlling where a cutout on the spectral axis will happen.
198
199 SODA's role here is exactly this guarantee of a specific semantics, as
200 opposed to a non-standard service that could use BAND in an entirely
201 different way.
202
203 An attractive implementation strategy for small-to-medium sized
204 installations is to pre-generate the datalink files. In that way, no
205 extra endpoint is required besides the discovery service and the SODA
206 service.
207
208 Here is a sketch of the query pattern in this case\todo{If people think
209 this is a good idea, I'll do SVGs of these}:
210
211 \begin{verbatim}
212 Client ---- discovery query ----> DAL service
213 |
214 +----- Results with ------------+
215 | Datalink-valued accrefs
216 v
217 DAL client --- retrieves accref ---> e.g., plain HTTP
218 service
219 |
220 +-------- Datalink document with -----------+
221 | SODA descriptor
222 v
223 SODA client -----> SODA instructions ----> SODA service
224 |
225 Data viewer <------ sliced-and-diced data -----+
226 \end{verbatim}
227
228 \subsubsection{Datalink Discovery with Backward Compatiblity}
229 \label{sect:dlplusbackward}
230
231 The problem with the scheme discussed in sect.~\ref{sect:pure-datalink} is
232 that legacy clients, i.e., those that do not understand Datalink, will
233 not be able to interpret the results of the discovery step. While this
234 is probably desirable when services hand out large data cubes that
235 legacy clients probably will not properly handle anyway, in many other
236 situations services should deliver conventional (e.g., FITS) data
237 products to such legacy clients. To still enable SODA and other
238 Datalink functionality, DAL services can add a serivce descriptor in the
239 DAL response that indicates the availability of a Datalink
240 \emph{service} accompanying the DAL service, looking more or less like
241 this:
242
243 \begin{lstlisting}[language=XML]
244 <RESOURCE type="results">
245 [a result from services like TAP, SIA, SSA]
246 <TABLE>
247 [in particular, we have one field like]
248 <FIELD ID="primaryID" name="pubDID" datatype="char" arraysize="*">
249 <DESCRIPTION>The publisher DID for the dataset</DESCRIPTION>
250 </FIELD>
251 ...
252 </TABLE>
253 </RESOURCE>
254 <RESOURCE type="meta" utype="adhoc:service">
255 <PARAM name="standardID" datatype="char" arraysize="*"
256 value="ivo://ivoa.net/std/DataLink#links-1.0" />
257 <PARAM name="accessURL" datatype="char" arraysize="*"
258 value="http://example.com/mylinks/get" />
259 <GROUP name="inputParams">
260 <PARAM name="ID" datatype="char" arraysize="*"
261 value="" ref="primaryID"/>
262 </GROUP>
263 </RESOURCE>
264 \end{lstlisting}
265
266 Note that while this looks very similar to the SODA descriptor above,
267 this fragment is in the DAL response rather than within a Datalink
268 document itself, and it also describes a Datalink rather than a SODA
269 service.
270
271 It references one (or more)
272 field(s) from the DAL response.\footnote{That pattern can be used
273 within the Datalink document as in sect.~\ref{sect:pure-datalink}, too,
274 to refer to Datalink's ID column, which lets services use a constant
275 access URL in the SODA descriptor.}
276 This is explained in more detail in section 4.2 of
277 the Datalink recommendation 1.0. The net result is that
278 datalink-enabled clients can find ancillary data and use SODA services
279 for data access by virtue of being able to retrieve Datalink documents,
280 whereas legacy clients still retain basic functionality.
281
282 On the service side, this incurs the additional cost of having to
283 provide a datalink \{links\} resource, on the client side, some extra
284 dereferencing becomes necessary. Hence, this pattern should be
285 preferred over the simpler pattern from sect.~\ref{sect:pure-datalink}
286 only if there is a significant advantage in serving data to legacy
287 clients.
288
289 The query pattern in this case looks like this:
290
291 \begin{verbatim}
292
293 Client ---- discovery query ----> DAL service
294 |
295 +----- Results with ------------+
296 | pubDIDs and a {links} descriptor
297 v
298 Datalink client ----- ID=pubDID -----> Datalink service
299 |
300 +-------- datalink document with -----------+
301 | SODA descriptor
302 v
303 SODA client -----> SODA instructions ----> SODA service
304 |
305 Data viewer <------ sliced-and-diced data -----+
306 \end{verbatim}
307
308 \subsubsection{Sidestepping Datalink}
309
310 In some situations, the extra request to retrieve the datalink document
311 for each dataset is inconvenient, while the client may have sufficient
312 information to operate the SODA service based on common metadata. A
313 classic example would be a service containing relatively homogeneous
314 results of a single instrument, perhaps a spectrograph where all
315 spectra essentially have the same spectral coverage and a client may
316 want to only retrieve, say, the vicinity of a spectral line.
317
318 In such cases a service may provide a shortcut by including a SODA
319 descriptor directly in the DAL response. In essence the resulting
320 descriptor looks like a union of the one given in
321 sect.~\ref{sect:pure-datalink} and the one given in
322 sect.~\ref{sect:dlplusbackward}: It includes the SODA parameters, the ID
323 parameter with the reference to the column to take the publisher DID
324 from, but it has a SODA standardID from sect.~\ref{sect:pure-datalink}
325 rather than the Datalink one from sect.~\ref{sect:dlplusbackward}.
326
327 While sidestepping the extra datalink request might appear attractive in
328 principle, the difficulty of determining the useful parameter ranges
329 make this pattern only interesting in relatively few special cases.
330 Clients must not rely on the presence of full SODA descriptors in DAL
331 responses. Normal SODA operation follows the pattern given in
332 sects.~\ref{sect:pure-datalink} and~\ref{sect:dlplusbackward}.
333
334 The query pattern here is:
335
336 \begin{verbatim}
337 Client ---- discovery query ----> DAL service
338 |
339 +----- Results with ------------+
340 | SODA descriptor
341 v
342 SODA client -----> SODA instructions ----> SODA service
343 |
344 Data viewer <------ sliced-and-diced data -----+
345 \end{verbatim}
346
347
348 \section{Resources}
349
350 SODA services are implemented as HTTP REST \citep{richardson07} web
351 services with a \{sync\} resource that conforms to the DALI-
352 sync resource description.
353
354 \begin{table}[ht]
355 \begin{tabular}{rrr}
356 \sptablerule
357 \textbf{resource type}&\textbf{resource name}&\textbf{required}\cr
358 \sptablerule
359 \{sync\}&service specific&\cr
360 \{async\}&service specific&\cr
361 {DALI-examples}&/examples&no\cr
362 {VOSI-availability}&/availability&yes\cr
363 {VOSI-capabilities}&/capabilities&yes\cr
364 \sptablerule
365 \end{tabular}
366 \caption{Endpoints for AccessData services}
367 \end{table}
368
369 A stand-alone SODA service may have one or both of the \{sync\} and \{async\} resources. For either type, it could have multiple resources (e.g. to support alternate authentication schemes). The SODA service may also include other custom or supporting resources.
370
371 Either the \{sync\} or \{async\} SODA capability may be included as part of other web services. For example, a single web service could contain the SIA-2.0 \{query\} capability, the DataLink-1.0 \{links\} capability, and the SODA \{sync\} capability. Such a service must also have the VOSI-availability and VOSI-capabilities resources to report on and describe all the implemented capabilities.
372
373 \subsection{\{sync\} resource}
374 \label{sect:syncres}
375
376 The \{sync\} resource is a synchronous web service resource
377 that conforms to the DALI-sync description. Implementors
378 are free to name this resource however
379 they like, except that the name must consist of one URI segment only (i.e.,
380 contain no slash). This is to allow clients, given the access URL,
381 can reliably find out the URL of the capabilities endpoint.
382 Clients, in turn, can find the resource path using the
383 VOSI-capabilities resource, but will in general be provided the access
384 URLs through a previous data discovery query or through direct user
385 input.
386
387 The \{sync\} resource performs the data access as specified by
388 the input parameters and returns the data directly in the
389 output stream. Synchronous data access is suitable when the
390 operations can be quickly performed and the data stream can
391 be setup and written to (by the service) in a short period
392 of time (e.g. before any timeouts).
393
394 \subsection{\{async\} resource}
395
396 The \{async\} resource is an asynchronous web service resource
397 that conforms to the DALI-async description. The considerations on
398 naming the resource given in sect.~\ref{sect:syncres} apply for it.
399
400 The \{async\} resource performs the data access as specified
401 by the input parameters and either (i) stores the results
402 for later transfer or (ii) pushes the results to a specified
403 destination (e.g. to a VOSpace location). Asynchronous data
404 access usually introduces resource constraints on the
405 service (which may be limited) and usually imposes a higher
406 latency before any results can be seen because the location
407 of results does not have to be valid until the data access
408 job is complete. Asynchronous data access is intended for
409 (but not limited to) use when the operations take
410 considerable time and results must be staged (e.g. some
411 multi-pass algorithms or operations that result in multiple
412 outputs).
413
414 \subsection{Examples: DALI-examples}
415
416 SODA services should provide a DALI-examples resource
417 with one example invocation that shows the variety
418 operations the service can perform. Example operations using
419 the \{sync\} resource and that output a small data stream are
420 preferred, as the examples may be used by automatic validators doing
421 relatively frequent (of order daily) queries.
422
423 Parameters to be passed to the service must be given using the DALI
424 \texttt{generic-parameter} term.
425
426
427 \subsection{Availability: VOSI-availability}
428
429 A SODA web service must have a VOSI-availability
430 resource \citep{std:VOSI} as described in DALI \citep{std:DALI}.
431
432 \subsection{Capabilities: VOSI-capabilities}
433
434 A web service that includes SODA capabilities must
435 have a VOSI-capabilities resource \citep{std:VOSI} as described in DALI
436 \citep{std:DALI}. The standardID for the \{sync\} resource is
437 $$\hbox{\texttt{ivo://ivoa.net/std/SODA\#sync-1.0}.}$$
438
439 The standardID for the \{async\} resource is
440
441 $$\hbox{\texttt{ivo://ivoa.net/std/SODA\#async-1.0}.}$$
442
443 All DAL services must implement the \texttt{/capabilities} resource.
444 The following capabilities document shows the minimal
445 metadata for a stand-alone SODA service and does not
446 require a registry extension schema:
447
448 \begin{lstlisting}[language=XML]
449 <?xml version="1.0"?>
450 <capabilities
451 xmlns:vosi="http://www.ivoa.net/xml/VOSICapabilities/v1.0"
452 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
453 xmlns:vod="http://www.ivoa.net/xml/VODataService/v1.1">
454 <capability standardID="ivo://ivoa.net/std/VOSI#capabilities">
455 <interface xsi:type="vod:ParamHTTP" version="1.0">
456 <accessURL use="full">http://example.com/data/capabilities</accessURL>
457 </interface>
458 </capability>
459 <capability standardID="ivo://ivoa.net/std/VOSI#availability">
460 <interface xsi:type="vod:ParamHTTP" version="1.0">
461 <accessURL use="full">
462 http://example.com/data/availability
463 </accessURL>
464 </interface>
465 </capability>
466 <capability standardid="ivo://ivoa.net/std/SODA#sync-1.0">
467 <interface xsi:type="vod:ParamHTTP" role="std" version="1.0">
468 <accessurl use="full">
469 http://example.com/data/sync
470 </accessurl>
471 </interface>
472 <!-- service details from extension schema could go here -->
473 </capability>
474 <capability standardid="ivo://ivoa.net/std/SODA#async-1.0">
475 <interface xsi:type="vod:ParamHTTP" role="std" version="1.0">
476 <accessurl use="full">
477 http://example.com/data/async
478 </accessurl>
479 </interface>
480 <!-- service details from extension schema could go here -->
481 </capability>
482 </capabilities>
483 \end{lstlisting}
484
485 Note that the \{sync\} and \{async\} resources do not have to be
486 named as shown in the accessURL(s) above. Multiple
487 capability elements for the \{sync\} and the \{async\} resources
488 may be included; this is typically used if the differ in
489 protocol (http vs. https) and/or authentication
490 requirements.
491
492 \begin{admonition}{Note}
493 As SODA builds upon several concepts of DataLink \citep{std:datalink},
494 that document should be read before trying to understand the following
495 material.
496 \end{admonition}
497 \subsection{Parameter Description and Three-Factor Semantics}
498
499 \subsubsection{Three-factor Semantics}
500
501 Parameters in SODA are defined by triples of name, UCD, and unit (the
502 ``SODA triple''). Data services are free to support as many such parameters
503 as is appropriate for their datasets, in addition to supporting
504 standard parameters. With the three factors, it is
505 unlikely that two operators will by accident use the same three factors
506 for parameters of differing semantics. In this way, when a new standard
507 parameter is adopted, no existing service should require changes to
508 remain compliant.
509
510 With standard parameters as defined in this document, clients can rely
511 on certain semantics and exploit that knowledge in the provision of
512 special UIs or APIs. Standard parameters defined so far are given
513 in table~\ref{table:standardpars}. Instructions for how to propose
514 additional standard parameters are given on the landing page of the IVOA
515 DAL working group\footnote{At the time of writing, this is
516 \href{http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaResReg}{http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaResReg}.}.
517
518 \begin{table}[mht]
519 \begin{tabular}{llll}
520 \sptablerule
521 \textbf{Name}&\textbf{UCD}&\textbf{Unit}&\textbf{Semantics}\cr
522 \sptablerule
523 ID&meta.ref.url;meta.curation&&cf.~sect.~\ref{sect:ID}\cr
524 POS&phys.angArea;obs&&cf.~sect.~\ref{sect:POS}\cr
525 BAND&em.wl&m&cf.~sect.~\ref{sect:BAND}\cr
526 TIME&time.interval;obs.exposure&d&cf.~sect.~\ref{sect:TIME}\cr
527 POL&meta.code;phys.polarization&&cf.~sect.~\ref{sect:POL}\cr
528 \sptablerule
529 \end{tabular}
530 \caption{SODA triples for the standard parameters defined here.}
531 \label{table:standardpars}
532 \end{table}
533
534 Both standard and non-standard parameters should follow DALI conventions
535 if at all possible. Roughly, float-valued parameters should be mapped
536 to interval-valued parameters (i.e., do not split up minimum and maximum
537 into separate parameters). Depending on their semantics, integer
538 parameters should either be intervals or enumerated parameters (which
539 typically can be repeated). String-valued parameters should always
540 be enumerated.
541
542 \subsubsection{Discovery of Supported Parameter. Implementation strategies}
543 \label{sect:discovery-supported}
544
545 There are two principal ways a client will be led to a SODA service:
546 From a service descriptor that is part of a DAL discovery response,
547 or by direct access mediated or not via a DataLink service. In the latter case an empty query to the service iwill answer by giving the service descriptor. . In both cases,
548 clients must obtain the set of supported parameters
549 from the applicable service descriptor.
550
551 In contrast to previous IVOA DAL protocols, SODA is not a data discovery
552 protocol but instead operates on concrete datasets. In a typical case,
553 some combinations of parameters (e.g., a positional or spectral cutout)
554 may yield no output at all, as the coverage of an individual dataset is
555 very limited. To provide meaningful user interfaces, clients would therefore
556 need detailed information on the service parameters, in particular as
557 regards their domains (i.e., set of values that are likely to yield
558 non-empty results). This, in regard, requires efficient and rich service APIs, not unlikely involving service-specific parameters.
559
560 But the double ways towards SODA service execution makes the providing of this information difficult to describe and is not done in this version.
561
562 As a consequence of this, when the SODA service is driven from a data discovery phase, parameters domains can be inferred by the end-user from the discovery reponse. In the other case, it may happen that these parameters have to be furnished blindly by the end-user.
563
564 \subsubsection{SODA Service Descriptor}
565 \label{sect:serv-desc}
566
567 The DataLink \citep{std:datalink} specification describes a mechanism for
568 describing a service within a VOTable resource and
569 recommends that services can describe themselves with a
570 special resource with \texttt{name="this"}. SODA responses for
571 empty sync queries should include a descriptor describing
572 both standard and custom query parameters (if applicable).
573 The descriptor for a service with standard parameters (see
574 sect.~\ref{sect:stdpars}) would be:
575
576 \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
577
578 <RESOURCE type="meta" utype="adhoc:service" name="this">
579
580 <PARAM name="standardID" datatype="char" arraysize="*"
581 value="ivo://ivoa.net/std/SODA#sync-1.0" />
582
583 <PARAM name="accessURL" datatype="char" arraysize="*"
584 value="http://example.com/SODA/sync" />
585
586 <GROUP name="inputParams">
587 <PARAM name="ID" ucd="meta.ref.url;meta.curation" datatype="char"
588 arraysize="*" />
589 <PARAM name="POS" ucd="phys.angArea;obs" datatype="char"
590 arraysize="*" />
591 <PARAM name="BAND" ucd="em.wl" unit="m" datatype="double"
592 arraysize="*"/>
593 <PARAM name="TIME" ucd="time.interval;obs.exposure"
594 unit="d" datatype="double"
595 arraysize="*" xtype="interval" />
596 <PARAM name="POL" ucd="meta.code;phys.polarization" datatype="char"
597 arraysize="*" />
598 </GROUP>
599 </RESOURCE>
600
601 \end{lstlisting}
602
603 This VOTable resource should be output for empty sync
604 queries; Thus all inputs and outputs would be fully
605 described.
606
607 \subsubsection{Client Handling of Discovered Parameters}
608
609 To keep SODA clients useful even when advanced data products are being
610 accessed, the following procedure should be followed by clients:
611
612 \begin{enumerate}
613 \item Obtain the parameter triples domains as
614 described in sect.~\ref{sect:discovery-supported}.
615 \item Identify standard parameters understood by the client and provide
616 suitable input widgets (e.g., a rubberband on top of a sky rendering,
617 or a \texttt{spatial\_cutout} method possibly allowing library-specific
618 coverage objects) for them.
619
620 \item For the remaining parameters, provide generic UI or API elements
621 (e.g., sliders or text boxes with domains annotated for intervals, pop-up
622 menus for enumerated values).
623 \end{enumerate}
624
625
626
627
628 \section{Parameters for \{sync\} and \{async\}}
629
630 \label{sect:stdpars}
631
632 The \{sync\} and \{async\} resources accept the same set of
633 parameters.
634
635
636 \subsection{Common Parameters}
637
638 \subsubsection{ID}
639 \label{sect:ID}
640
641 The ID parameter is used to specify the dataset or file to
642 be accessed. The values for the ID parameter are generally
643 discovered from data discovery or DataLink requests. The
644 values must be treated as opaque identifiers that are used
645 as-is. The DataLink specification \citep{std:datalink} describes mechanisms
646 for conveying opaque parameters and values in service
647 descriptor resources that can be used by clients to set the
648 ID parameter.
649
650 The ID parameter is single-valued in \{sync\} requests, so
651 \{sync\} soda requests access a single dataset or file.
652 Multiple ID parameters may be submitted in \{async\} requests
653 on order to bundle access to multiple datasets or files in a
654 single job.
655
656
657 The ID ucd is ``meta.ref.url;meta.curation'', and its unit is blank.
658 In addition its datatype ``char'', with
659 arraysize ``*''.
660
661
662 \subsection{Filtering Parameters}
663
664 Filtering parameters are used to extract subsets of larger
665 datasets or data files. In general, filtering parameters are
666 single-valued in \{sync\} requests and multi-valued in \{async\}
667 requests (exceptions noted below). When multiple values of
668 filtering parameters are used in an \{async\} job, each
669 combination of values produces zero or one result. For
670 example, if an \{async\} job included two POS and two BAND
671 values, there could be as many as four results (or fewer if
672 some combinations do not produce a result because the filter
673 does not intersect the bounds of the data).
674
675 \subsubsection{POS}
676 \label{sect:POS}
677
678 The POS parameter defines the positional region(s) to be
679 extracted from the data. The value is made up of a shape
680 keyword followed by coordinate values. The
681 allowed shapes are:
682
683 \begin{table}[h]
684 \begin{tabular}{ll}
685 \sptablerule
686 \textbf{Shape}&\textbf{Coordinate values}\cr
687 \sptablerule
688 \texttt{CIRCLE}&\texttt{<longitude> <latitude> <radius>}\cr
689 \texttt{RANGE}&\texttt{<longitude1> <longitude2> <latitude1> <latitude2>}\cr
690 \texttt{POLYGON}&\texttt{<longitude1> <latitude1> ... (at least 3 pairs)}\cr
691 \sptablerule
692 \end{tabular}
693 \caption{POS Values in Spherical Coordinates}
694 \end{table}
695
696 As in DALI, open intervals use -Inf or +Inf as one limit.
697
698 Examples for POS values:
699
700 \begin{itemize}
701 \item A circle at (12,34) with radius 0.5:
702
703 \begin{lstlisting}
704 POS=CIRCLE 12 34 0.5
705 \end{lstlisting}
706
707 \item A range of [12,14] in longitude and [34,36] in latitude:
708
709 \begin{lstlisting}
710 POS=RANGE 12 14 34 36
711 \end{lstlisting}
712
713 \item A polygon from (12,34) to (14,34) to (14,36) to (12,36) and
714 (implicitly) back to (12,34):
715
716 \begin{lstlisting}
717 POS=POLYGON 12 34 14 34 14 36 12 36
718 \end{lstlisting}
719
720 The inside is always assumed to be the smaller of the region
721 to the left and the region to the right so only polygons
722 smaller than half the sphere can be specified.
723
724 \item A band around the equator:
725
726 \begin{lstlisting}
727 POS=RANGE 0 360 -2 2
728 \end{lstlisting}
729
730 \item The north pole:
731
732 \begin{lstlisting}
733 POS=RANGE 0 360 89 +Inf
734 \end{lstlisting}
735 \end{itemize}
736
737 All longitude and latitude values (plus the radius of the
738 CIRCLE) are expressed in degrees in the ICRS. A future
739 version of this specification may allow the use of other
740 reference systems (specifically the native system of the
741 data).
742
743 The POS parameter is single-valued for \{sync\} requests and
744 multi-valued for \{async\} jobs.
745
746 Since it is string-valued, POS is unitless (although the values
747 contained in the the string are all in decimal degrees).
748 The ucd is ``phys.angArea;obs''. However the datatype
749 of the POS parameter is ``char''.
750
751 \subsubsection{BAND}
752 \label{sect:BAND}
753
754 The BAND parameter defines the energy interval(s) to be
755 extracted from the data. The value is an open or closed
756 numeric interval of values in the native spectral axis
757 coordinate system and units of the data. The intervals
758 always include the bounding values. As in DALI, open intervals use -Inf
759 or +Inf as one limit.
760
761 If there is one single value the interval is assumed to be
762 infinitely small (a scalar value).
763
764 \begin{itemize}
765 \item The closed interval [500,550]:
766
767 \begin{lstlisting}
768 BAND=500 550
769 \end{lstlisting}
770
771 \item The open interval (-inf,300]:
772
773 \begin{lstlisting}
774 BAND=-Inf 300
775 \end{lstlisting}
776
777 \item The open interval [750,inf):
778
779 \begin{lstlisting}
780 BAND=750 +Inf
781 \end{lstlisting}
782
783 \item The scalar value 550, equivalent to [550,550]:
784
785 \begin{lstlisting}
786 BAND=550
787 \end{lstlisting}
788
789 \end{itemize}
790
791 Extracting using a scalar value should normally extract a
792 single pixel along the energy axis of the data; extracting
793 using an interval should extract one or more pixels.
794
795 All energy values are expressed as barycentric wavelength in
796 meters. A future version of this specification may allow the
797 use of other reference systems (specifically the native
798 system of ther data).
799
800 The BAND parameter is single-valued for \{sync\} requests and
801 multi-valued for \{async\} jobs.
802
803 The ucd of the BAND parameter is ``em.wl'', the unit is ``m''. Its datatype is
804 double with an arraysize of 2, and the xtype is ``interval'' as defined in DALI.
805
806
807 \subsubsection{TIME}
808 \label{sect:TIME}
809
810 The TIME parameter defines the time interval(s) to be
811 extracted from the data. The value is an open or closed
812 interval with either numeric values (interpreted as Modified
813 Julian Dates). As in DALI, open intervals use -Inf or +Inf as one limit.
814
815 If there is one single value the numeric interval is assumed
816 to be infinitely small (a scalar value).
817
818 \begin{itemize}
819 \item An open interval from the MJD 55100.0 and all later times:
820
821 \begin{lstlisting}
822 TIME= 55100.0 +Inf
823 \end{lstlisting}
824
825 \item A range of MJD values:
826
827 \begin{lstlisting}
828 TIME=55123.456 55123.466
829 \end{lstlisting}
830
831 \item An instant in time using Modified Julian Date:
832
833 \begin{lstlisting}
834 TIME=55678.123456
835 \end{lstlisting}
836 \end{itemize}
837
838 Time values are always UTC.
839 The TIME parameter is single-valued for \{sync\} requests and
840 multi-valued for \{async\} jobs.
841
842 The ucd of the TIME parameter is ``time.interval;obs.exposure'' and the unit is ``d''. The
843 datatype is ``double'' with an arraysize of 2,
844 and the xtype is, again, ``interval'' as defined in
845 DALI
846
847
848 \subsubsection{POL}
849 \label{sect:POL}
850
851 The POL parameter defines the polarization state(s) (Stokes)
852 to be extracted from the data.
853
854 \begin{itemize}
855 \item Extract the unpolarized intensity:
856 \begin{lstlisting}
857 POL=I
858 \end{lstlisting}
859 \item Extract the standard circular polarization:
860 \begin{lstlisting}
861 POL=V
862 \end{lstlisting}
863
864 \item The POL parameter is multi-valued; multiple values can be
865 included in a single request and all will be extracted.
866 Extract only the IQU components:
867 \begin{lstlisting}
868 POL=I
869 POL=Q
870 POL=U
871 \end{lstlisting}
872 \end{itemize}
873
874 The POL is multi-valued for both \{sync\} and \{async\} requests.
875 Unlike general filtering parameters, all values of POL are
876 combined into a single filter; for example, if the request
877 includes the three values above, the job would generate one
878 result with some or all of these polarization states (per
879 combination of ID and other filtering parameters).
880
881 The ucd of the POL PARAMETER is ``meta.code;phys.polarization'' and the unit is blank. The
882 datatype is ``char'' with arraysize ``*''.
883
884
885
886 \section{\{sync\} Responses}
887
888 All responses from the \{sync\} resource follow the rules for
889 DALI-sync resources, except that the \{sync\} response allows
890 for error messages for individual input identifier values.
891
892 \subsection{Successful Requests}
893
894 Successfully executed requests should result in a response
895 with HTTP status code 200 (OK) and a response in the format
896 requested by the client or in the default format for the
897 service.
898
899 If the values specified for cutout parameters do not include
900 any pixels from the target dataset/file, the service must
901 respond with HTTP status code 204 (No Content) and no
902 response body.
903
904 The service should set the following HTTP headers to the
905 correct values where possible.
906
907 \begin{tabular}{rr}
908 \sptablerule
909 Content-Type&mime-type of the response\cr
910 Content-Encoding&encoding/compression of the response (if applicable)\cr
911 \sptablerule
912 \end{tabular}
913
914 Since the response is usually dynamically generated, the
915 Content-Length and Last-Modified headers cannot usually be
916 set.
917
918
919 \subsection{Errors}
920
921 The error handling specified for DALI-sync resources applies
922 to service failure. Error documents should be text using the
923 text/plain content-type and the text must begin with one of
924 the following strings:
925
926 \begin{table}[h]
927 \begin{tabular}{rr}
928 Error&General error (not covered below)\cr
929 AuthenticationError&Not authenticated\cr
930 AuthorizationError&Not authorized to access the resource\cr
931 ServiceUnavailable&Transient error (could succeed with retry)\cr
932 UsageError&Permanent error (retry pointless)\cr
933 \end{tabular}
934 \caption{error messages with their meaning}
935 \end{table}
936
937 \section{\{async\} Responses}
938
939 The \{async\} resource conforms to the DALI-async resource
940 description, which means the job is a UWS job with all the
941 job control features available. All result files are to be
942 listed as children of the UWS results resource. The service
943 provider is free to name each result.
944
945 \appendix
946
947 \section{Changes from Previous Versions}
948
949 \subsection{Changes from WD-SODA-1.0-20151212}
950
951 \begin{itemize}
952 \item POS is now unitless
953 \item Aligned parameter UCDs with what is in Obscore
954 \item Removed gratuitous xtypes.
955 \end{itemize}
956
957 \subsection{Changes from WD-SODA-1.0-20151120}
958
959 Change the name of the protocol. Suppression of SELECT and COORD. xtype description are in DALI. Reference to this has been added.
960
961 \subsection{Changes from WD-AccessData-1.0-20151021}
962
963 Added general introduction on PARAMETER description to
964 section 3. Modified SELECT and COORD sections in order to
965 detach them from SimDal. Added Appendix on xtype description
966 with BNF syntax.
967
968 \subsection{Changes from WD-AccessData-1.0-20140730}
969
970 \begin{itemize}
971 \item Removed REQUEST parameter since the DAL-WG decision to not
972 include it when there is only one value.
973
974 \item Clarified that ID and filierting parameters are single
975 valued for \{sync\} and multi-valued for \{async\}, wth POL
976 being multi-valued but still being treated as a single
977 filter.
978 \end{itemize}
979
980 \subsection{Changes from WD-AccessData-1.0-20140312}
981
982 This is the initial document version.
983
984 \bibliography{ivoatex/ivoabib}
985
986 \end{document}

msdemlei@ari.uni-heidelberg.de
ViewVC Help
Powered by ViewVC 1.1.26