\documentclass[11pt,a4paper]{ivoa}
\input tthdefs

\usepackage{todonotes}

\newcommand{\dousecase}[2]{%
  #1~\ref{uc:#2}}
\newcommand{\Usecase}[1]{%
  \dousecase{Use case}{#1}}
\newcommand{\usecase}[1]{%
  \dousecase{use case}{#1}}

\title{The IVOA Source Data Model}

\ivoagroup{Data Model}

\author[http://wiki.ivoa.net/twiki/bin/view/IVOA/JesusSalgado]{Jesus
Salgado}
\author[http://wiki.ivoa.net/twiki/bin/view/IVOA/GerardLemson]{Gerard
Lemson}
\author[http://wiki.ivoa.net/twiki/bin/view/IVOA/MarkusDemleitner]{Markus
Demleitner}

\editor{TBD}

% \previousversion[????URL????]{????Funny Label????}
\previousversion{This is the first public release}
       

\begin{document}
\begin{abstract}
Catalogues of astronomical objects make up an important part of the
Virtual Observatory (VO).  Almost since the beginning of the VO, the
Simple Cone Search (SCS) standard provided a simple model for such data.
The Source Data Model refines and formalises the data model implied by
SCS with the overall goal of enabling a greater degree of automation in
processing such catalogues.  This standard derives requirements on the
source data model from a set of use cases and then discusses the data
model derived in turn from the requirements.

\end{abstract}


\section*{Conformance-related definitions}

The words ``MUST'', ``SHALL'', ``SHOULD'', ``MAY'', ``RECOMMENDED'', and
``OPTIONAL'' (in upper or lower case) used in this document are to be
interpreted as described in IETF standard RFC2119 \citep{std:RFC2119}.

The \emph{Virtual Observatory (VO)} is a
general term for a collection of federated resources that can be used
to conduct astronomical research, education, and outreach.
The \href{http://www.ivoa.net}{International
Virtual Observatory Alliance (IVOA)} is a global
collaboration of separately funded projects to develop standards and
infrastructure that enable VO applications.


\section{Introduction}

The Simple Cone Search standard SCS \citep{std:SCS} very early in the
development of the Virtual Observatory introduced a simple but effective
model for describing a catalogue of astronomical sources, consisting of

\begin{enumerate}
\item A string-valued identifier.
\item A ``main'' position, given as ICRS Right Ascension
and declination in two floating-point fields.
\item Arbitrary further information.
\end{enumerate}

The individual items are identified through Unified Content Descriptors
\citep{std:UCD}.  To enable limited machine interpretation of the
further information, SCS relies on the mechanisms defined by VOTable
\citep{std:VOTable}, in particular again UCDs, but also units,
description, etc.

While SCS has served the VO community well over more than a decade,
several important use cases are not sufficiently covered by it, and the
limited amount of annotation precludes many applications that require
machines to have a deeper understanding of the catalogue's contents.  This
standard intends to supply the missing elements.

We begin by sketching the motivating use cases and derive requirements
from them in sect.~\ref{sect:ucreq}.  Based on this analysis, in
sect.~\ref{sect:dm} we give the VO-DML based definition of the SourceDM
components.  We conclude with considerations on phasing in SourceDM
usage and possibilities to automatically infer rudimentary SourceDM
annotation from legacy VOTable annotation.\todo{Complete the document structure}


\subsection{Role within the VO Architecture}

\begin{figure}
\centering

\includegraphics[width=0.9\textwidth]{archdiag.png}
\caption{Architecture diagram for SourceDM}
\label{fig:archdiag}
\end{figure}

Fig.~\ref{fig:archdiag} shows the role SourceDM plays within the
IVOA architecture \citep{note:VOARCH}.  It depends on the following
standards:

\begin{bigdescription}
\item[VO-DML \citep{prestd:VODML}] SourceDM is written in VO-DML, a
standard language to define data models, essentially providing an
easy-to-process subset of the common modelling language UML sufficient
to express data models useful in the Virtual Observatory context.

\item[Mapping \citep{prestd:Mapping}] The Mapping document defines how
to actually annotate catalogues written in VOTable (or possibly other
formats in the future) with the concepts defined in this document.

\item[PhotDM \citep{std:PhotDM}] The Photometry data model is re-used by
SourceDM to express photometric quantities.
\end{bigdescription}


\section{Use Cases and Requirements}
\label{sec:ucreq}

\subsection{Use Cases}
\label{sect:usecases}

% please label use cases as uc:hyphenated-form-of-header

Since a full model for an astronomical source would presumably result in
a model of a large part of astronomy and physics, it is important to be
guided by use cases when delimiting the scope of the model and
formulating requirements on the model.  This subsection will list the
use cases the community agreed upon.\todo{I suspect we want some non-use
cases, too, to avoid repetitive discussions about unrealistic wishes. MD}
We use a ``want to'' form for the use cases, with two parties involved:
Data providers who annotate their catalogues, and authors of programs or
libraries (``clients'') dealing with the annotated data.

\subsubsection{Standard Locations}
\label{uc:standard-locations}

Client authors want to to take SourceDM-annotated tables and easily
provide plots of the locations of the contained objects in charts or
over astrometrically calibrated images, preferably without having to
implement a large number of coordinate transforms; this is essentially
what tools can do right now with the ICRS positions given in SCS
responses.

\subsubsection{Identities}
\label{uc:identities}

Catalogue annotators want to express whether or not two instances
(``rows'') refer to the same physical object on the sky.  Client authors
want to be able to match known-identical sources, for instance to
aggregate information from several different catalogueues.

\subsubsection{Native Frames}
\label{uc:native-frames}

Catalogue annotators want to give coordinates in their catalogue's
native frames and for whatever epoch is convenient for them in order to
achieve maximum precision, avoid hard-to-control error propagation in
transformation requiring additinal information, and keep provenance
simple.  This becomes more important as we have more and more data
noticeably distant from the J2000 epoch most of today's major surveys
are close to, and more and more data precise enough that for a large
number of objects proper their motions since J2000 become relevant for
daily work.

\subsubsection{Precise Astrometry}
\label{uc:precise-astrometry}

Client authors want to recognise all information in catalogues relevant to
precision astrometry (e.g., distances or radial velocities as necessary in
foreshortening calculations).  This also involves the position of the
observer, at least for a few standard reference positions.  Of
course it would be beneficial to be able to express actual observer
positions, but in order to keep the model manageable, we suggest
deferring this to provenance.

\subsubsection{Crossmatching}
\label{uc:crossmatching}

Client authors want to implement smart crossmatching schemes between
different catalogues.  This includes reducing catalogues to common
reference frames and epochs, but also unifying non-positional source
characteristics (e.g., photometry).  In the presence of crossmatching
schemes based on principled statistical approaches, they want to
identify data aiding them in the estimation of the relevant
distributions.

\subsubsection{Combining Data}
\label{uc:combining-data}

Following a crossmatch (or just within a single table), tool authors
want to aggregate various pieces of information into new data points or
products.  Examples include:

\begin{itemize}
\item Colours or even SEDs from photometry points
\item Proper motions or even orbits from positional measurements
\item Time series from, say, radial velocity measurements
\end{itemize}

\subsubsection{Automatic Errors}
\label{uc:automatic-errors}

Where catalogues contain the necessary information, tool authors want to
provide sound error estimates for derived quanities as discussed in
\ref{uc:combining-data}.

\subsubsection{Annotation of Existing Catalogues}
\label{uc:annotation-of-existing-catalogues}

Catalogue annotators do not want -- and indeed, in general cannot --
change catalogue data.  SourceDM must therefore be flexible enough to
enable the meaningful annotation of a large majority of the catalogues
already in the VO.

\subsubsection{Standard Properties are Annotated}
\label{uc:standard-properties-are-annotated}

Client authors want to find scientifically relevant source properties
easily.  These include

\begin{itemize}
\item source type
\item spectral class
\item mass
\item physical size
\item angular size
\item redshift
\end{itemize}


\subsection{Requirements}

% please reference use cases with \Usecase{hyphenated-form} (or
% \usecase within a sentence)

This section derives concrete requirements from the use cases collected
in sect.~\ref{sect:usecases}.  Some requirements are formulated using
``should'' phrases.  ``Should'' as used in such requirements is not to
be understood in the sense of RFC2119.

\subsubsection{General Requirements}

Most of the use cases of course require basic metadata necessary to
interpret physical quanitites, in particular their units.

\Usecase{identities} requires a concept of a physical object
that a source refers to.  It also means that in relational
representations of collections of SourceDM instances, the physical
object's identifier should not be the primary key.\todo{This may be wrong;
perhaps we should mandate one instance per object per collection.  I'm
happy to strike this if it turns out to be dumb. MD}

Regarding \usecase{standard-properties-are-annotated}, UCDs may be
sufficient to annotate the concepts listed.  Thus, at this point we
derive no requirements from this use case.


\subsubsection{Requirements on Space and Time Metadata}

\Usecase{standard-locations} suggests that SourceDM instances
should maintain the SCS practice of including ICRS positions wherever
possible.  Given \usecase{precise-astrometry}, it is highly
desirable that these positions are nevertheless properly annotated, in
particular with information on the epoch for which the position is
pertinent, and where available the reference position.

\Usecase{crossmatching} means that the following spatial
concepts should be present in the model (even though of course many
catalogues cannot fill all of them):

\begin{itemize}
\item Positions (given the context, spherical positions are probably
sufficient in the model)
\item Derivatives of positions (i.e., proper motions)
\item Errors in the positions and their derivatives
\item Epochs of the positions
\end{itemize}

In view of \usecase{precise-astrometry} together with
\usecase{native-frames}, SourceDM should enable specifying frames with
enough detail that sufficiently capable clients can use native frames
for crossmatching.  Less capable clients could still crossmatch at least
partially using the standard locations from
\usecase{standard-locations}.

What this means in particular is that a single source should admit
multiple positional specifications.  In order to not leave the set of
SourceDM-representable instances in \usecase{combining-data}, SourceDM
should allow that these multiple specifications refer to different
epochs.

\Usecase{precise-astrometry} requires the third dimension, i.e.,

\begin{itemize}
\item distance
\item its derivative
\item errors on both quanitites
\end{itemize}

Taking \usecase{annotation-of-existing-catalogues} into account, various
forms of distance specification need to be admitted; in particular, many
important catalogues give parallaxes rather than distances.  Also note
that the derivative of the distance, the radial velocity, will typically
given in spatial coordinates even if the distance is given as (angular)
parallax.

In extragalactic astronomy, redshift is frequently used as a proxy for
distance, and it is of course also a measure for the radial velocity.
We still propose to not allow cosmological redshifts for distance or the
derivative of the distance.  In the nearby universe, the presence of a
random motions (which might indeed be the actual objectiv a a catalog)
makes this undesirable, in the more distant universe the interpretation
of the term distance becomes difficult and should be avoided in favour
of the redshift itself.  Of course, where authors actually give
redshift-based distances, these would be annotated as such.

One consequence of \usecase{identities} is that SourceDM should allow
statements of the type ``this row corresponds to source
\emph{identifier-cat1} in catalogue 1 and to source \emph{idenifier-cat2}
in catalogue 2''.

\Usecase{automatic-errors} requires suffient information on the errors.
In the astrometric case, this in particular concerns the mean epochs,
i.e., the epoch at which the covariance matrix of positions and proper
motions becomes diagonal.  The mean epochs in general are different on
two coordinates.

\subsection{Requirements on Photometry}

Lorem ipsum.\todo{Can we steal something from the PhotDM here?}


\appendix
\section{Changes from Previous Versions}

No previous versions yet.  
% these would be subsections "Changes from v. WD-..."
% Use itemize environments.


\bibliography{ivoatex/ivoabib}


\end{document}
