ViewVC logotype

Contents of /trunk/projects/grid/notes/schema-versioning/schemaVersioning.tex

Parent Directory Parent Directory | Revision Log Revision Log

Revision 3144 - (show annotations)
Thu Oct 29 09:14:10 2015 UTC (5 years ago) by harripa
File MIME type: application/x-tex
File size: 15671 byte(s)
further small tweaks just prior to publishing 1.0
1 \documentclass[10pt,a4paper]{ivoa}
2 \input tthdefs
4 \usepackage{color}
5 \usepackage{listings}
6 \lstloadlanguages{sh,make,[latex]tex,XML,XSLT}
7 \lstset{flexiblecolumns=true,numberstyle=\small,numbers=left,
8 identifierstyle=\texttt}
11 \usepackage{color}
12 \definecolor{gray}{rgb}{0.4,0.4,0.4}
13 \definecolor{darkblue}{rgb}{0.0,0.0,0.6}
14 \definecolor{cyan}{rgb}{0.0,0.6,0.6}
16 \lstset{
17 basicstyle=\ttfamily,
18 columns=fullflexible,
19 showstringspaces=false,
20 commentstyle=\color{gray}\upshape
21 }
23 \lstdefinelanguage{XML}
24 {
25 morestring=[b]",
26 morestring=[s]{>}{<},
27 morecomment=[s]{<?}{?>},
28 stringstyle=\color{black},
29 identifierstyle=\color{darkblue},
30 keywordstyle=\color{cyan},
31 % morekeywords={xmlns,version,type}% list your attributes here
32 }
35 \usepackage{todonotes}
37 \usepackage[utf8]{inputenc}
39 \definecolor{texcolor}{rgb}{0.4,0.1,0.1}
40 \newcommand{\texword}[1]{\texttt{\color{texcolor} #1}}
42 \iftth
43 \newcommand{\BibTeX}{BibTeX}
44 \fi
46 \iftth
47 \newcommand{\comicstuff}[1]{
48 \begin{html}<span class="comic">#1</span>\end{html}}
49 \else
50 \newcommand{\comicstuff}[1]{(HTML exclusive material)}
51 \fi
52 \customcss{custom.css}
54 \title{XML Schema Versioning Policies}
56 \ivoagroup{Standards and Processes}
58 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PaulHarrison]{Paul Harrison}
59 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarkusDemleitner]{Markus Demleitner}
60 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/BrianMajor]{Brian Major}
61 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PatDowler]{Pat Dowler}
63 \editor{Paul Harrison}
65 \previousversion{This is the first public release}
67 \begin{document}
69 \SVN$Rev$
70 \SVN$Date$
71 \SVN$URL$
73 \begin{abstract}
74 This note describes the recommended practice for the evolution of IVOA
75 standard XML schemata that are associated with IVOA standards. The criteria for
76 deciding what might be considered major and minor changes and the policies for dealing with each case are described.
77 \end{abstract}
81 \section*{Acknowledgments}
83 The content of this note is derived from discussions that occurred in a splinter
84 session at the June 2015 IVOA interoperability meeting in Sesto, Italy.
86 \section{Introduction}
88 Many of the standard protocols and data models developed by
89 the International Virtual Observatory Association (IVOA) have used XML
90 \citep{std:XML} for message or object serialization. The structure of
91 these XML files has
92 usually been constrained using the XML schema definition language
93 \citep{std:XSD}, or XSD for short.
94 The particular schema that has been associated with a standard
95 is defined by the ``target namespace'' (hereafter refered to as simply ``the
96 namespace'') of the schema - the namespace identifier itself is a URI that
97 typically has a form that contains the version number of the standard. There
98 exist in many programming languages XML parsers that can use the XSD schema
99 automatically to do a strong check whether an instance of an XML
100 document conforms to the given schema. If the XML document does conform to the
101 structure defined by the schema then it is known as ``valid'' and even a small
102 deviation from the specified structure will mean that the XML instance is
103 ``invalid''. This strong check of validity is extremely useful in the context of
104 interoperating services and clients within the VO as it guarantees that both
105 sides of an interaction will agree on the structure of the document and
106 hence its interpretation. To maintain
107 this behaviour, the conventional XSD practice is to give the schema a new
108 identity (namespace) if any changes are made to it so that both client and
109 server can agree on the exact version of a schema that they are using for
110 checking validity of an XML instance.
112 Once a VO service has been standardised there will typically be a growing number
113 of clients that are coded against the particular version of the
114 schema - any changes to the definition of the schema that is subsequently used
115 by a new version of the service to create instance documents will result in
116 immediate classification as ``invalid'' by the clients that have not themselves being
117 updated to use the new schema.
118 As explained above, in general this property is desirable for guaranteeing interoperability, but it does limit
119 the ability even to correct errors in the original schema definition without
120 causing disruption to the deployed clients.
122 A related problem occurs because, to namespace-aware XML parsers, the
123 element name is a tuple of the namespace URI and the tag content. For
124 instance, in the XML document
126 \begin{lstlisting}[language=XML]
127 <doc xmlns="http://example.com/1.0"/>
128 \end{lstlisting}
129 the fully qualified name of the element includes the namespace and is
130 conventionally written as \verb|{http://example.com/1.0}doc|, and clients will expect this full
131 form, whether or not they perform schema validation. As soon as the
132 namespace changes, any client expecting this element name will no longer
133 understand anything in the document. Rather typically, however, the
134 client would still be able to make sense of the document as far as it is
135 relevant to the client if it ignored the namespace part of the element
136 name. In consequence, many clients started discarding the namespace
137 part entirely, which then leads to new interoperability problems, for
138 instance, when elements from a different namespace are embedded within
139 such a document. Hence it is desirable to limit namespace changes to
140 cases when legacy clients would definitely break.
142 This note describes the
143 circumstances in which it is permissible to make changes to a schema and not
144 change its namespace.
147 \section{Schema Versioning}
148 It is the case that two XML instances are formally regarded by XML
149 conventions as not equivalent if the only textual difference between them is
150 that the namespace declaration is not the same. We
151 can use the observation that simply removing the namespace definition from both instances
152 would make the instances both textually the same and equivalent in the strict
153 XML sense to inform the class of changes can be to the schema definition without
154 necessarily needing to change the namespace, and how a client should treat such
155 changes.
156 \subsection{Minor changes}
157 In general the class of changes that might be considered minor are those
158 which allow legacy clients (i.e. without rewriting) to keep functioning with
159 content produced against the new schema.
160 Specific structual changes to the schema that allow this goal to be achieved
161 include:
163 \begin{itemize}
164 \item Not removing concepts (i.e. elements or attributes) from the old
165 schema.
166 \item Ensuring that any new concepts are optional.
167 \end{itemize}
168 Even with the restrictive conditions above it is still necessary that any
169 consumer of XML instance documents takes the approach that it does not do strict
170 schema validation against the version of the schema that it knows about, but
171 rather ignore everything that it does not understand. This approach is allowable
172 because any new concepts are optional even for consumers of the XML instance
173 that are aware of the latest version of the schema, and so clients cannot use
174 this information for fundamental changes in the behaviour of an IVOA protocol.
175 In other words the IVOA protocol remains backwards compatible and would only
176 warrant a ``point change'' in the standard version.
178 Conversely if the conditions above cannot be met by an evolution of an IVOA
179 standard then it is a indication that the standard is undergoing a ``major
180 change'' and backwards compatibility may be broken. Such a change would
181 involve both a change in the first part of the standard document's version
182 number as well as a change in the version used in the schema
183 namespace.
184 \subsubsection{Determining if the changes are indeed minor}
185 Although the conditions outlined above for minor changes should generally be
186 strictly adhered to in the design of minor extensions to standards, there are
187 occasions where a new version of a standard might try to correct an error made
188 in a previous version. For example a certain construct could have been
189 schema valid which should not have been allowed in a correctly authored
190 schema that expressed the intentions of the standard. In this case it is likely
191 that the correction would break the first of the above conditions, but hopefully
192 clients would not be written to expect the unintentionally valid construct and
193 would not be unduely impacted by the change. It would be up to the IVOA
194 Technical Coordination Group (TGC) to determine the likely impact of such a
195 change and whether the violation of the constraints for a minor change would be
196 allowable in such circumstances.
198 \subsection{Indicating the version number}
200 When allowing minor changes to the schema without changing the
201 namespace it becomes imperative that the the minor version number of the schema
202 is communicated via a means other than the namespace. This is true for
203 both instance documents and schema documents.
205 \subsubsection{Version numbers in instance documents}
207 It is important that
208 clients are able to know which version of the schema was used to create an XML
209 instance. A common use case envisioned is that in service responses,
210 this information can be used to discover a service's ability to
211 correctly respond to newly specified parts of the protocols.
213 Hence, in service responses, root elements must have
214 a \xmlel{version}
215 attribute exactly giving the value of the correspoing XSD file's
216 \xmlel{version} attribute.
218 When global elements from a different schema are included within other
219 XML files, each of these global elements should have a version attribute
220 of its own. Where the discovery of extended capabilities is not a
221 likely use case, such version attributes may be left out.
223 \subsubsection{Version numbers in schema files}
224 It is of similar importance to software writers to know exactly the version of
225 the schema that they are using. In the XSD definition there is an appropriate
226 \xmlel{version} attribute on the top level \xmlel{<schema>} element that can be
227 used for this purpose.
229 \begin{lstlisting}[language=XML]
230 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
231 targetNamespace="http://www.ivoa.net/xml/UWS/v1.0"
232 xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0"
233 xmlns:xlink="http://www.w3.org/1999/xlink"
234 elementFormDefault="qualified"
235 attributeFormDefault="unqualified"
236 version="1.1-PR-20150626"
237 >
238 \end{lstlisting}
240 This version attribute is not used formally by the schema validation machinery
241 but can be used as desired here to indicate a precise version of the schema
242 given that the namespace is only used to differentiate between major revisions
243 of the schema.
247 \section{Hosting the Schema on the IVOA Web Site}
248 As mentioned in the introduction the schemata are typically associated with a
249 particular version of a standard and so are typically included in an appendix of
250 the standard. In addition they are located on the ivoa website
251 \url{http://www.ivoa.net/xml} to allow software authors interactively to
252 obtain the latest version directly as a file.
254 The availability of the schemata via the IVOA web site also
255 allows XML parsers to be instructed to fetch the schema automatically when
256 validating an XML instance. This is done using the \xmlel{xsi:schemaLocation}
257 attribute which pairs a namespace with the URL of a schema that has definitions
258 in that namespace. Although there is no particular necessity to do so, it has
259 been standard practice to use namespaces that have a one-to-one correspondence
260 with a location on the IVOA web site. The advantage of this approach is that the
261 namespace is ``owned'' by the IVOA and there is therefore much less chance of
262 accidental clashes of namespaces.
264 Thus a current schema is usually available at two URLs on the IVOA site;
265 \begin{enumerate}
266 \item at a URL that corresponds to the namespace \newline e.g.
267 \texttt{http://www.ivoa.net/xml/UWS/v1.0}
268 \item at a URL that corresponds to the filename of the schema \newline e.g.
269 \texttt{http://www.ivoa.net/xml/UWS/UWS-v1.0.xsd}
270 \end{enumerate}
272 As this note now recommends that there are potentially many minor versions of a
273 schema all in the same namespace the actual file than the first from of the URL
274 above points to must be updated when a new minor version is published. The
275 various minor versions of a particular schema should be available at the
276 ``filename'' style URLs of the second kind. It should be noted that the
277 actual file pointed to by the ``namespace'' style URL should only be updated to
278 point to the new minor version when the associated standard has reached the
279 ``recommendation'' stage.
281 \subsection{Client use of schema}
282 Using this strategy means that client software that simply uses validation with
283 the parser utilizing this schema location hint will always pick up the latest
284 version of the schema, which should mean that the instance document will always
285 be valid assuming that the server has created a strictly valid document. Note
286 that this still applies even if the server is using an older (minor) version of
287 the schema, as new versions are only allowed to add optional features to the
288 schema.
290 It is often the case that client software is written to use a local copy of the
291 schema to avoid the overhead of continually downloading the schema. In this case
292 it is possible that the XML instance document will not be valid against the
293 local schema if the server is working against a newer minor version of the
294 schema - as it could have added a new optional element into the XML instance.
295 It is for this reason that the client side of an IVOA service should be
296 forgiving in ignoring elements that it is not expecting. It is possible for the
297 client to recognise that the XML instance is valid against a newer version of
298 the schema by comparing the version attribute of the root element of the XML
299 instance against the version attribute in the local XSD schema document.
301 \section{Summary of Recommendations}
302 This section summarizes the main recommendations contained within this document.
304 \begin{itemize}
305 \item The namespace URI should contain only the associated standard major number,
306 and therefore does not change when a minor version increment occurs.
307 e.g.\ \texttt{http://www.ivoa.net/xml/ABC/v1}. However, existing namespace URIs
308 which contain a ``\texttt{.0}'' minor increment suffix should not be changed to
309 conform to this rule.
311 \item Include a \xmlel{version} attribute in the top level element to allow
312 client version discovery - this version number should include the full standard version
313 number including the minor version increment. -- 1.1
315 \item Set the \xmlel{version} attribute of the \xmlel{<schema>} element in the
316 XSD to be equal to the full standard version including the minor version
317 increment and document status and version - e.g.\ 1.1-WD-20150607
319 \item Minor version changes should not break clients - i.e. in general they
320 should only add new elements/attributes, not remove formerly valid content -
321 correspondingly, this means that clients should quietly ignore things that they
322 do not know about in the XML - i.e. they should not automatically issue an error
323 if schema validation fails (though of course if they can determine that the
324 validation error is because of a missing required element then they should
325 issue an error).
326 \item In contrast servers must produce strictly valid documents, and service
327 validators must test strict validity against the relevant schema (discovered
328 from the namespace and the version element).
329 \end{itemize}
334 \appendix
337 \section{Changes from Previous Versions}
339 No previous versions yet.
340 % these would be subsections "Changes from v. WD-..."
341 % Use itemize environments.
344 \bibliography{ivoatex/ivoabib}
347 \end{document}


Name Value
svn:keywords Date Rev URL

ViewVC Help
Powered by ViewVC 1.1.26