Skip to content

Commit 56c005e

Browse files
committed
Update output.xml and Libdoc schema documentation.
Part of updating output.xml schema. (robotframework#3726)
1 parent c167ae2 commit 56c005e

File tree

2 files changed

+59
-81
lines changed

2 files changed

+59
-81
lines changed

atest/README.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,9 @@ directory in the project root and ``CLASSPATH`` is set automatically.
231231
Schema validation
232232
-----------------
233233

234+
output.xml schema
235+
~~~~~~~~~~~~~~~~~
236+
234237
Created output.xml has a `schema <../doc/schema>`_ that can be tested as part of
235238
acceptance tests. The schema is always used to validate selected outputs in
236239
`<robot/rebot/compatibility.robot>`_, but validating all outputs would slow down
@@ -240,6 +243,13 @@ It is, however, possible to enable validating all outputs by setting
240243
``ATEST_VALIDATE_OUTPUT`` environment variable to ``TRUE`` (case-insensitive).
241244
This is recommended especially if the schema is updated or output.xml changed.
242245

246+
Libdoc XML and JSON spec schemas
247+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
248+
249+
Libdoc can create spec files both in XML and JSON formats and they both have
250+
`schemas <../doc/schema>`_. All generated Libdoc specs are validated automatically
251+
in Libdoc tests.
252+
243253
Telnet tests
244254
------------
245255

doc/schema/README.rst

Lines changed: 49 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -1,95 +1,63 @@
1-
Robot Framework and Libdoc XML schemas
2-
======================================
1+
Robot Framework and Libdoc schema definitions
2+
=============================================
33

44
Introduction
55
------------
66

7-
While Robot Framework is running tests, it generates an XML output file
8-
containing all information about the execution. After execution is over it
9-
creates, by default, log and report files using the `Rebot tool`__
10-
internally. The same ``rebot`` functionality can also be used externally
11-
afterwards both as a standalone tool and programmatically__.
12-
13-
This document describes the format of the output file in high level and in the
14-
same folder there are detailed
15-
`XML schema definition <http://en.wikipedia.org/wiki/XML_Schema_(W3C)>`_ (XSD)
16-
files that can be used for validating that an XML file is Robot Framework
17-
compatible. The output file format can be useful both for people interested in
18-
parsing the output and for people interested to create Robot Framework
19-
compatible outputs.
7+
This directory contains schema definitions for Robot Frameworks output.xml files
8+
as well as for spec files created by Libdoc_.
209

21-
Also Robot Framework's library documentation tool Libdoc__ can generate output
22-
in XML and JSON format and this directory contains a schema documentation for it as
23-
well.
10+
output.xml schema
11+
-----------------
2412

25-
__ http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#rebot
13+
While Robot Framework is running tests, it generates an XML output file
14+
containing all information about the execution. After execution is over it
15+
creates, by default, log and report files based on the output.xml file.
16+
Logs and reports can be generated also afterwards both using the standalone
17+
Rebot_ tool and programmatically__. The output.xml file format can be useful
18+
both for people interested in parsing the output and for people interested
19+
to create Robot Framework compatible outputs.
20+
21+
This directory contains XSD_ schema definitions that are compatible with
22+
different Robot Framework versions. Newer output.xml files have ``schemaversion``
23+
attribute telling which version they support and older implicitly support schema
24+
version 1.
25+
26+
* `<robot.02.xsd>`__ - Compatible with Robot Framework >= 4.0.
27+
* `<robot.01.xsd>`__ - Compatible with Robot Framework < 4.0.
28+
29+
Due to XSD 1.1 not being widely adopted, these schema definitions use XSD 1.0.
30+
Newer schema definitions contain embedded documentation and comments explaining
31+
the structure in more detail. They also contain instructions how to make them
32+
XSD 1.1 compatible if needed.
33+
34+
.. _Rebot: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#rebot
2635
__ http://robot-framework.readthedocs.org/en/latest/autodoc/robot.html#robot.rebot.rebot
27-
__ http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#libdoc
28-
29-
Schema definitions
30-
------------------
31-
32-
Available schema files:
33-
34-
* `<robot-xsd10.xsd>`__ - Robot Framework XML output (XSD 1.0 compatible version)
35-
* `<robot-xsd11.xsd>`__ - Robot Framework XML output (XSD 1.1 compatible version)
36-
* `<libdoc.01.xsd>`__ - Libdoc XML spec version 1 (Robot Framework < 3.2) (XSD 1.0)
37-
* `<libdoc.02.xsd>`__ - Libdoc XML spec version 2 (Robot Framework == 3.2) (XSD 1.0)
38-
* `<libdoc.03.xsd>`__ - Libdoc XML spec version 3 (Robot Framework >= 4.0) (XSD 1.0)
39-
* `<libdoc_schema.json>`__ - Libdoc JSON schema (Robot Framework >= 4.0) (DRAFT-7__)
40-
41-
XSD 1.1 schemas are more complete than XSD 1.0 schemas but not as widely
42-
supported.
43-
44-
JSON Schema Draft-7 is used instead of the newer Draft 2019-09 due to missing Python
45-
validator implementations. See difference between Draft-7 and 2019-09 in Release-Nodes__.
46-
Robot Framework uses Python project `jsonschema`__ for validation in unit tests.
47-
48-
__ https://json-schema.org/draft/2019-09/json-schema-core.html
49-
__ https://json-schema.org/draft/2019-09/release-notes.html
50-
__ https://github.com/Julian/jsonschema
51-
36+
.. _XSD: http://en.wikipedia.org/wiki/XML_Schema_(W3C)
5237

53-
General Robot Framework XML output structure
54-
--------------------------------------------
38+
Libdoc schema
39+
-------------
5540

56-
These are the main elements in Robot Framework XML output files with descriptions of their
57-
sub-elements. Unless stated otherwise, all attributes are optional. Additionally
58-
``rebot`` does not care of the order of the XML elements, except for the order
59-
of suite, test, and kw elements. Starting from Robot Framework 2.9, empty
60-
elements and attributes are not written to the output XML. This means that,
61-
for example, test case having no documentation has no ``<doc>`` element either.
41+
Libdoc_ tool distributed with Robot Framework can generate machine readable spec files
42+
both in XML and JSON format. XML spec files have XSD_ 1.0 compatible schema definition
43+
and JSON spec schema is JSON Schema `DRAFT-7`__ compatible.
6244

63-
robot - root element
64-
* ``suite`` - root element always has one suite which contains the subsuites and tests
65-
* ``statistics`` - statistics contains statistics of the test run
66-
* ``errors`` - if there were any errors, they are listed in this element
45+
* `<libdoc.03.xsd>`__ - Compatible with Robot Framework >= 4.0.
46+
* `<libdoc.02.xsd>`__ - Compatible with Robot Framework == 3.2.
47+
* `<libdoc.01.xsd>`__ - Compatible with Robot Framework < 3.2.
48+
* `<libdoc_schema.json>`__ - Compatible with Robot Framework >= 4.0.
6749

68-
suite - suite element, name is given as an attribute
69-
* ``kw`` - suite can have two kw elements: setup and teardown, both are optional
70-
* ``suite`` - any number of sub suites in execution order
71-
* ``test`` - any number of tests in execution order
72-
* ``doc`` - optional documentation element
73-
* ``metadata`` - optional suite metadata
74-
* ``status`` - suite has to have a status element
50+
.. _Libdoc: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#libdoc
51+
__ https://json-schema.org/specification-links.html#draft-7
7552

76-
test - test element, name is given as an attribute
77-
* ``kw`` - keywords of the test in execution order
78-
* ``msg`` - optional test message
79-
* ``doc`` - optional test documentation
80-
* ``tags`` - optional test tags
81-
* ``timeout`` - optional test timeout. Before 3.0 this was an attribute.
82-
* ``status`` - test has to have a status
53+
Testing schemas
54+
---------------
8355

84-
kw - keyword element, name is given as an attribute. Type attribute describes the type of keyword. If this attribute is not present, type is assumed to be ``kw``.
85-
* ``tags`` - optional keyword tags (new in 2.9)
86-
* ``doc`` - optional keyword documentation
87-
* ``arguments`` - optional keyword arguments
88-
* ``assignment`` - possible assignment of keyword's return values to variables, each variable in ``var`` subelement (new in 2.9)
89-
* ``kw`` - possible sub-keywords in execution order
90-
* ``msg`` - any number of optional keyword messages
91-
* ``timeout`` - optional keyword timeout. Before 3.0 this was an attribute.
92-
* ``status`` - keyword has to have a status
93-
* ``sourcename`` - original name of the keyword when using embedded arguments, not set otherwise. New in 4.0.
56+
Both output.xml and Libdoc schema definitions are tested as part of `acceptance test
57+
runs <../../atest/README.rst>`__ by validating created outputs against the appropriate
58+
schemas. Most output.xml files created during test runs are not validated, however,
59+
because that would slow down test execution a bit too much. Full validation `can be
60+
enabled separately`__ and that should be done if the schema is updated or output.xml
61+
structure is changed.
9462

95-
For more details and full list of elements and attributes, please see the XML schema files above.
63+
__ https://github.com/robotframework/robotframework/blob/master/atest/README.rst#schema-validation

0 commit comments

Comments
 (0)