PolicyX file format documentation

Table of Contents

1. Overview

PolicyX is an XML file format for specifying POMDP and MOMDP[1] policy computed by an offline solver. PolicyX can specify alpha vector policy or policy graph (not implemented yet).

2. PolicyX Tutorial

The purpose of this section is to provide a tutorial-like approach to using the PolicyX format.

2.1. Example PolicyX document

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <Policy version="0.1" type="value" model="RockSampleSimple.pomdpx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="policyx.xsd">
    <AlphaVector vectorLength="2" numObsValue="3" numVectors="6">
        <Vector action="1" obsValue="0">9.5 9.5 </Vector>
        <Vector action="3" obsValue="0">19.025 -0.975 </Vector>
        <Vector action="2" obsValue="0">18.0737 9.025 </Vector>
        <Vector action="1" obsValue="1">10 10 </Vector>
        <Vector action="0" obsValue="1">17.1701 8.57375 </Vector>
        <Vector action="3" obsValue="2">0 0 </Vector>
    </AlphaVector>
 </Policy>

2.2. File format structure

The first line of the document is an XML processing instruction which defines that the document adheres to the XML 1.0 standard and that the encoding of the document is ISO-8859-1. Other encodings such as UTF-8 are also possible. It is followed by the root element <Policy> tag that specifies the policy.

Example

  <?xml version="1.0" encoding="ISO-8859-1"?>

2.2.1. <Policy> Tag

The second line of the document is the <Policy> tag, the root element of the PolicyX file. The <Policy> tag has the following attributes:

• version - version of PolicyX
• type - type of the policy. Either "value" for alpha vector policy or "graph" for policy graph
• model - Optional. Stores the file name of the model that this policy solves.
• xsi:noNamespaceSchemaLocation - The location of the XML schema for PolicyX file format, policyx.xsd. PolicyX documents should be validated with this schema to ensure correctness.

The <Policy> tag has either an alpha vector policy specified by <AlphaVector> tag or a policy graph specified by <DAG> tag as its child element.

Example

 <Policy version="0.1" type="value" model="RockSampleSimple.pomdpx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="policyx.xsd">

2.2.2. <AlphaVector> Tag

This tag contains the pool of alpha vectors that form the policy as child elements. Its child elements can be either <Vector> tag or <SparseVector> tag. A mixture of both <Vector> tag and <SparseVector> tag is allowed. The <AlphaVector> tag has the following attributes:

• vectorLength - The number of entries in each vector.
• numObsValue - The number of observed states for (MOMDP) mixed observability problems or "1" for POMDP problems.
• numVectors - The total number of alpha vectors in this policy. This attribute is optional.

Example

 <AlphaVector vectorLength="2" numObsValue="3" numVectors="6">
        <Vector action="1" obsValue="0">9.5 9.5 </Vector>
        <Vector action="3" obsValue="0">19.025 -0.975 </Vector>
        ...
        <Vector action="3" obsValue="2">0 0 </Vector>
 </AlphaVector>

2.2.3. <Vector> Tag

The <Vector> tag specifies a single alpha vector as a list of floating point numbers within the tag. The number of floating point entries in the tag must be the same as vectorLength specified earlier in the <AlphaVector> tag. On top of the floating point entries, the following attributes also need to be specified:

• action - the action number associated with this alpha vector
• obsValue - the observed state number associated with this alpha vector or '0' if the problem is POMDP.

Example

 <Vector action="2" obsValue="0">18.0737 9.025 </Vector>

2.2.4. <SparseVector> Tag

This tag specifies a single alpha vector as a list of index, value pair in its children elements. Entries whose indice are not specified within the tag are assumed to be zero. This is useful when many of the entries in the alpha vector are zero. The <SparseVector> tag has the same attributes as <Vector> tag.

Example

  <SparseVector action="1" obsValue="0">
            <Entry>0 9.5</Entry>
            <Entry>1 9.5</Entry>
  </SparseVector>

2.2.5. <Entry> Tag

This tag is the child element of <SparseVector> tag. Within the <Entry> tag, it contains index and value of an entry in an alpha vector.

Example

 <Entry>1 9.5</Entry>

2.2.6. <DAG> Tag

This tag specifies the policy graph. It is not implemented yet.

3. References

[1] S.C.W. Ong, S.W. Png, D. Hsu, and W.S. Lee. POMDPs for robotic tasks with mixed observability. In Proc. Robotics: Science and Systems, 2009.

4. Appendix A

Policy file for RockSample problem with alpha vectors specified in dense format using <Vector> tags.

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <Policy version="0.1" type="value" model="RockSampleSimple.pomdpx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="policyx.xsd">
    <AlphaVector vectorLength="2" numObsValue="3" numVectors="6">
        <Vector action="1" obsValue="0">9.5 9.5 </Vector>
        <Vector action="3" obsValue="0">19.025 -0.975 </Vector>
        <Vector action="2" obsValue="0">18.0737 9.025 </Vector>
        <Vector action="1" obsValue="1">10 10 </Vector>
        <Vector action="0" obsValue="1">17.1701 8.57375 </Vector>
        <Vector action="3" obsValue="2">0 0 </Vector>
    </AlphaVector>
 </Policy>

5. Appendix B

Policy file for RockSample problem with alpha vectors specified in sparse format using <SparseVector> tags.

  <?xml version="1.0" encoding="ISO-8859-1"?>
  <Policy version="0.1" type="value" model="RockSampleSimple.pomdpx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="policyx.xsd">
    <AlphaVector vectorLength="2" numObsValue="3" numVectors="6">
        <SparseVector action="1" obsValue="0">
            <Entry>0 9.5</Entry>
            <Entry>1 9.5</Entry>
        </SparseVector>
        <SparseVector action="3" obsValue="0">
            <Entry>0 19.025</Entry>
            <Entry>1 -0.975</Entry>
        </SparseVector>
        <SparseVector action="2" obsValue="0">
            <Entry>0 18.0737</Entry>
            <Entry>1 9.025</Entry>
        </SparseVector>
        <SparseVector action="1" obsValue="1">
            <Entry>0 10</Entry>
            <Entry>1 10</Entry>
        </SparseVector>
        <SparseVector action="0" obsValue="1">
            <Entry>0 17.1701</Entry>
            <Entry>1 8.57375</Entry>
        </SparseVector>
        <SparseVector action="3" obsValue="2" />
    </AlphaVector>
  </Policy>