Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Commit

Permalink
Violations of pre-binding syntax rules now cause failures, added corr…
Browse files Browse the repository at this point in the history 
…esponding test cases
  • Loading branch information
@HolgerKnublauch
HolgerKnublauch committed May 4, 2017
1 parent 1a12b59 commit c97ea5e
Show file tree
Hide file tree
Showing 13 changed files with 479 additions and 23 deletions.
35 changes: 27 additions & 8 deletions data-shapes-test-suite/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ <h3 id="Validate">Validate RDF data</h3>
</p>
<p>
To pass <a>full compliance</a>, the <code>actual</code> validation report must be <a href="https://www.w3.org/TR/rdf11-concepts/#graph-isomorphism">isomorphic</a> to <code>expected</code>.
If the test case has <code>sht:Failure</code> as its value for <code>mf:result</code> then the test is passed if the validation reported a failure.
Alternatively, implementation may still report pass if they have manually checked that their validation reports satisfy the requirements specified by the SHACL specification.
The latter also applies to scenarios that are well-formed SHACL validation results according to [[!shacl]] while the
test framework has stricter expectations. For example, not all implementation are required to produce <code>sh:sourceShape</code> triples.
Expand Down Expand Up @@ -304,13 +305,13 @@ <h3>Validate RDF Data Tests</h3>
<th>File</th><th>Status</th><th>Test Case</th><th><a href="#Corese">Corese</a><th><a href="#Netage">Netage</a></th><th><a href="#TopBraid">TopBraid</a></th>
</tr>
<tr>
<td></td><td></td><td></td><td class="percent">90 / 113 (80%)</td><td class="percent">93 / 113 (83%)</td><td class="percent">113 / 113 (100%)</td>
<td></td><td></td><td></td><td class="percent">97 / 119 (82%)</td><td class="percent">93 / 119 (79%)</td><td class="percent">119 / 119 (100%)</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/complex/personexample.ttl">core/complex/personexample</a></td><td class="status-td">proposed</td><td class="label-td">Test of personexample</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/complex/shacl-shacl.ttl">core/complex/shacl-shacl</a></td><td class="status-td">proposed</td><td class="label-td">frozen eat your own ( eat your own frozen dogfood )</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/complex/shacl-shacl.ttl">core/complex/shacl-shacl</a></td><td class="status-td">proposed</td><td class="label-td">frozen eat your own ( eat your own frozen dogfood )</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/misc/deactivated-001.ttl">core/misc/deactivated-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:deactivated 001</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
Expand Down Expand Up @@ -382,10 +383,10 @@ <h3>Validate RDF Data Tests</h3>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minInclusive-001.ttl">core/node/minInclusive-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minInclusive at node shape 001</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minInclusive-002.ttl">core/node/minInclusive-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minInclusive at node shape 002 - dateTime with timezone</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minInclusive-002.ttl">core/node/minInclusive-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minInclusive at node shape 002 - dateTime with timezone</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minInclusive-003.ttl">core/node/minInclusive-003</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minInclusive at node shape 003 - dateTime without timezone</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minInclusive-003.ttl">core/node/minInclusive-003</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minInclusive at node shape 003 - dateTime without timezone</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/minLength-001.ttl">core/node/minLength-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:minLength at node shape 001</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
Expand All @@ -412,7 +413,7 @@ <h3>Validate RDF Data Tests</h3>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/pattern-002.ttl">core/node/pattern-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:pattern at node shape 002</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/qualified-001.ttl">core/node/qualified-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of qualified parameters allowed in node shapes</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/qualified-001.ttl">core/node/qualified-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of qualified parameters allowed in node shapes</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/xone-001.ttl">core/node/xone-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:xone at node shape 001</td><td class="passed">passed</td><td class="passed">passed</td><td class="passed">passed</td>
Expand All @@ -427,7 +428,7 @@ <h3>Validate RDF Data Tests</h3>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-complex-001.ttl">core/path/path-complex-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of path complex (rdf:type/rdfs:subClassOf*) 001</td><td class="passed">passed</td><td class="failed">failed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-complex-002.ttl">core/path/path-complex-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of complex path validation results</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-complex-002.ttl">core/path/path-complex-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of complex path validation results</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-inverse-001.ttl">core/path/path-inverse-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of path sh:inversePath 001</td><td class="passed">passed</td><td class="failed">failed</td><td class="passed">passed</td>
Expand All @@ -445,10 +446,10 @@ <h3>Validate RDF Data Tests</h3>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-sequence-duplicate-001.ttl">core/path/path-sequence-duplicate-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of path sequence with duplicate 001</td><td class="passed">passed</td><td class="failed">failed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-strange-001.ttl">core/path/path-strange-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of strange path 001 two valid paths together</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-strange-001.ttl">core/path/path-strange-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of strange path 001 two valid paths together</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-strange-002.ttl">core/path/path-strange-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of strange path 002 valid and invalid paths together</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-strange-002.ttl">core/path/path-strange-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of strange path 002 valid and invalid paths together</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/path/path-unused-001.ttl">core/path/path-unused-001</a></td><td class="status-td">proposed</td><td class="label-td">Test with unused ill-formed path</td><td class="passed">passed</td><td class="nodata">no data</td><td class="passed">passed</td>
Expand Down Expand Up @@ -642,6 +643,24 @@ <h3>Validate RDF Data Tests</h3>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/pre-binding-006.ttl">sparql/pre-binding/pre-binding-006</a></td><td class="status-td">proposed</td><td class="label-td">Test of pre-binding in nested SELECT</td><td class="nodata">no data</td><td class="passed">passed</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-001.ttl">sparql/pre-binding/unsupported-sparql-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of unsupported MINUS</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-002.ttl">sparql/pre-binding/unsupported-sparql-002</a></td><td class="status-td">proposed</td><td class="label-td">Test of unsupported VALUES</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-003.ttl">sparql/pre-binding/unsupported-sparql-003</a></td><td class="status-td">proposed</td><td class="label-td">Test of unsupported SERVICE</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-004.ttl">sparql/pre-binding/unsupported-sparql-004</a></td><td class="status-td">proposed</td><td class="label-td">Test of unsupported SELECT</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-005.ttl">sparql/pre-binding/unsupported-sparql-005</a></td><td class="status-td">proposed</td><td class="label-td">Test of unsupported AS ?prebound</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/pre-binding/unsupported-sparql-006.ttl">sparql/pre-binding/unsupported-sparql-006</a></td><td class="status-td">proposed</td><td class="label-td">Test of ASK trying to reassign ?value</td><td class="nodata">no data</td><td class="nodata">no data</td><td class="passed">passed</td>
</tr>
<tr>
<td class="file-td"><a href="https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/sparql/property/sparql-001.ttl">sparql/property/sparql-001</a></td><td class="status-td">proposed</td><td class="label-td">Test of sh:sparql at property shape 001</td><td class="nodata">no data</td><td class="passed">passed</td><td class="passed">passed</td>
</tr>
Expand Down
30 changes: 30 additions & 0 deletions data-shapes-test-suite/reports/alltests.ttl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -560,6 +560,36 @@
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/pre-binding-006.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-001>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of unsupported MINUS" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-001.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-002>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of unsupported VALUES" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-002.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-003>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of unsupported SERVICE" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-003.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-004>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of unsupported SELECT" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-004.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-005>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of unsupported AS ?prebound" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-005.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/pre-binding/unsupported-sparql-006>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of ASK trying to reassign ?value" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/pre-binding/unsupported-sparql-006.ttl" ;
<http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#status> <http://www.w3.org/ns/shacl-test#proposed> ;
.
<urn:x-shacl-test:/sparql/property/sparql-001>
<http://www.w3.org/2000/01/rdf-schema#label> "Test of sh:sparql at property shape 001" ;
<http://www.w3.org/2000/01/rdf-schema#seeAlso> "sparql/property/sparql-001.ttl" ;
Expand Down
Loading

0 comments on commit c97ea5e

Please sign in to comment.