MarkLogic - Custom Search Snippet

Question

I am using Roxy to manage my project. Also using MarkLogic 8.0-6.1

I am trying to submit a searchTerm, and return a custom formatted search:snippet

Here are the complete steps that I am taking:

./../roxy/ml new test-app --server-version=8 --app-type=rest

Configure my build.properties

cd test-app/ ./ml local bootstrap

Now I have my project Structure.

Create File - test-app/rest-api/ext/show-search.xqy

xquery version "1.0-ml";

module namespace ss = "http://marklogic.com/rest-api/resource/show-search";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";
import module namespace json = "http://marklogic.com/xdmp/json" at "/MarkLogic/json/json.xqy";




declare
function ss:get(
  $context as map:map,
  $params  as map:map
) as document-node()*
{

  map:put($context, "output-types", "application/json"),
  map:put($context, "output-status", (200, "OK")),

  let $search-term := map:get($params, "searchTerm")
  let $query := search:search($search-term,
      <options xmlns="http://marklogic.com/appservices/search">
        <transform-results apply="raw"/>
      </options>
      )

  return document {$query} 
};

(:
 :)
declare 
function ss:put(   
    $context as map:map,
    $params  as map:map,
    $input   as document-node()*
) as document-node()?
{
  map:put($context, "output-types", "application/xml"),
  map:put($context, "output-status", (201, "Created")),
  document { "PUT called on the ext service extension" }
};

(:
 :)
declare 
function ss:post(
    $context as map:map,
    $params  as map:map,
    $input   as document-node()*
) as document-node()*
{
  map:put($context, "output-types", "application/xml"),
  map:put($context, "output-status", (201, "Created")),
  document { "POST called on the ext service extension" }
};

(:
 :)
declare 
function ss:delete(
    $context as map:map,
    $params  as map:map
) as document-node()?
{
  map:put($context, "output-types", "application/xml"),
  map:put($context, "output-status", (200, "OK")),
  document { "DELETE called on the ext service extension" }
};

The above GET request uses the transform-results apply=raw option, deploys, and functions properly (I have some test documents).

However I do not want to return the whole document, I want to return a whole section of the json that had a match, no matter where in that seciton the match happened (lower levels)

So I try to write my own snipper

Create File - test-app/rest-api/ext/show-search-snipper.xqy

xquery version "1.0-ml";

module namespace sss = "http://marklogic.com/rest-api/resource/show-search-snipper";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";
import module namespace json = "http://marklogic.com/xdmp/json" at "/MarkLogic/json/json.xqy";

declare
function sss:my-snippet(
    $result as node(),
    $ctsquery as schema-element(cts:query),
    $options as element(search:transform-results)?
) as element(search:snippet)
{
    <search:snippet>

    </search:snippet>
};

I then update the search:search call to the following

  let $query := search:search($search-term,
      <options xmlns="http://marklogic.com/appservices/search">
        <transform-results apply="my-snippet" ns="http://marklogic.com/rest-api/resource/show-search-snipper" at="show-search-snipper.xqy"/>
      </options>
      )

Now I should have everything I need (I think)

I run the deploy ./ml local deploy rest

and get the following

Minty-linux test-app # ./ml local deploy rest Loading REST properties in /opt/this-is-a-test/test-app/rest-api/config/properties.xml Loading REST options in /opt/this-is-a-test/test-app/rest-api/config/options

Loading REST extensions from /opt/this-is-a-test/test-app/rest-api/ext

ERROR: 400 "Bad Request" ERROR: {"errorResponse":{"statusCode":400, "status":"Bad Request", "messageCode":"RESTAPI-INVALIDCONTENT", "message":"RESTAPI-INVALIDCONTENT: (err:FOER0000) Invalid content: invalid show-search-snipper extension: show-search-snipper either is not a valid module or does not provide extension functions (delete, get, put, post) in the http://marklogic.com/rest-api/resource/show-search-snipper namespace"}}

So I tried moving the show-search-snipper.xqy file up 1 level (to test-app/rest-api/show-search-snipper.xqy`

Run the deployment Deployment Works No errors Hit the URL and receive the following

500 Internal Server Error INTERNAL ERROR RESTAPI-INVALIDREQ: (err:FOER0000) Invalid request: reason: Extension show-search does not exist. . See the MarkLogic server error log for further detail.

Although I know the extension was created properly, since it worked fine before the introduction of the custom snip function. (with apply="raw")

Any thoughts how I can apply my custom snip function or what I am doing wrong in deployment?

---

Answer 1

Should you decide to stick with a custom snippeter:

It looks like Roxy is trying to treat it your snippeter module as a resource extension, which it is not. Your snippeter should be just a vanilla module in the modules db.

IDK how to configure Roxy, unfortunately, but what you're aiming for is to get Roxy to either install it using PUT /v1/ext/directories/asset or a straight up insert (`PUT /v1/documents) on your modules db. See http://docs.marklogic.com/REST/PUT/v1/ext/[directories]/[asset].

Assuming Roxy uses /ext, then the path to your snippeter would NOT be the unqualified path you have in your options. It would be an absolute path rooted at /ext/. See http://docs.marklogic.com/guide/rest-dev/search#id_72390.

Answer 2

There's a simpler way -- you can do this with the extract-document-data search option.

I set up some sample data to work with like this:

xquery version "1.0-ml";

for $i in (1 to 10)
return 
  xdmp:document-insert(
    '/test-content-' || $i || '.json',
    xdmp:unquote('{ "important": { "foo": 1, "bar": 2 }, "not-important": { "stuff": 3, "blah": 4 } }')
  )

Having bootstrapped and deployed modules, I can get search results at http://localhost:8040/v1/search. However, I can start taking more control of the search results by using stored search options. Take a look in your project at rest-api/config/options/all.xml. They were already deployed for you when you run ml local deploy modules, so you now can search using http://localhost:8040/v1/search?options=all. Since you're using JSON data, I took it one step further and ran with: http://localhost:8040/v1/search?format=json&options=all.

I added this to rest-api/config/options/all.xml:

<extract-document-data selected="include">
  <extract-path>/important</extract-path>
</extract-document-data>

This tells the search options to include the specified path with all search results. After deploying and running the search again, one result looks like this:

{
  "index":1, 
  "uri":"/test-content-6.json", 
  "path":"fn:doc(\"/test-content-6.json\")", 
  "score":0, 
  "confidence":0, 
  "fitness":0, 
  "href":"/v1/documents?uri=%2Ftest-content-6.json", 
  "mimetype":"application/json", 
  "format":"json", 
  "matches":[{"path":"fn:doc(\"/test-content-6.json\")/object-node()", "match-text":[]}], 
  "extracted":{
    "kind":"array", 
    "content":[
      {"important":{"foo":1, "bar":2}}
    ]
  }
}, 

Notice the "extracted" part at the end there -- I get the "important" JSON property, as specified in the options.

For a full list of options you can set to control search, see the Query Options Reference appendix.