Skip to content

Toro Cloud Dev Center


Creating and connecting to a custom search index

This guide will walk you through creating and linking your own local or remote Solr core (also known as a collection). The process is simple: configure your Solr server and configure Martini so that it has the necessary information to connect to your Solr server.

Prerequisites

Understanding the following topics is a prerequisite to this guide:

Procedure

  1. Set required solr-prefixed instance properties.

    • solr.mode

      This property will tell Martini where to look for your Solr core (or collection). You have the option to choose from three possible values:

      Mode Description
      local Tells Martini that it will be using an embedded Solr instance which uses the host's local file system to store indexes.
      remote Tells Martini to connect to a remote instance of Solr.
      cloud Tells Martini to connect to a SolrCloud instance.
  2. Create your Solr core or collection using configuration files.

    1. Prepare the following Solr configuration files for your Solr core:

      • solrconfig.xml
      • schema.xml
      • core.properties (optional for remote and SolrCloud instances)

      If you plan to use SolrCloud, we recommend prefixing your collections' names. The format should be <prefix>_<core_name>.

      You can do this by setting the name property via the Solr Admin UI or by editing the core.properties file:

      1
      name=<prefix>_<core_name>
      

      And more importantly, you also have to set the solr.core-prefix instance property.

    2. Create your Solr core or collection using the configuration files you have created above. The process will differ depending on which mode your Solr instance is running on:

      • Local

        1. Package your configuration files such that they follow the structure below:

          1
          2
          3
          4
          5
          <core>
          ├── conf
          │   └── schema.xml
          │   └── solrconfig.xml
          └── core.properties
          
        2. Place this core configuration directory under the target package1's solr directory.

          1
          2
          3
          4
          5
          6
          7
          8
          <package>
          ├── ...
          └── solr
              └── <core>
                  ├── conf
                  │   └── schema.xml
                  │   └── solrconfig.xml
                  └── core.properties
          
      • Remote

        1. Package your configuration files such that they follow the structure below:

          1
          2
          3
          4
          <core>
          └── conf
              └── schema.xml
              └── solrconfig.xml
          
        2. Add this core configuration folder to your Solr instance's configuration directory. Typically, this is in the <solr-home>/server/solr directory.

          1
          2
          3
          4
          5
          6
          <solr-home>/server/solr
          ├── ...
          └── <core>
              └── conf
                  └── schema.xml
                  └── solrconfig.xml
          
        3. Open the Solr admin page, click on the Core Admin tab, click on the Add Core button, and then complete and submit the form to add your Solr core.

          Filled-up form for adding a new Solr core

          The following table describes the fields of this form:

          Field Description
          name The name of the Solr core.
          instanceDir The name of the Solr core directory.
          dataDir The name of the directory which will store the index files.
          config The name of the Solr core XML configuration file.
          schema The name of the Solr core XML schema file.
      • Cloud

        1. Zip the configuration files solrconfig.xml and schema.xml together. They must be top-level entries in the archive.

          1
          2
          cd ${solr-home}/server/solr/configsets/${collection}/conf
          zip -r myConfigSet.zip *
          
        2. Upload the resulting .zip file as a ConfigSet via the Solr ConfigSets API to any Solr instance belonging to your target cluster.

          1
          2
          3
          4
          curl -X POST \
          --header "Content-Type:application/octet-stream" \
          --data-binary @myConfigSet.zip \
          "http://<host>:<port>/solr/admin/configs?action=UPLOAD&name=myConfigSet"
          
        3. Open the Solr admin page, click on the Collections tab, click on the Add Collection button, and then populate and submit the form to add your Solr collection.

          Filled-up form for adding a new Solr collection

          The following table describes the fields of this form:

          Field Description
          name The name of the Solr core.
          config set The name of the config set that will be used.
          numShards The number of shards to be created as part of the collection.
          replicationFactor The number of replicas to be created for each shard.
  3. Register your Solr core or collection in package.xml.

    In order to register a custom Solr core or collection to Martini, you must edit the package's1 package.xml file to add a new solr-core element under the solr-cores array element. This new solr-core entry will be used to identify your custom Solr core.

    1
    2
    3
    4
    5
    6
    7
    <package>
        <!-- ... -->
        <solr-cores>
            <solr-core name="<core_name>" enabled="true" />
            <!-- ... -->
        </solr-cores>
    </package>
    

    While we require the format <prefix>_<core_name> for Solr collections, the new solr-core element's name attribute should only be <core_name> without <prefix>_ prepended.

  4. Restart your Martini instance or just the affected Martini package, depending on the scope of your changes.

    If you have made modifications on any solr-prefixed instance property, then your Martini instance will require a restart to pick up the changes. Otherwise, a quick restart of your package will do the trick.

    If your registration was a success, you should have log messages similar to the ones below:

    1
    2
    12/03/18 15:04:05.114 INFO  [<LocalSolrClient|CloudSolrClient|RemoteSolrClient>] Starting core: '<core_name>'
    12/03/18 15:04:25.402 INFO  [<LocalSolrClient|CloudSolrClient|RemoteSolrClient>] Starting core: '<core_name>' completed
    

Gloop models in Solr

If you have a simple Gloop model that you would like to index in Solr, you can execute the io.toro.martini.SolrMethods.gloopModelToSolrSchema(...) service. This will create the required files for Solr that will get you indexing Gloop models in no time!


  1. Particulary the package which will use the custom core or collection.