|
| 1 | +--- |
| 2 | +layout: page |
| 3 | +title: "Elasticsearch Interpreter" |
| 4 | +description: "" |
| 5 | +group: manual |
| 6 | +--- |
| 7 | +{% include JB/setup %} |
| 8 | + |
| 9 | + |
| 10 | +## Elasticsearch Interpreter for Apache Zeppelin |
| 11 | + |
| 12 | +### 1. Configuration |
| 13 | + |
| 14 | +<br/> |
| 15 | +<table class="table-configuration"> |
| 16 | + <tr> |
| 17 | + <th>Property</th> |
| 18 | + <th>Default</th> |
| 19 | + <th>Description</th> |
| 20 | + </tr> |
| 21 | + <tr> |
| 22 | + <td>elasticsearch.cluster.name</td> |
| 23 | + <td>elasticsearch</td> |
| 24 | + <td>Cluster name</td> |
| 25 | + </tr> |
| 26 | + <tr> |
| 27 | + <td>elasticsearch.host</td> |
| 28 | + <td>localhost</td> |
| 29 | + <td>Host of a node in the cluster</td> |
| 30 | + </tr> |
| 31 | + <tr> |
| 32 | + <td>elasticsearch.port</td> |
| 33 | + <td>9300</td> |
| 34 | + <td>Connection port <b>(important: this is not the HTTP port, but the transport port)</b></td> |
| 35 | + </tr> |
| 36 | + <tr> |
| 37 | + <td>elasticsearch.result.size</td> |
| 38 | + <td>10</td> |
| 39 | + <td>The size of the result set of a search query</td> |
| 40 | + </tr> |
| 41 | +</table> |
| 42 | + |
| 43 | +<center> |
| 44 | +  |
| 45 | +</center> |
| 46 | + |
| 47 | + |
| 48 | +> Note #1: you can add more properties to configure the Elasticsearch client. |
| 49 | +
|
| 50 | +> Note #2: if you use Shield, you can add a property named `shield.user` with a value containing the name and the password (format: `username:password`). For more details about Shield configuration, consult the [Shield reference guide](https://www.elastic.co/guide/en/shield/current/_using_elasticsearch_java_clients_with_shield.html). Do not forget, to copy the shield client jar in the interpreter directory (`ZEPPELIN_HOME/interpreters/elasticsearch`). |
| 51 | +
|
| 52 | + |
| 53 | +<hr/> |
| 54 | + |
| 55 | +### 2. Enabling the Elasticsearch Interpreter |
| 56 | + |
| 57 | +In a notebook, to enable the **Elasticsearch** interpreter, click the **Gear** icon and select **Elasticsearch**. |
| 58 | + |
| 59 | + |
| 60 | +<hr/> |
| 61 | + |
| 62 | + |
| 63 | +### 3. Using the Elasticsearch Interpreter |
| 64 | + |
| 65 | +In a paragraph, use `%elasticsearch` to select the Elasticsearch interpreter and then input all commands. To get the list of available commands, use `help`. |
| 66 | + |
| 67 | +```bash |
| 68 | +| %elasticsearch |
| 69 | +| help |
| 70 | +Elasticsearch interpreter: |
| 71 | +General format: <command> /<indices>/<types>/<id> <option> <JSON> |
| 72 | + - indices: list of indices separated by commas (depends on the command) |
| 73 | + - types: list of document types separated by commas (depends on the command) |
| 74 | +Commands: |
| 75 | + - search /indices/types <query> |
| 76 | + . indices and types can be omitted (at least, you have to provide '/') |
| 77 | + . a query is either a JSON-formatted query, nor a lucene query |
| 78 | + - size <value> |
| 79 | + . defines the size of the result set (default value is in the config) |
| 80 | + . if used, this command must be declared before a search command |
| 81 | + - count /indices/types <query> |
| 82 | + . same comments as for the search |
| 83 | + - get /index/type/id |
| 84 | + - delete /index/type/id |
| 85 | + - index /ndex/type/id <json-formatted document> |
| 86 | + . the id can be omitted, elasticsearch will generate one |
| 87 | +``` |
| 88 | +
|
| 89 | +> Tip: use (CTRL + .) for completion |
| 90 | +
|
| 91 | +
|
| 92 | +#### get |
| 93 | +With the `get` command, you can find a document by id. The result is a JSON document. |
| 94 | +
|
| 95 | +```bash |
| 96 | +| %elasticsearch |
| 97 | +| get /index/type/id |
| 98 | +``` |
| 99 | +
|
| 100 | +Example: |
| 101 | + |
| 102 | +
|
| 103 | +
|
| 104 | +#### search |
| 105 | +With the `search` command, you can send a search query to Elasticsearch. There are two formats of query: |
| 106 | +* You can provide a JSON-formatted query, that is exactly what you provide when you use the REST API of Elasticsearch. |
| 107 | + * See [Elasticsearch search API reference document](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html) for more details about the content of the search queries. |
| 108 | +* You can also provide the content of a `query_string` |
| 109 | + * This is a shortcut to a query like that: `{ "query": { "query_string": { "query": "__HERE YOUR QUERY__", "analyze_wildcard": true } } }` |
| 110 | + * See [Elasticsearch query string syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax) for more details about the content of such a query. |
| 111 | +
|
| 112 | +```bash |
| 113 | +| %elasticsearch |
| 114 | +| search /index1,index2,.../type1,type2,... <JSON document containing the query or query_string elements> |
| 115 | +``` |
| 116 | +
|
| 117 | +If you want to modify the size of the result set, you can add a line that is setting the size, before your search command. |
| 118 | +
|
| 119 | +```bash |
| 120 | +| %elasticsearch |
| 121 | +| size 50 |
| 122 | +| search /index1,index2,.../type1,type2,... <JSON document containing the query or query_string elements> |
| 123 | +``` |
| 124 | +
|
| 125 | +
|
| 126 | +Examples: |
| 127 | +* With a JSON query: |
| 128 | +```bash |
| 129 | +| %elasticsearch |
| 130 | +| search / { "query": { "match_all": {} } } |
| 131 | + |
| 132 | +| %elasticsearch |
| 133 | +| search /logs { "query": { "query_string": { "query": "request.method:GET AND status:200" } } } |
| 134 | +``` |
| 135 | +
|
| 136 | +* With query_string elements: |
| 137 | +```bash |
| 138 | +| %elasticsearch |
| 139 | +| search /logs request.method:GET AND status:200 |
| 140 | + |
| 141 | +| %elasticsearch |
| 142 | +| search /logs (404 AND (POST OR DELETE)) |
| 143 | +``` |
| 144 | +
|
| 145 | +> **Important**: a document in Elasticsearch is a JSON document, so it is hierarchical, not flat as a row in a SQL table. |
| 146 | +For the Elastic interpreter, the result of a search query is flattened. |
| 147 | +
|
| 148 | +Suppose we have a JSON document: |
| 149 | +```json |
| 150 | +{ |
| 151 | + "date": "2015-12-08T21:03:13.588Z", |
| 152 | + "request": { |
| 153 | + "method": "GET", |
| 154 | + "url": "/zeppelin/4cd001cd-c517-4fa9-b8e5-a06b8f4056c4", |
| 155 | + "headers": [ "Accept: *.*", "Host: apache.org"] |
| 156 | + }, |
| 157 | + "status": "403" |
| 158 | +} |
| 159 | +``` |
| 160 | +
|
| 161 | +The data will be flattened like this: |
| 162 | +
|
| 163 | +date | request.headers[0] | request.headers[1] | request.method | request.url | status |
| 164 | +-----|--------------------|--------------------|----------------|-------------|------- |
| 165 | +2015-12-08T21:03:13.588Z | Accept: \*.\* | Host: apache.org | GET | /zeppelin/4cd001cd-c517-4fa9-b8e5-a06b8f4056c4 | 403 |
| 166 | +
|
| 167 | +
|
| 168 | +Examples: |
| 169 | +* With a table containing the results: |
| 170 | + |
| 171 | +
|
| 172 | +
|
| 173 | +* You can also use a predefined diagram: |
| 174 | + |
| 175 | +
|
| 176 | +* With a JSON query: |
| 177 | + |
| 178 | +
|
| 179 | +* With a query string: |
| 180 | + |
| 181 | +
|
| 182 | +
|
| 183 | +#### count |
| 184 | +With the `count` command, you can count documents available in some indices and types. You can also provide a query. |
| 185 | +
|
| 186 | +```bash |
| 187 | +| %elasticsearch |
| 188 | +| count /index1,index2,.../type1,type2,... <JSON document containing the query OR a query string> |
| 189 | +``` |
| 190 | +
|
| 191 | +Examples: |
| 192 | +* Without query: |
| 193 | + |
| 194 | +
|
| 195 | +* With a query: |
| 196 | + |
| 197 | +
|
| 198 | +
|
| 199 | +#### index |
| 200 | +With the `index` command, you can insert/update a document in Elasticsearch. |
| 201 | +```bash |
| 202 | +| %elasticsearch |
| 203 | +| index /index/type/id <JSON document> |
| 204 | + |
| 205 | +| %elasticsearch |
| 206 | +| index /index/type <JSON document> |
| 207 | +``` |
| 208 | +
|
| 209 | +#### delete |
| 210 | +With the `delete` command, you can delete a document. |
| 211 | +
|
| 212 | +```bash |
| 213 | +| %elasticsearch |
| 214 | +| delete /index/type/id |
| 215 | +``` |
| 216 | +
|
| 217 | +
|
| 218 | +
|
| 219 | +#### Apply Zeppelin Dynamic Forms |
| 220 | +
|
| 221 | +You can leverage [Zeppelin Dynamic Form]({{BASE_PATH}}/manual/dynamicform.html) inside your queries. You can use both the `text input` and `select form` parameterization features |
| 222 | +
|
| 223 | +```bash |
| 224 | +%elasticsearch |
| 225 | +size ${limit=10} |
| 226 | +search /index/type { "query": { "match_all": {} } } |
| 227 | +``` |
| 228 | +
|
0 commit comments