If you have ever worked with Elasticsearch, then you are probably
familiar with one of the most important features of Elasticsearch - the
[Dynamic Field Mapping]:
By default, when a previously unseen field is found in a document,
Elasticsearch will add the new field to the type mapping.
Therefore, if you need to index documents with high similarity between
their field names (dense data), including field types, then this default
behaviour may be exactly what you want. However, if your documents have
high variation of field names (sparse data), or have same field names
but with different types, then you will need a different approach.
In this post I would like to show you how to create an Elasticsearch
index that can be used to index arbitrary JSON data, including data with
nested arrays and objects. All this, without exploding the index type
mapping with arbitrary properties originating from the indexed data.
Nevertheless, the indexed data will still be searchable by any of its
fields, including any nested fields by specifying their path in a “dot”
notation format. In addition, I will show how to use Elasticsearch
aggregations to fetch a list of all of the available fields and their
respective types of the indexed data.
Indexing and Searching Arbitrary Data in Elasticsearch