Inconsistent behavior of Elasticsearch with multiple must_not condition
Multi tool use
I am working on NodeJs based API and it is build with Elasticsearch and NodeJs.
I have to provide a functionality where API users can exclude some content based on the taxonomy terms.
freeform: Multi-value field - type : string
games: Multi-value field - type : string
players: Multi-value field - type : string
teams: Multi-value field - type : string
writer: Multi-value field - type : string
channels: Multi-value field - type : string
Format of API url will be something similar to below.
https://api.test.net/2/xxxxx/article/?exclude=term1+term2
Above should return the content which is not tagged to term1 and term2.
Suppose we have 3 articles, first article is tagged to term1, second article is tagged to term2 and third article is tagged to term1 and term2.
Expected Result:
https://api.test.net/2/xxxxx/article/?exclude=term1+term2 should return article first(tagged to term1), second article(tagged to term2) and not the third(Tagged to term1 and term2)
Actual Result:
Sometimes https://api.test.net/2/xxxxx/article/?exclude=term1+term2 returns all three articles and sometime first(tagged to term1) and second article(tagged to term2).
Update:
Below is the correct doc and I have added the query also but still I am seeing the inconsistency.
Doc:
{
"team":{
"mappings":{
"article":{
"properties":{
"body":{
"properties":{
"content":{
"type":"string"
},
"html":{
"type":"string"
},
"type":{
"type":"string"
}
}
},
"brand":{
"type":"string"
},
"changed":{
"type":"date",
"format":"dateOptionalTime"
},
"changedUtc":{
"type":"long"
},
"content":{
"type":"string"
},
"created":{
"type":"date",
"format":"dateOptionalTime"
},
"createdUtc":{
"type":"long"
},
"domain":{
"type":"string"
},
"headline":{
"type":"string"
},
"langcode":{
"type":"string",
"index":"not_analyzed"
},
"nid":{
"type":"string"
},
"published":{
"type":"date",
"format":"dateOptionalTime"
},
"publishedUtc":{
"type":"long"
},
"raw":{
"type":"string",
"index":"no"
},
"revisionTimestamp":{
"type":"date",
"format":"dateOptionalTime"
},
"revisionTimestampUtc":{
"type":"long"
},
"status":{
"type":"string"
},
"subheadline":{
"type":"string"
},
"syndication":{
"properties":{
"brand":{
"type":"string"
},
"options":{
"properties":{
"actionFrom":{
"type":"string"
},
"publish":{
"type":"string"
},
"status":{
"type":"string"
}
}
},
"type":{
"type":"string"
}
}
},
"taxonomy":{
"properties":{
"coaches":{
"properties":{
"value":{
"type":"string"
}
}
},
"freeform":{
"properties":{
"value":{
"type":"string"
}
}
},
"games":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"players":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"section":{
"properties":{
"value":{
"type":"string"
}
}
},
"teams":{
"properties":{
"city":{
"type":"string"
},
"id":{
"type":"string"
},
"nickname":{
"type":"string"
},
"tricode":{
"type":"string"
},
"urlName":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"writer":{
"properties":{
"emailAddress":{
"type":"string"
},
"id":{
"type":"string"
},
"responsive_web":{
"type":"string"
},
"title":{
"type":"string"
},
"value":{
"type":"string"
}
}
}
}
},
"teaser":{
"type":"string"
},
"title":{
"type":"string"
},
"type":{
"type":"string"
},
"url":{
"type":"string",
"index":"not_analyzed"
},
"uuid":{
"type":"string",
"index":"not_analyzed"
},
"vid":{
"type":"string"
}
}
}
}
}
}
Query:
{
"filter":{
"bool":{
"should":[
[
{
"term":{
"status":1
}
}
]
],
"must":[
]
}
},
"from":0,
"size":10,
"_source":{
"include":[
"uuid",
"nid",
"type",
"title",
"headline",
"shortHeadline",
"teaser",
"url",
"published",
"changed",
"subheadline",
"listImage",
"brand",
"videoId",
"videoSource",
"duration",
"taxonomy",
"includeTerms",
"excludeTerms",
"media",
"credit",
"caption",
"description",
"videoCaptions",
"franchiseId",
"showAirTime",
"slateImage",
"mainImage",
"thumbnailImage",
"showTitle",
"status",
"pubstatus"
]
},
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"langcode:"en""
}
},
{
"bool":{
"must_not":{
"query_string":{
"query":"(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
}
]
}
},
"sort":[
{
"publishedUtc":{
"order":"desc"
}
},
{
"_score":{
"order":"desc"
}
}
]
}
elasticsearch
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
add a comment |
I am working on NodeJs based API and it is build with Elasticsearch and NodeJs.
I have to provide a functionality where API users can exclude some content based on the taxonomy terms.
freeform: Multi-value field - type : string
games: Multi-value field - type : string
players: Multi-value field - type : string
teams: Multi-value field - type : string
writer: Multi-value field - type : string
channels: Multi-value field - type : string
Format of API url will be something similar to below.
https://api.test.net/2/xxxxx/article/?exclude=term1+term2
Above should return the content which is not tagged to term1 and term2.
Suppose we have 3 articles, first article is tagged to term1, second article is tagged to term2 and third article is tagged to term1 and term2.
Expected Result:
https://api.test.net/2/xxxxx/article/?exclude=term1+term2 should return article first(tagged to term1), second article(tagged to term2) and not the third(Tagged to term1 and term2)
Actual Result:
Sometimes https://api.test.net/2/xxxxx/article/?exclude=term1+term2 returns all three articles and sometime first(tagged to term1) and second article(tagged to term2).
Update:
Below is the correct doc and I have added the query also but still I am seeing the inconsistency.
Doc:
{
"team":{
"mappings":{
"article":{
"properties":{
"body":{
"properties":{
"content":{
"type":"string"
},
"html":{
"type":"string"
},
"type":{
"type":"string"
}
}
},
"brand":{
"type":"string"
},
"changed":{
"type":"date",
"format":"dateOptionalTime"
},
"changedUtc":{
"type":"long"
},
"content":{
"type":"string"
},
"created":{
"type":"date",
"format":"dateOptionalTime"
},
"createdUtc":{
"type":"long"
},
"domain":{
"type":"string"
},
"headline":{
"type":"string"
},
"langcode":{
"type":"string",
"index":"not_analyzed"
},
"nid":{
"type":"string"
},
"published":{
"type":"date",
"format":"dateOptionalTime"
},
"publishedUtc":{
"type":"long"
},
"raw":{
"type":"string",
"index":"no"
},
"revisionTimestamp":{
"type":"date",
"format":"dateOptionalTime"
},
"revisionTimestampUtc":{
"type":"long"
},
"status":{
"type":"string"
},
"subheadline":{
"type":"string"
},
"syndication":{
"properties":{
"brand":{
"type":"string"
},
"options":{
"properties":{
"actionFrom":{
"type":"string"
},
"publish":{
"type":"string"
},
"status":{
"type":"string"
}
}
},
"type":{
"type":"string"
}
}
},
"taxonomy":{
"properties":{
"coaches":{
"properties":{
"value":{
"type":"string"
}
}
},
"freeform":{
"properties":{
"value":{
"type":"string"
}
}
},
"games":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"players":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"section":{
"properties":{
"value":{
"type":"string"
}
}
},
"teams":{
"properties":{
"city":{
"type":"string"
},
"id":{
"type":"string"
},
"nickname":{
"type":"string"
},
"tricode":{
"type":"string"
},
"urlName":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"writer":{
"properties":{
"emailAddress":{
"type":"string"
},
"id":{
"type":"string"
},
"responsive_web":{
"type":"string"
},
"title":{
"type":"string"
},
"value":{
"type":"string"
}
}
}
}
},
"teaser":{
"type":"string"
},
"title":{
"type":"string"
},
"type":{
"type":"string"
},
"url":{
"type":"string",
"index":"not_analyzed"
},
"uuid":{
"type":"string",
"index":"not_analyzed"
},
"vid":{
"type":"string"
}
}
}
}
}
}
Query:
{
"filter":{
"bool":{
"should":[
[
{
"term":{
"status":1
}
}
]
],
"must":[
]
}
},
"from":0,
"size":10,
"_source":{
"include":[
"uuid",
"nid",
"type",
"title",
"headline",
"shortHeadline",
"teaser",
"url",
"published",
"changed",
"subheadline",
"listImage",
"brand",
"videoId",
"videoSource",
"duration",
"taxonomy",
"includeTerms",
"excludeTerms",
"media",
"credit",
"caption",
"description",
"videoCaptions",
"franchiseId",
"showAirTime",
"slateImage",
"mainImage",
"thumbnailImage",
"showTitle",
"status",
"pubstatus"
]
},
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"langcode:"en""
}
},
{
"bool":{
"must_not":{
"query_string":{
"query":"(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
}
]
}
},
"sort":[
{
"publishedUtc":{
"order":"desc"
}
},
{
"_score":{
"order":"desc"
}
}
]
}
elasticsearch
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13
add a comment |
I am working on NodeJs based API and it is build with Elasticsearch and NodeJs.
I have to provide a functionality where API users can exclude some content based on the taxonomy terms.
freeform: Multi-value field - type : string
games: Multi-value field - type : string
players: Multi-value field - type : string
teams: Multi-value field - type : string
writer: Multi-value field - type : string
channels: Multi-value field - type : string
Format of API url will be something similar to below.
https://api.test.net/2/xxxxx/article/?exclude=term1+term2
Above should return the content which is not tagged to term1 and term2.
Suppose we have 3 articles, first article is tagged to term1, second article is tagged to term2 and third article is tagged to term1 and term2.
Expected Result:
https://api.test.net/2/xxxxx/article/?exclude=term1+term2 should return article first(tagged to term1), second article(tagged to term2) and not the third(Tagged to term1 and term2)
Actual Result:
Sometimes https://api.test.net/2/xxxxx/article/?exclude=term1+term2 returns all three articles and sometime first(tagged to term1) and second article(tagged to term2).
Update:
Below is the correct doc and I have added the query also but still I am seeing the inconsistency.
Doc:
{
"team":{
"mappings":{
"article":{
"properties":{
"body":{
"properties":{
"content":{
"type":"string"
},
"html":{
"type":"string"
},
"type":{
"type":"string"
}
}
},
"brand":{
"type":"string"
},
"changed":{
"type":"date",
"format":"dateOptionalTime"
},
"changedUtc":{
"type":"long"
},
"content":{
"type":"string"
},
"created":{
"type":"date",
"format":"dateOptionalTime"
},
"createdUtc":{
"type":"long"
},
"domain":{
"type":"string"
},
"headline":{
"type":"string"
},
"langcode":{
"type":"string",
"index":"not_analyzed"
},
"nid":{
"type":"string"
},
"published":{
"type":"date",
"format":"dateOptionalTime"
},
"publishedUtc":{
"type":"long"
},
"raw":{
"type":"string",
"index":"no"
},
"revisionTimestamp":{
"type":"date",
"format":"dateOptionalTime"
},
"revisionTimestampUtc":{
"type":"long"
},
"status":{
"type":"string"
},
"subheadline":{
"type":"string"
},
"syndication":{
"properties":{
"brand":{
"type":"string"
},
"options":{
"properties":{
"actionFrom":{
"type":"string"
},
"publish":{
"type":"string"
},
"status":{
"type":"string"
}
}
},
"type":{
"type":"string"
}
}
},
"taxonomy":{
"properties":{
"coaches":{
"properties":{
"value":{
"type":"string"
}
}
},
"freeform":{
"properties":{
"value":{
"type":"string"
}
}
},
"games":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"players":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"section":{
"properties":{
"value":{
"type":"string"
}
}
},
"teams":{
"properties":{
"city":{
"type":"string"
},
"id":{
"type":"string"
},
"nickname":{
"type":"string"
},
"tricode":{
"type":"string"
},
"urlName":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"writer":{
"properties":{
"emailAddress":{
"type":"string"
},
"id":{
"type":"string"
},
"responsive_web":{
"type":"string"
},
"title":{
"type":"string"
},
"value":{
"type":"string"
}
}
}
}
},
"teaser":{
"type":"string"
},
"title":{
"type":"string"
},
"type":{
"type":"string"
},
"url":{
"type":"string",
"index":"not_analyzed"
},
"uuid":{
"type":"string",
"index":"not_analyzed"
},
"vid":{
"type":"string"
}
}
}
}
}
}
Query:
{
"filter":{
"bool":{
"should":[
[
{
"term":{
"status":1
}
}
]
],
"must":[
]
}
},
"from":0,
"size":10,
"_source":{
"include":[
"uuid",
"nid",
"type",
"title",
"headline",
"shortHeadline",
"teaser",
"url",
"published",
"changed",
"subheadline",
"listImage",
"brand",
"videoId",
"videoSource",
"duration",
"taxonomy",
"includeTerms",
"excludeTerms",
"media",
"credit",
"caption",
"description",
"videoCaptions",
"franchiseId",
"showAirTime",
"slateImage",
"mainImage",
"thumbnailImage",
"showTitle",
"status",
"pubstatus"
]
},
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"langcode:"en""
}
},
{
"bool":{
"must_not":{
"query_string":{
"query":"(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
}
]
}
},
"sort":[
{
"publishedUtc":{
"order":"desc"
}
},
{
"_score":{
"order":"desc"
}
}
]
}
elasticsearch
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
I am working on NodeJs based API and it is build with Elasticsearch and NodeJs.
I have to provide a functionality where API users can exclude some content based on the taxonomy terms.
freeform: Multi-value field - type : string
games: Multi-value field - type : string
players: Multi-value field - type : string
teams: Multi-value field - type : string
writer: Multi-value field - type : string
channels: Multi-value field - type : string
Format of API url will be something similar to below.
https://api.test.net/2/xxxxx/article/?exclude=term1+term2
Above should return the content which is not tagged to term1 and term2.
Suppose we have 3 articles, first article is tagged to term1, second article is tagged to term2 and third article is tagged to term1 and term2.
Expected Result:
https://api.test.net/2/xxxxx/article/?exclude=term1+term2 should return article first(tagged to term1), second article(tagged to term2) and not the third(Tagged to term1 and term2)
Actual Result:
Sometimes https://api.test.net/2/xxxxx/article/?exclude=term1+term2 returns all three articles and sometime first(tagged to term1) and second article(tagged to term2).
Update:
Below is the correct doc and I have added the query also but still I am seeing the inconsistency.
Doc:
{
"team":{
"mappings":{
"article":{
"properties":{
"body":{
"properties":{
"content":{
"type":"string"
},
"html":{
"type":"string"
},
"type":{
"type":"string"
}
}
},
"brand":{
"type":"string"
},
"changed":{
"type":"date",
"format":"dateOptionalTime"
},
"changedUtc":{
"type":"long"
},
"content":{
"type":"string"
},
"created":{
"type":"date",
"format":"dateOptionalTime"
},
"createdUtc":{
"type":"long"
},
"domain":{
"type":"string"
},
"headline":{
"type":"string"
},
"langcode":{
"type":"string",
"index":"not_analyzed"
},
"nid":{
"type":"string"
},
"published":{
"type":"date",
"format":"dateOptionalTime"
},
"publishedUtc":{
"type":"long"
},
"raw":{
"type":"string",
"index":"no"
},
"revisionTimestamp":{
"type":"date",
"format":"dateOptionalTime"
},
"revisionTimestampUtc":{
"type":"long"
},
"status":{
"type":"string"
},
"subheadline":{
"type":"string"
},
"syndication":{
"properties":{
"brand":{
"type":"string"
},
"options":{
"properties":{
"actionFrom":{
"type":"string"
},
"publish":{
"type":"string"
},
"status":{
"type":"string"
}
}
},
"type":{
"type":"string"
}
}
},
"taxonomy":{
"properties":{
"coaches":{
"properties":{
"value":{
"type":"string"
}
}
},
"freeform":{
"properties":{
"value":{
"type":"string"
}
}
},
"games":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"players":{
"properties":{
"id":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"section":{
"properties":{
"value":{
"type":"string"
}
}
},
"teams":{
"properties":{
"city":{
"type":"string"
},
"id":{
"type":"string"
},
"nickname":{
"type":"string"
},
"tricode":{
"type":"string"
},
"urlName":{
"type":"string"
},
"value":{
"type":"string"
}
}
},
"writer":{
"properties":{
"emailAddress":{
"type":"string"
},
"id":{
"type":"string"
},
"responsive_web":{
"type":"string"
},
"title":{
"type":"string"
},
"value":{
"type":"string"
}
}
}
}
},
"teaser":{
"type":"string"
},
"title":{
"type":"string"
},
"type":{
"type":"string"
},
"url":{
"type":"string",
"index":"not_analyzed"
},
"uuid":{
"type":"string",
"index":"not_analyzed"
},
"vid":{
"type":"string"
}
}
}
}
}
}
Query:
{
"filter":{
"bool":{
"should":[
[
{
"term":{
"status":1
}
}
]
],
"must":[
]
}
},
"from":0,
"size":10,
"_source":{
"include":[
"uuid",
"nid",
"type",
"title",
"headline",
"shortHeadline",
"teaser",
"url",
"published",
"changed",
"subheadline",
"listImage",
"brand",
"videoId",
"videoSource",
"duration",
"taxonomy",
"includeTerms",
"excludeTerms",
"media",
"credit",
"caption",
"description",
"videoCaptions",
"franchiseId",
"showAirTime",
"slateImage",
"mainImage",
"thumbnailImage",
"showTitle",
"status",
"pubstatus"
]
},
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"langcode:"en""
}
},
{
"bool":{
"must_not":{
"query_string":{
"query":"(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
}
]
}
},
"sort":[
{
"publishedUtc":{
"order":"desc"
}
},
{
"_score":{
"order":"desc"
}
}
]
}
elasticsearch
elasticsearch
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
edited Dec 28 '18 at 20:34
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
asked Nov 28 '18 at 21:46
Mohammad Anwar
636
636
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13
add a comment |
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13
add a comment |
1 Answer
1
active
oldest
votes
I'm assuming that you have mapping something similar to this:
{
"mappings": {
"_doc": {
"properties": {
"taxonomy": {
"properties": {
"game": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"team": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
The required query will be:
{
"query": {
"bool": {
"filter": [
{
"query_string": {
"query": "(t1 OR t2) NOT (t1 AND t2)",
"fields": [
"taxonomy.game.keyword",
"taxonomy.team.keyword"
]
}
}
]
}
}
}
Based on the document you provided your field list will be:
"fields":[
"taxonomy.teams.id",
"taxonomy.freeform.value",
"taxonomy.games.id",
"taxonomy.players.id",
"taxonomy.writer.value",
"taxonomy.coaches.value"
]
Please note that I assumed id, value are keyword type. In case they are not then please change the mapping. Also if the mapping is dynamically created then there will be a sub-field name keyword under id, value, in a way similar to the example mapping I have used which has keyword field under game and team.
The above works for elasticsearch 6.x
Update based on query added to question:
The must_not part in your query should be logically:
(f1=t1 AND f1=t2) OR (f2=t1 AND f2=t2) OR (f3=t1 AND f3=t2) ....
where f i represents field and t i represents term.
So the must_not part should be:
{
"must_not": {
"query_string": {
"query": "(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
|
show 2 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53528609%2finconsistent-behavior-of-elasticsearch-with-multiple-must-not-condition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I'm assuming that you have mapping something similar to this:
{
"mappings": {
"_doc": {
"properties": {
"taxonomy": {
"properties": {
"game": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"team": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
The required query will be:
{
"query": {
"bool": {
"filter": [
{
"query_string": {
"query": "(t1 OR t2) NOT (t1 AND t2)",
"fields": [
"taxonomy.game.keyword",
"taxonomy.team.keyword"
]
}
}
]
}
}
}
Based on the document you provided your field list will be:
"fields":[
"taxonomy.teams.id",
"taxonomy.freeform.value",
"taxonomy.games.id",
"taxonomy.players.id",
"taxonomy.writer.value",
"taxonomy.coaches.value"
]
Please note that I assumed id, value are keyword type. In case they are not then please change the mapping. Also if the mapping is dynamically created then there will be a sub-field name keyword under id, value, in a way similar to the example mapping I have used which has keyword field under game and team.
The above works for elasticsearch 6.x
Update based on query added to question:
The must_not part in your query should be logically:
(f1=t1 AND f1=t2) OR (f2=t1 AND f2=t2) OR (f3=t1 AND f3=t2) ....
where f i represents field and t i represents term.
So the must_not part should be:
{
"must_not": {
"query_string": {
"query": "(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
|
show 2 more comments
I'm assuming that you have mapping something similar to this:
{
"mappings": {
"_doc": {
"properties": {
"taxonomy": {
"properties": {
"game": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"team": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
The required query will be:
{
"query": {
"bool": {
"filter": [
{
"query_string": {
"query": "(t1 OR t2) NOT (t1 AND t2)",
"fields": [
"taxonomy.game.keyword",
"taxonomy.team.keyword"
]
}
}
]
}
}
}
Based on the document you provided your field list will be:
"fields":[
"taxonomy.teams.id",
"taxonomy.freeform.value",
"taxonomy.games.id",
"taxonomy.players.id",
"taxonomy.writer.value",
"taxonomy.coaches.value"
]
Please note that I assumed id, value are keyword type. In case they are not then please change the mapping. Also if the mapping is dynamically created then there will be a sub-field name keyword under id, value, in a way similar to the example mapping I have used which has keyword field under game and team.
The above works for elasticsearch 6.x
Update based on query added to question:
The must_not part in your query should be logically:
(f1=t1 AND f1=t2) OR (f2=t1 AND f2=t2) OR (f3=t1 AND f3=t2) ....
where f i represents field and t i represents term.
So the must_not part should be:
{
"must_not": {
"query_string": {
"query": "(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
|
show 2 more comments
I'm assuming that you have mapping something similar to this:
{
"mappings": {
"_doc": {
"properties": {
"taxonomy": {
"properties": {
"game": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"team": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
The required query will be:
{
"query": {
"bool": {
"filter": [
{
"query_string": {
"query": "(t1 OR t2) NOT (t1 AND t2)",
"fields": [
"taxonomy.game.keyword",
"taxonomy.team.keyword"
]
}
}
]
}
}
}
Based on the document you provided your field list will be:
"fields":[
"taxonomy.teams.id",
"taxonomy.freeform.value",
"taxonomy.games.id",
"taxonomy.players.id",
"taxonomy.writer.value",
"taxonomy.coaches.value"
]
Please note that I assumed id, value are keyword type. In case they are not then please change the mapping. Also if the mapping is dynamically created then there will be a sub-field name keyword under id, value, in a way similar to the example mapping I have used which has keyword field under game and team.
The above works for elasticsearch 6.x
Update based on query added to question:
The must_not part in your query should be logically:
(f1=t1 AND f1=t2) OR (f2=t1 AND f2=t2) OR (f3=t1 AND f3=t2) ....
where f i represents field and t i represents term.
So the must_not part should be:
{
"must_not": {
"query_string": {
"query": "(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
I'm assuming that you have mapping something similar to this:
{
"mappings": {
"_doc": {
"properties": {
"taxonomy": {
"properties": {
"game": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"team": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
The required query will be:
{
"query": {
"bool": {
"filter": [
{
"query_string": {
"query": "(t1 OR t2) NOT (t1 AND t2)",
"fields": [
"taxonomy.game.keyword",
"taxonomy.team.keyword"
]
}
}
]
}
}
}
Based on the document you provided your field list will be:
"fields":[
"taxonomy.teams.id",
"taxonomy.freeform.value",
"taxonomy.games.id",
"taxonomy.players.id",
"taxonomy.writer.value",
"taxonomy.coaches.value"
]
Please note that I assumed id, value are keyword type. In case they are not then please change the mapping. Also if the mapping is dynamically created then there will be a sub-field name keyword under id, value, in a way similar to the example mapping I have used which has keyword field under game and team.
The above works for elasticsearch 6.x
Update based on query added to question:
The must_not part in your query should be logically:
(f1=t1 AND f1=t2) OR (f2=t1 AND f2=t2) OR (f3=t1 AND f3=t2) ....
where f i represents field and t i represents term.
So the must_not part should be:
{
"must_not": {
"query_string": {
"query": "(taxonomy.freeform.value:"sociosqu" AND taxonomy.freeform.value:"nullam") OR (taxonomy.games.id:"sociosqu" AND taxonomy.games.id:"nullam") OR (taxonomy.players.id:"sociosqu" AND taxonomy.players.id:"nullam") OR (taxonomy.teams.id:"sociosqu" AND taxonomy.teams.id:"nullam") OR (taxonomy.writer.value:"sociosqu" AND taxonomy.writer.value:"nullam") OR (taxonomy.channels.value:"sociosqu" AND taxonomy.channels.value:"nullam") OR (taxonomy.section.value:"sociosqu" AND taxonomy.section.value:"nullam") "
}
}
}
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
edited Dec 28 '18 at 3:03
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
answered Nov 30 '18 at 2:31
Nishant Saini
1,030818
1,030818
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
|
show 2 more comments
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
I have updated the correct doc and have added the query as per your suggestion but still I am seeing the inconsistency in the results.
– Mohammad Anwar
Dec 27 '18 at 18:05
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
@MohammadAnwar What version of elasticsearch you are using?
– Nishant Saini
Dec 28 '18 at 2:39
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I have updated answer based on your query, though its still not clear when a document is considered to be tagged? You haven't shared a sample doc yet for the same.
– Nishant Saini
Dec 28 '18 at 3:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am using below version. "version": { "number": "1.5.0", "build_hash": "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp": "2015-03-23T14:30:58Z", "build_snapshot": false, "lucene_version": "4.10.4" }
– Mohammad Anwar
Dec 28 '18 at 15:26
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
I am new to Elastic Search so don't know what is sample doc. If possible please let me know how to get the sample doc and I will share it with you.
– Mohammad Anwar
Dec 28 '18 at 15:29
|
show 2 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53528609%2finconsistent-behavior-of-elasticsearch-with-multiple-must-not-condition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
a4GvK0TyE5NyI,u,rt YZgnKClWztILcwS2 AVJ86Fi0TG hrwjgAOX9vVVlL,u2,3Lzi 6rcqD,mXVvM,c44HmkS26f1,YDE94
What is the type of the multivalued fields?
– Nishant Saini
Nov 29 '18 at 0:14
@NishantSaini I have added type in the description and it is string for all the fields.
– Mohammad Anwar
Nov 29 '18 at 14:29
What do you mean by tagged? If term1 appears in any one of the 7 fields you mentiones, will that doc be considered as tagged as term1?
– Nishant Saini
Nov 29 '18 at 17:13
Also can you share a sample doc?
– Nishant Saini
Nov 29 '18 at 17:13
@NishantSaini I have added the doc to the question as per your request.
– Mohammad Anwar
Nov 29 '18 at 18:13