Elasticsearch Stopwords are not working and /_analyze giving correct response
up vote
0
down vote
favorite
I am using elastic search 6.5.x and I added multiple under custom analyzer. When it comes to stopword analyzer, I am able to get the response correctly when I check with _analyze, When comes to the reality on my index the search box giving me the results even the stopwords are entered. I am posting my index,mapping any Solution for this.
Index:
PUT some-index
{
"settings": {
"index": {
"number_of_shards": 4,
"number_of_replicas": 1,
"refresh_interval": "60s",
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["standard", "lowercase", "my_snow", "my_stop","my_synonym"]
}
},
"filter" : {
"my_snow" : {
"type" : "snowball",
"language": "English"
},
"my_stop": {
"type":"stop",
"stopwords_path":"/home/elasticsearch-6.5.0/config/stopwords/stopwords.txt"
},
"my_synonym" : {
"type" : "synonym",
"synonyms_path" : "/home/elasticsearch-6.5.0/config//analysis/synonyms.txt"
}
}}
}
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"title": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"content": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"url": {
"type": "keyword",
"index": "false",
"store": true
},
"host": {
"type": "keyword",
"index": "true",
"store": true
},
"seed": {
"type": "keyword",
"index": "true",
"store": true
}
}}
}
}
Query I need to use
POST /some-index/_search
{
"query": {
"bool" : {
"must" : {
"query_string" : {
"query" : "test" /* Tested with and,between*/
}
},
"filter" : {
"term" : { "seed" : "ghi" }
}
}
}
}
Tested with /_analyze Works great
POST some-index/_analyze
{
"analyzer": "my_analyzer",
"text":"Hi this is Ramy."
}
elasticsearch asked Nov 16 at 21:25
an__snatcher
488
add a comment |
up vote
0
down vote
favorite
I am using elastic search 6.5.x and I added multiple under custom analyzer. When it comes to stopword analyzer, I am able to get the response correctly when I check with _analyze, When comes to the reality on my index the search box giving me the results even the stopwords are entered. I am posting my index,mapping any Solution for this.
Index:
PUT some-index
{
"settings": {
"index": {
"number_of_shards": 4,
"number_of_replicas": 1,
"refresh_interval": "60s",
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["standard", "lowercase", "my_snow", "my_stop","my_synonym"]
}
},
"filter" : {
"my_snow" : {
"type" : "snowball",
"language": "English"
},
"my_stop": {
"type":"stop",
"stopwords_path":"/home/elasticsearch-6.5.0/config/stopwords/stopwords.txt"
},
"my_synonym" : {
"type" : "synonym",
"synonyms_path" : "/home/elasticsearch-6.5.0/config//analysis/synonyms.txt"
}
}}
}
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"title": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"content": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"url": {
"type": "keyword",
"index": "false",
"store": true
},
"host": {
"type": "keyword",
"index": "true",
"store": true
},
"seed": {
"type": "keyword",
"index": "true",
"store": true
}
}}
}
}
Query I need to use
POST /some-index/_search
{
"query": {
"bool" : {
"must" : {
"query_string" : {
"query" : "test" /* Tested with and,between*/
}
},
"filter" : {
"term" : { "seed" : "ghi" }
}
}
}
}
Tested with /_analyze Works great
POST some-index/_analyze
{
"analyzer": "my_analyzer",
"text":"Hi this is Ramy."
}
asked Nov 16 at 21:25
an__snatcher
488
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the*.txtfiles in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.
– Kamal
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample dataTitle: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler
– an__snatcher
2 days ago
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am using elastic search 6.5.x and I added multiple under custom analyzer. When it comes to stopword analyzer, I am able to get the response correctly when I check with _analyze, When comes to the reality on my index the search box giving me the results even the stopwords are entered. I am posting my index,mapping any Solution for this.
Index:
PUT some-index
{
"settings": {
"index": {
"number_of_shards": 4,
"number_of_replicas": 1,
"refresh_interval": "60s",
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["standard", "lowercase", "my_snow", "my_stop","my_synonym"]
}
},
"filter" : {
"my_snow" : {
"type" : "snowball",
"language": "English"
},
"my_stop": {
"type":"stop",
"stopwords_path":"/home/elasticsearch-6.5.0/config/stopwords/stopwords.txt"
},
"my_synonym" : {
"type" : "synonym",
"synonyms_path" : "/home/elasticsearch-6.5.0/config//analysis/synonyms.txt"
}
}}
}
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"title": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"content": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"url": {
"type": "keyword",
"index": "false",
"store": true
},
"host": {
"type": "keyword",
"index": "true",
"store": true
},
"seed": {
"type": "keyword",
"index": "true",
"store": true
}
}}
}
}
Query I need to use
POST /some-index/_search
{
"query": {
"bool" : {
"must" : {
"query_string" : {
"query" : "test" /* Tested with and,between*/
}
},
"filter" : {
"term" : { "seed" : "ghi" }
}
}
}
}
Tested with /_analyze Works great
POST some-index/_analyze
{
"analyzer": "my_analyzer",
"text":"Hi this is Ramy."
}
asked Nov 16 at 21:25
an__snatcher
488
I am using elastic search 6.5.x and I added multiple under custom analyzer. When it comes to stopword analyzer, I am able to get the response correctly when I check with _analyze, When comes to the reality on my index the search box giving me the results even the stopwords are entered. I am posting my index,mapping any Solution for this.
Index:
PUT some-index
{
"settings": {
"index": {
"number_of_shards": 4,
"number_of_replicas": 1,
"refresh_interval": "60s",
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["standard", "lowercase", "my_snow", "my_stop","my_synonym"]
}
},
"filter" : {
"my_snow" : {
"type" : "snowball",
"language": "English"
},
"my_stop": {
"type":"stop",
"stopwords_path":"/home/elasticsearch-6.5.0/config/stopwords/stopwords.txt"
},
"my_synonym" : {
"type" : "synonym",
"synonyms_path" : "/home/elasticsearch-6.5.0/config//analysis/synonyms.txt"
}
}}
}
},
"mappings": {
"doc": {
"_source": {
"enabled": true
},
"properties": {
"title": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"content": {
"type": "text",
"index": "true",
"store": true,
"analyzer": "my_analyzer"
},
"url": {
"type": "keyword",
"index": "false",
"store": true
},
"host": {
"type": "keyword",
"index": "true",
"store": true
},
"seed": {
"type": "keyword",
"index": "true",
"store": true
}
}}
}
}
Query I need to use
POST /some-index/_search
{
"query": {
"bool" : {
"must" : {
"query_string" : {
"query" : "test" /* Tested with and,between*/
}
},
"filter" : {
"term" : { "seed" : "ghi" }
}
}
}
}
Tested with /_analyze Works great
POST some-index/_analyze
{
"analyzer": "my_analyzer",
"text":"Hi this is Ramy."
}
asked Nov 16 at 21:25
an__snatcher
488
asked Nov 16 at 21:25
an__snatcher
488
asked Nov 16 at 21:25
an__snatcher
488
asked Nov 16 at 21:25
an__snatcher
488
asked Nov 16 at 21:25
an__snatcher
488
488
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the*.txtfiles in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.
– Kamal
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample dataTitle: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler
– an__snatcher
2 days ago
add a comment |
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the*.txtfiles in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.
– Kamal
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample dataTitle: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler
– an__snatcher
2 days ago
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the
*.txt files in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.– Kamal
2 days ago
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the
*.txt files in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.– Kamal
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample data
Title: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler – an__snatcher
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample data
Title: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler – an__snatcher
2 days ago
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53345607%2felasticsearch-stopwords-are-not-working-and-analyze-giving-correct-response%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
hey @an__snatcher, could you let me know how many servers are being used, did you reboot all the servers after adding the
*.txtfiles in all of them? I've tried query at my end and it works as expected. Could you also post a sample document which you think shouldn't be showing up but yet shows up in the response.– Kamal
2 days ago
Hi @Kamal. If the server means an Elastic Server then I am using one server and Do I need to reboot this server before indexing the data or else after pushing the data into the index. I am sharing some sample data
Title: About Storm Crawler Content: StormCrawler is an open source SDK for building distributed web crawlers based on Apache Storm. The project is under Apache license v2 and consists of a collection of reusable resources and components, written mostly in Java. url: http://stormcrawler.net/ host: www.stormcrawler.net seed: stormcrawler– an__snatcher
2 days ago