I am changing indexing of Cosmos DB but it is not working as it is supposed to be
up vote
0
down vote
favorite
I am Using Azure SQL API and my data is structured in the following way:
{
"deviceId": "123_123",
"comms": 0,
"engineSpdEnc": 0,
"currentTime": 1542185998605,
"deviceName": "mydevice2",
"siteId": 0,
"messageType": 2,
"data": {
"v5B3Freq": 0,
"v5B3Amp": 0,
"v5B4Freq": 0,
"v5B4Amp": 0,
"v5B5Freq": 0,
"v5B5Amp": 0,
"v6B6Freq": 0,
"v6B6Amp": 0,
"v6B7Freq": 0,
"v6B7Amp": 0,
"inletPres": 0
},
"EventProcessedUtcTime": "2018-11-14T09:01:42.6897624Z",
"PartitionId": 1,
"EventEnqueuedUtcTime": "2018-11-14T08:59:58.645Z",
"IoTHub": {
"MessageId": null,
"CorrelationId": null,
"ConnectionDeviceId": "device1",
"ConnectionDeviceGenerationId": "636758197942626855",
"EnqueuedTime": "2018-11-14T08:59:58.649Z",
"StreamId": null
},
"id": "1734dd0c-1bb5-d424-4946-e2c957bb3858",
"_rid": "lblPAOEu3xYCAAAAAAAAAA==",
"_self": "dbs/lblPAA==/colls/lblPAOEu3xY=/docs/lblPAOEu3xYCAAAAAAAAAA==/",
"_etag": ""08008e15-0000-0000-0000-5bebe47c0000"",
"_attachments": "attachments/",
"_ts": 1542186108 }
And by using Azure portal I have changed the indexing policy from default to following:
{
"indexingMode": "lazy",
"automatic": false,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": 3
},
{
"kind": "Range",
"dataType": "String",
"precision": 3
},
{
"kind": "Spatial",
"dataType": "Point"
}
]
}
],
"excludedPaths": [
{
"path": "/data/*"
}
]
}
According to this I have disable auto indexing policy and excluded the path in the /data/* that means
If I am going to query:
select * from c where c.data.v6B7Amp = 0
It should return nothing to me as there is no indexing forc.data.pressure , but I am getting all the records supposed to be in it.
Is it because I am using Azure portal to changing the indexing or anything else?
azure azure-cosmosdb azure-cosmosdb-sqlapi asked Nov 18 at 14:25
Amjath Khan
438
|
show 3 more comments
up vote
0
down vote
favorite
I am Using Azure SQL API and my data is structured in the following way:
{
"deviceId": "123_123",
"comms": 0,
"engineSpdEnc": 0,
"currentTime": 1542185998605,
"deviceName": "mydevice2",
"siteId": 0,
"messageType": 2,
"data": {
"v5B3Freq": 0,
"v5B3Amp": 0,
"v5B4Freq": 0,
"v5B4Amp": 0,
"v5B5Freq": 0,
"v5B5Amp": 0,
"v6B6Freq": 0,
"v6B6Amp": 0,
"v6B7Freq": 0,
"v6B7Amp": 0,
"inletPres": 0
},
"EventProcessedUtcTime": "2018-11-14T09:01:42.6897624Z",
"PartitionId": 1,
"EventEnqueuedUtcTime": "2018-11-14T08:59:58.645Z",
"IoTHub": {
"MessageId": null,
"CorrelationId": null,
"ConnectionDeviceId": "device1",
"ConnectionDeviceGenerationId": "636758197942626855",
"EnqueuedTime": "2018-11-14T08:59:58.649Z",
"StreamId": null
},
"id": "1734dd0c-1bb5-d424-4946-e2c957bb3858",
"_rid": "lblPAOEu3xYCAAAAAAAAAA==",
"_self": "dbs/lblPAA==/colls/lblPAOEu3xY=/docs/lblPAOEu3xYCAAAAAAAAAA==/",
"_etag": ""08008e15-0000-0000-0000-5bebe47c0000"",
"_attachments": "attachments/",
"_ts": 1542186108 }
And by using Azure portal I have changed the indexing policy from default to following:
{
"indexingMode": "lazy",
"automatic": false,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": 3
},
{
"kind": "Range",
"dataType": "String",
"precision": 3
},
{
"kind": "Spatial",
"dataType": "Point"
}
]
}
],
"excludedPaths": [
{
"path": "/data/*"
}
]
}
According to this I have disable auto indexing policy and excluded the path in the /data/* that means
If I am going to query:
select * from c where c.data.v6B7Amp = 0
It should return nothing to me as there is no indexing forc.data.pressure , but I am getting all the records supposed to be in it.
Is it because I am using Azure portal to changing the indexing or anything else?
azure azure-cosmosdb azure-cosmosdb-sqlapi asked Nov 18 at 14:25
Amjath Khan
438
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01
|
show 3 more comments
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am Using Azure SQL API and my data is structured in the following way:
{
"deviceId": "123_123",
"comms": 0,
"engineSpdEnc": 0,
"currentTime": 1542185998605,
"deviceName": "mydevice2",
"siteId": 0,
"messageType": 2,
"data": {
"v5B3Freq": 0,
"v5B3Amp": 0,
"v5B4Freq": 0,
"v5B4Amp": 0,
"v5B5Freq": 0,
"v5B5Amp": 0,
"v6B6Freq": 0,
"v6B6Amp": 0,
"v6B7Freq": 0,
"v6B7Amp": 0,
"inletPres": 0
},
"EventProcessedUtcTime": "2018-11-14T09:01:42.6897624Z",
"PartitionId": 1,
"EventEnqueuedUtcTime": "2018-11-14T08:59:58.645Z",
"IoTHub": {
"MessageId": null,
"CorrelationId": null,
"ConnectionDeviceId": "device1",
"ConnectionDeviceGenerationId": "636758197942626855",
"EnqueuedTime": "2018-11-14T08:59:58.649Z",
"StreamId": null
},
"id": "1734dd0c-1bb5-d424-4946-e2c957bb3858",
"_rid": "lblPAOEu3xYCAAAAAAAAAA==",
"_self": "dbs/lblPAA==/colls/lblPAOEu3xY=/docs/lblPAOEu3xYCAAAAAAAAAA==/",
"_etag": ""08008e15-0000-0000-0000-5bebe47c0000"",
"_attachments": "attachments/",
"_ts": 1542186108 }
And by using Azure portal I have changed the indexing policy from default to following:
{
"indexingMode": "lazy",
"automatic": false,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": 3
},
{
"kind": "Range",
"dataType": "String",
"precision": 3
},
{
"kind": "Spatial",
"dataType": "Point"
}
]
}
],
"excludedPaths": [
{
"path": "/data/*"
}
]
}
According to this I have disable auto indexing policy and excluded the path in the /data/* that means
If I am going to query:
select * from c where c.data.v6B7Amp = 0
It should return nothing to me as there is no indexing forc.data.pressure , but I am getting all the records supposed to be in it.
Is it because I am using Azure portal to changing the indexing or anything else?
azure azure-cosmosdb azure-cosmosdb-sqlapi asked Nov 18 at 14:25
Amjath Khan
438
I am Using Azure SQL API and my data is structured in the following way:
{
"deviceId": "123_123",
"comms": 0,
"engineSpdEnc": 0,
"currentTime": 1542185998605,
"deviceName": "mydevice2",
"siteId": 0,
"messageType": 2,
"data": {
"v5B3Freq": 0,
"v5B3Amp": 0,
"v5B4Freq": 0,
"v5B4Amp": 0,
"v5B5Freq": 0,
"v5B5Amp": 0,
"v6B6Freq": 0,
"v6B6Amp": 0,
"v6B7Freq": 0,
"v6B7Amp": 0,
"inletPres": 0
},
"EventProcessedUtcTime": "2018-11-14T09:01:42.6897624Z",
"PartitionId": 1,
"EventEnqueuedUtcTime": "2018-11-14T08:59:58.645Z",
"IoTHub": {
"MessageId": null,
"CorrelationId": null,
"ConnectionDeviceId": "device1",
"ConnectionDeviceGenerationId": "636758197942626855",
"EnqueuedTime": "2018-11-14T08:59:58.649Z",
"StreamId": null
},
"id": "1734dd0c-1bb5-d424-4946-e2c957bb3858",
"_rid": "lblPAOEu3xYCAAAAAAAAAA==",
"_self": "dbs/lblPAA==/colls/lblPAOEu3xY=/docs/lblPAOEu3xYCAAAAAAAAAA==/",
"_etag": ""08008e15-0000-0000-0000-5bebe47c0000"",
"_attachments": "attachments/",
"_ts": 1542186108 }
And by using Azure portal I have changed the indexing policy from default to following:
{
"indexingMode": "lazy",
"automatic": false,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": 3
},
{
"kind": "Range",
"dataType": "String",
"precision": 3
},
{
"kind": "Spatial",
"dataType": "Point"
}
]
}
],
"excludedPaths": [
{
"path": "/data/*"
}
]
}
According to this I have disable auto indexing policy and excluded the path in the /data/* that means
If I am going to query:
select * from c where c.data.v6B7Amp = 0
It should return nothing to me as there is no indexing forc.data.pressure , but I am getting all the records supposed to be in it.
Is it because I am using Azure portal to changing the indexing or anything else?
azure azure-cosmosdb azure-cosmosdb-sqlapi azure azure-cosmosdb azure-cosmosdb-sqlapi asked Nov 18 at 14:25
Amjath Khan
438
asked Nov 18 at 14:25
Amjath Khan
438
edited Nov 18 at 16:05
asked Nov 18 at 14:25
Amjath Khan
438
asked Nov 18 at 14:25
Amjath Khan
438
asked Nov 18 at 14:25
Amjath Khan
438
438
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01
|
show 3 more comments
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01
|
show 3 more comments
2 Answers
2
active
oldest
votes
up vote
1
down vote
Firstly, you don't need to turn automatic indexing off or set the indexingMode to lazy, unless you have a reason to.
It appears that equality checks can work even if the path is excluded.
Where the excluded path will kick in is when you try to do something like an order by against that field.
Here is an example using your data data and your indexing policy:
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
|
show 7 more comments
up vote
0
down vote
When a path is excluded from indexing, query will fallback to do a full scan of all the documents in the collection to filter the results. That's the reason you are seeing results for your query.
answered Nov 19 at 20:48
Bala Peru
1
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53361918%2fi-am-changing-indexing-of-cosmos-db-but-it-is-not-working-as-it-is-supposed-to-b%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
Firstly, you don't need to turn automatic indexing off or set the indexingMode to lazy, unless you have a reason to.
It appears that equality checks can work even if the path is excluded.
Where the excluded path will kick in is when you try to do something like an order by against that field.
Here is an example using your data data and your indexing policy:
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
|
show 7 more comments
up vote
1
down vote
Firstly, you don't need to turn automatic indexing off or set the indexingMode to lazy, unless you have a reason to.
It appears that equality checks can work even if the path is excluded.
Where the excluded path will kick in is when you try to do something like an order by against that field.
Here is an example using your data data and your indexing policy:
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
|
show 7 more comments
up vote
1
down vote
up vote
1
down vote
Firstly, you don't need to turn automatic indexing off or set the indexingMode to lazy, unless you have a reason to.
It appears that equality checks can work even if the path is excluded.
Where the excluded path will kick in is when you try to do something like an order by against that field.
Here is an example using your data data and your indexing policy:
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
Firstly, you don't need to turn automatic indexing off or set the indexingMode to lazy, unless you have a reason to.
It appears that equality checks can work even if the path is excluded.
Where the excluded path will kick in is when you try to do something like an order by against that field.
Here is an example using your data data and your indexing policy:
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
answered Nov 19 at 9:42
Nick Chapsas
2,4661314
2,4661314
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
|
show 7 more comments
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
Reason i am looking to off all the indexes is so that , my throughput of insertion increases from Azure Stream analytics JOB. As my Edge Device is sending 20 messages at a given time it is showing throttled Cosmos DB service . I feel too much of indexing as i have 150 sensors in it. I am just wondered cosmos DB can't even write 20 messages per connection each record of size 8KB size ... that too from azures Stream analytics JOB service only
– Amjath Khan
Nov 19 at 11:21
1
1
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
@AmjathKhan With automatic indexing turned off, you can still selectively add specific items to the index, but it's unclear if you're doing so, that's why I raised it.
– Nick Chapsas
Nov 19 at 11:25
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
In my Edge Device I am collection 20 device data but that data each message i cannot partition by device-id as ( only sending data to Event Hub can be partition Key ) added at sender's side. From Stream Analytics job i am pushing data to COSMOS DB .. How can i improve performance of Cosmos DB
– Amjath Khan
Nov 19 at 11:26
1
1
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
@AmjathKhan I can't answer that in an answer comments. you'd have to ask a new question specific to this new problem. I only answered the indexing question here. Feel free to open a new one explaining your problem in detail and we'll get back to you
– Nick Chapsas
Nov 19 at 11:27
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
Can you help me with sample how can i add selective item to index with automatic index off .. @Nick
– Amjath Khan
Nov 19 at 11:29
|
show 7 more comments
up vote
0
down vote
When a path is excluded from indexing, query will fallback to do a full scan of all the documents in the collection to filter the results. That's the reason you are seeing results for your query.
answered Nov 19 at 20:48
Bala Peru
1
add a comment |
up vote
0
down vote
When a path is excluded from indexing, query will fallback to do a full scan of all the documents in the collection to filter the results. That's the reason you are seeing results for your query.
answered Nov 19 at 20:48
Bala Peru
1
add a comment |
up vote
0
down vote
up vote
0
down vote
When a path is excluded from indexing, query will fallback to do a full scan of all the documents in the collection to filter the results. That's the reason you are seeing results for your query.
answered Nov 19 at 20:48
Bala Peru
1
When a path is excluded from indexing, query will fallback to do a full scan of all the documents in the collection to filter the results. That's the reason you are seeing results for your query.
answered Nov 19 at 20:48
Bala Peru
1
answered Nov 19 at 20:48
Bala Peru
1
answered Nov 19 at 20:48
Bala Peru
1
answered Nov 19 at 20:48
Bala Peru
1
1
add a comment |
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53361918%2fi-am-changing-indexing-of-cosmos-db-but-it-is-not-working-as-it-is-supposed-to-b%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Indexing !== Filtering :).
– Gaurav Mantri
Nov 18 at 14:44
Did you give some time to the indexing process to complete? Whenever you change the indexing policy everything has to be reindexed and included/excluded from scratch, and based on your database's size, this can take time.
– Nick Chapsas
Nov 18 at 14:53
I know it is not filtering but cosmos DB is tree based indexed database .as a result when we keep c.data.pressure it should not able to find in where condition. I want to have complete manual indexing where only top level is indexed so that write throghput is more.
– Amjath Khan
Nov 18 at 14:53
My collection is empty . changed the policy and uploaded the json file and checked it.@NickChapsas
– Amjath Khan
Nov 18 at 14:56
@NickChapsas , I am right to expect ....the behavior to give no results for where filtering of c.data.pressure < 25 ..
– Amjath Khan
Nov 18 at 15:01