Mysql query running very slow with sum and group by option
Mysql query running very slow.
there are 1000000 rows in the table.I have added explain query to the question. it is taking 10-11 seconds. I have multiple conditions which keep on changing basis user's input. Is it possible to provide unique index.
explain query
id -1
select_type -SIMPLE
table -sellers
type -ref
possible_keys -quarter,quarter_2,comp_idx1,index_all,qc
key -qc
key_len -32
ref -const
rows -399782
Extra -Using where
SELECT `cm`,
`week`,
`quarter`,
SUM( `gmv`) as gmv,
SUM( `qty_sold`) as qty_sold,
SUM( `live_listing`) as live_listing ,
SUM( `txn`) as txn,
SUM( `new_listings`) as new_listings,
SUM( `ended_listings`) as ended_listings
FROM `sellers`
where quarter=:quarter and
category=:category
group by cm
CREATE TABLE `sellers` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`seller_id` varchar(20) NOT NULL,
`am` varchar(30) NOT NULL,
`rm` varchar(30) NOT NULL,
`cm` varchar(30) NOT NULL,
`week` varchar(10) NOT NULL,
`quarter` varchar(10) NOT NULL,
`site` varchar(20) NOT NULL,
`category` varchar(20) NOT NULL,
`format` varchar(20) NOT NULL,
`price_band` varchar(20) NOT NULL,
`gmv` double(11,2) NOT NULL,
`qty_sold` int(11) NOT NULL,
`live_listing` int(11) NOT NULL,
`txn` int(11) NOT NULL,
`new_listings` int(11) NOT NULL,
`ended_listings` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `seller_id` (`seller_id`),
KEY `quarter` (`quarter`),
KEY `week` (`week`),
KEY `id` (`id`),
KEY `category` (`category`),
KEY `site` (`site`),
KEY `cm` (`cm`),
KEY `rm` (`rm`),
KEY `am` (`am`)
) ENGINE=InnoDB AUTO_INCREMENT=1692618 DEFAULT CHARSET=utf8
mysql
|
show 2 more comments
Mysql query running very slow.
there are 1000000 rows in the table.I have added explain query to the question. it is taking 10-11 seconds. I have multiple conditions which keep on changing basis user's input. Is it possible to provide unique index.
explain query
id -1
select_type -SIMPLE
table -sellers
type -ref
possible_keys -quarter,quarter_2,comp_idx1,index_all,qc
key -qc
key_len -32
ref -const
rows -399782
Extra -Using where
SELECT `cm`,
`week`,
`quarter`,
SUM( `gmv`) as gmv,
SUM( `qty_sold`) as qty_sold,
SUM( `live_listing`) as live_listing ,
SUM( `txn`) as txn,
SUM( `new_listings`) as new_listings,
SUM( `ended_listings`) as ended_listings
FROM `sellers`
where quarter=:quarter and
category=:category
group by cm
CREATE TABLE `sellers` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`seller_id` varchar(20) NOT NULL,
`am` varchar(30) NOT NULL,
`rm` varchar(30) NOT NULL,
`cm` varchar(30) NOT NULL,
`week` varchar(10) NOT NULL,
`quarter` varchar(10) NOT NULL,
`site` varchar(20) NOT NULL,
`category` varchar(20) NOT NULL,
`format` varchar(20) NOT NULL,
`price_band` varchar(20) NOT NULL,
`gmv` double(11,2) NOT NULL,
`qty_sold` int(11) NOT NULL,
`live_listing` int(11) NOT NULL,
`txn` int(11) NOT NULL,
`new_listings` int(11) NOT NULL,
`ended_listings` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `seller_id` (`seller_id`),
KEY `quarter` (`quarter`),
KEY `week` (`week`),
KEY `id` (`id`),
KEY `category` (`category`),
KEY `site` (`site`),
KEY `cm` (`cm`),
KEY `rm` (`rm`),
KEY `am` (`am`)
) ENGINE=InnoDB AUTO_INCREMENT=1692618 DEFAULT CHARSET=utf8
mysql
What is the definition of "very slow" ? How much time is it taking currently ? Also, addEXPLAIN
statement result to the question for the current query.
– Madhur Bhaiya
Nov 20 at 7:16
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26
|
show 2 more comments
Mysql query running very slow.
there are 1000000 rows in the table.I have added explain query to the question. it is taking 10-11 seconds. I have multiple conditions which keep on changing basis user's input. Is it possible to provide unique index.
explain query
id -1
select_type -SIMPLE
table -sellers
type -ref
possible_keys -quarter,quarter_2,comp_idx1,index_all,qc
key -qc
key_len -32
ref -const
rows -399782
Extra -Using where
SELECT `cm`,
`week`,
`quarter`,
SUM( `gmv`) as gmv,
SUM( `qty_sold`) as qty_sold,
SUM( `live_listing`) as live_listing ,
SUM( `txn`) as txn,
SUM( `new_listings`) as new_listings,
SUM( `ended_listings`) as ended_listings
FROM `sellers`
where quarter=:quarter and
category=:category
group by cm
CREATE TABLE `sellers` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`seller_id` varchar(20) NOT NULL,
`am` varchar(30) NOT NULL,
`rm` varchar(30) NOT NULL,
`cm` varchar(30) NOT NULL,
`week` varchar(10) NOT NULL,
`quarter` varchar(10) NOT NULL,
`site` varchar(20) NOT NULL,
`category` varchar(20) NOT NULL,
`format` varchar(20) NOT NULL,
`price_band` varchar(20) NOT NULL,
`gmv` double(11,2) NOT NULL,
`qty_sold` int(11) NOT NULL,
`live_listing` int(11) NOT NULL,
`txn` int(11) NOT NULL,
`new_listings` int(11) NOT NULL,
`ended_listings` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `seller_id` (`seller_id`),
KEY `quarter` (`quarter`),
KEY `week` (`week`),
KEY `id` (`id`),
KEY `category` (`category`),
KEY `site` (`site`),
KEY `cm` (`cm`),
KEY `rm` (`rm`),
KEY `am` (`am`)
) ENGINE=InnoDB AUTO_INCREMENT=1692618 DEFAULT CHARSET=utf8
mysql
Mysql query running very slow.
there are 1000000 rows in the table.I have added explain query to the question. it is taking 10-11 seconds. I have multiple conditions which keep on changing basis user's input. Is it possible to provide unique index.
explain query
id -1
select_type -SIMPLE
table -sellers
type -ref
possible_keys -quarter,quarter_2,comp_idx1,index_all,qc
key -qc
key_len -32
ref -const
rows -399782
Extra -Using where
SELECT `cm`,
`week`,
`quarter`,
SUM( `gmv`) as gmv,
SUM( `qty_sold`) as qty_sold,
SUM( `live_listing`) as live_listing ,
SUM( `txn`) as txn,
SUM( `new_listings`) as new_listings,
SUM( `ended_listings`) as ended_listings
FROM `sellers`
where quarter=:quarter and
category=:category
group by cm
CREATE TABLE `sellers` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`seller_id` varchar(20) NOT NULL,
`am` varchar(30) NOT NULL,
`rm` varchar(30) NOT NULL,
`cm` varchar(30) NOT NULL,
`week` varchar(10) NOT NULL,
`quarter` varchar(10) NOT NULL,
`site` varchar(20) NOT NULL,
`category` varchar(20) NOT NULL,
`format` varchar(20) NOT NULL,
`price_band` varchar(20) NOT NULL,
`gmv` double(11,2) NOT NULL,
`qty_sold` int(11) NOT NULL,
`live_listing` int(11) NOT NULL,
`txn` int(11) NOT NULL,
`new_listings` int(11) NOT NULL,
`ended_listings` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `seller_id` (`seller_id`),
KEY `quarter` (`quarter`),
KEY `week` (`week`),
KEY `id` (`id`),
KEY `category` (`category`),
KEY `site` (`site`),
KEY `cm` (`cm`),
KEY `rm` (`rm`),
KEY `am` (`am`)
) ENGINE=InnoDB AUTO_INCREMENT=1692618 DEFAULT CHARSET=utf8
mysql
mysql
edited Nov 20 at 9:22
asked Nov 20 at 6:54
Manish Malviya
3292819
3292819
What is the definition of "very slow" ? How much time is it taking currently ? Also, addEXPLAIN
statement result to the question for the current query.
– Madhur Bhaiya
Nov 20 at 7:16
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26
|
show 2 more comments
What is the definition of "very slow" ? How much time is it taking currently ? Also, addEXPLAIN
statement result to the question for the current query.
– Madhur Bhaiya
Nov 20 at 7:16
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26
What is the definition of "very slow" ? How much time is it taking currently ? Also, add
EXPLAIN
statement result to the question for the current query.– Madhur Bhaiya
Nov 20 at 7:16
What is the definition of "very slow" ? How much time is it taking currently ? Also, add
EXPLAIN
statement result to the question for the current query.– Madhur Bhaiya
Nov 20 at 7:16
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26
|
show 2 more comments
2 Answers
2
active
oldest
votes
You can add an index to 'cm' column. It will be faster than previously
It is already there.
– Manish Malviya
Nov 20 at 7:03
add a comment |
In a particular query, MySQL is able to use only one index at a time. In your case, you have multiple columns to check against. You will need to add a Composite Index on (quarter, category, cm)
, for better performance:
ALTER TABLE sellers
ADD INDEX comp_idx1(quarter, category, cm);
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? AddEXPLAIN
statement results to the question so that it can be analyzed further.
– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
|
show 2 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53387705%2fmysql-query-running-very-slow-with-sum-and-group-by-option%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can add an index to 'cm' column. It will be faster than previously
It is already there.
– Manish Malviya
Nov 20 at 7:03
add a comment |
You can add an index to 'cm' column. It will be faster than previously
It is already there.
– Manish Malviya
Nov 20 at 7:03
add a comment |
You can add an index to 'cm' column. It will be faster than previously
You can add an index to 'cm' column. It will be faster than previously
answered Nov 20 at 6:58
Arup Garai
1016
1016
It is already there.
– Manish Malviya
Nov 20 at 7:03
add a comment |
It is already there.
– Manish Malviya
Nov 20 at 7:03
It is already there.
– Manish Malviya
Nov 20 at 7:03
It is already there.
– Manish Malviya
Nov 20 at 7:03
add a comment |
In a particular query, MySQL is able to use only one index at a time. In your case, you have multiple columns to check against. You will need to add a Composite Index on (quarter, category, cm)
, for better performance:
ALTER TABLE sellers
ADD INDEX comp_idx1(quarter, category, cm);
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? AddEXPLAIN
statement results to the question so that it can be analyzed further.
– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
|
show 2 more comments
In a particular query, MySQL is able to use only one index at a time. In your case, you have multiple columns to check against. You will need to add a Composite Index on (quarter, category, cm)
, for better performance:
ALTER TABLE sellers
ADD INDEX comp_idx1(quarter, category, cm);
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? AddEXPLAIN
statement results to the question so that it can be analyzed further.
– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
|
show 2 more comments
In a particular query, MySQL is able to use only one index at a time. In your case, you have multiple columns to check against. You will need to add a Composite Index on (quarter, category, cm)
, for better performance:
ALTER TABLE sellers
ADD INDEX comp_idx1(quarter, category, cm);
In a particular query, MySQL is able to use only one index at a time. In your case, you have multiple columns to check against. You will need to add a Composite Index on (quarter, category, cm)
, for better performance:
ALTER TABLE sellers
ADD INDEX comp_idx1(quarter, category, cm);
answered Nov 20 at 7:31
Madhur Bhaiya
19.5k62236
19.5k62236
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? AddEXPLAIN
statement results to the question so that it can be analyzed further.
– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
|
show 2 more comments
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? AddEXPLAIN
statement results to the question so that it can be analyzed further.
– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
It fails if I remove category condition. we have various (5-6) parameter to put in the condition. I have added one more index after removing category from the filter. ALTER TABLE sellers ADD INDEX qc(quarter, cm) still it is not working.
– Manish Malviya
Nov 20 at 8:39
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? Add
EXPLAIN
statement results to the question so that it can be analyzed further.– Madhur Bhaiya
Nov 20 at 8:41
@ManishMalviya what do you mean by "its not working" ? Please specify the duration for which queries are running ? Add
EXPLAIN
statement results to the question so that it can be analyzed further.– Madhur Bhaiya
Nov 20 at 8:41
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Explain sql results. and query taking 10 sec. id - 1 select_type -SIMPLE table sellers type ref possible_keys quarter,quarter_2,comp_idx1,index_all,qc key qc key_len 32 ref const rows 399782 Extra Using where
– Manish Malviya
Nov 20 at 8:58
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
Put it on your question to provide more info..
– dwir182
Nov 20 at 9:08
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
@dwir182 Added.
– Manish Malviya
Nov 20 at 9:23
|
show 2 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53387705%2fmysql-query-running-very-slow-with-sum-and-group-by-option%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What is the definition of "very slow" ? How much time is it taking currently ? Also, add
EXPLAIN
statement result to the question for the current query.– Madhur Bhaiya
Nov 20 at 7:16
Please add these details to the question, by editing it. All the formatting is lost in the comments and make it unreadable.
– Madhur Bhaiya
Nov 20 at 8:47
use order by null after group byu statement
– p.ganesh
Nov 20 at 8:51
Generally, you should include within a GROUP BY any and all non-aggregated columns within the SELECT
– Strawberry
Nov 20 at 9:02
@Strawberry I didn't get that.
– Manish Malviya
Nov 20 at 9:26