Can arangodb handle 20 billion records?
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
add a comment |
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
I just want to know, can Arangodb handle 20 billions of records in a single collection?
What is the best way to insert 1 billion records at a time?
arangodb arangodb-php
arangodb arangodb-php
asked Nov 19 at 12:20
Yash Pandya
61
asked Nov 19 at 12:20
Yash Pandya
61
asked Nov 19 at 12:20
Yash Pandya
61
asked Nov 19 at 12:20
Yash Pandya
61
asked Nov 19 at 12:20
Yash Pandya
61
61
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
add a comment |
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
add a comment |
up vote
0
down vote
up vote
0
down vote
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
Of course. The underlying storage engine choice should definitely be RocksDB. How big would every individual document in average be?
The best way would be any sort of import/restore. Is this going to be a cluster or a single server deployment?
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
answered Nov 20 at 13:14
Kaveh Vahedipour
2,1601416
2,1601416
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
add a comment |
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
The individual document can be of size 1 kb. Currently I am using it as a single server and mmfiles as a storage engine.
– Yash Pandya
Nov 20 at 13:40
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
MMFiles is not an option unless you are looking into buying 20 exabyte of RAM. Memory mapped files performance goes towards zero, when data size including indexes cannot be matched in RAM size.
– Kaveh Vahedipour
Nov 20 at 17:45
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Thanks for the reply. You suggested the best way to insert records is import/restore, but to use that my whole data should be ready before insertion,right? Suppose my data is generated at run time using some sort of script, at that time what should I use?
– Yash Pandya
Nov 22 at 14:40
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
Absolutely. By all means insert all you data.
– Kaveh Vahedipour
Nov 22 at 14:48
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53374517%2fcan-arangodb-handle-20-billion-records%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
