What is the right way to import data to tensorflow?
up vote
0
down vote
favorite
I am new to Tensorflow and trying to make my own little project. I would like to import my CSV file as a dataset and then I would like to split it into training and testing sets and also to be able to make batches from my dataset.
My CSV file contains 3 columns of numbers so I managed to find these lines of code
filenames = ['mydata.csv']
record_defaults = [tf.float32] * 3
dataset = tf.contrib.data.CsvDataset(filenames, record_defaults, header=True, select_cols=[1,2,3])
How do I convert this object to tensor or dataset, so I can either split the data or create batches of data?
python tensorflow
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
asked Nov 17 at 15:23
Crash
276
add a comment |
up vote
0
down vote
favorite
I am new to Tensorflow and trying to make my own little project. I would like to import my CSV file as a dataset and then I would like to split it into training and testing sets and also to be able to make batches from my dataset.
My CSV file contains 3 columns of numbers so I managed to find these lines of code
filenames = ['mydata.csv']
record_defaults = [tf.float32] * 3
dataset = tf.contrib.data.CsvDataset(filenames, record_defaults, header=True, select_cols=[1,2,3])
How do I convert this object to tensor or dataset, so I can either split the data or create batches of data?
python tensorflow
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
asked Nov 17 at 15:23
Crash
276
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am new to Tensorflow and trying to make my own little project. I would like to import my CSV file as a dataset and then I would like to split it into training and testing sets and also to be able to make batches from my dataset.
My CSV file contains 3 columns of numbers so I managed to find these lines of code
filenames = ['mydata.csv']
record_defaults = [tf.float32] * 3
dataset = tf.contrib.data.CsvDataset(filenames, record_defaults, header=True, select_cols=[1,2,3])
How do I convert this object to tensor or dataset, so I can either split the data or create batches of data?
python tensorflow
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
asked Nov 17 at 15:23
Crash
276
I am new to Tensorflow and trying to make my own little project. I would like to import my CSV file as a dataset and then I would like to split it into training and testing sets and also to be able to make batches from my dataset.
My CSV file contains 3 columns of numbers so I managed to find these lines of code
filenames = ['mydata.csv']
record_defaults = [tf.float32] * 3
dataset = tf.contrib.data.CsvDataset(filenames, record_defaults, header=True, select_cols=[1,2,3])
How do I convert this object to tensor or dataset, so I can either split the data or create batches of data?
python tensorflow
python tensorflow
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
asked Nov 17 at 15:23
Crash
276
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
asked Nov 17 at 15:23
Crash
276
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
edited Nov 17 at 16:41
Matthieu Brucher
6,7891331
6,7891331
asked Nov 17 at 15:23
Crash
276
asked Nov 17 at 15:23
Crash
276
asked Nov 17 at 15:23
Crash
276
276
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
0
down vote
Use a tool to split your data like sklearn.model_selection.train_test_split:
X_train, X_test, y_train, y_test = train_test_split(
dataset[:2], dataset[2], test_size=0.33, random_state=42)
For instance if your dataset consists of two features columns and one output label.
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
add a comment |
up vote
0
down vote
As explained on the tensorflow guide here , you have the dataset and after this you can preprocess your data using the Dataset.map() transformation for a certain defined function. Batching and shuffling could also be done after wards using dataset.batch(Batch_size) and dataset.shuffle(buffer_size=Buffer_Size). you can read the guide for further details.
answered Nov 19 at 14:52
D.negn
262
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Use a tool to split your data like sklearn.model_selection.train_test_split:
X_train, X_test, y_train, y_test = train_test_split(
dataset[:2], dataset[2], test_size=0.33, random_state=42)
For instance if your dataset consists of two features columns and one output label.
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
add a comment |
up vote
0
down vote
Use a tool to split your data like sklearn.model_selection.train_test_split:
X_train, X_test, y_train, y_test = train_test_split(
dataset[:2], dataset[2], test_size=0.33, random_state=42)
For instance if your dataset consists of two features columns and one output label.
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
add a comment |
up vote
0
down vote
up vote
0
down vote
Use a tool to split your data like sklearn.model_selection.train_test_split:
X_train, X_test, y_train, y_test = train_test_split(
dataset[:2], dataset[2], test_size=0.33, random_state=42)
For instance if your dataset consists of two features columns and one output label.
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
Use a tool to split your data like sklearn.model_selection.train_test_split:
X_train, X_test, y_train, y_test = train_test_split(
dataset[:2], dataset[2], test_size=0.33, random_state=42)
For instance if your dataset consists of two features columns and one output label.
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
answered Nov 17 at 16:43
Matthieu Brucher
6,7891331
6,7891331
add a comment |
add a comment |
up vote
0
down vote
As explained on the tensorflow guide here , you have the dataset and after this you can preprocess your data using the Dataset.map() transformation for a certain defined function. Batching and shuffling could also be done after wards using dataset.batch(Batch_size) and dataset.shuffle(buffer_size=Buffer_Size). you can read the guide for further details.
answered Nov 19 at 14:52
D.negn
262
add a comment |
up vote
0
down vote
As explained on the tensorflow guide here , you have the dataset and after this you can preprocess your data using the Dataset.map() transformation for a certain defined function. Batching and shuffling could also be done after wards using dataset.batch(Batch_size) and dataset.shuffle(buffer_size=Buffer_Size). you can read the guide for further details.
answered Nov 19 at 14:52
D.negn
262
add a comment |
up vote
0
down vote
up vote
0
down vote
As explained on the tensorflow guide here , you have the dataset and after this you can preprocess your data using the Dataset.map() transformation for a certain defined function. Batching and shuffling could also be done after wards using dataset.batch(Batch_size) and dataset.shuffle(buffer_size=Buffer_Size). you can read the guide for further details.
answered Nov 19 at 14:52
D.negn
262
As explained on the tensorflow guide here , you have the dataset and after this you can preprocess your data using the Dataset.map() transformation for a certain defined function. Batching and shuffling could also be done after wards using dataset.batch(Batch_size) and dataset.shuffle(buffer_size=Buffer_Size). you can read the guide for further details.
answered Nov 19 at 14:52
D.negn
262
answered Nov 19 at 14:52
D.negn
262
answered Nov 19 at 14:52
D.negn
262
answered Nov 19 at 14:52
D.negn
262
262
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53352604%2fwhat-is-the-right-way-to-import-data-to-tensorflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
