JQ can't parse u2022 character
up vote
0
down vote
favorite
I'm trying to perform a bulk upload to Elasticsearch (around 1mln documents). In order to do that, I'm using jq to reformat the JSON file extracted from MySQL database and curl to post the data to Elasticsearch:
cat dataset.json | jq -r -c '. | { "index" : { } }, .' | curl -u login:password -H "Content-Type: application/json" -XPOST "https://.../skills/default/_bulk?pretty" --data-binary @-
I get an error:
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 276249, column 317
I found that the character that jq can't parse is u2022. I tried adding "-r" jq command but the error stil occurs. How can I handle this for all occurrences of u2022?
json unicode jq
edited Nov 18 at 2:55
peak
28.6k73752
asked Nov 17 at 15:17
Alit
165
add a comment |
up vote
0
down vote
favorite
I'm trying to perform a bulk upload to Elasticsearch (around 1mln documents). In order to do that, I'm using jq to reformat the JSON file extracted from MySQL database and curl to post the data to Elasticsearch:
cat dataset.json | jq -r -c '. | { "index" : { } }, .' | curl -u login:password -H "Content-Type: application/json" -XPOST "https://.../skills/default/_bulk?pretty" --data-binary @-
I get an error:
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 276249, column 317
I found that the character that jq can't parse is u2022. I tried adding "-r" jq command but the error stil occurs. How can I handle this for all occurrences of u2022?
json unicode jq
edited Nov 18 at 2:55
peak
28.6k73752
asked Nov 17 at 15:17
Alit
165
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to perform a bulk upload to Elasticsearch (around 1mln documents). In order to do that, I'm using jq to reformat the JSON file extracted from MySQL database and curl to post the data to Elasticsearch:
cat dataset.json | jq -r -c '. | { "index" : { } }, .' | curl -u login:password -H "Content-Type: application/json" -XPOST "https://.../skills/default/_bulk?pretty" --data-binary @-
I get an error:
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 276249, column 317
I found that the character that jq can't parse is u2022. I tried adding "-r" jq command but the error stil occurs. How can I handle this for all occurrences of u2022?
json unicode jq
edited Nov 18 at 2:55
peak
28.6k73752
asked Nov 17 at 15:17
Alit
165
I'm trying to perform a bulk upload to Elasticsearch (around 1mln documents). In order to do that, I'm using jq to reformat the JSON file extracted from MySQL database and curl to post the data to Elasticsearch:
cat dataset.json | jq -r -c '. | { "index" : { } }, .' | curl -u login:password -H "Content-Type: application/json" -XPOST "https://.../skills/default/_bulk?pretty" --data-binary @-
I get an error:
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 276249, column 317
I found that the character that jq can't parse is u2022. I tried adding "-r" jq command but the error stil occurs. How can I handle this for all occurrences of u2022?
json unicode jq
json unicode jq
edited Nov 18 at 2:55
peak
28.6k73752
asked Nov 17 at 15:17
Alit
165
edited Nov 18 at 2:55
peak
28.6k73752
asked Nov 17 at 15:17
Alit
165
edited Nov 18 at 2:55
peak
28.6k73752
edited Nov 18 at 2:55
peak
28.6k73752
edited Nov 18 at 2:55
peak
28.6k73752
28.6k73752
asked Nov 17 at 15:17
Alit
165
asked Nov 17 at 15:17
Alit
165
asked Nov 17 at 15:17
Alit
165
165
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30
add a comment |
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Here's verification that u2022 is properly handled by various versions of jq in a Mac environment:
$ echo '"u2022"' | jq-1.4 .
"•"
$ echo '"•"' | jq-1.6 .
"•"
$ echo '"•"' | jq-1.5 .
"•"
$ echo '"•"' | jq-1.4 .
"•"
$
Perhaps the problem is related to a bug that was fixed since the release of jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1311).
If you are having difficulties with jq version 1.6 (the current version), please provide a minimal complete verifiable example
with further details about the computing environment.
answered Nov 17 at 22:34
peak
28.6k73752
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Here's verification that u2022 is properly handled by various versions of jq in a Mac environment:
$ echo '"u2022"' | jq-1.4 .
"•"
$ echo '"•"' | jq-1.6 .
"•"
$ echo '"•"' | jq-1.5 .
"•"
$ echo '"•"' | jq-1.4 .
"•"
$
Perhaps the problem is related to a bug that was fixed since the release of jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1311).
If you are having difficulties with jq version 1.6 (the current version), please provide a minimal complete verifiable example
with further details about the computing environment.
answered Nov 17 at 22:34
peak
28.6k73752
add a comment |
up vote
0
down vote
Here's verification that u2022 is properly handled by various versions of jq in a Mac environment:
$ echo '"u2022"' | jq-1.4 .
"•"
$ echo '"•"' | jq-1.6 .
"•"
$ echo '"•"' | jq-1.5 .
"•"
$ echo '"•"' | jq-1.4 .
"•"
$
Perhaps the problem is related to a bug that was fixed since the release of jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1311).
If you are having difficulties with jq version 1.6 (the current version), please provide a minimal complete verifiable example
with further details about the computing environment.
answered Nov 17 at 22:34
peak
28.6k73752
add a comment |
up vote
0
down vote
up vote
0
down vote
Here's verification that u2022 is properly handled by various versions of jq in a Mac environment:
$ echo '"u2022"' | jq-1.4 .
"•"
$ echo '"•"' | jq-1.6 .
"•"
$ echo '"•"' | jq-1.5 .
"•"
$ echo '"•"' | jq-1.4 .
"•"
$
Perhaps the problem is related to a bug that was fixed since the release of jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1311).
If you are having difficulties with jq version 1.6 (the current version), please provide a minimal complete verifiable example
with further details about the computing environment.
answered Nov 17 at 22:34
peak
28.6k73752
Here's verification that u2022 is properly handled by various versions of jq in a Mac environment:
$ echo '"u2022"' | jq-1.4 .
"•"
$ echo '"•"' | jq-1.6 .
"•"
$ echo '"•"' | jq-1.5 .
"•"
$ echo '"•"' | jq-1.4 .
"•"
$
Perhaps the problem is related to a bug that was fixed since the release of jq 1.5 (see e.g. https://github.com/stedolan/jq/issues/1311).
If you are having difficulties with jq version 1.6 (the current version), please provide a minimal complete verifiable example
with further details about the computing environment.
answered Nov 17 at 22:34
peak
28.6k73752
edited Nov 18 at 8:28
answered Nov 17 at 22:34
peak
28.6k73752
answered Nov 17 at 22:34
peak
28.6k73752
answered Nov 17 at 22:34
peak
28.6k73752
28.6k73752
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53352558%2fjq-cant-parse-u2022-character%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Please follow the Minimal, Complete, and Verifiable example guidelines.
– peak
Nov 18 at 8:30