Bitwise operations on strings Python3.7
up vote
1
down vote
favorite
I have a large string of hex numbers, on which I want to perform some bitwise operations. I am trying to extract different bytes from it and then to apply bitwise OR, AND, XOR operations. Unfortunately, I don't know an easy way to do that, so every time I want to perform bitwise operations I am converting the hex into an integer. This is a simplified code.
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
hex_data = "0x" + data[20:32]
four_bytes = hex_data[:10]
fifth_byte = "0x" + hex_data[10:12]
lshift_fourb = hex(int(four_bytes, 16) << 1)
bitwise_or_res = hex(int(lshift_fourb, 16) | int(fifth_byte, 16))
Is there an easy way to omit the constant conversion to integer and hex back and forth in order to do the same operations. I prefer to use either hex or binary since I need to extract certain bytes from the input data string and the hex(int(hex_number, 16))
seems a bit too repetitive and tedious. If I don't convert it to an integer, Python is complaining that it cannot execute the |
or ^
on strings.
python python-3.x bit-manipulation bitwise-operators
|
show 4 more comments
up vote
1
down vote
favorite
I have a large string of hex numbers, on which I want to perform some bitwise operations. I am trying to extract different bytes from it and then to apply bitwise OR, AND, XOR operations. Unfortunately, I don't know an easy way to do that, so every time I want to perform bitwise operations I am converting the hex into an integer. This is a simplified code.
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
hex_data = "0x" + data[20:32]
four_bytes = hex_data[:10]
fifth_byte = "0x" + hex_data[10:12]
lshift_fourb = hex(int(four_bytes, 16) << 1)
bitwise_or_res = hex(int(lshift_fourb, 16) | int(fifth_byte, 16))
Is there an easy way to omit the constant conversion to integer and hex back and forth in order to do the same operations. I prefer to use either hex or binary since I need to extract certain bytes from the input data string and the hex(int(hex_number, 16))
seems a bit too repetitive and tedious. If I don't convert it to an integer, Python is complaining that it cannot execute the |
or ^
on strings.
python python-3.x bit-manipulation bitwise-operators
1
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
1
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
1
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
1
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36
|
show 4 more comments
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have a large string of hex numbers, on which I want to perform some bitwise operations. I am trying to extract different bytes from it and then to apply bitwise OR, AND, XOR operations. Unfortunately, I don't know an easy way to do that, so every time I want to perform bitwise operations I am converting the hex into an integer. This is a simplified code.
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
hex_data = "0x" + data[20:32]
four_bytes = hex_data[:10]
fifth_byte = "0x" + hex_data[10:12]
lshift_fourb = hex(int(four_bytes, 16) << 1)
bitwise_or_res = hex(int(lshift_fourb, 16) | int(fifth_byte, 16))
Is there an easy way to omit the constant conversion to integer and hex back and forth in order to do the same operations. I prefer to use either hex or binary since I need to extract certain bytes from the input data string and the hex(int(hex_number, 16))
seems a bit too repetitive and tedious. If I don't convert it to an integer, Python is complaining that it cannot execute the |
or ^
on strings.
python python-3.x bit-manipulation bitwise-operators
I have a large string of hex numbers, on which I want to perform some bitwise operations. I am trying to extract different bytes from it and then to apply bitwise OR, AND, XOR operations. Unfortunately, I don't know an easy way to do that, so every time I want to perform bitwise operations I am converting the hex into an integer. This is a simplified code.
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
hex_data = "0x" + data[20:32]
four_bytes = hex_data[:10]
fifth_byte = "0x" + hex_data[10:12]
lshift_fourb = hex(int(four_bytes, 16) << 1)
bitwise_or_res = hex(int(lshift_fourb, 16) | int(fifth_byte, 16))
Is there an easy way to omit the constant conversion to integer and hex back and forth in order to do the same operations. I prefer to use either hex or binary since I need to extract certain bytes from the input data string and the hex(int(hex_number, 16))
seems a bit too repetitive and tedious. If I don't convert it to an integer, Python is complaining that it cannot execute the |
or ^
on strings.
python python-3.x bit-manipulation bitwise-operators
python python-3.x bit-manipulation bitwise-operators
asked Nov 19 at 10:27
Georgе Stoyanov
7712
7712
1
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
1
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
1
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
1
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36
|
show 4 more comments
1
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
1
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
1
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
1
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36
1
1
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
1
1
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
1
1
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
1
1
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36
|
show 4 more comments
2 Answers
2
active
oldest
votes
up vote
2
down vote
accepted
How about this:
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
size = (len(data)-2)//2
data_bytes = int(data,16).to_bytes(size,byteorder='big')
Now you can do this:
data_bytes[4] & data_bytes[5]
this seems to work, but I don't know why, when I printdata_bytes
it gives meb'+Cxf5xe3~m'
I am expecting something likeb'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes fromdata_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.
– Georgе Stoyanov
Nov 19 at 11:26
1
I have understood why the format of the data_bytes, I need to print them with.hex()
at the end, otherwise, it is trying to match the ASCII codes.
– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to executedata_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.
– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers usingint.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.
– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printingdata_bytes
produces the desired result. Note that I've edited the code in the answer.
– Yakov Dan
Nov 19 at 13:04
|
show 2 more comments
up vote
2
down vote
As you mentioned, the problem is that strings of hex digits are more readable and good for slicing and at least some shifting operations, but don't support bitwise operations. On the other hand, integers support bitwise operations but no slicing. The closest to what you want may be to create a custom class implementing both features (doing conversion whenever needed). This won't save you from implementing (and executing) the more complex code, but the rest of your application may be more readable because the conversions are "hidden".
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
How about this:
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
size = (len(data)-2)//2
data_bytes = int(data,16).to_bytes(size,byteorder='big')
Now you can do this:
data_bytes[4] & data_bytes[5]
this seems to work, but I don't know why, when I printdata_bytes
it gives meb'+Cxf5xe3~m'
I am expecting something likeb'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes fromdata_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.
– Georgе Stoyanov
Nov 19 at 11:26
1
I have understood why the format of the data_bytes, I need to print them with.hex()
at the end, otherwise, it is trying to match the ASCII codes.
– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to executedata_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.
– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers usingint.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.
– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printingdata_bytes
produces the desired result. Note that I've edited the code in the answer.
– Yakov Dan
Nov 19 at 13:04
|
show 2 more comments
up vote
2
down vote
accepted
How about this:
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
size = (len(data)-2)//2
data_bytes = int(data,16).to_bytes(size,byteorder='big')
Now you can do this:
data_bytes[4] & data_bytes[5]
this seems to work, but I don't know why, when I printdata_bytes
it gives meb'+Cxf5xe3~m'
I am expecting something likeb'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes fromdata_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.
– Georgе Stoyanov
Nov 19 at 11:26
1
I have understood why the format of the data_bytes, I need to print them with.hex()
at the end, otherwise, it is trying to match the ASCII codes.
– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to executedata_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.
– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers usingint.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.
– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printingdata_bytes
produces the desired result. Note that I've edited the code in the answer.
– Yakov Dan
Nov 19 at 13:04
|
show 2 more comments
up vote
2
down vote
accepted
up vote
2
down vote
accepted
How about this:
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
size = (len(data)-2)//2
data_bytes = int(data,16).to_bytes(size,byteorder='big')
Now you can do this:
data_bytes[4] & data_bytes[5]
How about this:
data = "0x4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37"
size = (len(data)-2)//2
data_bytes = int(data,16).to_bytes(size,byteorder='big')
Now you can do this:
data_bytes[4] & data_bytes[5]
edited Nov 19 at 13:02
answered Nov 19 at 10:57
Yakov Dan
429211
429211
this seems to work, but I don't know why, when I printdata_bytes
it gives meb'+Cxf5xe3~m'
I am expecting something likeb'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes fromdata_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.
– Georgе Stoyanov
Nov 19 at 11:26
1
I have understood why the format of the data_bytes, I need to print them with.hex()
at the end, otherwise, it is trying to match the ASCII codes.
– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to executedata_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.
– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers usingint.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.
– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printingdata_bytes
produces the desired result. Note that I've edited the code in the answer.
– Yakov Dan
Nov 19 at 13:04
|
show 2 more comments
this seems to work, but I don't know why, when I printdata_bytes
it gives meb'+Cxf5xe3~m'
I am expecting something likeb'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes fromdata_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.
– Georgе Stoyanov
Nov 19 at 11:26
1
I have understood why the format of the data_bytes, I need to print them with.hex()
at the end, otherwise, it is trying to match the ASCII codes.
– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to executedata_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.
– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers usingint.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.
– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printingdata_bytes
produces the desired result. Note that I've edited the code in the answer.
– Yakov Dan
Nov 19 at 13:04
this seems to work, but I don't know why, when I print
data_bytes
it gives me b'+Cxf5xe3~m'
I am expecting something like b'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes from data_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.– Georgе Stoyanov
Nov 19 at 11:26
this seems to work, but I don't know why, when I print
data_bytes
it gives me b'+Cxf5xe3~m'
I am expecting something like b'x2bx43xf5xe3x7ex6d
and also how I can extract the first 4 bytes from data_bytes
. I am also not able to perform bitwise operations on bytes and integers, so I need either to convert the bytes to integers or convert the integers to bytes.– Georgе Stoyanov
Nov 19 at 11:26
1
1
I have understood why the format of the data_bytes, I need to print them with
.hex()
at the end, otherwise, it is trying to match the ASCII codes.– Georgе Stoyanov
Nov 19 at 11:34
I have understood why the format of the data_bytes, I need to print them with
.hex()
at the end, otherwise, it is trying to match the ASCII codes.– Georgе Stoyanov
Nov 19 at 11:34
I mean when I try to execute
data_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.– Georgе Stoyanov
Nov 19 at 11:41
I mean when I try to execute
data_byte[3] << bytes(1)
it gives me an error that the left shift operation is not supported for 'bytes' objects.– Georgе Stoyanov
Nov 19 at 11:41
As a workaround I need to convert the bytes to integers using
int.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.– Georgе Stoyanov
Nov 19 at 11:47
As a workaround I need to convert the bytes to integers using
int.from_bytes(data_bytes, byteorder="big")
in order to perform the bitwise left or right shift.– Georgе Stoyanov
Nov 19 at 11:47
at least on my machine, printing
data_bytes
produces the desired result. Note that I've edited the code in the answer.– Yakov Dan
Nov 19 at 13:04
at least on my machine, printing
data_bytes
produces the desired result. Note that I've edited the code in the answer.– Yakov Dan
Nov 19 at 13:04
|
show 2 more comments
up vote
2
down vote
As you mentioned, the problem is that strings of hex digits are more readable and good for slicing and at least some shifting operations, but don't support bitwise operations. On the other hand, integers support bitwise operations but no slicing. The closest to what you want may be to create a custom class implementing both features (doing conversion whenever needed). This won't save you from implementing (and executing) the more complex code, but the rest of your application may be more readable because the conversions are "hidden".
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
add a comment |
up vote
2
down vote
As you mentioned, the problem is that strings of hex digits are more readable and good for slicing and at least some shifting operations, but don't support bitwise operations. On the other hand, integers support bitwise operations but no slicing. The closest to what you want may be to create a custom class implementing both features (doing conversion whenever needed). This won't save you from implementing (and executing) the more complex code, but the rest of your application may be more readable because the conversions are "hidden".
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
add a comment |
up vote
2
down vote
up vote
2
down vote
As you mentioned, the problem is that strings of hex digits are more readable and good for slicing and at least some shifting operations, but don't support bitwise operations. On the other hand, integers support bitwise operations but no slicing. The closest to what you want may be to create a custom class implementing both features (doing conversion whenever needed). This won't save you from implementing (and executing) the more complex code, but the rest of your application may be more readable because the conversions are "hidden".
As you mentioned, the problem is that strings of hex digits are more readable and good for slicing and at least some shifting operations, but don't support bitwise operations. On the other hand, integers support bitwise operations but no slicing. The closest to what you want may be to create a custom class implementing both features (doing conversion whenever needed). This won't save you from implementing (and executing) the more complex code, but the rest of your application may be more readable because the conversions are "hidden".
answered Nov 19 at 11:26
jtd
986
986
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
add a comment |
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
exactly my point. Hex is perfect for slicing but then I need to convert it all the time in order to execute bitwise operations.
– Georgе Stoyanov
Nov 19 at 11:28
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53372602%2fbitwise-operations-on-strings-python3-7%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Convert the string to an integer once, then do all operations in the integer domain, and only convert back to a string once at the end.
– mkrieger1
Nov 19 at 10:30
1
@mkrieger1 Integer, however, doesn't support slicing, thus calculating hex_data and four_bytes would have to be calculated with modulo. Possible, but tedious as well
– Michal Polovka
Nov 19 at 10:33
but how I am supposed to extract specific bytes from an integer?
– Georgе Stoyanov
Nov 19 at 10:33
1
By shifting and masking.
– mkrieger1
Nov 19 at 10:34
1
You can also write a few helper functions to factor out the repetitive stuff.
– mkrieger1
Nov 19 at 10:36