sed like stream editor for open files using open syscall
I'm working with a software which reads a trace file with multiple entries, each of which are of format:
<...>,<...>,<64 character string>
This 64 character string is in my case constitute entirely of '0'.
Would it be possible to have something like sed for file streams (not stdin, file stream opened using open syscall) which would allow me to feed the said program a file with following entries:
<...>,<...>,&
and the program would replace '&' with 64 '0's. The problem with using sed on the trace file itself before executing the program is the limited space available on the machine.
My current plan is to use ptrace to intercept read syscalls and make it appear for the program as if it is reading a normal file, would that be a good idea?
(edit)
The program parses the file only once.
linux
asked Dec 20 '18 at 14:06
lollol
236
migrated from superuser.com Dec 20 '18 at 16:53
This question came from our site for computer enthusiasts and power users.
|
show 2 more comments
I'm working with a software which reads a trace file with multiple entries, each of which are of format:
<...>,<...>,<64 character string>
This 64 character string is in my case constitute entirely of '0'.
Would it be possible to have something like sed for file streams (not stdin, file stream opened using open syscall) which would allow me to feed the said program a file with following entries:
<...>,<...>,&
and the program would replace '&' with 64 '0's. The problem with using sed on the trace file itself before executing the program is the limited space available on the machine.
My current plan is to use ptrace to intercept read syscalls and make it appear for the program as if it is reading a normal file, would that be a good idea?
(edit)
The program parses the file only once.
linux
asked Dec 20 '18 at 14:06
lollol
236
migrated from superuser.com Dec 20 '18 at 16:53
This question came from our site for computer enthusiasts and power users.
1
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
Is the file opened by the program a file named on the command line? In that case you should still be able to give it/dev/stdinas the filename while pipingsedto it.
– Kusalananda
Dec 20 '18 at 17:22
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02
|
show 2 more comments
I'm working with a software which reads a trace file with multiple entries, each of which are of format:
<...>,<...>,<64 character string>
This 64 character string is in my case constitute entirely of '0'.
Would it be possible to have something like sed for file streams (not stdin, file stream opened using open syscall) which would allow me to feed the said program a file with following entries:
<...>,<...>,&
and the program would replace '&' with 64 '0's. The problem with using sed on the trace file itself before executing the program is the limited space available on the machine.
My current plan is to use ptrace to intercept read syscalls and make it appear for the program as if it is reading a normal file, would that be a good idea?
(edit)
The program parses the file only once.
linux
asked Dec 20 '18 at 14:06
lollol
236
I'm working with a software which reads a trace file with multiple entries, each of which are of format:
<...>,<...>,<64 character string>
This 64 character string is in my case constitute entirely of '0'.
Would it be possible to have something like sed for file streams (not stdin, file stream opened using open syscall) which would allow me to feed the said program a file with following entries:
<...>,<...>,&
and the program would replace '&' with 64 '0's. The problem with using sed on the trace file itself before executing the program is the limited space available on the machine.
My current plan is to use ptrace to intercept read syscalls and make it appear for the program as if it is reading a normal file, would that be a good idea?
(edit)
The program parses the file only once.
linux
linux
asked Dec 20 '18 at 14:06
lollol
236
asked Dec 20 '18 at 14:06
lollol
236
edited Dec 20 '18 at 18:18
lol
asked Dec 20 '18 at 14:06
lollol
236
asked Dec 20 '18 at 14:06
lollol
236
asked Dec 20 '18 at 14:06
lollol
236
236
migrated from superuser.com Dec 20 '18 at 16:53
This question came from our site for computer enthusiasts and power users.
migrated from superuser.com Dec 20 '18 at 16:53
This question came from our site for computer enthusiasts and power users.
1
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
Is the file opened by the program a file named on the command line? In that case you should still be able to give it/dev/stdinas the filename while pipingsedto it.
– Kusalananda
Dec 20 '18 at 17:22
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02
|
show 2 more comments
1
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
Is the file opened by the program a file named on the command line? In that case you should still be able to give it/dev/stdinas the filename while pipingsedto it.
– Kusalananda
Dec 20 '18 at 17:22
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02
1
1
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
Is the file opened by the program a file named on the command line? In that case you should still be able to give it
/dev/stdin as the filename while piping sed to it.– Kusalananda
Dec 20 '18 at 17:22
Is the file opened by the program a file named on the command line? In that case you should still be able to give it
/dev/stdin as the filename while piping sed to it.– Kusalananda
Dec 20 '18 at 17:22
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02
|
show 2 more comments
1 Answer
1
active
oldest
votes
sed 's/,[^,]*$/,&/' <file | program /dev/stdin
This assumes that the program must be given the filename on the command line and that it does not try to do seeks backwards or forwards in the file (standard input is not generally seekable).
The special file /dev/stdin will contain the contents of the standard input stream of the process. Above, we use this fact to send the modified contents of file to the program using a standard Unix pipe on the command line, even though the program normally does not read its standard input stream.
The sed expression will replace everything after the last comma on each line of file with a &.
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490155%2fsed-like-stream-editor-for-open-files-using-open-syscall%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
sed 's/,[^,]*$/,&/' <file | program /dev/stdin
This assumes that the program must be given the filename on the command line and that it does not try to do seeks backwards or forwards in the file (standard input is not generally seekable).
The special file /dev/stdin will contain the contents of the standard input stream of the process. Above, we use this fact to send the modified contents of file to the program using a standard Unix pipe on the command line, even though the program normally does not read its standard input stream.
The sed expression will replace everything after the last comma on each line of file with a &.
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
add a comment |
sed 's/,[^,]*$/,&/' <file | program /dev/stdin
This assumes that the program must be given the filename on the command line and that it does not try to do seeks backwards or forwards in the file (standard input is not generally seekable).
The special file /dev/stdin will contain the contents of the standard input stream of the process. Above, we use this fact to send the modified contents of file to the program using a standard Unix pipe on the command line, even though the program normally does not read its standard input stream.
The sed expression will replace everything after the last comma on each line of file with a &.
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
add a comment |
sed 's/,[^,]*$/,&/' <file | program /dev/stdin
This assumes that the program must be given the filename on the command line and that it does not try to do seeks backwards or forwards in the file (standard input is not generally seekable).
The special file /dev/stdin will contain the contents of the standard input stream of the process. Above, we use this fact to send the modified contents of file to the program using a standard Unix pipe on the command line, even though the program normally does not read its standard input stream.
The sed expression will replace everything after the last comma on each line of file with a &.
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
sed 's/,[^,]*$/,&/' <file | program /dev/stdin
This assumes that the program must be given the filename on the command line and that it does not try to do seeks backwards or forwards in the file (standard input is not generally seekable).
The special file /dev/stdin will contain the contents of the standard input stream of the process. Above, we use this fact to send the modified contents of file to the program using a standard Unix pipe on the command line, even though the program normally does not read its standard input stream.
The sed expression will replace everything after the last comma on each line of file with a &.
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
answered Dec 20 '18 at 18:18
KusalanandaKusalananda
124k16234385
124k16234385
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
add a comment |
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
What if the program seeks through the file? Can the output be written to some other file?
– lol
Dec 20 '18 at 18:21
1
1
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
@lol I'm not 100% certain, but skipping forwards may possibly be ok, but I would avoid it (I simply haven't tested this). The ideal situation would be to fully preprocess the input file and then give it to the program.
– Kusalananda
Dec 20 '18 at 18:35
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490155%2fsed-like-stream-editor-for-open-files-using-open-syscall%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
@Mawg you are right, it belongs to unix.stackexchange more than it does here. Is it possible to move this question there? Thanks
– lol
Dec 20 '18 at 14:32
A moderator can, but I can't :-/ I have flagged it, so hopefully it will be migrated. Wait a day or so, and if no migration, then delete this & post again there, I guess. If they can't help on Unix & Linux, we can probably recommend an app for it on S/W recommendations.
– Mawg
Dec 20 '18 at 14:36
Is the file opened by the program a file named on the command line? In that case you should still be able to give it
/dev/stdinas the filename while pipingsedto it.– Kusalananda
Dec 20 '18 at 17:22
@Kusalananda That worked, thanks! Consider answering the question.
– lol
Dec 20 '18 at 17:55
@Kusalananda Would this work if the program didn't read the input file line by line?
– lol
Dec 20 '18 at 18:02