What are the foundations of probability and how are they dependent upon a $sigma$-field?
up vote
9
down vote
favorite
I am reading Christopher D. Manning's Foundations of Statistical Natural Language Processing which gives an introduction on Probability Theory where it talks about $sigma$-fields. It says,
The foundations of probability theory depend on the set of events $mathscr{F}$ forming a $sigma$-field".
I understand the definition of a $sigma$-field, but what are these foundations of probability theory, and how are these foundations dependent upon a $sigma$-field?
probability-theory measure-theory
edited yesterday
Mike Pierce
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
add a comment |
up vote
9
down vote
favorite
I am reading Christopher D. Manning's Foundations of Statistical Natural Language Processing which gives an introduction on Probability Theory where it talks about $sigma$-fields. It says,
The foundations of probability theory depend on the set of events $mathscr{F}$ forming a $sigma$-field".
I understand the definition of a $sigma$-field, but what are these foundations of probability theory, and how are these foundations dependent upon a $sigma$-field?
probability-theory measure-theory
edited yesterday
Mike Pierce
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
1
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
5
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
1
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
2
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday
add a comment |
up vote
9
down vote
favorite
up vote
9
down vote
favorite
I am reading Christopher D. Manning's Foundations of Statistical Natural Language Processing which gives an introduction on Probability Theory where it talks about $sigma$-fields. It says,
The foundations of probability theory depend on the set of events $mathscr{F}$ forming a $sigma$-field".
I understand the definition of a $sigma$-field, but what are these foundations of probability theory, and how are these foundations dependent upon a $sigma$-field?
probability-theory measure-theory
edited yesterday
Mike Pierce
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
I am reading Christopher D. Manning's Foundations of Statistical Natural Language Processing which gives an introduction on Probability Theory where it talks about $sigma$-fields. It says,
The foundations of probability theory depend on the set of events $mathscr{F}$ forming a $sigma$-field".
I understand the definition of a $sigma$-field, but what are these foundations of probability theory, and how are these foundations dependent upon a $sigma$-field?
probability-theory measure-theory
probability-theory measure-theory
edited yesterday
Mike Pierce
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
edited yesterday
Mike Pierce
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
edited yesterday
Mike Pierce
11.4k103583
edited yesterday
Mike Pierce
11.4k103583
edited yesterday
Mike Pierce
11.4k103583
11.4k103583
asked Dec 15 at 23:03
eddard.stark
1593
asked Dec 15 at 23:03
eddard.stark
1593
asked Dec 15 at 23:03
eddard.stark
1593
1593
1
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
5
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
1
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
2
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday
add a comment |
1
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
5
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
1
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
2
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday
1
1
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
5
5
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
1
1
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
2
2
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday
add a comment |
1 Answer
1
active
oldest
votes
up vote
16
down vote
accepted
Probability when there are only finitely many outcomes is a matter of counting. There are $36$ possible results from a roll of two dice and $6$ of them sum to $7$ so the probability of a sum of $7$ is $6/36$. You've measured the size of the set of outcomes that you are interested in.
It's harder to make rigorous sense of things when the set of possible results is infinite. What does it mean to choose two numbers at random in the interval $[1,6]$ and ask for their sum? Any particular pair, like $(1.3, pi)$, will have probability $0$.
You deal with this problem by replacing counting with integration. Unfortunately, the integration you learn in first year calculus ("Riemann integration") isn't powerful enough to derive all you need about probability. (It is enough to determine the probability that your two rolls total exactly $7$ is $0$, and to find the probability that it's at least $7$.)
For the definitions and theorems of rigorous probability theory (those are the "foundations" you ask about) you need "Lebesgue integration". That requires first carefully specifying the sets that you are going to ask for the probabilities of - and not every set is allowed, for technical reasons without which you can't make the mathematics work the way you want. It turns out that the set of sets whose probability you are going to ask about carries the name "$sigma$-field" or "sigma algebra". (It's not a field in the arithmetic sense.)
The essential point is that it's closed under countable set operations. That's what the "$sigma$" says. Your text may not provide a formal definition - you may not need it for NLP applications.
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3042059%2fwhat-are-the-foundations-of-probability-and-how-are-they-dependent-upon-a-sigm%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
16
down vote
accepted
Probability when there are only finitely many outcomes is a matter of counting. There are $36$ possible results from a roll of two dice and $6$ of them sum to $7$ so the probability of a sum of $7$ is $6/36$. You've measured the size of the set of outcomes that you are interested in.
It's harder to make rigorous sense of things when the set of possible results is infinite. What does it mean to choose two numbers at random in the interval $[1,6]$ and ask for their sum? Any particular pair, like $(1.3, pi)$, will have probability $0$.
You deal with this problem by replacing counting with integration. Unfortunately, the integration you learn in first year calculus ("Riemann integration") isn't powerful enough to derive all you need about probability. (It is enough to determine the probability that your two rolls total exactly $7$ is $0$, and to find the probability that it's at least $7$.)
For the definitions and theorems of rigorous probability theory (those are the "foundations" you ask about) you need "Lebesgue integration". That requires first carefully specifying the sets that you are going to ask for the probabilities of - and not every set is allowed, for technical reasons without which you can't make the mathematics work the way you want. It turns out that the set of sets whose probability you are going to ask about carries the name "$sigma$-field" or "sigma algebra". (It's not a field in the arithmetic sense.)
The essential point is that it's closed under countable set operations. That's what the "$sigma$" says. Your text may not provide a formal definition - you may not need it for NLP applications.
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
add a comment |
up vote
16
down vote
accepted
Probability when there are only finitely many outcomes is a matter of counting. There are $36$ possible results from a roll of two dice and $6$ of them sum to $7$ so the probability of a sum of $7$ is $6/36$. You've measured the size of the set of outcomes that you are interested in.
It's harder to make rigorous sense of things when the set of possible results is infinite. What does it mean to choose two numbers at random in the interval $[1,6]$ and ask for their sum? Any particular pair, like $(1.3, pi)$, will have probability $0$.
You deal with this problem by replacing counting with integration. Unfortunately, the integration you learn in first year calculus ("Riemann integration") isn't powerful enough to derive all you need about probability. (It is enough to determine the probability that your two rolls total exactly $7$ is $0$, and to find the probability that it's at least $7$.)
For the definitions and theorems of rigorous probability theory (those are the "foundations" you ask about) you need "Lebesgue integration". That requires first carefully specifying the sets that you are going to ask for the probabilities of - and not every set is allowed, for technical reasons without which you can't make the mathematics work the way you want. It turns out that the set of sets whose probability you are going to ask about carries the name "$sigma$-field" or "sigma algebra". (It's not a field in the arithmetic sense.)
The essential point is that it's closed under countable set operations. That's what the "$sigma$" says. Your text may not provide a formal definition - you may not need it for NLP applications.
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
add a comment |
up vote
16
down vote
accepted
up vote
16
down vote
accepted
Probability when there are only finitely many outcomes is a matter of counting. There are $36$ possible results from a roll of two dice and $6$ of them sum to $7$ so the probability of a sum of $7$ is $6/36$. You've measured the size of the set of outcomes that you are interested in.
It's harder to make rigorous sense of things when the set of possible results is infinite. What does it mean to choose two numbers at random in the interval $[1,6]$ and ask for their sum? Any particular pair, like $(1.3, pi)$, will have probability $0$.
You deal with this problem by replacing counting with integration. Unfortunately, the integration you learn in first year calculus ("Riemann integration") isn't powerful enough to derive all you need about probability. (It is enough to determine the probability that your two rolls total exactly $7$ is $0$, and to find the probability that it's at least $7$.)
For the definitions and theorems of rigorous probability theory (those are the "foundations" you ask about) you need "Lebesgue integration". That requires first carefully specifying the sets that you are going to ask for the probabilities of - and not every set is allowed, for technical reasons without which you can't make the mathematics work the way you want. It turns out that the set of sets whose probability you are going to ask about carries the name "$sigma$-field" or "sigma algebra". (It's not a field in the arithmetic sense.)
The essential point is that it's closed under countable set operations. That's what the "$sigma$" says. Your text may not provide a formal definition - you may not need it for NLP applications.
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
Probability when there are only finitely many outcomes is a matter of counting. There are $36$ possible results from a roll of two dice and $6$ of them sum to $7$ so the probability of a sum of $7$ is $6/36$. You've measured the size of the set of outcomes that you are interested in.
It's harder to make rigorous sense of things when the set of possible results is infinite. What does it mean to choose two numbers at random in the interval $[1,6]$ and ask for their sum? Any particular pair, like $(1.3, pi)$, will have probability $0$.
You deal with this problem by replacing counting with integration. Unfortunately, the integration you learn in first year calculus ("Riemann integration") isn't powerful enough to derive all you need about probability. (It is enough to determine the probability that your two rolls total exactly $7$ is $0$, and to find the probability that it's at least $7$.)
For the definitions and theorems of rigorous probability theory (those are the "foundations" you ask about) you need "Lebesgue integration". That requires first carefully specifying the sets that you are going to ask for the probabilities of - and not every set is allowed, for technical reasons without which you can't make the mathematics work the way you want. It turns out that the set of sets whose probability you are going to ask about carries the name "$sigma$-field" or "sigma algebra". (It's not a field in the arithmetic sense.)
The essential point is that it's closed under countable set operations. That's what the "$sigma$" says. Your text may not provide a formal definition - you may not need it for NLP applications.
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
edited Dec 16 at 22:17
Lord_Farin
15.5k636108
15.5k636108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
answered Dec 16 at 1:00
Ethan Bolker
40.8k546108
40.8k546108
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
add a comment |
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
I am reading it a couple of times. I think it is going to take me some time to understand all of it. Thanks for the answer anyways.
– eddard.stark
Dec 16 at 1:43
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3042059%2fwhat-are-the-foundations-of-probability-and-how-are-they-dependent-upon-a-sigm%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
I am reading "Foundations of Statistical Natural Language Processing" by Christopher D. Manning
– eddard.stark
Dec 16 at 0:47
5
But this is not a textbook on probability theory. This is a book on Natural Language Processing which includes some preliminaries of probability theory -- this is not the same thing. If you want to understand the foundations of probability, consult a textbook on that, not on something that merely uses probabilities.
– Clement C.
Dec 16 at 0:48
1
@eddard Some people are concerned with the scope of your question being too broad ("foundations of probability theory" is a topic of considerable size). Would you mind editing the post to be only about the second bullet point?
– Lord_Farin
Dec 16 at 22:00
2
@Lord_Farin someone tried to focus it and improve the title. Then someone didn't like it and rolled it back. Not sure why, since that would make this a good question.
– Don Hatch
yesterday