Are there neural networks with very few nodes that decently solve non-trivial problems?
$begingroup$
I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.
neural-networks
edited 10 hours ago
nbro
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.
neural-networks
edited 10 hours ago
nbro
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
1
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago
add a comment |
$begingroup$
I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.
neural-networks
edited 10 hours ago
nbro
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.
neural-networks
neural-networks
edited 10 hours ago
nbro
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 10 hours ago
nbro
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 10 hours ago
nbro
1,416621
edited 10 hours ago
nbro
1,416621
edited 10 hours ago
nbro
1,416621
1,416621
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
asked 12 hours ago
Guillermo MosseGuillermo Mosse
1435
1435
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Guillermo Mosse is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
1
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago
add a comment |
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
1
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
1
1
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
Smallest Neural Network to Learn the Ising Criticality
Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons
Just my cents:
- reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
$endgroup$
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "658"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Guillermo Mosse is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fai.stackexchange.com%2fquestions%2f10944%2fare-there-neural-networks-with-very-few-nodes-that-decently-solve-non-trivial-pr%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
Smallest Neural Network to Learn the Ising Criticality
Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons
Just my cents:
- reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
$endgroup$
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
add a comment |
$begingroup$
Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
Smallest Neural Network to Learn the Ising Criticality
Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons
Just my cents:
- reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
$endgroup$
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
add a comment |
$begingroup$
Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
Smallest Neural Network to Learn the Ising Criticality
Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons
Just my cents:
- reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
$endgroup$
Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me
Smallest Neural Network to Learn the Ising Criticality
Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons
Just my cents:
- reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
edited 8 hours ago
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
answered 10 hours ago
Nicola BerniniNicola Bernini
1262
1262
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
add a comment |
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
2
2
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right?
$endgroup$
– Guillermo Mosse
5 hours ago
add a comment |
Guillermo Mosse is a new contributor. Be nice, and check out our Code of Conduct.
Guillermo Mosse is a new contributor. Be nice, and check out our Code of Conduct.
Guillermo Mosse is a new contributor. Be nice, and check out our Code of Conduct.
Guillermo Mosse is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Artificial Intelligence Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fai.stackexchange.com%2fquestions%2f10944%2fare-there-neural-networks-with-very-few-nodes-that-decently-solve-non-trivial-pr%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Universal approximation theorem: a neural network with one hidden layer can approximate any "reasonable" function given a sufficient number of nodes in the hidden layer.
$endgroup$
– nbro
11 hours ago
1
$begingroup$
I think it will not be easy to answer your question. What do you mean by "non-trivial problem"?
$endgroup$
– nbro
11 hours ago
$begingroup$
I want few nodes even in hidden layers.
$endgroup$
– Guillermo Mosse
5 hours ago
$begingroup$
@nbro my sense of "non-trivial" in this context is intractable or unsolved.
$endgroup$
– DukeZhou♦
3 hours ago