To crawl vs crawl url [on hold]
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}
up vote
0
down vote
favorite
I'm not a native speaker and I try to understand what is the difference between the word To crawland Crawl the URL. When I type the phrase crawl in the Google I receive a picture of a soldier or little kid who is crawling. What about the phrases e.g. Crawl the URL or Getting URLs Crawledor How to Crawl a Website. Can somebody explain me the meaning in different and easier words?
british-english
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as off-topic by Jason Bassford, TimLymington, Lawrence, Chris H, cobaltduck Dec 3 at 12:46
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "Please include the research you’ve done, or consider if your question suits our English Language Learners site better. Questions that can be answered using commonly-available references are off-topic." – TimLymington, Chris H, cobaltduck
If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
up vote
0
down vote
favorite
I'm not a native speaker and I try to understand what is the difference between the word To crawland Crawl the URL. When I type the phrase crawl in the Google I receive a picture of a soldier or little kid who is crawling. What about the phrases e.g. Crawl the URL or Getting URLs Crawledor How to Crawl a Website. Can somebody explain me the meaning in different and easier words?
british-english
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as off-topic by Jason Bassford, TimLymington, Lawrence, Chris H, cobaltduck Dec 3 at 12:46
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "Please include the research you’ve done, or consider if your question suits our English Language Learners site better. Questions that can be answered using commonly-available references are off-topic." – TimLymington, Chris H, cobaltduck
If this question can be reworded to fit the rules in the help center, please edit the question.
1
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm not a native speaker and I try to understand what is the difference between the word To crawland Crawl the URL. When I type the phrase crawl in the Google I receive a picture of a soldier or little kid who is crawling. What about the phrases e.g. Crawl the URL or Getting URLs Crawledor How to Crawl a Website. Can somebody explain me the meaning in different and easier words?
british-english
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I'm not a native speaker and I try to understand what is the difference between the word To crawland Crawl the URL. When I type the phrase crawl in the Google I receive a picture of a soldier or little kid who is crawling. What about the phrases e.g. Crawl the URL or Getting URLs Crawledor How to Crawl a Website. Can somebody explain me the meaning in different and easier words?
british-english
british-english
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Dec 3 at 12:07
GoldenAge
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Dec 3 at 12:07
GoldenAge
104
asked Dec 3 at 12:07
GoldenAge
104
104
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
GoldenAge is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as off-topic by Jason Bassford, TimLymington, Lawrence, Chris H, cobaltduck Dec 3 at 12:46
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "Please include the research you’ve done, or consider if your question suits our English Language Learners site better. Questions that can be answered using commonly-available references are off-topic." – TimLymington, Chris H, cobaltduck
If this question can be reworded to fit the rules in the help center, please edit the question.
put on hold as off-topic by Jason Bassford, TimLymington, Lawrence, Chris H, cobaltduck Dec 3 at 12:46
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "Please include the research you’ve done, or consider if your question suits our English Language Learners site better. Questions that can be answered using commonly-available references are off-topic." – TimLymington, Chris H, cobaltduck
If this question can be reworded to fit the rules in the help center, please edit the question.
1
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37
add a comment |
1
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37
1
1
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
From Wikipedia:
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers copy pages for processing by a search engine which indexes the downloaded pages so users can search more efficiently.
A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies. If the crawler is performing archiving of websites it copies and saves the information as it goes. The archives are usually stored in such a way they can be viewed, read and navigated as they were on the live web, but are preserved as ‘snapshots'
A web crawler is also known as a spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter
The connection with crawling and spiders is related to the word web, used in world wide web and alluding to a spider web, which can be seen as interconnected strands linking different pieces into one large "thing".
answered Dec 3 at 12:26
bookmanu
4,3701334
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
From Wikipedia:
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers copy pages for processing by a search engine which indexes the downloaded pages so users can search more efficiently.
A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies. If the crawler is performing archiving of websites it copies and saves the information as it goes. The archives are usually stored in such a way they can be viewed, read and navigated as they were on the live web, but are preserved as ‘snapshots'
A web crawler is also known as a spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter
The connection with crawling and spiders is related to the word web, used in world wide web and alluding to a spider web, which can be seen as interconnected strands linking different pieces into one large "thing".
answered Dec 3 at 12:26
bookmanu
4,3701334
add a comment |
up vote
0
down vote
From Wikipedia:
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers copy pages for processing by a search engine which indexes the downloaded pages so users can search more efficiently.
A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies. If the crawler is performing archiving of websites it copies and saves the information as it goes. The archives are usually stored in such a way they can be viewed, read and navigated as they were on the live web, but are preserved as ‘snapshots'
A web crawler is also known as a spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter
The connection with crawling and spiders is related to the word web, used in world wide web and alluding to a spider web, which can be seen as interconnected strands linking different pieces into one large "thing".
answered Dec 3 at 12:26
bookmanu
4,3701334
add a comment |
up vote
0
down vote
up vote
0
down vote
From Wikipedia:
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers copy pages for processing by a search engine which indexes the downloaded pages so users can search more efficiently.
A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies. If the crawler is performing archiving of websites it copies and saves the information as it goes. The archives are usually stored in such a way they can be viewed, read and navigated as they were on the live web, but are preserved as ‘snapshots'
A web crawler is also known as a spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter
The connection with crawling and spiders is related to the word web, used in world wide web and alluding to a spider web, which can be seen as interconnected strands linking different pieces into one large "thing".
answered Dec 3 at 12:26
bookmanu
4,3701334
From Wikipedia:
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers copy pages for processing by a search engine which indexes the downloaded pages so users can search more efficiently.
A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies. If the crawler is performing archiving of websites it copies and saves the information as it goes. The archives are usually stored in such a way they can be viewed, read and navigated as they were on the live web, but are preserved as ‘snapshots'
A web crawler is also known as a spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter
The connection with crawling and spiders is related to the word web, used in world wide web and alluding to a spider web, which can be seen as interconnected strands linking different pieces into one large "thing".
answered Dec 3 at 12:26
bookmanu
4,3701334
edited Dec 3 at 12:32
answered Dec 3 at 12:26
bookmanu
4,3701334
answered Dec 3 at 12:26
bookmanu
4,3701334
answered Dec 3 at 12:26
bookmanu
4,3701334
4,3701334
add a comment |
add a comment |
1
Have you looked up a dictionary? Compare meaning #1 and meaning #4. Your question is off-topic (no research). Our Help Centre advises: "don’t ask any questions about the following topics. They are out of scope for this site. #The meaning of words, or synonyms for words, unless you have first looked them up in a dictionary or thesaurus." See How to Ask for further guidance. :-)
– Chappo
Dec 3 at 12:22
In particular definition 1.1 at @Chappo's link is almost the entire answer, as the spider metaphor is widely used for web crawlers, following the threads that make up the web
– Chris H
Dec 3 at 12:37