Framework GoColly - Redirect to HTTPS doesn't work
up vote
0
down vote
favorite
I am using the framework GoColly to get the body of some urls. Right now I am testing the Netflix website and I have this code below
u = "http://netflix.com"
uri, err := url.Parse(u)
if err != nil {
fmt.Println(err.Error())
continue
}
c := colly.NewCollector()
c.AllowedDomains = string{"netflix.com"}
c.OnResponse(func(r *colly.Response) {
q.r.Set("success:"+u, string(r.Body))
})
c.OnError(func(r *colly.Response, err error) {
log.Println(err.Error())
q.r.Set("failed:"+u, err.Error())
})
c.Visit(uri.String())
c.Wait()
and when I execute it retuns this error Get https://www.netflix.com/: Not following redirect to www.netflix.com because its not in AllowedDomains
any thoughts?
go asked Nov 16 at 23:38
lucas alcantara
737
add a comment |
up vote
0
down vote
favorite
I am using the framework GoColly to get the body of some urls. Right now I am testing the Netflix website and I have this code below
u = "http://netflix.com"
uri, err := url.Parse(u)
if err != nil {
fmt.Println(err.Error())
continue
}
c := colly.NewCollector()
c.AllowedDomains = string{"netflix.com"}
c.OnResponse(func(r *colly.Response) {
q.r.Set("success:"+u, string(r.Body))
})
c.OnError(func(r *colly.Response, err error) {
log.Println(err.Error())
q.r.Set("failed:"+u, err.Error())
})
c.Visit(uri.String())
c.Wait()
and when I execute it retuns this error Get https://www.netflix.com/: Not following redirect to www.netflix.com because its not in AllowedDomains
any thoughts?
go asked Nov 16 at 23:38
lucas alcantara
737
1
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting tohttps://www.netflix.com(notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.
– Carson Hoffman
Nov 16 at 23:46
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am using the framework GoColly to get the body of some urls. Right now I am testing the Netflix website and I have this code below
u = "http://netflix.com"
uri, err := url.Parse(u)
if err != nil {
fmt.Println(err.Error())
continue
}
c := colly.NewCollector()
c.AllowedDomains = string{"netflix.com"}
c.OnResponse(func(r *colly.Response) {
q.r.Set("success:"+u, string(r.Body))
})
c.OnError(func(r *colly.Response, err error) {
log.Println(err.Error())
q.r.Set("failed:"+u, err.Error())
})
c.Visit(uri.String())
c.Wait()
and when I execute it retuns this error Get https://www.netflix.com/: Not following redirect to www.netflix.com because its not in AllowedDomains
any thoughts?
go asked Nov 16 at 23:38
lucas alcantara
737
I am using the framework GoColly to get the body of some urls. Right now I am testing the Netflix website and I have this code below
u = "http://netflix.com"
uri, err := url.Parse(u)
if err != nil {
fmt.Println(err.Error())
continue
}
c := colly.NewCollector()
c.AllowedDomains = string{"netflix.com"}
c.OnResponse(func(r *colly.Response) {
q.r.Set("success:"+u, string(r.Body))
})
c.OnError(func(r *colly.Response, err error) {
log.Println(err.Error())
q.r.Set("failed:"+u, err.Error())
})
c.Visit(uri.String())
c.Wait()
and when I execute it retuns this error Get https://www.netflix.com/: Not following redirect to www.netflix.com because its not in AllowedDomains
any thoughts?
go go asked Nov 16 at 23:38
lucas alcantara
737
asked Nov 16 at 23:38
lucas alcantara
737
asked Nov 16 at 23:38
lucas alcantara
737
asked Nov 16 at 23:38
lucas alcantara
737
asked Nov 16 at 23:38
lucas alcantara
737
737
1
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting tohttps://www.netflix.com(notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.
– Carson Hoffman
Nov 16 at 23:46
add a comment |
1
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting tohttps://www.netflix.com(notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.
– Carson Hoffman
Nov 16 at 23:46
1
1
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting to
https://www.netflix.com (notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.– Carson Hoffman
Nov 16 at 23:46
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting to
https://www.netflix.com (notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.– Carson Hoffman
Nov 16 at 23:46
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
accepted
you specified domain here c.AllowedDomains = string{"netflix.com"} without www subdomain, thats the resone it's not found within allowed domains.
c.AllowedDomains = string{"www.netflix.com"}
answered 2 days ago
danicheeta
856316
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
you specified domain here c.AllowedDomains = string{"netflix.com"} without www subdomain, thats the resone it's not found within allowed domains.
c.AllowedDomains = string{"www.netflix.com"}
answered 2 days ago
danicheeta
856316
add a comment |
up vote
0
down vote
accepted
you specified domain here c.AllowedDomains = string{"netflix.com"} without www subdomain, thats the resone it's not found within allowed domains.
c.AllowedDomains = string{"www.netflix.com"}
answered 2 days ago
danicheeta
856316
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
you specified domain here c.AllowedDomains = string{"netflix.com"} without www subdomain, thats the resone it's not found within allowed domains.
c.AllowedDomains = string{"www.netflix.com"}
answered 2 days ago
danicheeta
856316
you specified domain here c.AllowedDomains = string{"netflix.com"} without www subdomain, thats the resone it's not found within allowed domains.
c.AllowedDomains = string{"www.netflix.com"}
answered 2 days ago
danicheeta
856316
answered 2 days ago
danicheeta
856316
answered 2 days ago
danicheeta
856316
answered 2 days ago
danicheeta
856316
856316
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53346761%2fframework-gocolly-redirect-to-https-doesnt-work%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Have you confirmed the problem is actually redirecting from HTTP to HTTPS, or possibly that it's redirecting to
https://www.netflix.com(notice the www.)? That seems like what the error is suggesting is the issue. In the context of this library, a subdomain (www) may not be considered a part of the larger domain.– Carson Hoffman
Nov 16 at 23:46