Argthtjtr

How do I get hrefs from hrefs using Python in class and method format?
I have tried:

root_url = 'https://www.iea.org'



class IEAData:

       def __init__(self):

             try:--

             except:





       def get_links(self, url):

            all_links = 

            page = requests.get(root_url)

            soup = BeautifulSoup(page.text, 'html.parser')

            for href in soup.find_all(class_='omrlist'):

               all_links.append(root_url + href.find('a').get('href'))

            return all_links

            #print(all_links)



iea_obj = IEAData()

yearLinks = iea_obj.get_links(root_url + '/oilmarketreport/reports/')



reportLinks = 



for url in yearLinks:

    links =iea_obj.get_links(yearLinks)

    print(links)

Recommended: links variable must have all month hrefs but not getting, so please tell me how I should do it.

There were a couple of issues with your code. Your get_links() function was not using the url that was passed to it. When looping over the returned links, you were passing yearLinks rather than the url.

The following should get you going:

from bs4 import BeautifulSoup                        

import requests



root_url = 'https://www.iea.org'



class IEAData:

    def get_links(self, url):

        all_links = 

        page = requests.get(url)

        soup = BeautifulSoup(page.text, 'html.parser')



        for li in soup.find_all(class_='omrlist'):

           all_links.append(root_url + li.find('a').get('href'))

        return all_links



iea_obj = IEAData()

yearLinks = iea_obj.get_links(root_url + '/oilmarketreport/reports/')



for url in yearLinks:

    links = iea_obj.get_links(url)

    print(url, links)

This would give you output starting:

https://www.iea.org/oilmarketreport/reports/2018/ ['https://www.iea.org/oilmarketreport/reports/2018/0118/', 'https://www.iea.org/oilmarketreport/reports/2018/0218/', 'https://www.iea.org/oilmarketreport/reports/2018/0318/', 'https://www.iea.org/oilmarketreport/reports/2018/0418/', 'https://www.iea.org/oilmarketreport/reports/2018/0518/', 'https://www.iea.org/oilmarketreport/reports/2018/0618/', 'https://www.iea.org/oilmarketreport/reports/2018/0718/', 'https://www.iea.org/oilmarketreport/reports/2018/0818/', 'https://www.iea.org/oilmarketreport/reports/2018/1018/']

https://www.iea.org/oilmarketreport/reports/2017/ ['https://www.iea.org/oilmarketreport/reports/2017/0117/', 'https://www.iea.org/oilmarketreport/reports/2017/0217/', 'https://www.iea.org/oilmarketreport/reports/2017/0317/', 'https://www.iea.org/oilmarketreport/reports/2017/0417/', 'https://www.iea.org/oilmarketreport/reports/2017/0517/', 'https://www.iea.org/oilmarketreport/reports/2017/0617/', 'https://www.iea.org/oilmarketreport/reports/2017/0717/', 'https://www.iea.org/oilmarketreport/reports/2017/0817/', 'https://www.iea.org/oilmarketreport/reports/2017/0917/', 'https://www.iea.org/oilmarketreport/reports/2017/1017/', 'https://www.iea.org/oilmarketreport/reports/2017/1117/', 'https://www.iea.org/oilmarketreport/reports/2017/1217/']

I'm fairly new to programming, and I'm still learning and trying to understand how classes and whatnot all work together. But gave it a shot (that's how we learn, right?)

Not sure if this is what you're looking for as your output. I changed 2 things and was able to put all the links from within the yearLinks into a list. Note that it'll also include the PDF links as well as the months links that I think you wanted. If you don't want those PDF links, and exclusively the months, then just don't include the pdf.

So here's the code I did it with, and maybe you can use that to fit into how you have it structured.

root_url = 'https://www.iea.org'





class IEAData:



    def get_links(self, url):



       all_links = 

       page = requests.get(url)

       soup = bs4.BeautifulSoup(page.text, 'html.parser')

       for href in soup.find_all(class_='omrlist'):

           all_links.append(root_url + href.find('a').get('href'))

       return all_links

       #print(all_links)





iea_obj = IEAData()

yearLinks = iea_obj.get_links(root_url + '/oilmarketreport/reports/')



reportLinks = 



for url in yearLinks:

    links = iea_obj.get_links(url)



    # uncomment line below if you do not want the .pdf links

    #links = [ x for x in links if ".pdf" not in x ]

    reportLinks += links