Bdtjtk

I am working on a project that revolves scraping a lot of data. I am working on a rather long script right now but running into a problem with my for loop.

I am trying to scrape information out of a 9 row table. I have tried to set up a for loop so that it scrapes the same information from each row. In order to access the first row, I split the table into a list. The first row starts with the third indice.

Here is my Code:

When I run it I get an, "AttributeError" at the line "Aa" is on. The error reads, "'NoneType' object has no attribute 'text'"

This doesn't happen when I feed that line of code individually into the console, I get the desired text. And when I take out the for loop I am able to scrape the first indaplaybox.

from urllib.request import urlopen as uReq

from bs4 import BeautifulSoup as soup



my_url ='Myurl/=' + page



uClient = uReq(my_url)

page_html = uClient.read()

uClient.close()



page_soup = soup(page_html, "html.parser")

boxes = page_soup.findAll("table",{"class":"TableTable tableBody"})

box = boxes[0]

playboxes = box.find_all('tr')

indaplaybox = playboxes[3]



filename = "QBS.csv"

f = open(filename, "a")



headers= "Aa, Ab, Ac, Adn"

f.write(headers)



for indaplaybox in playboxes:



    Aa = indaplaybox.find('td', attrs = {'style' : 'font-weight: bold;'}).text



    c = indaplaybox.find('td', attrs = {'class' : 'tablePlayName'})

    cl = c.text.split() 

    Ab = cl[0] + " " + cl[1]

    Ac = cl[2]

    Ad = indaplaybox.div.a.text







    print("Aa:" + Aa)

    print("Ab:" + Ab)

    print("Ac:" + Ac)

    print("Ad:" + Ad)





    with open (filename, "a") as myfile:

        myfile.write(Aa + "," + Ab + "," + Ac.replace(",", "|") + "," + Ad + "n")

f.close()

I want to loop through the playbox indices 3-11.

I am not well versed with indices, so tried to do something like:

p = [str(i) for i in range (3,12)] 

indaplaybox = playboxes[p]



for indaplaybox in playboxes:



    rest of code

But that doesn't work, because what is probably obvious to most is that list indices must be integers.

I could really use some help thinking through how to get this for loop running smoothly. Thanks!

You can do:

Method 1:

# p has all the values from playboxes at these indexes

p = [playboxes[i] for i in range(3,12)]



# now simple loop

for indaplaybox in p:

    ......

Method 2:

for indaplaybox in playboxes[3:12]:

    ....

p = [str(i) for i in range (3,12)] 

for i in p:

    indaplaybox = playboxes[i]

    ...

    rest of the code