Recently I was experimenting with a project to scrape some information from LinkedIn and was very close to completing my script but I was having difficulties getting my for loop to work since the square brackets from the list values were causing BeautifulSoup4 not to work.
This is the error I was running into. As you can when I print the value for the URL I’m iterating through its being read/stored with the square brackets (‘[‘, ‘]’) which is mucking things up for my scraping module.
After a bunch of searching and multiple attempts (turn the item into a string and try to replace out the square brackets, print without square brackets and save as new value for a variable, etc) I finally found something that worked!
It was as simple as this.
for ListUrl in urlList: url = ''.join(ListUrl)
I joined the value from the list (in this case just a singular value since I was iterating through one item in the list at the time) to the other values (in this case no other values) with nothing (” without anything in between means you’re not assigning a value for your separator, if I’d used ‘-‘ instead for example there would be a hyphen in front of my URL and connecting any other URLs that were being joined together to the first one).
This removed the square brackets from my list value and turned it into a string at the same time. Awesome!
Hopefully this helps others who are experiencing the same struggle, :D.