'unicode' object has no attribute 'prettify'

Question

I am using BeautifulSoup to parse an html article. I use some functions to clear the html, so I can keep only the main article.

Also, I want to save the Soup Output to a file. The error I get is the following:

soup = soup.prettify("utf-8")
AttributeError: 'unicode' object has no attribute 'prettify'

Source Code:

#!/usr/bin/env python
import urllib2
from bs4 import BeautifulSoup
import nltk
import argparse

def cleaner():
    url = "https://www.ceid.upatras.gr/en/announcements/job-offers/full-stack-web-developer-papergo"
    ourUrl  = urllib2.urlopen(url).read()
    soup = BeautifulSoup(ourUrl)

    #remove scripts
    for script in soup.find_all('script'):
        script.extract()
    soup = soup.find("div", class_="clearfix")

    #below code will delete tags except /br
    soup = soup.encode('utf-8')
    soup = soup.replace('<br/>' , '^')
    soup = BeautifulSoup(soup)
    soup = (soup.get_text())
    soup=soup.replace('^' , '<br/>')

    print soup
    with open('out.txt','w',encoding='utf-8-sig') as f:
        f.write(soup.prettify())

if __name__ == '__main__':
    cleaner()

Answer 1

This is because soup is not a BeautifulSoup or Tag instance anymore after these lines:

soup = (soup.get_text())
soup = soup.replace('^' , '<br/>')

It becomes a unicode string which, of course, does not have a .prettify() method.

Depending on what your desired output is, you should be able to make use of .get_text(), .replace_with(), .unwrap(), .extract() and other BeautifulSoup methods to clean up your HTML instead of trying to deal with it as a regular string.