Pull Text zwischen zwei BeautifulSoup Elemente

Question

Die allgemeine Frage wurde an einigen Stellen gestellt und beantwortet: http://www.resolvinghere.com/sof/18408799.shtml

How to get all text between just two specified tags using BeautifulSoup?

Aber bei dem Versuch, zu implementieren, erhalte ich wirklich umständlich Saiten.

Mein Setup: Ich versuche Transkript Text aus den Presidential Debatten zu ziehen, und ich dachte, ich würde hier beginnen: http://www.presidency.ucsb.edu/ws/index.php?pid=111500

I

Das gerade das Transkript mit

transcript = soup.find_all("span", class_="displaytext")[0] 

isolieren Formatierung des Transkripts ist nicht ideal. Alle paar Zeilen Text hat eine <p> und sie bezeichnen eine Änderung in Lautsprechern mit einem verschachtelten <b>. zB:

<p><b>TRUMP:</b> First of all, I have to say, as a businessman, I get along with everybody. I have business all over the world. [<i>booing</i>]</p>, 
<p>I know so many of the people in the audience. And by the way, I'm a self-funder. I don't have — I have my wife and I have my son. That's all I have. I don't have this. [<i>applause</i>]</p>, 
<p>So let me just tell you, I get along with everybody, which is my obligation to my company, to myself, et cetera.</p>, 
<p>Obviously, the war in Iraq was a big, fat mistake. All right? Now, you can take it any way you want, and it took — it took Jeb Bush, if you remember at the beginning of his announcement, when he announced for president, it took him five days.</p>, 
<p>He went back, it was a mistake, it wasn't a mistake. It took him five days before his people told him what to say, and he ultimately said, "It was a mistake." The war in Iraq, we spent $2 trillion, thousands of lives, we don't even have it. Iran has taken over Iraq, with the second-largest oil reserves in the world.</p>, 
<p>Obviously, it was a mistake.</p>, 
<p><b>DICKERSON:</b> So...</p> 

Aber wie ich schon sagte, ist kein neues Problem. Definieren Sie ein Start- und End-Tag, durchlaufen Sie die Elemente, so lange wie aktuell! = Next, fügen Sie den Text hinzu.

Also teste ich auf ein einzelnes Element, um die Details richtig zu machen.

startTag = transcript.find_all('b')[165] 
endTag = transcript.find_all('b')[166] 
content = [] 
content += startTag.string 
content 

Und die Ergebnisse, die wir bekommen sind [u'R', u'U', u'B', u'I', u'O', u':'] statt [u'RUBIO:'].

Was fehlt mir?

Answer 1

Die Idee wäre, alle b Elemente innerhalb des Transcripts zu finden, dann jedes der b Element Eltern und finden Sie die nächsten Absätze, bis es einen mit b Element im Inneren gibt. Umsetzung:

from bs4 import BeautifulSoup, Tag 
import requests 

url = "http://www.presidency.ucsb.edu/ws/index.php?pid=111500" 
response = requests.get(url) 

soup = BeautifulSoup(response.content, "html5lib") 
transcript = soup.find("span", class_="displaytext") 
for item in transcript.find_all("b")[3:]: # skipping first irrelevant parts 
    part = [" ".join(sibling.get_text(strip=True) if isinstance(sibling, Tag) else sibling.strip() 
        for sibling in item.next_siblings)] 
    for paragraph in item.parent.find_next_siblings("p"): 
     if paragraph.b: 
      break 

     part.append(paragraph.get_text(strip=True)) 

    print(item.get_text(strip=True)) 
    print("\n".join(part)) 
    print("-----") 

Drucke:

DICKERSON: 
Good evening. I'm John Dickerson. This holiday weekend, as America honors our first president, we're about to hear from six men who hope to be the 45th. The candidates for the Republican nomination are here in South Carolina for their ninth debate, one week before this state holds the first-in-the-South primary. 
George Washington ... 
----- 
DICKERSON: 
Before we get started, candidates, here are the rules. When we ask you a question, you will have one minute to answer, and 30 seconds more if we ask a follow-up. If you're attacked by another candidate, you get 30 seconds to respond. 
... 
----- 
TRUMP: 
Well, I can say this. If the president, and if I were president now, ...