Wie kann ich diese Schleife schneller in R laufen lassen?

Question

Es ist ein Wörterbuch Datenrahmen words.dict von etwa 44.000 Wörter, und der folgende Code soll alle Wörter im Datensatz dataset.num für ihre numerische IDs aus dem Wörterbuch ersetzen.

data.num:

dput(head(dataset.num)) 
c("rt breaking will from here forward be know as", "i hope you like wine and cocktails", "this week we are upgrading our servers there may be periodic disruptions to the housing application portal sorry for any inconvenience", "hanging out in foiachat anyone have fav management software on the gov t side anything from intake to redaction onwards", "they left out kourtney instead they let chick from big bang talk", "i am encoding film like for the billionth time already ") 

words.dict:

dput(head(words.dict,20) 
structure(list(id = c(10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 3L, 20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L), word = structure(1:20, .Label =c("already", "am", "and", "any", "anyone", "anything", "application", "are", "as", "bang", "be", "big", "billionth", "breaking", "chick", "cocktails","disruptions", "encoding", "fav", "film", "foiachat", "for", "forward", "from", "gov", "hanging", "have", "here", "hope", "housing", "i", "in", "inconvenience", "instead", "intake", "know", "kourtney", "left", "let", "like", "management", "may", "on", "onwards", "our", "out", "periodic", "portal", "redaction", "rt", "servers", "side", "software", "sorry", "t", "talk", "the", "there", "they", "this", "time", "to", "upgrading", "we", "week", "will", "wine", "you"), class = "factor")), .Names = c("id", "word"), row.names = c(10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 3L, 20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L), class = "data.frame") 

Loop:

for (i in 1:nrow(words.dict)) 

    dataset.num <- gsub(paste0("\\b(", words.dict[i,"word"], ")\\b"),words.dict[i,1], dataset.num) 

Während ich th abgeschnitten e data, dataset.num ist ein Zeichenvektor von fast 40.000 Zeilen (jede Zeile enthält durchschnittlich 20 Wörter). Der Code funktioniert gut bei kleinen Daten, aber nicht so schnell bei großen Daten mit begrenzter Verarbeitungsgeschwindigkeit.

Was würden Sie vorschlagen, um die Effizienz zu verbessern & Leistung des Codes?

Answer 1

Hier ist ein anderer Ansatz, der vielleicht besser skaliert, obwohl ich es nicht wirklich getestet habe.

sapply(strsplit(dataset.num, "\\s+"), function(y) { 
    i <- match(y, words.dict$word) 
    y[!is.na(i)] <- words.dict$id[na.omit(i)] 
    paste(y, collapse = " ") 
}) 
#[1] "rt 22 will from here forward 3 know 18"                   
#[2] "i hope you like wine 12 24"                      
#[3] "this week we 17 upgrading our servers there may 3 periodic 25 to the housing 16 portal sorry for 13 inconvenience" 
#[4] "hanging out in foiachat 14 have 27 management software on the gov t side 15 from intake to redaction onwards"  
#[5] "they left out kourtney instead they let 23 from 20 19 talk"              
#[6] "i 11 26 28 like for the 21 time 10" 

Beachten Sie, dass stringi::stri_split zu beschleunigen die Zeichenfolge Splitting nutzen könnten.