Wie konvertiert man JavaRDD <Row> zu JavaRDD <String> in Apache Spark Java

Question

Ich versuche, JavaRDD Aktionen auf Row Typ Daten durchzuführen. Aber ich bin nicht in der Lage JavaRDD < Row> Daten

Schema zu analysieren oder iterieren:

root 
|-- categories: array (nullable = true) 
| |-- element: string (containsNull = true) 
|-- discount: long (nullable = true) 
|-- expiration: string (nullable = true) 
|-- id: long (nullable = true) 
|-- maxCashback: string (nullable = true) 
|-- minTicket: long (nullable = true) 
|-- name: string (nullable = true) 
|-- rules: struct (nullable = true) 
| |-- cardRequired: boolean (nullable = true) 
| |-- cardType: array (nullable = true) 
| | |-- element: string (containsNull = true) 
| |-- usageLimit: long (nullable = true) 
| |-- vendor: array (nullable = true) 
| | |-- element: string (containsNull = true) 

Daten:

+--------------------+--------+----------+---+-----------+---------+--------------------+--------------------+ 
|   categories|discount|expiration| id|maxCashback|minTicket|    name|    rules| 
+--------------------+--------+----------+---+-----------+---------+--------------------+--------------------+ 
|  [Movie, Event]| null|31-03-2018| 1|  100|  1|ICICI Bank Credit...|[true,WrappedArra...| 
|    [Movie]|  10|30-11-2017| 2|  100|  2|RBL Credit Card O...|[true,WrappedArra...| 
|    [Movie]| null|30-11-2017| 3|  150|  2|SBI RUPAY PLATINU...|[true,WrappedArra...| 
|    [Movie]| null|31-10-2017| 4|  150|  2|IDEA Select Prepa...|[true,WrappedArra...| 
|[Movie, Event, Sp...|  10|31-10-2017| 5|  150|  1|Mobikwik Wallet O...|[true,WrappedArra...| 
|[Movie, Event, Sp...| null|  null| 6|   {}|  1|  Payback Point|[null,WrappedArra...| 
+--------------------+--------+----------+---+-----------+---------+--------------------+--------------------+ 

Code Snippet:

JavaRDD<Row> applicableOffers = offers.toJavaRDD(); 
applicableOffers.foreach((a)->{ 

      int fieldNoTicket = a.fieldIndex("minTicket"); 
      int filedNoCashback=a.fieldIndex("maxCashback"); 
      int fieldNoDiscount=a.fieldIndex("discount"); 

      System.out.println("a : " +a); 
     }); 

Ausgang:

a : [WrappedArray(Movie, Event),null,31-03-2018,1,100,1,ICICI Bank Credit Card Offer,[true,WrappedArray(Credit),null,WrappedArray(ICICI)]] 
a : [WrappedArray(Movie),10,30-11-2017,2,100,2,RBL Credit Card Offer,[true,WrappedArray(Credit),15,WrappedArray(RBL)]] 
a : [WrappedArray(Movie),null,30-11-2017,3,150,2,SBI RUPAY PLATINUM DEBIT CARD OFFER,[true,WrappedArray(Platinum Debit),null,WrappedArray(SBI)]] 
a : [WrappedArray(Movie),null,31-10-2017,4,150,2,IDEA Select Prepaid Offer,[true,WrappedArray(SIM),null,WrappedArray(IDEA)]] 
a : [WrappedArray(Movie, Event, Sports),10,31-10-2017,5,150,1,Mobikwik Wallet Offer,[true,WrappedArray(eWallet),null,WrappedArray(Mobikwik)]] 
a : [WrappedArray(Movie, Event, Sports),null,null,6,{},1,Payback Point,[null,WrappedArray(Credit, Debit),null,WrappedArray(ICICI,SBI,Canara)]] 

Alles, was ich tun muss, ist eine Aktion auszuführen, die den Rabatt für 1000 USD berechnet und Wert und Name eines Angebots in Apache Spark Java ausgibt.

Answer 1

Ich habe es geschafft, einen Workaround zu finden. Indem Sie fieldIndex(colName) verwenden, um den Index zu erfassen, gefolgt von getLong(index), um auf die Elemente zuzugreifen.

int orderValue=1000; // USD 1000 is order value 

applicableOffers.foreach((a) -> { 

     int name = a.fieldIndex("name"); 
     int discount = a.fieldIndex("discount"); 

     String offerName = a.getString(name); 
     Long discount = a.getLong(discount); 

     System.out.println("Offer:" + offerName + " Total:" + computeCashBack(orderValue,discount)); 
    });