Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-13162

UI does not render code correctly for python spark compute

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.4
    • Component/s: UI
    • Labels:
      None
    • Sprint:
      5.0 03/06
    • Rank:
      1|i00bab:

      Description

      To reproduce

      1. Deploy dynamic spark-compute v2.0.6 from cask market
      2. Use Python spark compute node
      3. Place the following code in the text area for code

      from pyspark import *
      from pyspark.sql import *
      from pyspark.ml.evaluation import RegressionEvaluator
      from pyspark.ml.recommendation import ALS
      def toCSVLine(data):  
        return ','.join(str(d) for d in data)  
      sc = SparkContext()
      spark =  SQLContext(sc)
      lines = spark.read.text("${input.path}").rdd
      parts = lines.map(lambda row: row.value.split("::"))
      ratingsRDD = parts.map(lambda p: Row(userId=int(p[0]), movieId=int(p[1]),                                     
                                           rating=float(p[2]), timestamp=long(p[3])))
      ratings = spark.createDataFrame(ratingsRDD)
      (training, test) = ratings.randomSplit([0.8, 0.2])
      
      # Build the recommendation model using ALS on the training data
      als = ALS(maxIter=5, regParam=0.01, userCol="userId", itemCol="movieId", ratingCol="rating")
      model = als.fit(training)
      
      # Evaluate the model by computing the RMSE on the test data
      predictions = model.transform(test)
      evaluator = RegressionEvaluator(metricName="rmse", labelCol="rating",
                                      predictionCol="prediction")
      rmse = evaluator.evaluate(predictions)
      predictions.map(toCSVLine).saveAsTextFile("${output.dir}")
      print("Root-mean-square error = " + str(rmse))
      

      4. Reopen the Python spark compute node

      The UI will render the code poorly formatted. Attached screenshots

        Attachments

          Activity

            People

            • Assignee:
              ajai Ajai Narayan
              Reporter:
              sree Sreevatsan Raman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: