Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10579

CVS Parser plugin can't handle complex comments


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Pipelines
    • Labels:
    • Sprint:
      App Eng Sprint 3
    • Release Notes:
      Added the functionality to add to error dataset for malformed rows in csv while parsing through csv parser.
    • Rank:


      Jobs fail when importing standard csv file with quoted comments or if line is not formatted well. If a delimited characters is present in data, the import fails. If a line Typical data that includes free form text input requires additional work to import either by hand clensing the data or writing complex parser as cdap program outside of the plugin.

      Would like ability to kick error to error table automatically rather than fail the entire job.

      Test File https://data.consumerfinance.gov/api/views/s6ew-h6mp/rows.csv?accessType=DOWNLOAD

      Example of failure at line 57732:
      03/24/2015,Credit card,,Other,,"Received Capital One charge card offer XXXX. Applied, was accepted ( {$500.00} limit ), activated card and used for XXXX presents. Charge card # XXXX. Right after activating card ... Capital One sent me another card with same {$500.00} limit ... never activated ... never used that card. First bill from above card # came due XXXX and minimum payment due was {$15.00}. I sent in {$20.00} via USPMO and sent in before due date. With the XXXX non-activated, non used credit card ... ..they also sent me bill for some yearly fees when never even activated the card. So called them up ... ... .told them did not want the card and sent back to them. Well ... .get my next bill from the card # above ( XXXX ) ... .they did not credit me for the {$20.00} payment and charged me outrageous over the limit fees, late fees, etc ... and now {$70.00} payment due. So, I called up, their rep stated they accidentally applied my {$20.00} payment to wrong account number and would be corrected. So, I sent in a {$70.00} payment via USPMO along with note to make sure account corrected and both payments applied to correct account number ( minimum due and wanted to keep that card and also repair my credit from bankruptcy in XXXX or XXXX. ) So ... XXXX bill came in the mail ... .and once again ... ..they did not apply my payment of {$70.00}, nor the previous payment of {$20.00} ... .which they stated they would correct. They charged me more outrageous over the limit fees, late fees, etc, along with some stupid note about "" spreading yourself too thin. '' I thought wow ... .this is total bs ... ..so I called numerous times, wrote numerous times ... with no success of them correcting my account, when they acknowledged their mistake ( s ) ... ... then they wanted {$120.00} for minimum payment ... .I thought this is totally illegal. Sent my payments in on time and sent in more than minimum to even begin with. Kept calling and writing, with no resolution like they stated. I totally quit paying at that point ... Did not know what else to do at that time. Sometime around end of XXXX started getting threatening phone calls from some collection agency ... ... I mean really threatening. Like about every other day. Well, I ended up extremely ill, in and out of hospitals around that time, doctors, more hospitals, XXXX, XXXX, XXXX ... ..etc ... ..ended up in XXXX and XXXX entire summer, due to XXXX, XXXX ( XXXX at time ) XXXX ... XXXX, XXXX, and more ( Have all the records. ) Anyway ... come XXXX or XXXX, XXXX, XXXX, XXXX put a freeze on my bank account. ( currently XXXX XXXX ... Never knew took me to court & did an illegal Judgement. Called Capital One ... they new NOTHING ... Called XXXX etc..asked for verification XXXX XXXX ), etc..of my account, payments, court order, etc. Did receive back some info of my account. It clearly shows that they put my more than minimum payments onto a totally different account ... ..I now have the proof, of their illegal practices regarding Capital One and XXXX, XXXX and XXXX. I also have my stubs from my payments sent to them via USPMO for further proof. Now, just recently I have been calls at my home from unknown number stating to call XXXX stating something to the effect of ( my name ) and "" property at ( my home ) address '' being sent from the clerks office. My Mother, who is owner of this home, has also been getting such calls and days in row ... regarding me and the "" property at ( my home address. ) and clerk of court. Now she is the owner of this home which we rent ... and her property has nothing to do with my Capital One account which they totally illegally screwed me on..along with XXXX, XXXX XXXX XXXX illegal collection practices. So, I did look on website, and the Attorney for XXXX, XXXX, XXXX did do a Judgement renewal previous to us getting these harassing, threatening, illegal calls. Need help here.
      ",,Capital One,OH,440XX,,Consent provided,Web,03/30/2015,Closed with explanation,Yes,No,1297939
      03/04/2015,Debt collection,"Other (i.e. phone, health club, etc.)",Cont'd attempts collect debt no


          Issue Links



              • Assignee:
                shashank Shashank
                ted Ted Coyle
              • Votes:
                0 Vote for this issue
                2 Start watching this issue


                • Created: