This project has moved and is read-only. For the latest updates, please go here.

5010 files with newlines after every 80 characters

May 15, 2013 at 2:05 PM
Some new providers are sending files that seem to be chunked at 80 bytes with newlines between each chunk..

Is there a way to tell parser.parseMultiple to strip the extraneous CR/LF characters?

If not, my approach is to convert the file stream to a string, replace the CR/LF characters with "" and convert it back to a memory stream. Unless anyone can suggest a better way.

May 15, 2013 at 3:02 PM
The latest version has an overload on the X12Parser constructor for specifying an array of characters to ignore.
If you are using the ClaimTransformationService, in C# this would look like:
var service = new ClaimTransformationService(new X12Parser(new SpecificationFinder(), true, new char[] { '\n','\r'}));
Some senders do use these characters as segment delimiters, so you would need to make sure you either don't have any of those or you compensate for those differently.
May 15, 2013 at 6:56 PM
When you say segment delimiters, you mean they replace a delimiter such as "*" with a newline?

If so, is there a reliable way to identify these types of files from the data? Such as "ISA<Newline>00<Newline>".

Or will I need to keep a provider profile, that specifies the format of the data they send?

May 15, 2013 at 7:08 PM
The ISA segment that starts each file is fixed width. The 106th character of the file always represents the segment terminator. This is typically a ~ in most of the specs that I have seen, but there is no requirement for it to be a tilde. Since a separator character cannot also be used in the content of the x12 message, many systems will choose other delimiter characters that they know will not cause conflicts.

If you wanted a pre-step to inspect the 106th character to make sure it's not '\r' or '\n', then you could then use the ignoredChars parameter safely to compensate for your scenario.
May 15, 2013 at 7:21 PM
Do you believe it might be possible that someone is using newline as a segment delimiter and chunk the file at 80 bytes?

I sure hope not!

May 15, 2013 at 8:18 PM
I do not believe what you are receiving is a valid X12 file at all, since the ISA segment which is the first 105 characters should be fixed width, you obviously have a line break within this segment which already breaks the X12 standard. So I am not sure you will even be looking at the segment separator in position 106 without normalizing this file first.

If it were a valid file it wouldn't have these returns every 80 characters and allow it to be a delimter. Per the 837P spec:
Once specified in the interchange header, the delimiters are not to be used in a data element value elsewhere in the interchange.
What they have done may have been some post processing for some other downstream internal process that can only read the file 80 characters at a time.
May 15, 2013 at 9:40 PM
In case anyone else cares the vb version of the C#
var service = new ClaimTransformationService(new X12Parser(new SpecificationFinder(), true, new char[] { '\n','\r'}));
Dim service As New ClaimTransformationService(New X12Parser(New SpecificationFinder(), True, New Char(1) {vbCr, vbLf}))
Jun 3, 2013 at 7:14 PM
I finally got around to testing this. Using the line;
        Dim transformationService As New ClaimTransformationService(New X12Parser(New SpecificationFinder(), True, New Char() {ControlChars.Lf, ControlChars.Cr}))
I receive the error;

6/3/2013 2:00:48 PM Error on File: I:\datafile.5010.txt Message: P is not a valid subelement separator in position 105 of the file.

The first 109 characters of the file look like this;
ISA*00*          *00*          *ZZ*800222771      *ZZ*133052274      *130412*070
Based on this, it would appear that either I do not have the transformationService line coded properly, or, the OopFactory routines are not throwing away the carriage return/line feed at positions 81/82 prior to looking for the delimiter at position 105 (0 based).

Any thoughts?
Aug 9, 2013 at 8:30 PM
The extra Parameter to ignore CRLF did not work for me and I was getting the "P is not a valid subelement separator in position 106" message.

I just wrote a utility method to remove CRLF's and it worked.
        String after = "";

        // Read file and modify
            using (StreamReader sr = new StreamReader(filename))
                String before = sr.ReadToEnd();
                after = before.Remove(new Char[] { '\r', '\n' });