Domain Driven Design

Jul 14, 2013 at 2:38 PM
I have been looking over a great number of X12 parsers. They all seem to parse to XML. and in some cases like OopFactory, can even write to a database.

But I am most familiar with Domain Driven Design. In other words, at the end, I want to see my X12 810 as an Entity such as from the Entity Framework.

The reason I want this is so that I can then do validation and business processing directly from that entity, and then when it is all done, I can push it into the database as an Invoice.

In other words, I don't want a staging database, as then it's just another layer. I think it is much easier to work with an Invoice Entity, then an EDI entity.

But of course, the EDI message still has to get parsed so it can be used to create such an entity, and the OopFactory X12 parser seems to be the best I have found so far.

Do you think I am on the right track here? Any suggestions for using OF to do the parsing part of this project?

Greg
Coordinator
Jul 14, 2013 at 9:57 PM
oh boy! since you asked...

This x12 parser and all the others you have reviewed ARE compatible with Domain Driven Design. You are implying that they are not.

Before I can answer your question I must first explain that there are at least 4 levels (if not more) of x12 standardization:
  • Level 1 - the message format. Each transaction set (there are over 300 of them) has a standard permissible syntax for a given version (4010 vs 5010). this allows multiple parties to split these messages apart and rearrange them to their final destination without needing to know much about how it will be applied.
  • Level 2 - industry published implementation guides. These will conform to level 1 standardization and has some domain knowledge in that it species how to interpret different identifier codes within a specific industries application of that transaction set. For example, 837P, 837I and 837D all conform to 837 X12 but are three different implementations.
  • Level 3 - a trading partner's implementation guide. This looks very much like level 2 but is governed by a company rather than a standards organization.
  • Level 4 – a trading agreement. This is usually like level 3 but might contain any specifications related to the B2B transactions.
Because of this it is very easy to develop one mechanism for persisting all x12 to a relational database at the Level 1 awareness, but each subsequent level requires knowledge of the implementation guide(s) (there could be many across companies and versions) for that transaction set (thus the need for domain driven design). That doesn’t make it difficult, only tedious. And there are many companies willing to charge you an arm and a leg to help you with that.

The next sequential step in the pipeline after getting the data into a hierarchical structure (whether that be XML or the x12-aware objects) is to transform it into its domain model. The Oopfactory.X12.Hippa.dll assembly is an EXAMPLE of this that uses XSLT transformations, but I found that most enterprise developers are quite weak in XSLT. The database feature added early this year was to allow for the transformation to occur without knowing XSLT. You can accomplish the same logically by using this tool to stage your data and performing the same logical transformation to domain specific tables using SQL.

The reason I said “the next sequential step” in the previous paragraph is to point out the order in which the message is processed. Per domain driven design principles you should design your model first and then concern yourself later with how it will be hydrated from the persistence store that is provided to you (in this case an x12 file, though now you can do it from the database instead). It sounds like this is where you are currently asking for some guidance.
I have actually spent years doing many implementations for clients using this pattern where I would transform the X12 using code into a domain model that I could act upon in code for validation etc., but there was a fatal flaw with this approach (which the database feature mitigates). When I transform the x12 first and save the result to more closely represent my domain objects, there was a loss in fidelity because I had always parsed out much less than the standard defined (in this case for an 837 health claim which has 3 different 600 page documents to describe it). Since I might have missed a few elements that I didn’t need at the time, that information was lost, because I didn’t store it to the database because my current domain model didn’t need it. As requirements got added and we needed to know other segments that were getting sent in the files, it was tedious to reparse the files to find those previously ignored segments (and many times we didn’t do it).
With the staging database, it is much easier to query and do analysis work on the full set of information sent to you. Many clients don’t really know what they want from a file until you tell them what is getting sent in a file and this is easier for you to do if you don’t have to think about your domain model before getting your x12 into database.
So, like you have stated a transformation has to be done. But before you dismiss using the database for staging let me tell you the pros and cons of using the database versus using xslt directly.

Transformation in XSLT:
PROS
  • Requires less layers of abstraction (as you have stated).
  • Requires less server storage for data that you will eventual save in another format
CONS
  • Very few enterprise develops are experienced and comfortable with XSLT code
  • Any data not transformed by the XSLT will be lost (unless you reparse the files)
  • It is difficult to query for what is already in the files except for loading each file individual and running your XSLT.
Transformation in SQL:
PROS
  • Many more enterprise developers will be able to understand a SQL query statement that transforms one table format to another
    The full contents of the X12 file is ALWAYS parsed
  • You can use the staging database for data-mining beyond what you initially needed for your domain object
    CONS
  • An extra layer of transformation has to occurred (though this is written for you, so this is only a minus if you don’t like having another database floating around or you like XLST more than you like SQL)
If you don’t want your production process to have a staging database, you could use the SQL feature just for analysis tool and use the XSLT method to do your transformations. There are plenty of examples of how to do this in the OopFactory.X12.Hippa project that is in the source code.
Hopefully this gives you some ideas as to the best approach for you.
Coordinator
Jul 14, 2013 at 10:14 PM
BTW, I am not suggesting that you attempt to parse the entire X12 message into your domain model. This would violate this design pattern since you are letting the persistence view dictate your domain.
But it is much easier to rehydrate legacy data with previously ignored segments if you already have it all in a staging database.

Also, I didn't use Entity Framework because I wanted this tool to be ORM agnostic. No one can agree on which one is the best to use and even Microsoft keeps re-inventing the wheel on this one.
Jul 15, 2013 at 8:31 PM
dstrubhar,

Thank you very much for your response. You hit the nail on the head when you said "It's not difficult, it just tedious!" This is my experience exactly. I am still thinking about everything you have gone over in the above. I appreciate your wisdom!

Greg
Jul 18, 2013 at 11:25 PM
Just to add my two cents. I think it is a bigger issue than just being tedious.

I think this would take the project out of the hands of people who want to use this project as is without modifying the source code. The reason I say that is because not all trading partners use the same specs. If you look at an 810 spec from AT&T and then look at one from WALMART they can be fairly different. This is because even though the segments and elements remain the same the purpose/identity of the data inside of an element may not be the same. So if your parsed out entity is going to end up being generic to accommodate this issue than you are back to square 1 and might as well have left it as it was.

I like the fact that with this library I can determine who the trading partner was and then IF i need an entity i can choose how i want it parsed based on that specific trading partner.
Coordinator
Jul 18, 2013 at 11:42 PM
I'm not sure why you say "this would take the project out of the hands of people who want to use this project as is without modifying the source code".

This project is only meant to be a data layer. So if you are using the same data as other 810 users, but just have different logic, than that's exactly what the library is intended for.

You would focus on the domain objects, you would need to map it to the x12 or edi objects in this library, but you can do this without modifying any source code at all.

There are even ways that you can redefine the x12 specification (by injecting your own) so that it can account for versions that might not be in the library (whether it is because it is an implementation of x12 instead of the standard or it is a future version not already embedded).

There are libraries outside the OopFactory.X12.dll that are examples of how to do the mapping, but it's not really part of the "parser" per se. It's part of a possible implementation.

This project really isn't to do anything more than save you the trouble of how to parse x12 into its' intended hierarchical format. In the end it's still just raw data with no implications of how that data will be consumed. The OopFactory.X12.Hippa.dll assembly of examples of how to transform this into poco objects or into FO-XML for presentation, but again no real business logic.

Everyone who uses this has to do some kind of transformation before the data is useful. This might be transforming it to flat files (and using some other technology to consume and process the transaction), or to .NET business objects or to database tables. But the intent is not to remove the need to do the transformation, but at least remove all the upfront work with understanding the relationships between the segments in the files. This can be done by "using" the library without "changing" the code.
Jul 18, 2013 at 11:49 PM
I agree with you but perhaps it did not come across that way.


dstrubhar wrote:
This project really isn't to do anything more than save you the trouble of how to parse x12 into its' intended hierarchical format.
This is exactly what I appreciate it for. I don't want it giving me some object called obj810 with specific properties that i may or may not care about and that perhaps map into data with my trading partner that means something different. I prefer to define that on my own.

I thought the OP was basically asking if this library could do that mapping for them and I was just saying it would be almost impossible to come up with something meaningful to everyone and not get 100 posts a day about how come this data parsed into a field called purchase order and it wasn't the purchase order in scenario xyz.