How to remove the Header row of a CSV file (Infor MEC/IEC)

Integrating CSV files with MEC is a very old but frequently used method. A CSV file may have a header and the data as in the below format. If we use this file (converting FLAT To XML) directly as it is, it will feed the data with the header. In this post, I'm going to share one of the possible methods to get rid of the CSV header.

FACI,PRDCT,MO,OPERNO,MANUQTY
A01,Y3003-Y01-010,3000018,10,2
A01,Y3003-Y01-010,3000018,50,2

Using the Flat file definition tool I will define the following structure.

Then I will get the following XML message into the MEC process.

<WORKBOOK>
	<ManuObj>
		<FACI>FACI</FACI>
		<PRDCT>Product</PRDCT>
		<MO>MO</MO>
		<OPRNO>Operation No</OPRNO>
		<MANUQTY>ManuQty</MANUQTY>
	</ManuObj>
	<ManuObj>
		<FACI>A01</FACI>
		<PRDCT>Y3001-Y02-016</PRDCT>
		<MO>0003000164</MO>
		<OPRNO>0010</OPRNO>
		<MANUQTY>1</MANUQTY>
	</ManuObj>
	<ManuObj>
		<FACI>A01</FACI>
		<PRDCT>Y3001-Y02-016</PRDCT>
		<MO>0003000164</MO>
		<OPRNO>0040</OPRNO>
		<MANUQTY>1</MANUQTY>
	</ManuObj>
</WORKBOOK>

You can notice the XML message contains the header. To get rid of the header one method is;

1. Read all the XML elements into a Java array in the Mapper.
2. Then process the data from the first position of the Array (remember 0^th position will have the header segment).

But here I will show you a different method that is using XSLT transformation. Very easy.

Steps are:

1.  Define the XLST definition. You can reuse this definition with two small modifications. Change the "WORKBOOK" value in <xsl:element name="WORKBOOK"> to the root element of your XML message. The change the xpath of  <xsl:copy-of select=""/> to match your respetive xpath.
   
   

   <?xml version="1.0"?>
   <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                   version="1.0">
   	<xsl:output indent="yes"/>
   	<xsl:template match="/">
   		<xsl:element name="WORKBOOK">
   			<xsl:copy-of select="WORKBOOK/ManuObj[position()>1]"/>
   		</xsl:element>
   	</xsl:template>
   </xsl:stylesheet>
   

2. Go to the Partner Admin, Manage->XSLT Definitions..,  create the new XSLT Definition.
   
   
3. Go the Process tab in your Partner Admin. Right click and select XSL Transform.
   
   
4. Select the XSLT definition you created in Step 2.
   
   
   
   
   
5. Make sure XSL Transform process step will be after the FLAT To XML process step in the process tab and Save. (You can add XML Transform after the XSL Transform to implement the logic, transformation you want).
   
   
   
   
   
   
   
   
   
6. Test the process and see messages in Admin page. 
   
   
   
    Happy Coding.

Control Properties in MEC

When we want to keep some parameters in the MEC we often use “.properties” file.  But alternatively we also can use Control properties in the Partner Administrator.

The control properties can be accessed from the MEC mapper using the following code.

String strVariable = getManifestInfo("agr:outputFolderPath");

Hint:

You can add control properties to the individual “Agreement” or to the “Group”.

Depending on the situation, we can choose the suitable method to use. Below comparison will help to choose appropriate method.

Comparison:

	Control Properties	Properties File
Format	Key,Value format	Key, Value format
Deployment	Will be ported to the environment with import/export function.	Possibility to miss the file to copy unless it is documented properly.
Code	Very easy. No explicit initiation of objects. One line required to access the variable.	Not very easy. Need to read the file and initiate property object. Require few extra coding to access first variable.
Usage	Properties to be used in agreement or group level.	Properties to be used as common to share across all the agreements. E.g. data base connections,

MEC: How to Set Message Counter for EDI Message

When you sending/creating EDI messages it is necessary to include unique message interchange number. This is to ensure each message that we are sending is unique.

In EANCOM/EDIFACT

Element 0020 of UNB segment:
UNB+UNOA:4+xxxxxxx:14+xxxxxx+20160905:0831+00000000000057+    +ORDERS++1'

In X12

Element ISA13 of ISA segment:
ISA*00* *00* *ZZ*167520391 *ZZ*39319445 *991201*1248*U*00200*000000001*0*P*>

This number need to be persistent. The middleware we are using for EDI transformation should cater this requirement. I used MS-BizTalk  Server for two EDI projects, which cater the same. (just by a configuration)

M3 e-Collaborator (MEC) does the same thing in a different way. In the MEC database (e.g. MEC_Storage_TST) has a table (UTIL_Message_Counters) for this purpose.

Table structure is like this:

Note 1: To be able to use this class you must first create this table (use the SQL script MeC_Utilities_db_script.sql).

You don’t have to enter a record or write code to save/get data. Instead, you have to write 2 lines of java code in your mapper to get the Value. ( Unique Message Interchange Number)

Note 2: Values in Key fields are case sensitive.

Steps:

1. Initialize the message counter for the map.
   myMap is reference to the Map.
   

   MessageCounter mc = new MessageCounter(myMap);
   
   

2. Get a new counter value using one of 3 overloads
   
   

   /*
   * getNewValue(String counterId)
   * counterID = Choose suitable name (max 20 chars)
   *
   * Returns new counter for the map
   */
    String newVal = mc.getNewValue("MsgCounter");
   

   
   Or
   

   /*
   * getNewValue(String counterId, int keyFields);
   * counterID = Choose suitable name (max 20 chars)
   *
   * Returns new counter for the map AND parnter ID
   */
    String newVal = mc.getNewValue("MsgCounter",mc.MAP_NAME + mc.PARTNER_ID);
   

   
   Or
   

   /*
   * getNewValue(String counterId, int keyFields, String cono, String divi, String date) 
   * counterID = Choose suitable name (max 20 chars)
   *
   * Returns new counter for the map AND parnter ID AND cono AND divi AND date.
   * you can omit cono and divi by setting null 
   * below code resets the counter for each day (format CCYYMMDD).
   */
    String newVal = mc.getNewValue("MsgCounter",mc.MAP_NAME + mc.PARTNER_ID, null, null, DATE);
   

If, everything OK, you will return the new counter other wise –1.

If you look at the table (UTIL_Message_Counters) , a record has been added and value will be incremented each time when getNewValue() is called.

MEC: Call Web Service in a cleaner way

Background

In the MEC mapper, calling a web service is a general scenario. There are few ways to do it. One method is build the SOAP request Payload in a string and submit via POST method. (as below code snippet)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

java.net.URL obj = new java.net.URL(iURL); java.net.HttpURLConnection con = (java.net.HttpURLConnection) obj.openConnection(); con.setRequestMethod("POST"); con.setRequestProperty ("Content-Type","text/xml"); con.setRequestProperty ("Accept","text/xml"); String itemStr = "<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">" + "<soap:Body>" + "<GetItem xmlns=\"http://tempuri.org/\">" + "<itemNumber>KelumG</itemNumber>" + "</GetItem>" + "</soap:Body></soap:Envelope>"; con.setDoOutput(true); java.io.DataOutputStream wr = new java.io.DataOutputStream(con.getOutputStream()); wr.writeBytes(itemStr); wr.flush(); wr.close();

This is very primitive and takes lot of times of developer. Also it hinders the code readability & maintainability.

From this post I’m going to tell you a cleaner way, rather simple way. We, programmers, know easy way to call web service is using Proxy objects/classes.

Proxy objects are type of representations of the web service for the programming language we are using.

Solution

1. Create Proxy class from the web service.
   The service I’m using for this post is http://localhost:4599/HelloService.asmx?wsdl  (refer below P.S. section). There are few tools to create proxy classes in java (wsimport, axis2, java2wsdl etc. ). Here I’m using wsimport with following argument.
   
   -d KG : store generated class into KG folder
   
   wsimport -d KG http://localhost:4599/HelloService.asmx?wsdl command to create proxy classes.
   
   
2. Create jar file for the generated files.
   Using command line: https://docs.oracle.com/javase/tutorial/deployment/jar/build.html
   Using Eclipse: http://help.eclipse.org/neon/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Ftasks%2Ftasks-33.htm
   
   Note:
   There are limitations for customer JARs. ( Page # 85, Business Document Mapper, Version 11.4.3.0)
   Package
   To use a Java class from a Custom JAR in a mapping, the class has to use a package. The default package, that is no package, will not work. Note that you have to enter this package for all references to the custom class in the code for Java functions within a mapping.
   
   Javadoc, encoding
   If you want to get Javadoc in the mapping Java editor when using a Java class from a Custom JAR, the Java source code, that is, the .java file, must be included in the Custom JAR. The .java file must use the character encoding UTF-16, otherwise no Javadoc will be shown in the mapper. This is because the files for the mapping use that encoding.
   
3. In the map add reference to this custom JAR file.
   
4. Simply call the service just like a method.

   1 2 3 4 5 6

   private void callWebSvc() throws Throwable { // Please implement me org.tempuri.HelloService hs = new org.tempuri.HelloService(new java.net.URL(iUrl)); org.tempuri.HelloServiceSoap hsoap= hs.getHelloServiceSoap(); oItem = hsoap.getItem(iItem); }

   
   

---

P.S:

For this post I have created a mock web service (.asmx ) using C# and hosted in IIS. Sample code as below.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

namespace WebServiceApp { [WebService(Namespace = "http://tempuri.org/")] [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)] [System.ComponentModel.ToolboxItem(false)] public class HelloService : System.Web.Services.WebService { [WebMethod] public string GetItem(string itemNumber) { return new ItemService().GetItem(itemNumber); } } public class ItemService { public string GetItem(string itno) { return string.Format("Item {0}",itno); } }

MEC: Change out file name dynamically

Background

Manipulating the out file name generated by the middle ware is a common scenario. By default file names are generated with a GUID (13a7a838-a706-4273-b190-9d58a79bf04e). It is more practical if the file name is human readable.

Solution

1. In the Mapper add User function to change the file name. (Note: this function should the last function in the map)
   
2. Add similar code to the function. This code set file name to the manifest information of the map.
   

   1 2 3 4 5 6

   String date = new java.text.SimpleDateFormat("yyMMdd").format(new java.util.GregorianCalendar().getTime()); String time = new java.text.SimpleDateFormat("HHmmss").format(new java.util.GregorianCalendar().getTime()); String fileName = "EDI_832_OUT_"+ date+time; setManifestInfo("map:FileName", fileName);

3. Then, configure the relevant Send (in partner agreement) object to take the given file name.
   
4. And you are done. !!!

Development with MEC older version

Presently I’m working with a MEC older version (Ver 9.1).

This version is not part of an Eclipse plugin. It’s a standalone application that use for MEC mapper development. Very tedious to work with.

MEC Mapper different than the Eclipse version. But Flat file definition tool, it’s manager & the partner agreement tool looks the same.

Mapping manger looks like this.

1. Load button to retrieve all the existing mappings.
2. New button to create a new one
3. Generate & Publish buttons to compile the map & deploy to the server
4. Import & export button: Once you create a map, you cannot change it. Instead, import existing version and export with a new changes and new version number

Mapper looks like this.

1. Mapping flow is organized in “tree-view” like structure.
2. You can generate the map from this screen. But to publish you have to close this and go back to Mapping manger & publish.
3. Two options to save (to DB & to File)
4. Validate map option also available on this screen.

How MEC Message Flow Work

Like any other middle-ware application MEC has unique architectural components. Below diagram depicts them.

1. Inbound message will be consumed by the MEC and goes through detection rules defined in the partner agreement. Detection is either channel or XML detection. If inbound message is a flat file, channel detection will be used.
   For XML detection, inbound message must have meta data to configure the detection.
   
   Note below payload has RequestHeader as meta data.
   

   <ItemInfo>
       <RequestHeader>
           <environment>A</environment>
           <program>B</program>
           <company>A</company>
       </RequestHeader>
       <RequestBody>
           <!-- Items goes here-->
       </RequestBody>
   </ItemInfo>
   
   

   
   detection configuration is:
   
   • Detection information  & rules are saved in MEC storage. MEC will evaluate all saved detections (in a order) until its find the correct one.
2. Once the detection has passed, process the message based on the defined processing steps. (in the given order)
   
   
   • These steps are also saved on the MEC Storage
3. Generate the output message.

M3 Integrations (MEC)

I was able to refresh my knowledge of   M3 integration with MEC (M3 E-collaborator).  MEC is a middle-ware specifically design for M3 integration development. Over the past few years it has been evolved.

MEC has following components:

1. Flat-File definition tool -  Using this tool you can create XML schema file. Also it can generate sample Flat File & XML file for the schema. 
2. Flat-File Repository manger - Is the repository for the local and server definitions. 
3. Mapper - Use to transform data to a output file or update M3 directly for the given input schema and the defined logic. It has certain functions to perform this level of transformation.  (Later, with my next posts I'm hopping to reveal them). If you are familiar with Ms-BizTalk server, it's orchestration and MEC mapper is  similar in behavior. 
4. Partner Admin tool - Use to configure agreement for the integration such as, 

• Detection
• Processes
• Error handling
• and other admin tasks (e.g. communication, web-service definitions, EventHub subscriptions etc.) 

Its mapper look as follows, 

MEC Mapper