Data mapping

Data mapping is a very common task when developing services. This task is performed with many integration patterns, some of them quite common, such as the following:

• Message Translator: http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageTranslator.html
• Normalizer: http://www.enterpriseintegrationpatterns.com/patterns/messaging/Normalizer.html
• Canonical Data Model: http://www.enterpriseintegrationpatterns.com/patterns/messaging/CanonicalDataModel.html

This task entails translating, transforming, or converting data from one data format to another, such as XML to JSON. For instance, we may receive a message in our service in the XML format, and we have to send the information within this XML payload to a JSON endpoint and to an XML endpoint in which the XML format is different from the XML format specified in our service contract.

In such situations, we will have to transform the message, or create a new one from the original. To achieve this task, WSO2EI has a mediator called Data Mapper Mediator. Our life can be even easier with the graphical tool that WSO2 Developer Studio gives us.

We need these three things to perform a data mapping:

• Input schema file: This will allow the mediator and the diagram editor to know which information is there in the input file. This can be an XML schema file (*.xsd), for instance, or a JSON schema. When using JSON schema, we can create it using the WOS2 Developing tool by adding the JSON element needed to define our JSON schema to the blank output schema.
• Output schema file: This allows the mediator and the diagram editor how the target information is stored. The procedure explained for the input schema files can be applied here for the output schema files as well.
• Mapping configuration file: This is the file that has the information about how the translation takes place. This is a JavaScript file; therefore, the data mapping is performed using a JavaScript engine.

The schema files supporter for both input and output schema files are as follows:

• XML: An XML file whose content has the structure we need. When using XML files, it is important to note that every single tag must be qualified with its namespace, so tags such as <book>This book</book> are not allowed in this tool. We should use &<ns:book>This book</ns:book>, assuming that ns is the namespace of this tag in the XML file.
• JSON: A JSON file containing the structure needed.
• CSV: A CSV file with column definition in the first record.
• XSD: A proper schema file (*.xsd).
• JSONSCHEMA: AWSO2 data mapper JSON schema.
• CONNECTOR: A connector to use the JSON schemas defined for their operations.

This tool, called WSO2 data mapper, is an independent component that is not tied to any WSO2 product. So, to create our first data mapping, we just need to add the data mapper mediator to a sequence. You may remember the sequence we used to show how to debug our service. In this case, we will use a similar sequence that transforms the incoming message before it responds to us.

Assuming that we are editing a sequence, we call it InMappingSequence. The steps to use the data mapper are as listed:

1. Drag and drop the Data Mapper to the sequence:

1. Double-click on the mediator in our sequence. We set the name for it as My1stDataMapper.

1. The input and output files we mentioned earlier must be stored in the WSO2EI registry, so when we double-click on the mediator, we are asked whether to create a new data mapper configuration or use an existing one. We choose Create new Configuration and set a name for it. We also need to choose a Registry Resource Project to save the configuration. In this case, we will use the one we created earlier, and click on OK:

1. Now, we will see the canvas with the Input and Output files:

In this canvas, we have three sections:

• Palette: Here, we find the operators we use to build the logic of our data mapping. We have several categories of operators.
• Input: Here, we load the input file that defines the input message.
• Output: Here, we load the output file that defines the output message.

In our example, we will build a service that receives a greetings message and responds to our greetings. We will achieve this with a data mapper that will use these operators, which are quite common and useful:

• Constant
• Compare
• IfElse

We will use XML messages for input and output messages. For the input message, we will use an XML Schema Definition (*.xsd file) you can find in the book resources, and for the output, we will build the response from the editor.

To build this data mapper, we need to follow these steps:

1. Load the input schema by right-clicking on input and clicking on Load Input:

1. Choose XML value for the Resource Type and select the resource from the file system. We will select the greetings.xsd file you can find in this book's resources and click on OK:

The input box will show the XML defined by the schema we have loaded. This is a simple XML that looks like this:

. If we do not send a message in the differen<greetings> 
   <iSay>Hi!</iSay> 
</greetings> 

1. Now, we will create the response that will look like this:

 <response> 
   <youSay>Hi!</youSay> 
   <iSay>What's up</iSay> 
</response> 

To do this, we right-click inside the output box and choose Add new root element.

1. In the next window, we will introduce this data:
   • Name: response
   • SchemaType: object
   • ID: No value
   • isnullable: Unchecked
   • Namespace: No value; we leave the default namespace

We click on OK.

1. Now, we have the root element called response:

1. We need two more elements, both children of response. To add them, click on response and right-click on the response label. Then, click on addField.
2. We add the following information for the youSay XML tag:
   • Name: youSay
   • SchemaType: string
   • isNullable: Unchecked
   • Namespaces: Blank

1. Click on OK:

1. Repeat steps 8 and 9 to add the iSay XML tag. Now, we have finished building the response:

Now, we will build the logic to build the response message from the incoming message. Our service will work as follows:

1. If we send a message such as Howareyou?, it will respond with our greetings in an XML tag called youSay, plus the Fine,thanks! response in an XML tag called iSay.

This is the request:

<greetings> 
   <iSay>Hi!</iSay> 
</greetings> 

This is the response:

<greetings> 
   <youSay>Hi!</youSay> 
   <iSay>What's up!</iSay> 
</greetings>

1. If we do not send a message in the different form How are you?, the response will always be What's up!.

This is the request:

<greetings> 
   <iSay>Hi!</iSay> 
</greetings> 

This is the response:

<greetings> 
   <youSay>Hi!</youSay> 
   <iSay>What's up!</iSay> 
</greetings> 

So, to implement this functionality, the use case will be as shown:

• Copy the <iSay> value from the request to the <youSay> value in the output message
• Consider that <iSay>from the input message is How are you?:
  • Set the <iSay> value to Fine,thanks! in the output message
• Otherwise, do this:
  • Set the <iSay> value to What's up! in the output message

Now, we will implement this logic:

1. We have the input and output messages already defined from the previous steps, and they look like this:

We begin assigning the <iSay> value from the input to the <youSay> value in the output. We just drag the iSay field and drop it over the youSay field. Then, a link between both the fields will appear:

1. Now, we will add the comparison between the iSay field and the How are you? string:

In the Palette, we open the Common tab, and drag the Constant operator and drop it in the canvas:

1. Right-click over the square in the left-hand side of the Constant window, and select Configure constant operator.
2. Set the Constant Type to String and assign How are you? to the constant value. Click on OK.
3. Then, the Constant operator will look like this:

1. The next step is to create the comparison:

We can do that by dropping the Compare operator in the canvas from the palette:

1. We need to set the two strings we are comparing in this operator. This is quite easy, since we just need to drag and drop the values over the left boxes in the comparison window. Note that you can read In in these boxes. So, we set the iSay field from the input message over the first In box and our canvas will now be like this:

We drag the right arrow in the Constant window over the second In box in the Compare window:

Thus, we have set the two values of the comparison. Our canvas should look as follows:

1. We have the initial comparison (step 1 in the use case). The next step is to build the if-then structure. We add it in the canvas by dropping the IfElse operator from the Palette on the canvas. This operator can be found in the Conditional tab:

This operator has three inputs: condition, then, and else. You can read the name of the input in the box; these values will be the following:

• Condition: This input will be the output of the comparison operator we created earlier
• Then: This input will be a constant with the value Fine, thanks!
• Else: This input will be a constant with the value What's up!

1. We set the condition by dragging the right arrow in the compare operator over the condition input:

1. Now, we will create a new constant operator with the value Fine, thanks! and assign it to the Then input. The result will be as follows:

1. We create the left constant operator we need for the Else input and assign it to the Else input. In this case, we use the What's up! value (note that we need to escape the quote). Finally, the If Else operator will look as shown:

1. The last step is to assign the If Else output to the iSay field in the output message. We just need to drag the right arrow in the If Else operator and drop it on the iSay field:

The whole picture will look like this:

Now, we save the file and return to the sequence where we add the datamapper mediator. We add the Respond mediator, and save and close the sequence. It will look as illustrated:

You can easily test this by creating a proxy and setting this sequence as the input sequence. You can use it for the out sequence and fault sequences we created earlier. You can find a proxy, called My1stProxyMapping, in the resources to test it:

If we call this proxy, we will note the expected behavior:

You can find more about the data mapper mediator in the official documentation.