Estimated reading time: 3 minutes
Are you working on a data analytics project where you need to feed your data to a location that is able to process an XML file?
The ability to get your data into a structured format like XML has many benefits:
(A) You can transfer the data to a web service for processing.
(B) Multiple different formats of your raw data can be standardised, enabling quick conversion and processing.
(C) XML files can be read by multiple different programs, as long as you deliver them in the correct format.
(D) The receiver of data can easily read the XML file and store it on their database.
The ability to use this method to read and transfer data is a very powerful way to help a data analyst process large data sets.
In fact, if you are using cloud-based applications to analyse the information you are storing, this will quickly enable you to deliver the data.
What packages do I need in Python?
The first step is to import the following:
import pandas as pd
from lxml import etree as et
Next we want to read in the source data
In this instance, we are reading an excel file
raw_data = pd.read_excel(r'Link to where your data is stored including the full file name')
Now we want to start building the XML structure
The FIRST STEP is to define the root
root = et.Element('root')
The root is the parent of all the data items (tags) contained in the XML file and is needed as part of the structure
The SECOND STEP is to define the tag names that will store each row of the source data
for row in raw_data.iterrows(): ==> This is a loop that takes runs through each record and populates for each tag.
root_tags = et.SubElement(root, 'ExportData') #=== > Root name
# These are the tag names for each row (SECTION 1)
Column_heading_1 = et.SubElement(root_tags, 'Name')
Column_heading_2 = et.SubElement(root_tags, 'Area')
Column_heading_3 = et.SubElement(root_tags, 'NoPurchases')
Column_heading_4 = et.SubElement(root_tags, 'Active')
###These are the values that will be populated for each row above
# The values inside the [] are the raw file column headings.(SECTION 2)
Column_heading_1.text = str(row[1]['Name'])
Column_heading_2.text = str(row[1]['Area'])
Column_heading_3.text = str(row[1]['No Purchases'])
Column_heading_4.text = str(row[1]['Active'])
The raw file looks like this:
Name | Area | No Purchases | Active |
John | Dublin | 2 | Y |
Mary | Galway | 3 | N |
Joe | Limerick | 4 | N |
Jimmy | Kilkenny | 55 | Y |
Jennifer | Belfast | 6 | N |
Susan | Waterford | 3 | Y |
Jake | Cork | 1 | Y |
Bobby | Dundalk | 11 | N |
Sarah | Sligo | 9 | N |
Cian | Ennis | 8 | Y |
The THIRD STEP is to create the XML file and populate it with the data from the source file
# This Section outputs the data to an xml file
# Unless you tell it otherwise it saves it to the same folder as the script.
tree = et.ElementTree(root) ==> The variable tree is to hold all the values of "root"
et.indent(tree, space="\t", level=0) ===> This just formats in a way that the XML is readable
tree.write('output.xml', encoding="utf-8") ==> The data is saved to an XML file
The XML output should look like the below
<root>
<ExportData>
<Name>John</Name>
<Area>Dublin</Area>
<NoPurchases>2</NoPurchases>
<Active>Y</Active>
</ExportData>
<ExportData>
<Name>Mary</Name>
<Area>Galway</Area>
<NoPurchases>3</NoPurchases>
<Active>N</Active>
</ExportData>
<ExportData>
<Name>Joe</Name>
<Area>Limerick</Area>
<NoPurchases>4</NoPurchases>
<Active>N</Active>
</ExportData>
<ExportData>
<Name>Jimmy</Name>
<Area>Kilkenny</Area>
<NoPurchases>55</NoPurchases>
<Active>Y</Active>
</ExportData>
<ExportData>
<Name>Jennifer</Name>
<Area>Belfast</Area>
<NoPurchases>6</NoPurchases>
<Active>N</Active>
</ExportData>
<ExportData>
<Name>Susan</Name>
<Area>Waterford</Area>
<NoPurchases>3</NoPurchases>
<Active>Y</Active>
</ExportData>
<ExportData>
<Name>Jake</Name>
<Area>Cork</Area>
<NoPurchases>1</NoPurchases>
<Active>Y</Active>
</ExportData>
<ExportData>
<Name>Bobby</Name>
<Area>Dundalk</Area>
<NoPurchases>11</NoPurchases>
<Active>N</Active>
</ExportData>
<ExportData>
<Name>Sarah</Name>
<Area>Sligo</Area>
<NoPurchases>9</NoPurchases>
<Active>N</Active>
</ExportData>
<ExportData>
<Name>Cian</Name>
<Area>Ennis</Area>
<NoPurchases>8</NoPurchases>
<Active>Y</Active>
</ExportData>
</root>
Additional XML data can be added
- Add more rows – All you need to do is add onto the source file and save. When you rerun the logic it will read in the extra information.
- Add more columns – All you need to do is go to the second step above add in a tag name to SECTION 1. Seperately you will need to add an additional column with data to the source file, and then add that column name to SECTION 2 as well