Are you on a limited budget but looking for free ways to extract data from files without using expensive online tools or companies that you will have to pay? Join us here for an overview of some tools and techniques that you most likely have access to already.
You are probably sitting there hearing about big data and databases, data analytics and machine learning and wonder where a data analyst fits in?
Here we will look to break it down step by step.
Sometimes a data analyst can be confused with a business analyst; there are subtle differences:
Business Analyst: Their role is to document the user’s requirements in a document that is descriptive of what the user wants.
In this case, a document that all parties can agree to is created, and it can be used as part of the project sign off.
Data Analyst: On the other hand, a data analyst will take the business requirements and translate them into data deliverables.
They use the document to ensure the project has the right data to meet the project objectives in the right place at the right time.
Data Mapping
In different data projects there will be a need to reconcile the data between systems, a data analysis will help here.
In a data mapping exercise, the data analyst will be expected to look at one or more sources and map them to a destination system.
This ensures a match between the two datasets.
Which results in the ability to reconcile the two systems.
Allows the ability to use data in multiple systems, knowing the consistency is in place.
Consistency of the data types between the systems.
It ensures that data validation errors are kept to a minimum.
Often a Data Analyst will build a traceability matrix, which tracks the data item from creation through to consumption.
Data Quality
In most companies, there will be teams (depending on their size) dedicated to this, and their input will be pivotal to existing and future data use.
It is an important task that could impact internal and external reporting and a company’s ability to make decisions accurately.
Some of the areas that might be looked at include:
(A) Investigate duplicate data – There could be a number of reasons this has to be checked:
Data manually entered multiple times.
An automated process ran multiple times.
A change to an IT system has unknowingly duplicated data.
(B) Finding errors – This could be completed in conjunction with data reporting outlined below.
Normally companies will clearly have rules that pick up the data errors that are not expected.
A data analyst will analyse why these errors are occurring.
(C) Checking for missing data.
Data feeds have failed. A request to reload the data will be required.
Data that was not requested as part of the business requirements confirm that this is the case.
(D) Enhancing the data with additional information – Is there additional information that can be added that can enrich the dataset?
(E) Checking data is in the correct format – There are scenarios where this can go wrong, and example is a date field is populated with text.
Data Reporting
In some of the areas above, we touched on the importance of the quality of data.
Ultimately there may be a need to track:
Data Quality – Build reports to capture the quality of data based on predefined business measurements.
Real-time Reporting – No new customers or customers who have left an organisation.
Track Targets – Is the target set by the business been met daily, weekly, monthly?
Management Reporting – Build reports that provide input to management packs that provide an overview of how the business performs.
Data Testing
Organisations go through change projects where new data is being introduced or enhanced.
As a result the data analyst will have a number of tasks to complete:
Write Test Scripts – Write all scripts for record counts, transformations and table to table comparisons.
Datatype Validation – Ensures all new data will be the same as the other data where it is stored.
No loss of data – Check all data is imported correctly with no data truncated.
Record count – Write an SQL script that would complete a source to the destination reconciliation.
Data Transformation – Ensure any transformations are applied correctly.
Supporting data projects
Ad hoc projects are common , and sometimes become a priority for businses as they deal with requirements that result as part of an immediate business need.
Data Analysts will be called upon to support projects where there is a need to ensure the data required is of a standard that meets the project deliverables:
Some common areas where this might occur includes:
Extract data where it has been found to have been corrupted.
Investigate data changes, to analyse where a data breach may have occurred.
An external regulatory body has requested information to back up some reports submitted.
A customer has requested all the company’s information on them; usually the case for a GDPR request.
Here is our first live stream, we discuss the channel, the areas of data analytics we cover, and the future direction and what the plans are for 2021.
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Cookie settingsACCEPT
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.