Estimated reading time: 4 minutes
Livestream #2 – What does a data analyst do?
You are probably sitting there hearing about big data and databases, data analytics and machine learning and wonder where a data analyst fits in?
Here we will look to break it down step by step.
Sometimes a data analyst can be confused with a business analyst; there are subtle differences:
- Business Analyst: Their role is to document the user’s requirements in a document that is descriptive of what the user wants.
- In this case, a document that all parties can agree to is created, and it can be used as part of the project sign-off.
- Data Analyst: On the other hand, a data analyst will take the business requirements and translate them into data deliverables.
- They use the document to ensure the project has the right data to meet the project objectives in the right place at the right time.
Data Mapping
In different data projects, there will be a need to reconcile the data between systems, a data analysis will help here.
In a data mapping exercise, the data analyst will be expected to look at one or more sources and map them to a destination system.
- This ensures a match between the two datasets.
- Which results in the ability to reconcile the two systems.
- Allows the ability to use data in multiple systems, knowing the consistency is in place.
- Consistency of the data types between the systems.
- It ensures that data validation errors are kept to a minimum.
Often a Data Analyst will build a traceability matrix, which tracks the data item from creation through to consumption.
Data Quality
In most companies, there will be teams (depending on their size) dedicated to this, and their input will be pivotal to existing and future data use.
Data quality is an important task that could impact internal and external reporting and a company’s ability to make decisions accurately.
Some of the areas that might be looked at include:
(A) Investigate duplicate data – There could be a number of reasons this has to be checked:
- Data was manually entered multiple times.
- An automated process ran multiple times.
- A change to an IT system has unknowingly duplicated data.
(B) Finding errors – This could be completed in conjunction with the data reporting outlined below.
- Normally companies will clearly have rules that pick up the data errors that are not expected.
- A data analyst will analyse why these errors are occurring.
(C) Checking for missing data.
- Data feeds have failed. A request to reload the data will be required.
- Data that was not requested as part of the business requirements confirm that this is the case.
(D) Enhancing the data with additional information – Is there additional information that can be added that can enrich the dataset?
(E) Checking data is in the correct format – There are scenarios where this can go wrong, and an example is a date field is populated with text.
Data Reporting
In some of the areas above, we touched on the importance of the quality of data.
Ultimately there may be a need to track:
- Data Quality – Build reports to capture the quality of data based on predefined business measurements.
- Real-time Reporting – No new customers or customers who have left an organisation.
- Track Targets – Is the target set by the business been met daily, weekly, or monthly?
- Management Reporting – Build reports that provide input to management packs that provide an overview of how the business performs.
Data Testing
Organisations go through change projects where new data is being introduced or enhanced.
As a result, the data analyst will have a number of tasks to complete:
- Write Test Scripts – Write all scripts for record counts, transformations and table-to-table comparisons.
- Datatype Validation – Ensures all new data will be the same as the other data where it is stored.
- No loss of data – Check all data is imported correctly with no data truncated.
- Record count – Write an SQL script that would complete a source-to-the-destination reconciliation.
- Data Transformation – Ensure any transformations are applied correctly.
Supporting data projects
Ad hoc projects are common, and sometimes become a priority for businesses as they deal with requirements that result as part of an immediate business need.
Data Analysts will be called upon to support projects where there is a need to ensure the data required is of a standard that meets the project deliverables:
Some common areas where this might occur include:
- Extract data where it has been found to have been corrupted.
- Investigate data changes, to analyse where a data breach may have occurred.
- An external regulatory body has requested information to back up some reports submitted.
- A customer has requested all the company’s information on them; usually the case for a GDPR request.