Skip to content
  • YouTube
  • FaceBook
  • Twitter
  • Instagram

Data Analytics Ireland

Data Analytics and Video Tutorials

  • Home
  • About Us
    • Latest
    • Write for us
    • Learn more information about our website
  • Useful Links
  • Glossary
  • All Categories
  • Faq
  • Livestream
  • Toggle search form
  • How many python functions are there? Python
  • What is the r programming language R Programming
  • Python Tutorial: Add a column to a data frame Python Dataframe
  • how to remove spaces from a string regular expressions
  • How can I filter my data in Tableau? data visualisation
  • TypeError: the first argument must be callable Python
  • What Are Constraints in SQL? SQL
  • how do I merge two dictionaries in Python? Python

How to remove unwanted characters

Posted on April 25, 2020February 11, 2021 By admin No Comments on How to remove unwanted characters

Estimated reading time: 2 minutes

Removing the unwanted, that is holding you up.
A situation has arisen of you having information, which has erroneous data inside it, what do you do?

Data issues are a common scenario faced by many data analytics professionals and the industry as a whole. Data quality now has become more critical, especially as we move more processes online and the digital landscape increases.

Most pieces of data go through a process of been transferred somewhere between systems to be used or reports rely on the accuracy of them. If the data in the source system has quality issues, the problem if not addressed before going somewhere else can then push the data quality issues more throughout an organisation, like a spiders web it expands further.

The next step, looking to fix the problem and planning for it.

To combat this problem professionals need to come up with a plan on how to tackle this, either:

  • Fix at source
  • Take the data in before moving it on, and investigate the problems.
  •  Reject the file or part thereof.

All three options above have scenarios around them with costs and implications, depending on the industry, you need to pick the most appropriate way to handle. As an example, in the banking industry payment files can sometimes have data in them that is rejected entirely or in part.

But the bank may decide they will only discard the records with the wrong data and process everything else.

How to go about it and how regular expressions can help

In this video, we look to go through an example of how to cleanse a data set;

(A) We use a list to check what problems we need to find.

(B) Using functions again to process through the data to find the problem and extract them.

(C) Regular expressions also appear as they look to find the special characters in the data set.

The concept of regular expressions is used extensively across several programming languages; it is a good way to test data and find erroneous values. If you are thinking about machine learning, it is quite important to get a more thorough knowledge of how they work. Here is a good link for further reading if you need more information Regular Expression how to

Thanks for watching and if you like, please share and subscribe through the buttons on this page!

Data Analytics Ireland

Python Data Cleansing Tags:Data Analytics, Data Cleansing, Data Quality, Learn python online, Learn python programming, Python Tutorial

Post navigation

Previous Post: YouTube channel lists – Python Data Cleansing
Next Post: How to remove characters from an imported CSV file

Related Posts

  • How to data cleanse a database table Python Data Cleansing
  • How To Check For Unwanted Characters Using Loops With Python Python Data Cleansing
  • YouTube channel lists – Python Data Cleansing Python Data Cleansing

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • TypeError: ‘list’ object is not an iterator Python
  • Tkinter python tutorial Python
  • how do I merge two dictionaries in Python? Python
  • ValueError: pattern contains no capture groups Value Error
  • TypeError: cannot unpack non-iterable int object Python
  • planning your machine learning model machine learning
  • What are dimensions in Tableau? data visualisation
  • Explain different types of data Normalization. SQL

Copyright © 2023 Data Analytics Ireland.

Powered by PressBook Premium theme