How To Check For Unwanted Characters Using Loops With Python

Estimated reading time: 3 minutes

On this website, we have posted about how to remove unwanted characters from your data and How to remove characters from an imported CSV file both will show you different approaches.

In this blog posting, we are going to approach the process by using loops within a function. Essentially we are going to pass a list and then we are going to loop through the strings to check the data against it.

Step 1 – Import the data and create a data frame

The first bit of work we need to complete is to load the data. Here we create a dictionary with their respective key-value pairs.

In order to prepare the data to be processed through the function in step 2, we then load it into a data frame.

import pandas as pd
#Create a dataset locally
data = {'Number':  ["00&00$000", '111$11111','2222€2222','3333333*33','444444444£','555/55555','666666@666666'],
        'Error' : ['0','0','0','0','0','0','0']}

#Create a dataframe and tell it where to get its values from
df = pd.DataFrame (data, columns = ['Number','Error'])

Step 2 – Create the function that checks for invalid data

This is the main piece of logic that gives the output. As you can see there is a list “L” that is fed to the function run.

One thing to note is that *l is passed to the function, as there is more than one value in the list, otherwise the program would not execute properly.

To start off we create a data frame, which extracts using a regular expression the characters we don’t want to have.

Next, we then need to drop a column that is generated with NAN values, as these are not required.

Then we updated the original data fame with the values that we found.

Just in case if there are any NAN values in this updated column “Error”, we remove them on the next line.

The main next is the loop that creates a new column called “Fix”. This holds the values that will be populated into the fix after the data we don’t want is removed and is data cleansed.

The data we do not need is in str.replace.

#Function to loop through the dataset and see if any of the list values below are found and replace them
def run(*l):
    #This line extracts all the special characters into a new column
    #Using regular expressions it finds values that should not appear
    df2 = df['Number'].str.extract('(\D+|^(0)$)') # New dataframe to get extraction we need
    print(df2)
    df2 = df2.drop(1, axis=1) # Drops the column with NAN in it, not required

    df['Error'] = df2[0] # Updates original dataframe with values that need to be removed.
    #This line removes anything with a null value
    df.dropna(subset=['Error'], inplace=True)
    #This line reads in the list and assigns it a value i, to each element of the list.
    #Each i value assigned also assigns an index number to the list value.
    #The index value is then used to check whether the value associated with it is in the df['Number'] column 
    #and then replaces if found
    for i in l:
        df['Fix']= df['Number'].str.replace(i[0],"").str.replace(i[1],"").str.replace(i[2],"").str.replace(i[3],"") \
        .str.replace(i[4],"").str.replace(i[5],"").str.replace(i[6],"")
        print("Error list to check against")
        print(i[0])
        print(i[1])
        print(i[2])
        print(i[3])
        print(i[4])
        print(i[5])
        print(i[6])
    print(df)

#This is the list of errors you want to check for
l = ['$','*','€','&','£','/','@']

Step 3 – Run the program

To run this program, we just execute the below code. All this does is read in the list “L” to the function “run” and then the output in step 4 is produced

run(l)

Step 4 – Output

Error list to check against
$
*
€
&
£
/
@
          Number Error           Fix
0      00&00$000     &       0000000
1      111$11111     $      11111111
2      2222€2222     €      22222222
3     3333333*33     *     333333333
4     444444444£     £     444444444
5      555/55555     /      55555555
6  666666@666666     @  666666666666

Process finished with exit code 0

How to use parameters in Python

Estimated reading time: 5 minutes

Are you using a function and looking to pass values into it to be processed? Well if this is the case you are trying to pass arguments to that function’s parameters so they will return a value for you.

Parameters are just values that are associated with the function. For each function, you are going to have one or more parameters it is expecting to process.

As a result, there are a number of ways they are created, let’s explore how this can be achieved.

Passing arguments directly into a function parameters

In a lot of instances, the argument values to be processed are going to be passed into the function parameters, which then executes some logic and returns a value.

Typical ways this may happen:

  1. A password is entered ( the inputted argument), the function then confirms if it is valid or not.
  2. You may need to check if a value entered ( the inputted argument) is above or below a certain value. It is passed to a function parameter and then the function does the necessary checks, and confirms the result in the output.

The above are two examples, but let’s see this in action.

Below we have a function. The purpose is to take in two arguments passed to it and conduct a calculation on them.

The calculation happens in the variable number 1 or number 2.

Variable number 1 and variable number 2 get their values from parameters a and b respectively.

The line multiplenumber(1,2) is just calling the function and passing the arguments to the parameters a,b which then processes them and the output is printed.

def multiplenumber(a,b):
    number1 = a*2
    number2 = b*3
    print(number1,number2) #===> These values are different to the variables below as they happen as a result of the processing inside the function.

number1=1
number2=2
multiplenumber(number1,number2)
print(number1,number2) #===> These values are outside the function and do not have any relationship to the variables inside the function above, hence are different.

Result:
2 6
1 2

The result of the above piece of logic is called “passing by value“, which essentially means the data is passed from the variables ( number 1 and Number 2) to the function parameters, but when the function is completed processing, it does not impact the original variables outside the function.

This is because the original variables are immutable.

Passing arguments directly into function parameters from a list

In the above example, as outlined, there will be no impact on the variables that pass the arguments into the function, they remain the same.

On the other hand, if the original source of the arguments is a mutable object, it will have its values changed.

Let’s look at a situation where this may occur.

def addnumber(a):
    number=['3']
    a.append(number)
    print(a) #===> This prints the original value of a with the addition of the value 3

number1=[1,2]
addnumber(number1)
print(number1) #===> As number1 is passed to the function, and number1 is allowed to be changed then this will print out the new list value once the function has completed processing.

Result:
[1, 2, ['3']]
[1, 2, ['3']]

So in this instance, as number1 is a list, and this is mutable, its value will change once the function completes its processing.

This can also be called “passing by reference“.

Passing arguments between functions

In another video, we completed called how to pass data between functions this discusses how you can pass the data between two functions and print it out.

Effectively in the below the arguments are name, country, city, and these values are passed to functionb

def functiona():
    name = "Joseph"
    country = "Mars"
    city = "atlantis"
    #this captures the three variables above
    return name,country,city
functiona()

def functionb():
    myname,mycountry,mycity = functiona()
    print(myname)
    print(mycountry)
    print(mycity)

functionb()

Result:
Joseph
Mars
atlantis

Passing arguments using *args as the parameter

In order to understand this, before we look at some code we need to understand what are *args?

*args essentially allows you to pass multiple positional arguments to a function, and this can vary over time.

As a result, you don’t have to assign the positional arguments to more than one variable. The *args take in all the arguments, no matter how long or short they are and then return a value.

def addnumbers(*args):
    result = 2
    for x in args:
        result = result * x
    return result

print(addnumbers(1,2,3))

Result:
12

Passing arguments using **kwargs as the parameter

In contrast to the above, this will process a set of values that have a name attached to them.

As can be seen in the previous example the arguments passed had no name attached to them.

Let’s look closely at an example. In this scenario, the arguments are assigned a value of a,b,c.

def addnumbers(**kwargs):
    result = 2
    for x in kwargs.values():
        result = result * x
    return result

print(addnumbers(a=1,b=2,c=3))

Result:
12

Here it is clear that the logic takes in each value of a,b,c processes it and then returns the result required.

In essence, both *args and **kwargs give you the same answer, it just depends on other parts of your code and how it is processing your data.

For example, in the **kwargs it could be that the values of a,b,c could be populated by other variables within the program.

So in that scenario, the variable that has been passed on can change and not be static.

So in summary:

a. Arguments can be passed multiple ways to functions, the parameters then just process them.

b. Parameters also don’t have to be assigned to a variable, they can be read in through *args.

c. On the other hand you can have multiple arguments read in at once by using the **kwargs.

TypeError: ‘float’ object is not callable

So you have been presented with the TypeError: ‘float’ object is not callable in Python, but unsure how to fix? Well read on so we can get you a solution.

First things an overview of a Float

A float is any number that can contain a decimal place separating an integer from the fractions to the right of the decimal point.

As an example, in the below code, we have the exact same number. But when a computer comes to read it, it treats them differently, due to the fact that b has a decimal point and a fraction to the right of it.

a = 1
b = 1.0

print (type(a))
print (type(b))

Output:
<class 'int'>
<class 'float'>

So how in a piece of code would this error occur?

In the below code, the error occurs. The reason behind this is that float is an actual function that changes a value to a float data type.

As a result, you cannot assign a variable name as “float” and then call it separately in its code. Python specifically does not allow a programmer to use a function as a variable name.

a= float(10)
b= float(11)
c= float(12)
float = a

d = float(a/b*c)
print(d)

Output:
d = float(a/b*c)
TypeError: 'float' object is not callable

So how is this fixed?

In order to fix this, we rename the variable “float” to something else, and make sure that the new name is not another function name!

As can be seen when we do this, the below output runs as expected with no errors:

a= float(10)
b= float(11)
c= float(12)
float_value = a   ===> This line was corrected from above from "float" to "float_value"

d = float(a/b*c)
print(d)

Output: 
10.909090909090908

In summary the crux of this problem is that when creating a variable, it cannot be a function name.

You can see similar posts here on TypeErrors.

Have you seen these posts?

TypeError: ‘float’ object is not callable

TypeError: ‘int’ object is not callable

TypeError: ‘int’ object is not callable

Estimated reading time: 1 minute

We have posted several python solutions to TypeErrors here on the website. Here is another one we hope you will find useful in solving a programming issue you may have.

So what does the error mean?

In the below code, we have four variables that have been assigned an integer.

a= 10
b= 11
c= 12
int= 13

d = int(a/b*c)
print(d)

As can be seen we also have a variable d that is assigned to a function int , that is using the variables a,b,c.

As int is a function it cannot be assigned as a variable, and for this reason the below error will appear:

d = int(a/b*c)
TypeError: 'int' object is not callable

So how can this be fixed?

The solution to this is quite straight forward, it is important not to assign a function as a variable. The fix you would apply is as follows:

Incorrect code:

a= 10
b= 11
c= 12
int = 13  ===> change this line to fix the error

d = float(a/b*c)
print(d)

Corrected code:
a= 10
b= 11
c= 12
int_value = 13  ===> corrected line of code
d = float(a/b*c)
print(d)

Giving you the result:
10

As can also be seen with TypeError: ‘str’ object is not callable assign variables to functions should be avoided at all cost.

R Tutorial: How to pass data between functions

When starting to look at functions and having tested them in Python and Javascript, it was quickly apparent how programming languages are so similar.

Except for the syntax you use in each; the programming is quite similar.

The purpose of this video is to:

  • Start on using functions from the ground up.
  • Don’t over-complicate the example; keep it easy enough to follow.

How to write the code to pass data between functions

As this is a short video, the code that went into making it is pretty straight forward

# create a function
function.a <- function(){
  newvarb <- 2
}

function.b <- function(){
  newvarb <- function.a()*2 # this takes in the value of function a and multiplies it by two
}
print(function.b()) # Prints out the value of function b

Below is the video that will take you through each line, and show the output that we are looking to achieve

How can we use this in our projects

No matter what programming language you use or choose to learn, the concept of functions will appear in some shape or form. Their ability to quickly run a repeatable process and return a value, which can be called from anywhere in a program allows the programmer to reduce their coding time swiftly and reduce repetitive tasks that only need to run once.

This video has an equivalent in Python, and you can see it here  Python Functions – passing data between them

Data Analytics Ireland

how to pass data between functions

Estimated reading time: 2 minutes

In this python program, we are learning how to pass data between functions. 

In light of this, you will use and see functions in many programming languages and data analytics projects.

As a result, the ability to understand them has become important.

Functions serve a number of benefits:

  • You can pass a number of arguments to them to be processed.
  • It reduces repetition as the function can be called from many places with a program.
  • They are easily identified by using the def keyword in your code.
  • A return statement can give you the output of the function to show on the screen or pass to another function.

It compliments r-tutorial-how-to-pass-data-between-functions/  as a result this is a handy bit of functionality used widely across many different programming languages.

Below is a video that will help to give an understanding of how to pass data between functions when trying to learn python:

 

In many of the Data Analytics Ireland    YouTube channel videos, there is an emphasis on creating content that eliminates duplication of code within the code.

We have also started incorporating classes as well and you can see here How to create a class in Python, a tutorial on how to create one.

Classes by their nature have methods, which are called on the objects that created them ( the class), and can alter their state, whereas a function will run and just return a value.

It is important to understand the distinction as while the two will most likely achieve the same outcome, it is the ability to change the class state that will differentiate the two.