How to run Python validation from Javascript

Estimated reading time: 6 minutes

More and more, website developers are looking to blend different programming technologies to allow them to use functionality, that maybe is not available to them in the current programming language they utilise.

In previous posts how to pass python variables to Javascript and how to run python directly from javascript, we touched on how to use Python and Javascript interchangeably to pass data.

Here in this blog post, we are going to look at data validation using Python, with data captured by Javascript. The purpose here is to allow another way to validate data other than relying on javascript.

Python Validation run from Javascript

There are a number of reasons for this:

  1. You may want to use Python to process the data and provide output.
  2. The data could be passed from Python into a database, you need to run some checks before you proceed.
  3. Python is your core language, you only use Javascript to capture the data and pass it from a website.

There are four validation checks

  1. Validation if a username exists.
  2. Validation if a number has been entered on a username field.
  3. Validation of a password entered.
  4. A validation check to make sure that an email address entered is in the correct format.

We have two files with the requisite code in them as follows:

  1. app.py ===> As this application is built in Python Flask, this file holds the logic to open the website files, and it holds the Python validation checks.
  2. index.html ====> This file holds the HTML that creates the pages, labels, and input boxes that appear in the web browser. It also includes the Javascript that will capture the data to be passed to Python for validation.

Code Overview – App.PY

Below, I will look to explain what is going on as follows:

The variable Regex is used in the email validation to check if the email entered is correct.

def inputcheck ===> This is just checking the username passed over and performing some validations on it.

def inputvalidation ====> This also is checking the username but looking at if numbers only are entered, or if it is empty.

def passvalidation ====> In this piece of logic, it checks if the password is empty, less than five characters, is numbers only.

def emailvalidation ====> All this is doing is checking if the data received from the Javascript is in the correct email format. It references the regex variable above, which is used to confirm if the format is correct or otherwise.

Within each of the functions, there are if statements which are the core validation checks used by Python.

Also, all the popup menus use ctypes, which allows you to access the windows libraries that hold message boxes, customise them and call within your program.

import json
import ctypes
import re
from flask import request,redirect,url_for

from flask import Flask, render_template

app = Flask(__name__)

regex = '^[a-z0-9]+[\._]?[a-z0-9]+[@]\w+[.]\w{2,3}$'

@app.route('/')
def index():
    return render_template('index.html')


@app.route('/inputcheck', methods=['GET','POST'])
def inputcheck():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    username_list = ['joe','john']
    if result['username'] in username_list:
        MessageBox = ctypes.windll.user32.MessageBoxW(0, 'Username ' + result['username'] + ' ' + 'exists', "Username check", 0x00001000)
        return render_template('various.html')
    elif result['username'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'You cannot enter an empty value', 'Username check', 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Username ' + result['username'] + ' ' + 'does not exist', 'Username check', 0x00001000)

@app.route('/inputvalidation', methods=['GET','POST'])
def inputvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if result['username'].isdecimal():
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username cannot be numbers',"Number check", 0x00001000)
    elif result['username'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'The username cannot be empty', "Number check",0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username looks ok', "Number check", 0x00001000)
    return render_template('index.html')

@app.route('/passvalidation', methods=['GET','POST'])
def passvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if result['password'].isdecimal():
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your password cannot be numbers',"Password check", 0x00001000)
    elif result['password'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'The password cannot be empty', "Password empty check",0x00001000)
    elif len(result['password']) < 5:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username should be greater than five characters', "Password length check", 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your password looks ok', "Number check", 0x00001000)
    return render_template('index.html')

@app.route('/emailvalidation', methods=['GET','POST'])
def emailvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if re.search(regex, result['email']):
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your email is in the correct format', "Email check", 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your email is invalid, please correct', "Email check", 0x00001000)
    return render_template('index.html')



if __name__ == "__main__":
    app.run(debug=True)


Code Overview – Index.html

The below code is the page code, that allows the capturing of data through a web browser HTML page.

Between the <style></style> tags these are the CSS properties that format the labels and buttons on the page.

Within the <div></div> tags are where we create the labels and buttons to show on the screen. Also within these, are the references to the on click event function that should run once the buttons are clicked.

Within the <script></script> tags, this is where the javascript is written which captures the data entered into the input boxes within the <div> tags.

Also, you will see within this javascript, there are sections called $.ajax, which is where the data captured by the javascript is stored, and then passed onto the Python script file (app.py)

Note that in each ajax , url:”/APAGENAME” is referenced. These are the sections within the app.py that the data is passed to, and then the Python logic kicks in and validates the data.

<html lang="en">

<head>

    <title>Data Analytics Ireland</title></head>
<style>
.button1 {
    position: absolute;
    top: 50%;
    left: 55%;
    width: 900px;
    height: 300px;
    margin-left: -300px;
    margin-top: -80px;
}



.labels {
    position: absolute;
    top: 50%;
    left: 55%;
    width: 900px;
    height: 300px;
    margin-left: -300px;
    margin-top: -150px;
}

</style>

<body>



<div class="labels" id="mem1" >
<label  for="username">username:</label> <input type="text" id="username" name="username">
<label for="password">password:</label><input type="text" id="password" name="password">
    <label for="email">email:</label><input type="text" id="email" name="password">
</div>


<div class="button1" id="mem2" >
<button type="submit" onclick='myfunction();'>Click here to validate your username</button>
<button type="submit" onclick='myfunctionval();'>Click here to check if a number</button>
    <button type="submit" onclick='myfunctionpass();'>Click here to check password</button>
      <button type="submit" onclick='myfunctionemail();'>Click here to check your email</button>
</div>


<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>

<script>
    function myfunction() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/inputcheck",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>
<script>
    function myfunctionval() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/inputvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>

<script>
    function myfunctionpass() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/passvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>

<script>
    function myfunctionemail() {

        const email = document.getElementById("email").value;


        const dict_values = {email} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/emailvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>




</body>
</html>

How to Pass Python Variables to Javascript

Estimated reading time: 4 minutes

In our recent blog posting How to Pass a Javascript Variable to Python using JSON, we demonstrated how to easily use AJAX to pass whatever data you wanted and then manipulate it with Python.

In this blog posting, we are going to show how to do this the other way around. The scenario is that you have an application and or website that wants to use data generated through Python, but let Javascript then use it within the application.

As Python can be connected to numerous databases and files ( txt, excel) etc, this piece of logic is very useful for the programmer looking to integrate both programming languages.

Let’s start looking at the code, and see how this can be achieved.

Step 1 – What Files are generated?

This program uses Python Flask to create a web page, that has a drop-down menu. The two files used to generate this are as follows:

(A) app.py – This is the python file that creates a website and loads a template HTML file as outlined below.

(B) Index.html – This is the template file that loads into the browser and runs all the javascript. The javascript loaded here also loads the python data passed over from app.py

Step 2 – APP.PY code overview

The Python library that enables webpage creation is called Flask, and as can be seen below it has to be imported.

In addition, we need to also import render_template which tells the program to go to the templates folder and load “Index.HTML”

The variable that is been passed to JavaScript is called name, and these are the values that you will see in the web browser when it is loaded.

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    name = ['Joe','John','Jim','Paul','Niall','Tom']
    return render_template('index.html', name=name)

if __name__ == "__main__":
    app.run(debug=True)

Step 3 – Index.HTML overview

Here is the template HTML file that runs in the browser. You can add CSS etc to this to make it look nicer and more user friendly.

As you can see it has the usual HTML tags appear as part of a website.

Well look at some of the code further:

In this bit <select id =’select’> </select>, this is the dropdown menu that will appear when Index.html is opened. It will store all the values passed from python. Note that its id is “select”, this will be used later on.

The main parts to focus on next is between <script></script>. This is what reads in the python data and populates it to the dropdown menu.

In step 2 we mentioned that there was a variable called “name”, with values to be passed over.

This is acheived on this line:

var select = document.getElementById(“select”), test = {{ name | tojson }};

Notice that name appears here, and this is referencing back to the exact same value that was discussed in step 2.

For the rest of the lines, I have explained with comments what each does.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Pass Python variable to Javascript</title>
</head>
<body>

<select id ='selectvalue'>
</select>
<script>
    //name = ['Joe','John','Jim','Paul','Niall','Tom']

    var selectvalue = document.getElementById("selectvalue"), test = {{ name | tojson }};
    //The increment operator (++) increments (adds one to) its operand and returns a value.
    for(var i = 0; i < test.length; i++) // This line checks for the length of the data you feeding in i.e the no of items
         {
var selection = document.createElement("OPTION"), // This line creates a variable to store the different values fed in from the JSON object "TEST"
txt = document.createTextNode(test[i]); // This just reads each value from the test JSON variable above
selection.appendChild(txt); // This line appends each value as it is read.
selection.setAttribute("value",test[i]); // This line sets each value read in as a value for the drop down
selectvalue.insertBefore(selection,selectvalue.lastChild); //This reads eah value into the dropdown based on the order in the "TEST" above.
 }
</script>
</body>
</html>

Step 4 – What the output looks like!

From step 2, these are values we asked to be used in Javascript to populate a dropdown:

name = [‘Joe’,’John’,’Jim’,’Paul’,’Niall’,’Tom’]

Python variable passed to Javascript

How to pass multiple lists to a function and compare

Estimated reading time: 3 minutes

Have you ever been faced with a situation where you have multiple lists you want to compare, and need a function that will read them in and show you what is different between them all?

In this scenario, this can be handy where you have large data sets and need a quick way to find those differences and fix them where appropriate.

Comparing two lists

Let’s look and see what is going on below.

First of all, we have defined two lists. The only difference between the two is that one has a value of six, the other does not.

Next, we have the function “comparelists”. What this is doing is taking in the two lists as parameters (a,b), and then processing them.

The lists are passed as arguments to the parameters in this line ===> comparelists(list1,list2)

The parameters a are assigned to list1, and b is assigned to list2.

The main bit of the function is the list comprehension, and it is doing the following:

  1. x for x is basically creating a variable x, and starting a loop that goes through all the values of b.
  2. Each iteration of x is stored and compared with a.
  3. “If x not in a” completes the comparison and if true returns the value, otherwise moves to the next value.

As a result of this logic, it can be seen that six is the only value returned, and this is what we are expecting based on a visual inspection.

#Reading in two lists
list1 = [1,2,3,4,5] # parameter a below
list2 = [1,2,3,4,5,6] # parameter b below

def comparelists(a,b):
    z = [x for x in b if x not in a] #list comprehension
    print(z)

comparelists(list1,list2)

Output:

[6]

Comparing more than two lists

In the logic above, we saw that two lists can be compared. But what if you want to compare one list against two other lists?

That is easily done, with some tweaking to the code.

As before the three lists are defined, created as arguments in comparelists(list1,list2,list3), and then passed to the function parameters a,b,c.

The only difference in this logic is that the list comprehension is written a little different as follows:

  1. x for x is basically creating a variable x, and starting a loop that goes through all the values of b. ====> This is the same as above
  2. Each iteration of x is stored and compared with a and c ===> This line is different as comparing to two lists now.
  3. “if x not in a and x not in c” ====> Here we have two comparisons
    • Checking for the value x from b in a
    • Checking for the value x from b in c
  4. The value of six is correct as it is not in either a or c.
  5. Note the if statement is specific as to what to check for with the “and” statement. This can be changed to suit your needs.
list1 = [1,2,3,4,5] # parameter a below
list2 = [1,2,3,4,5,6] # parameter b below
list3 = [1,2,3,7,8,9] # parameter c below

def comparelists(a,b,c):
    z = [x for x in b
         if x not in a and x not in c] #list comprehension
    print(z)

comparelists(list1,list2,list3)

Output:
[6]

How to use parameters in Python

Estimated reading time: 5 minutes

Are you using a function and looking to pass values into it to be processed? Well if this is the case you are trying to pass arguments to that function’s parameters so they will return a value for you.

Parameters are just values that are associated with the function. For each function, you are going to have one or more parameters it is expecting to process.

As a result, there are a number of ways they are created, let’s explore how this can be achieved.

Passing arguments directly into a function parameters

In a lot of instances, the argument values to be processed are going to be passed into the function parameters, which then executes some logic and returns a value.

Typical ways this may happen:

  1. A password is entered ( the inputted argument), the function then confirms if it is valid or not.
  2. You may need to check if a value entered ( the inputted argument) is above or below a certain value. It is passed to a function parameter and then the function does the necessary checks, and confirms the result in the output.

The above are two examples, but let’s see this in action.

Below we have a function. The purpose is to take in two arguments passed to it and conduct a calculation on them.

The calculation happens in the variable number 1 or number 2.

Variable number 1 and variable number 2 get their values from parameters a and b respectively.

The line multiplenumber(1,2) is just calling the function and passing the arguments to the parameters a,b which then processes them and the output is printed.

def multiplenumber(a,b):
    number1 = a*2
    number2 = b*3
    print(number1,number2) #===> These values are different to the variables below as they happen as a result of the processing inside the function.

number1=1
number2=2
multiplenumber(number1,number2)
print(number1,number2) #===> These values are outside the function and do not have any relationship to the variables inside the function above, hence are different.

Result:
2 6
1 2

The result of the above piece of logic is called “passing by value“, which essentially means the data is passed from the variables ( number 1 and Number 2) to the function parameters, but when the function is completed processing, it does not impact the original variables outside the function.

This is because the original variables are immutable.

Passing arguments directly into function parameters from a list

In the above example, as outlined, there will be no impact on the variables that pass the arguments into the function, they remain the same.

On the other hand, if the original source of the arguments is a mutable object, it will have its values changed.

Let’s look at a situation where this may occur.

def addnumber(a):
    number=['3']
    a.append(number)
    print(a) #===> This prints the original value of a with the addition of the value 3

number1=[1,2]
addnumber(number1)
print(number1) #===> As number1 is passed to the function, and number1 is allowed to be changed then this will print out the new list value once the function has completed processing.

Result:
[1, 2, ['3']]
[1, 2, ['3']]

So in this instance, as number1 is a list, and this is mutable, its value will change once the function completes its processing.

This can also be called “passing by reference“.

Passing arguments between functions

In another video, we completed called how to pass data between functions this discusses how you can pass the data between two functions and print it out.

Effectively in the below the arguments are name, country, city, and these values are passed to functionb

def functiona():
    name = "Joseph"
    country = "Mars"
    city = "atlantis"
    #this captures the three variables above
    return name,country,city
functiona()

def functionb():
    myname,mycountry,mycity = functiona()
    print(myname)
    print(mycountry)
    print(mycity)

functionb()

Result:
Joseph
Mars
atlantis

Passing arguments using *args as the parameter

In order to understand this, before we look at some code we need to understand what are *args?

*args essentially allows you to pass multiple positional arguments to a function, and this can vary over time.

As a result, you don’t have to assign the positional arguments to more than one variable. The *args take in all the arguments, no matter how long or short they are and then return a value.

def addnumbers(*args):
    result = 2
    for x in args:
        result = result * x
    return result

print(addnumbers(1,2,3))

Result:
12

Passing arguments using **kwargs as the parameter

In contrast to the above, this will process a set of values that have a name attached to them.

As can be seen in the previous example the arguments passed had no name attached to them.

Let’s look closely at an example. In this scenario, the arguments are assigned a value of a,b,c.

def addnumbers(**kwargs):
    result = 2
    for x in kwargs.values():
        result = result * x
    return result

print(addnumbers(a=1,b=2,c=3))

Result:
12

Here it is clear that the logic takes in each value of a,b,c processes it and then returns the result required.

In essence, both *args and **kwargs give you the same answer, it just depends on other parts of your code and how it is processing your data.

For example, in the **kwargs it could be that the values of a,b,c could be populated by other variables within the program.

So in that scenario, the variable that has been passed on can change and not be static.

So in summary:

a. Arguments can be passed multiple ways to functions, the parameters then just process them.

b. Parameters also don’t have to be assigned to a variable, they can be read in through *args.

c. On the other hand you can have multiple arguments read in at once by using the **kwargs.

TypeError: list indices must be integers or slices, not tuple

Estimated reading time: 3 minutes

When working with Python lists in your data analytics projects, when you trying to read the data, a common problem occurs if you have a list of lists, and it is not properly formatted.

In this instance, Python will not be able to read one or more lists and as a result, will throw this error.

In order to understand how this problem occurs, we need to understand how to create a list of lists.

How to create a lists of lists

Let’s look at a simple list:

a = [1,2,3]
print(a)
print(type(a))

Result:
[1, 2, 3]
<class 'list'>

Let’s create a second list called b:

b = [4,5,6]
print(b)
print(type(b))

Result:
[4, 5, 6]
<class 'list'>

So if we want to join the lists together into one list ( hence a list of lists) then:

a = [1,2,3]
b = [4,5,6]

list_of_lists = []
list_of_lists.append(a)
list_of_lists.append(b)
print(list_of_lists)
print(type(list_of_lists))

Result:
[[1, 2, 3], [4, 5, 6]]
<class 'list'>

So as can be seen the two lists are contained within a master list called “list_of_lists”.

So why does the error list indices must be integers or slices, not tuple occur?

Reason 1 ( Missing commas between lists)

If you manually type them in and forget the comma between the lists this will cause your problem:

a=[[1,2,3][4,5,6]]
print(a)

Result (Error):
Traceback (most recent call last):
  line 10, in <module>
    a=[[1,2,3][4,5,6]]
TypeError: list indices must be integers or slices, not tuple

But if you put a comma between the two lists then it returns no error:

a=[[1,2,3],[4,5,6]]
print(a)

Result (no error):
[[1, 2, 3], [4, 5, 6]]
Process finished with exit code 0

Reason 2 ( Comma in the wrong place)

Sometimes you have a list, and you only want to bring back some elements of the list, but not others:

In the below, we are trying to bring back the first two elements of the list.

a= [1,2,3,4,5]
print(a[0:,2])

Result:
Traceback (most recent call last):
   line 14, in <module>
    print(a[0:,2])
TypeError: list indices must be integers or slices, not tuple

The reason that the same error happens is the additional comma in a[0:,2], causes the error to appear as Python does not know how to process it.

This is easily fixed by removing the additional comma as follows:

a= [1,2,3,4,5]
print(a[0:2])

Result:
[1, 2]
Process finished with exit code 0

So why is there a mention of tuples in the error output?

The final piece of the jigsaw needs to understand why there is a reference to a tuple in the error output?

If we return to a looking at a list of lists and look at their index values:

a=[[1,2,3],[4,5,6]]
z = [index for index, value in enumerate(a)]
print(z)

Result:
[0, 1]
Process finished with exit code 0

As can be seen, the index values are 0,1, which is correct.

As before removing the comma gives the error we are trying to solve for:

a=[[1,2,3][4,5,6]]
z = [index for index, value in enumerate(a)]
print(z)

Result:
Traceback (most recent call last):
  line 16, in <module>
    a=[[1,2,3][4,5,6]]
TypeError: list indices must be integers or slices, not tuple

BUT the reference to the tuple comes from the fact that when you pass two arguments (say an index value) a Tuple is created, but in this instance, as the comma is missing the tuple is not created and the error is called.

This stems from the __getitem__ for a list built-in class cannot deal with tuple arguments that are not integers ( i.e. 0,1 like we have returned here) , so the error is thrown, it is looking for the two index values to pass into a tuple.

How to Compare Column Headers in CSV to a List in Python

Estimated reading time: 3 minutes

So you have numerous different automation projects in Python. In order to ensure a clean and smooth straight-through processing, checks need to be made to ensure what was received is in the right format.

Most but not all files used in an automated process will be in the CSV format. It is important there that the column headers in these files are correct so you can process the file correctly.

This ensures a rigorous process that has errors limited.

How to compare the headers

The first step would be to load the data into a Pandas data frame:

import pandas as pd

df = pd.read_csv("csv_import.csv") #===> Include the headers
print(df)

The actual original file is as follows:

Next we need to make sure that we have a list that we can compare to:

header_list = ['Name','Address_1','Address_2','Address_3','Address_4','City','Country']

The next step will allow us to save the headers imported in the file to a variable:

import_headers = df.axes[1] #==> 1 is to identify columns
print(import_headers)

Note that the axis chosen was 1, and this is what Python recognises as the column axes.

Finally we will apply a loop as follows:

a = [i for i in import_headers if i not in header_list]
print(a)

In this loop, the variable “a” is taking the value “i” which represents each value in the import_headers variable and through a loop checks each one against the header_list to see if it is in it.

It then prints out the values not found.

Pulling this all together gives:

import pandas as pd

df = pd.read_csv("csv_import.csv") #===> Include the headers
print(df)

#Expected values to receive in CSV file
header_list = ['Name','Address_1','Address_2','Address_3','Address_4','City','Country']

import_headers = df.axes[1] #==> 1 is to identify columns
print(import_headers)


a = [i for i in import_headers if i not in header_list]
print(a)

Resulting in the following output:

As can be seen the addresses below where found not to be valid, as they where not contained within our check list “header_list”

TypeError: ‘list’ object is not an iterator

We have covered off many TypeErrors on this website, here we will go through which using a list with and it is not an iterator gives you errors.

In order to understand this error better, we need to first understand what is an iterator in Python?

An iterator is a Python object that has the following characteristics:

  • You can count the no of values that are contained within it.
  • It also can be iterated through, so you need to apply an iteration method to it.

How does this error occur?

Normally this error occurs when you try to iterate over a list, but you have not made the list iterable.

There are two things required to make this happen:

(A) The iter() returns an iterator object

(B) The next() method moves to the next value.

Without both the code will fail and the error you are about will occur!

In the below code we have a list:

a = ['q', 'w', 'e', 'r', 't', 'y']

with the following:

b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)

As can be seen in the above code we have one component for the iteration , we expect two as per the above.

As a result we get the error:

Traceback (most recent call last):
  File "list_object_is_not_an_iterator.py", line 13, in <module>
    b = next(a)
TypeError: 'list' object is not an iterator

In order to fix this ,all we need to do is apply the iterator to the list as follows:

a = iter(['q', 'w', 'e', 'r', 't', 'y']) ====> We added in the iter() here, enclosing the list within it

b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
#b = next(a)


print(b)

Giving output:
y

As a result of this, we now have the two required methods that will not give this error.

What is going on within the iterator?

In the above code we have asked to print b. What the iterator is doing is going to the first value of b, in this case q and print.

But because we have a variable b on multiple lines, with the method “next()” in it, the logic is moving through each value of the list till it gets to the end.

What can be done though is , reduce the length of the returned b variables to print as follows:

a = iter(['q', 'w', 'e', 'r', 't', 'y'])
b = next(a)
print(b)
returns:
q

BUT
a = iter(['q', 'w', 'e', 'r', 't', 'y'])
b = next(a)
b = next(a)
print(b)
returns:
w

As can be seen it returns the next value in the list. You can keep adding the b variables.

What happens when you get to the end of the list?

So now we have the below, and we are returning the last value:

a = iter(['q', 'w', 'e', 'r', 't', 'y'])
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)

Returns:
y

The reason for this is that we have the required no of variables with the next method, which equals the length of the list.

If we add in one more b variable:

a = iter(['q', 'w', 'e', 'r', 't', 'y'])
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a)
b = next(a) ===> Additional b variable

Returns: 
Traceback (most recent call last):
  File "list_object_is_not_an_iterator.py", line 19, in <module>
    b = next(a)
StopIteration

The purpose of StopIteration is to not allow a continuous loop and recognise that the end of the list has been reached.

Implementing Iterators

Iterators could be used in the following circumstances:

(A) You have a defined list of object values to work with.

(B) If sequence is important an iterator will help to process values in the order they appear in a list.

how do I declare a null value in python?

In writing about this question, it is important to understand a few things , before we step into the code.

Null values appear in almost every programming language, but they can have a different meaning.

A null value basically means it is empty. So when you return a value that is null, it has nothing in it.

Sometimes it can be confused with the value zero, but as zero is an actual integer, it is not an empty value in a field.

Python uses None to define the keyword null, that returns an empty value.

How can we show null in an output?

Lets look at the below output. From observation it can be seen that a,b,d retuns an int value, but that is quite straight forward.

Let’s focus on c. When it is printed out the value on the screen is showing nothing, and its data type is str. But why is that, surely it is None or empty as we were expecting?

Well Python will return as a string , unless it is explicitly declared as empty. The next section will take you through that.

a = 1
b = 1
c = ""
d = a-b
print(a)
print(b)
print(c)
print(d)
print(type(a))
print(type(b))
print(type(c))
print(type(d))

Returns:
1
1

0
<class 'int'>
<class 'int'>
<class 'str'>
<class 'int'>

So based on the above, how do I declare a null value in python?

We have modified the code above, and declared c as None, and in this instance the code now recognises the output as empty.

a = 1
b = 1
c = None
d = a-b
print(a)
print(b)
print(c)
print(d)
print(type(a))
print(type(b))
print(type(c))
print(type(d))

Result:
1
1
None
0
<class 'int'>
<class 'int'>
<class 'NoneType'>
<class 'int'>

What are the other scenarios that None will be returned?

Python also returns values on None, even though they have not been explicitly declared.

In the below if you try to print nothing, it will by default return an empty value.

On the other hand using an if statement you can check if a value is a None.

The final example is where you are using a function, to check if a value is in a list. If the value does not appear in the list it returns nothing.

This is especially handy , if you want to completely sure that that the returned values are nothing.

It gives you a level of comfort that that the code will not pass anything to other parts of the programme.

a = print()
print(a)

variable = None

if variable is None:
    print("Correct")
else:
    print("Incorrect")

variable1 = "today"
if variable1 is None:
    print("Correct")
else:
    print("Incorrect")


def returnnone():
    list = [1,2,3,4,5]
    for i in list:
        if i == 6:
            print("Six found")
        else:
            print(None)
            
returnnone()

Result:
None
Correct
Incorrect
None
None
None
None
None

Click here to get more information on other relevant data analytics posts.

how to compare two lists in Python

Estimated reading time: 2 minutes

Often you are going to be asked to compare lists, and you need a quick way to complete.

Here we are going to take through three ways to complete this, if you have more comment below.

Looping to find common values between lists

A simple loop can help you find data that is common to two lists:

# compare for similar values
list1 = ["1", "2", "3", "4"]
list2 = ["1", "2", "3", "4", "5"]

for i in list1:
    for j in list2:
        if i in j:
            print(i)

which yields:

1
2
3
4

Compare for an item in one list and not in the other

There maybe times you wish to find only the values that are in one list and not the other.

Below we use a one line piece of code using list comprehension, which does the same as a loop:

list1 = ["1", "2", "3", "4"]
list2 = ["1", "2", "3", "4", "5"]
for item in [x for x in list2 if x not in list1]:
    print(item)

which gives the result of:

5

comparing lists using the set method

The third way uses python sets, which essentially finds the intersection between two lists, like a Venn diagram.

Here we use set to find what values are not common to each list by using subtraction:

list1 = ["1", "2", "3", "4"]
list2 = ["1", "2", "3", "4", "5"]
a = set(list1)
b = set(list2)
c = b-a
print(c)

which gives you:

{'5'}

Alternatively you could find what is common to both:

list1 = ["1", "2", "3", "4"]
list2 = ["1", "2", "3", "4", "5"]
a = set(list1)
b = set(list2)
c = a.intersection(b)
print(c)

and your result will be:

{'1', '3', '4', '2'}

Remember that using sets will return them unordered, if you want them ordered then apply the following to the above code:

a = set(list1)
b = set(list2)
c = a.intersection(b)
d=sorted(c)
print(type(d))
print(d)

and the output will be:

<class 'list'>
['1', '2', '3', '4']

One thing to note that the sorted method above returns the set as a list not as a set.

There are plenty of resources online where you can learn about sets.

Python Tutorial: How to sort lists

Estimated reading time: 2 minutes

Following on from our post on how to use Python lists have you ever wondered how to sort lists for your Python project?

Our latest video on lists will go through some of the techniques available so that you can get an idea of how to structure your data and sort.

Getting to understand how to implement

In this latest video we will look at:

  • sort() method
  • sorted() function
  • sorting a list through a function

 

Adding in those extra bits to help make the process smoother

Have you thought about sorting ascending/descending?

  • There is also a discussion on this topic as well, and while an index is available for the list, which you may feel does not merit sorting, there could be other logical reasons to implement sorting.
  • Leaving out the reverse = True/False in the sorted method can have an impact, though if you require it left out of the list you have created, automatic ascending will be the default.

On this channel, we have discussed a number of different ways to manage your data. In thinking about sorting a list, why would you want to do this?

Some common reasons are:

  • To visually see if there are duplicates, either on the screen or printed out.
  • If other objects are dependant on the list, say a combo box, then having duplicates visible can help to reduce the size of their contents.
  • Iteration – If you are looking to iterate over a list, it will be quicker if it is sorted.

If you want to learn about lists, using them, and how how they can be iterated over, why not visit Data Analytics Ireland YouTube channel, there are lots of videos there that will help explain the concepts discussed here further.

To get some more links on this topic click here python sort method, it is a blog posting from our website that has some useful links and explanations for you.