ValueError: Columns must be same length as key

Estimated reading time: 3 minutes

Are you looking to learn python , and in the process coming across this error and trying to understand why it occurs?

In essence, this usually occurs when you have more than one data frames and in the process of writing your program you are trying to use the data frames and their data, but there is a mismatch in the no of items in each that the program cannot process until it is fixed.

A common scenario where this may happen is when you are joining data frames or splitting out data, these will be demonstrated below.

Scenario 1 – Joining data frames

Where we have df1[[‘a’]] = df2 we are assigning the values on the left side of the equals sign to what is on the right.

When we look at the right-hand side it has three columns, the left-hand side has one.

As a result the error “ValueError: Columns must be same length as key” will appear, as per the below.

import pandas as pd

list1 = [1,2,3]
list2 = [[4,5,6],[7,8,9]]

df1 = pd.DataFrame(list1,columns=['column1'])
df2 = pd.DataFrame(list2,columns=['column2','column3','column4'])

df1[['a']] = df2

The above code throws the below error:

The objective here is to have all the columns from the right-hand side, beside the columns from the left-hand side as follows:

What we have done is make both sides equal regards the no of columns to be shown from df2
Essentially we are taking the column from DF1, and then bringing in the three columns from DF2.
The columna, columnb, columnc below correspond to the three columns in DF2, and will store the data from them.

The fix for this issue is : df1[[‘columna’,’columnb’,’columnc’]] = df2

print (df1)

Scenario 2 – Splitting out data

There may be an occasion where you have a python list, and you need to split out the values of that list into separate columns.

new_list1 = ['1 2 3']
df1_newlist = pd.DataFrame(new_list1,columns=['column1'])

In the above, we have created a list, with three values that are part of one string. Here what we are looking to do is create a new column with the below code:

df1_newlist[["column1"]] = df1_newlist["column1"].str.split(" ", expand=True) #Splitting based on the space between the values.

print(df1_newlist)

When we run the above it throws the following valueerror:

The reason it throws the error is that the logic has three values to be split out into three columns, but we have only defined one column in df1_newlist[[“column1”]]

To fix this, we run the below code:

df1_newlist[["column1","column2","column3"]] = df1_newlist["column1"].str.split(" ", expand=True) #Splitting based on the space between the values.

print(df1_newlist)

This returns the following output, with the problem fixed!

Create a HTML Table From Python Using Javascript

Estimated reading time: 5 minutes

So you are using Python and you have data stored in a data frame? You would like to present that on a webpage HTML table, but unsure how to achieve this. Well, read on to see two different methods that will help you.

Pass the data to Javascript which then passes to the HTML the data needed to create the table

In both methods, we are using Python Flask which has an app.py file and HTML files created to present the outcomes.

Step 1 – Read the data and create the data frame

For those familiar with Python Flask, we create the imports that allow us to create the webpage interface to present the data.

Then we create a list called data, which stored the information we need. After this, we create the data frame “df”.

Finally, we create a JSON file,and do some data cleansing.

from flask import Flask, render_template, json
import pandas as pd


app = Flask(__name__)

data = [['Joe','Dublin','100'],['Francois','Paris','100'],['Michael','Liverpool','100']]
df = pd.DataFrame(data, columns = ['Name', 'City','Age'])


json_output = df.to_json()
json_output = json.loads(json_output.replace("\'", '"')) # Removes back slash from JSON generated

Step 2 – Create the output HTML files

Method 1 – Here all this is doing is creating the function to create the webpage “index.html”. Note that name_json=json_output captures the data from step one, and this is what is passed over to the HTML page as a JSON.

In method 2 – We are using to_html which Renders a DataFrame as an HTML table, pure and simple.

As can be seen, it stores the data onto an HTML page that is stored in the templates folder.

#Method 1
@app.route('/')
def index():

    return render_template('index.html', name_json=json_output)

#This is  the start of method 2#
html = df.to_html()
#write html to file
im2_file = open("templates/index_method2.html", "w")
im2_file.write(html)
im2_file.close()

@app.route('/index_method2')
def index_method2():
    return render_template('index_method2.html')


if __name__ == "__main__":
    app.run(debug=True)

Step 3 – Create the HTML tables through javascript

So steps 1 and 2 were getting the data ready so it can be viewed on the web pages, so how are they built?

So let’s walk down through the code, note that a good bit of this is the HTML that is used to present the data on the page.

The first thing to notice is the <style></style> tags. this is for method 1 and applies the boxes around the output.

<pre id=”json”></pre> – This line shows the JSON data as follows:

JSON Output

In the below few lines this is the HTML that creates the table to store the data from method 1:

<table id="json_table"> ===> Method 1 table
    <tr>
        <td> Name</td>
        <td> City</td>
        <td> Age</td>
  </tr>
</table>

The next section has the Javascript that will populate both for method 1 and method 2, I will go through it now:

So the first line is creating a variable that references the method 1 table, and where the JSON data will be loaded to.

The second line is converting the JSON into a format that can be read and shown as the “JSON” screenshot above.

In Line 3 & 4, all we are doing here is creating a variable to store the output of the loop that follows in the subsequent lines.

The final set of lines in the script except for the very last line ( which relates to Method 2) catch the data that is captured as part of the loop and feed it to the table that appears on index.html, as follows:

HTML table created from a python data frame using Javascript.

The final line:

window.open(‘http://127.0.0.1:5000/index_method2’, ‘_blank’);

In the Javascript section, this relates to Method 2 and takes the data directly from the app.py file and outputs it to http://127.0.0.1:5000/index_method2.html

This is a very quick and easy way to create the HTML table, and as can be seen, involves less coding than method 1.

<script>
    var selectvalue = document.getElementById("json_table"), test={{ name_json | tojson }}; ===> First line
    document.getElementById("json").textContent = JSON.stringify(test, undefined, 2); ===> Second line

    const keys = Object.keys(test); ===> Line 3
    for (let i = -1; i < keys.length-1; i) ===> Line 4

    {
        const key = keys[i++];

        console.log(key, test[key]);

        a = JSON.stringify(test["Name"][i])
        b = JSON.stringify(test["City"][i])
        c = JSON.stringify(test["Age"][i])
        const tbl = document.getElementById("json_table");
        const row = tbl.insertRow();
        const cell1 = row.insertCell();
        const cell2 = row.insertCell();
        const cell3 = row.insertCell();
        cell1.innerHTML = a;
        cell2.innerHTML = b;
        cell3.innerHTML = c;

    }

    window.open('http://127.0.0.1:5000/index_method2', '_blank');

  </script>

The Full HTML code

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">

  <title>Document</title>
</head>
<style>
table, td {
  border: 1px solid;
}
</style>
<body>
<pre id="json"></pre>
<div>Method 1 pass the data via JSON to Javascript </div>
<table id="json_table">
    <tr>
        <td> Name</td>
        <td> City</td>
        <td> Age</td>
  </tr>
</table>

  <script>
    var selectvalue = document.getElementById("json_table"), test={{ name_json | tojson }};
    document.getElementById("json").textContent = JSON.stringify(test, undefined, 2);

    const keys = Object.keys(test);
    for (let i = -1; i < keys.length-1; i)

    {
        const key = keys[i++];

        console.log(key, test[key]);

        a = JSON.stringify(test["Name"][i])
        b = JSON.stringify(test["City"][i])
        c = JSON.stringify(test["Age"][i])
        const tbl = document.getElementById("json_table");
        const row = tbl.insertRow();
        const cell1 = row.insertCell();
        const cell2 = row.insertCell();
        const cell3 = row.insertCell();
        cell1.innerHTML = a;
        cell2.innerHTML = b;
        cell3.innerHTML = c;

    }

    window.open('http://127.0.0.1:5000/index_method2', '_blank');

  </script>


</body>
</html>

#how to create a html table from a python dataframe using javascript

The Final output

Method 1
Method 2

How To Check For Unwanted Characters Using Loops With Python

Estimated reading time: 3 minutes

On this website, we have posted about how to remove unwanted characters from your data and How to remove characters from an imported CSV file both will show you different approaches.

In this blog posting, we are going to approach the process by using loops within a function. Essentially we are going to pass a list and then we are going to loop through the strings to check the data against it.

Step 1 – Import the data and create a data frame

The first bit of work we need to complete is to load the data. Here we create a dictionary with their respective key-value pairs.

In order to prepare the data to be processed through the function in step 2, we then load it into a data frame.

import pandas as pd
#Create a dataset locally
data = {'Number':  ["00&00$000", '111$11111','2222€2222','3333333*33','444444444£','555/55555','666666@666666'],
        'Error' : ['0','0','0','0','0','0','0']}

#Create a dataframe and tell it where to get its values from
df = pd.DataFrame (data, columns = ['Number','Error'])

Step 2 – Create the function that checks for invalid data

This is the main piece of logic that gives the output. As you can see there is a list “L” that is fed to the function run.

One thing to note is that *l is passed to the function, as there is more than one value in the list, otherwise the program would not execute properly.

To start off we create a data frame, which extracts using a regular expression the characters we don’t want to have.

Next, we then need to drop a column that is generated with NAN values, as these are not required.

Then we updated the original data fame with the values that we found.

Just in case if there are any NAN values in this updated column “Error”, we remove them on the next line.

The main next is the loop that creates a new column called “Fix”. This holds the values that will be populated into the fix after the data we don’t want is removed and is data cleansed.

The data we do not need is in str.replace.

#Function to loop through the dataset and see if any of the list values below are found and replace them
def run(*l):
    #This line extracts all the special characters into a new column
    #Using regular expressions it finds values that should not appear
    df2 = df['Number'].str.extract('(\D+|^(0)$)') # New dataframe to get extraction we need
    print(df2)
    df2 = df2.drop(1, axis=1) # Drops the column with NAN in it, not required

    df['Error'] = df2[0] # Updates original dataframe with values that need to be removed.
    #This line removes anything with a null value
    df.dropna(subset=['Error'], inplace=True)
    #This line reads in the list and assigns it a value i, to each element of the list.
    #Each i value assigned also assigns an index number to the list value.
    #The index value is then used to check whether the value associated with it is in the df['Number'] column 
    #and then replaces if found
    for i in l:
        df['Fix']= df['Number'].str.replace(i[0],"").str.replace(i[1],"").str.replace(i[2],"").str.replace(i[3],"") \
        .str.replace(i[4],"").str.replace(i[5],"").str.replace(i[6],"")
        print("Error list to check against")
        print(i[0])
        print(i[1])
        print(i[2])
        print(i[3])
        print(i[4])
        print(i[5])
        print(i[6])
    print(df)

#This is the list of errors you want to check for
l = ['$','*','€','&','£','/','@']

Step 3 – Run the program

To run this program, we just execute the below code. All this does is read in the list “L” to the function “run” and then the output in step 4 is produced

run(l)

Step 4 – Output

Error list to check against
$
*
€
&
£
/
@
          Number Error           Fix
0      00&00$000     &       0000000
1      111$11111     $      11111111
2      2222€2222     €      22222222
3     3333333*33     *     333333333
4     444444444£     £     444444444
5      555/55555     /      55555555
6  666666@666666     @  666666666666

Process finished with exit code 0

How to Create Multiple XML Files From Excel With Python

So you have an Excel file that has hundreds or maybe thousands of records, but you want to convert it to an XML file, and have each row as a separate file?

This can be achieved very easily using Python, in three easy steps.

First of all, what is XML?

XML stands for “extensible markup language”, and is used extensively with websites, Webservices and where companies want to transfer data between each other in a predefined format that follows a recognised standard.

To understand more about what is xml, follow the link. One thing to note is that it may reference the financial services industry, but XML is used widely across multiple industries too, as we are now digitising a lot of our processes, the transfer of data needs to follow a standard that everyone can work with seamlessly.

Step 1 – Let’s look at our raw data

Below is an excel file with three columns and eight rows. The objective is to create eight separate XML files for each row.

raw data to populate multiple excel files

Step 2 – Import the data and create a master XML file

Below is the logic I used in How to Create an XML File from Excel using Python, I have attached comments where necessary to explain what each line is doing.

The purpose at this point is to create one file with all the data from excel called output.xml. We will then use this to create the separate files we need and then delete it.

The next steps will take you through that.

import pandas as pd
from lxml import etree as et
import os


# STEP 1 - This step creates a master XML file

raw_data = pd.read_excel(r'\Users\haugh\OneDrive\dataanalyticsireland\YOUTUBE\how_to_read_each_excel_row_into_a_separate_XML_file\excel_raw_data_city.xlsx')

root = et.Element('root')

for row in raw_data.iterrows(): #==> This is a loop that takes runs through each record and populates for each tag.
    root_tags = et.SubElement(root, 'ExportData') #=== > Root name
# These are the tag names for each row (SECTION 1)
    Column_heading_1 = et.SubElement(root_tags, 'City')
    Column_heading_2 = et.SubElement(root_tags, 'Area')
    Column_heading_3 = et.SubElement(root_tags, 'Population')


###These are the values that will be populated for each row above
# The values inside the [] are the raw file column headings.(SECTION 2)
    Column_heading_1.text = str(row[1]['City'])
    Column_heading_2.text = str(row[1]['Area'])
    Column_heading_3.text = str(row[1]['Population'])

# This Section outputs the data to an xml file
# Unless you tell it otherwise it saves it to the same folder as the script.
tree = et.ElementTree(root) #==> The variable tree is to hold all the values of "root"
et.indent(tree, space="\t", level=0) #===> This just formats in a way that the XML is readable
tree.write('output.xml', encoding="utf-8") #==> The data is saved to an XML file

Step 3 – Loop through each row and create the separate files

In the below code, it opens the output.xml file and using the loop looks for all the data between the tags “ExportData”.

Then it copies them and then creates a new file.

Once one is done, it moves to the next set of tags and repeats the process, till it reaches the end of the output.xml file.

master_file = et.iterparse('output.xml', events=('end', ))
index = 0
for i, j in master_file:
    if j.tag == 'ExportData':
        index += 1
        filename = format(str(index) + ".xml")
        with open(filename, 'wb') as f:
            f.write(et.tostring(j))

Step 4 – Delete the output.xml file – purely optional

The objective here was to create separate XML files, based on the input of the original Excel file.

We have also created an additional file called output.xml to get us to the end goal.

If we have no need for this file then it can be removed with the below code, be sure that it is at the very end of your logic.

The import statement for os.unlink is in step two.

os.unlink(r'\Users\haugh\OneDrive\dataanalyticsireland\YOUTUBE\how_to_read_each_excel_row_into_a_separate_XML_file\output.xml')

How To Fix TypeError: unhashable type ‘slice’

Estimated reading time: 2 minutes

So you have a Python dictionary, and you want to retrieve data from it and print it to a screen. There are some characteristics of Python that first of all should be understood:

  1. They are mutable
  2. They also have the ability to grow and shrink as required.
  3. Data is accessed within the dictionary via keys.

The last point is very important as dictionaries do not have an index value, and this is why you get the TypeError you are here to solve for.

So let us recreate the problem

In the below code we have a dictionary called “userdata”, with its key-value pairs.

The loop is trying to retrieve the index value 1 for all the values in dai_data.

As can be seen, dai_data is trying to retrieve the last three index values within the dictionary.

As noted above the only way to access dictionary values is through their key values.

userdata = {
	  "name": "Data Analytics Ireland",
	  "Country": "Ireland",
	  "City": "Dublin",
	  "Age": "1000 years!",
}

dai_data = userdata[:3]

for i in dai_data:
	     print(i[1])

Output:
Traceback (most recent call last):
  File "", line 8, in <module>
    dai_data = userdata[:3]
TypeError: unhashable type: 'slice'

So how do we fix this problem?

First of all, values are accessed through the key within the dictionary

In the below dictionary the key values are: Name, Country, City, Age

userdata = {
	  "Name": "Data Analytics Ireland",
	  "Country": "Ireland",
	  "City": "Dublin",
	  "Age": "1000 years!",
}

print(userdata["Name"])
print(userdata["Country"])
print(userdata["City"])
print(userdata["Age"])

Output:
Data Analytics Ireland
Ireland
Dublin
1000 years!

As a result, now we are able to access the values associated with each key.

Did you know you could add a list to one of your key-value pairs?

In the above example, we focused on a single value, but we could also make a key equal to a list.

userdata = {
	  "Name": ["Joe","Jim"],
	  "Country": "Ireland",
	  "City": ["Dublin","Cork","Limerick"],
	  "Age": "1000 years!"
}

print(userdata["Name"])
print(userdata["Country"])
print(userdata["City"])
print(userdata["Age"])

Output:
['Joe', 'Jim']
Ireland
['Dublin', 'Cork', 'Limerick']
1000 years!

TypeError: Array() Argument 1 Must Be A Unicode Character, Not List

In a recent post, we discussed arrays and what they mean and how they differ from lists. When you have a list in an array, one of the things that you need to define is what data type it is. If you don’t then the array will throw the error that you are on this page to resolve.

On the Python.org website, below are the list of values that can be populated to indicate what the data type of a list is in an array.

If you don’t add in the value you will get the below error:

ValueError: bad typecode (must be b, B, u, h, H, i, I, l, L, q, Q, f or d)

Let’s look at an example

import array as test_array

a = test_array.array([1,2,3])

print(a)

Gives an error of:
TypeError: array() argument 1 must be a unicode character, not list

As can be seen, the above logic tries to print a list within an array. The problem here is that an array can only be of one data type, and it has to be specified on the array creation.

How do we fix this?

It is quite simple! Before the list, we simply specify which type code we would like to apply from the above list. In this instance we are going to assign it as “signed int” that being the value “i” as follows:

import array as test_array

a = test_array.array("i",[1,2,3])

print(a)

Prints the following with no error:
array('i', [1, 2, 3])

Process finished with exit code 0

I can change the value “i” to any of the values I want from the above list, just picked that one to show as an example.

How To Run Python Validation From Javascript

Estimated reading time: 6 minutes

More and more, website developers are looking to blend different programming technologies to allow them to use functionality, that maybe is not available to them in the current programming language they utilise.

In previous posts how to pass python variables to Javascript and how to run python directly from javascript, we touched on how to use Python and Javascript interchangeably to pass data.

Here in this blog post, we are going to look at data validation using Python, with data captured by Javascript. The purpose here is to allow another way to validate data other than relying on javascript.

Python Validation run from Javascript

There are a number of reasons for this:

  1. You may want to use Python to process the data and provide output.
  2. The data could be passed from Python into a database, you need to run some checks before you proceed.
  3. Python is your core language, you only use Javascript to capture the data and pass it from a website.

There are four validation checks

  1. Validation if a username exists.
  2. Validation if a number has been entered on a username field.
  3. Validation of a password entered.
  4. A validation check to make sure that an email address entered is in the correct format.

We have two files with the requisite code in them as follows:

  1. app.py ===> As this application is built in Python Flask, this file holds the logic to open the website files, and it holds the Python validation checks.
  2. index.html ====> This file holds the HTML that creates the pages, labels, and input boxes that appear in the web browser. It also includes the Javascript that will capture the data to be passed to Python for validation.

Code Overview – App.PY

Below, I will look to explain what is going on as follows:

The variable Regex is used in the email validation to check if the email entered is correct.

def inputcheck ===> This is just checking the username passed over and performing some validations on it.

def inputvalidation ====> This also is checking the username but looking at if numbers only are entered, or if it is empty.

def passvalidation ====> In this piece of logic, it checks if the password is empty, less than five characters, is numbers only.

def emailvalidation ====> All this is doing is checking if the data received from the Javascript is in the correct email format. It references the regex variable above, which is used to confirm if the format is correct or otherwise.

Within each of the functions, there are if statements which are the core validation checks used by Python.

Also, all the popup menus use ctypes, which allows you to access the windows libraries that hold message boxes, customise them and call within your program.

import json
import ctypes
import re
from flask import request,redirect,url_for

from flask import Flask, render_template

app = Flask(__name__)

regex = '^[a-z0-9]+[\._]?[a-z0-9]+[@]\w+[.]\w{2,3}$'

@app.route('/')
def index():
    return render_template('index.html')


@app.route('/inputcheck', methods=['GET','POST'])
def inputcheck():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    username_list = ['joe','john']
    if result['username'] in username_list:
        MessageBox = ctypes.windll.user32.MessageBoxW(0, 'Username ' + result['username'] + ' ' + 'exists', "Username check", 0x00001000)
        return render_template('various.html')
    elif result['username'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'You cannot enter an empty value', 'Username check', 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Username ' + result['username'] + ' ' + 'does not exist', 'Username check', 0x00001000)

@app.route('/inputvalidation', methods=['GET','POST'])
def inputvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if result['username'].isdecimal():
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username cannot be numbers',"Number check", 0x00001000)
    elif result['username'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'The username cannot be empty', "Number check",0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username looks ok', "Number check", 0x00001000)
    return render_template('index.html')

@app.route('/passvalidation', methods=['GET','POST'])
def passvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if result['password'].isdecimal():
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your password cannot be numbers',"Password check", 0x00001000)
    elif result['password'] == '':
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'The password cannot be empty', "Password empty check",0x00001000)
    elif len(result['password']) < 5:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your username should be greater than five characters', "Password length check", 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your password looks ok', "Number check", 0x00001000)
    return render_template('index.html')

@app.route('/emailvalidation', methods=['GET','POST'])
def emailvalidation():
    output = request.get_json()
    result = json.loads(output) #this converts the json output to a python dictionary
    if re.search(regex, result['email']):
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your email is in the correct format', "Email check", 0x00001000)
    else:
        MessageBox = ctypes.windll.user32.MessageBoxW(None, 'Your email is invalid, please correct', "Email check", 0x00001000)
    return render_template('index.html')



if __name__ == "__main__":
    app.run(debug=True)


Code Overview – Index.html

The below code is the page code, that allows the capturing of data through a web browser HTML page.

Between the <style></style> tags these are the CSS properties that format the labels and buttons on the page.

Within the <div></div> tags are where we create the labels and buttons to show on the screen. Also within these, are the references to the on click event function that should run once the buttons are clicked.

Within the <script></script> tags, this is where the javascript is written which captures the data entered into the input boxes within the <div> tags.

Also, you will see within this javascript, there are sections called $.ajax, which is where the data captured by the javascript is stored, and then passed onto the Python script file (app.py)

Note that in each ajax , url:”/APAGENAME” is referenced. These are the sections within the app.py that the data is passed to, and then the Python logic kicks in and validates the data.

<html lang="en">

<head>

    <title>Data Analytics Ireland</title></head>
<style>
.button1 {
    position: absolute;
    top: 50%;
    left: 55%;
    width: 900px;
    height: 300px;
    margin-left: -300px;
    margin-top: -80px;
}



.labels {
    position: absolute;
    top: 50%;
    left: 55%;
    width: 900px;
    height: 300px;
    margin-left: -300px;
    margin-top: -150px;
}

</style>

<body>



<div class="labels" id="mem1" >
<label  for="username">username:</label> <input type="text" id="username" name="username">
<label for="password">password:</label><input type="text" id="password" name="password">
    <label for="email">email:</label><input type="text" id="email" name="password">
</div>


<div class="button1" id="mem2" >
<button type="submit" onclick='myfunction();'>Click here to validate your username</button>
<button type="submit" onclick='myfunctionval();'>Click here to check if a number</button>
    <button type="submit" onclick='myfunctionpass();'>Click here to check password</button>
      <button type="submit" onclick='myfunctionemail();'>Click here to check your email</button>
</div>


<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>

<script>
    function myfunction() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/inputcheck",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>
<script>
    function myfunctionval() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/inputvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>

<script>
    function myfunctionpass() {

        const username = document.getElementById("username").value;
        const password = document.getElementById("password").value;


        const dict_values = {username, password} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/passvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>

<script>
    function myfunctionemail() {

        const email = document.getElementById("email").value;


        const dict_values = {email} //Pass the javascript variables to a dictionary.
        const s = JSON.stringify(dict_values); // Stringify converts a JavaScript object or value to JSON.
        console.log(s); // Prints the variables to console window, which are in the JSON format
        //window.alert(s)
        $.ajax({
            url:"/emailvalidation",
            type:"POST",
            contentType: "application/json",
            data: JSON.stringify(s)});
}
</script>




</body>
</html>

How to Pass Python Variables to Javascript

Estimated reading time: 4 minutes

In our recent blog posting How to Pass a Javascript Variable to Python using JSON, we demonstrated how to easily use AJAX to pass whatever data you wanted and then manipulate it with Python.

In this blog posting, we are going to show how to do this the other way around. The scenario is that you have an application and or website that wants to use data generated through Python, but let Javascript then use it within the application.

As Python can be connected to numerous databases and files ( txt, excel) etc, this piece of logic is very useful for the programmer looking to integrate both programming languages.

Let’s start looking at the code, and see how this can be achieved.

Step 1 – What Files are generated?

This program uses Python Flask to create a web page, that has a drop-down menu. The two files used to generate this are as follows:

(A) app.py – This is the python file that creates a website and loads a template HTML file as outlined below.

(B) Index.html – This is the template file that loads into the browser and runs all the javascript. The javascript loaded here also loads the python data passed over from app.py

Step 2 – APP.PY code overview

The Python library that enables webpage creation is called Flask, and as can be seen below it has to be imported.

In addition, we need to also import render_template which tells the program to go to the templates folder and load “Index.HTML”

The variable that is been passed to JavaScript is called name, and these are the values that you will see in the web browser when it is loaded.

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    name = ['Joe','John','Jim','Paul','Niall','Tom']
    return render_template('index.html', name=name)

if __name__ == "__main__":
    app.run(debug=True)

Step 3 – Index.HTML overview

Here is the template HTML file that runs in the browser. You can add CSS etc to this to make it look nicer and more user friendly.

As you can see it has the usual HTML tags appear as part of a website.

Well look at some of the code further:

In this bit <select id =’select’> </select>, this is the dropdown menu that will appear when Index.html is opened. It will store all the values passed from python. Note that its id is “select”, this will be used later on.

The main parts to focus on next is between <script></script>. This is what reads in the python data and populates it to the dropdown menu.

In step 2 we mentioned that there was a variable called “name”, with values to be passed over.

This is acheived on this line:

var select = document.getElementById(“select”), test = {{ name | tojson }};

Notice that name appears here, and this is referencing back to the exact same value that was discussed in step 2.

For the rest of the lines, I have explained with comments what each does.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Pass Python variable to Javascript</title>
</head>
<body>

<select id ='selectvalue'>
</select>
<script>
    //name = ['Joe','John','Jim','Paul','Niall','Tom']

    var selectvalue = document.getElementById("selectvalue"), test = {{ name | tojson }};
    //The increment operator (++) increments (adds one to) its operand and returns a value.
    for(var i = 0; i < test.length; i++) // This line checks for the length of the data you feeding in i.e the no of items
         {
var selection = document.createElement("OPTION"), // This line creates a variable to store the different values fed in from the JSON object "TEST"
txt = document.createTextNode(test[i]); // This just reads each value from the test JSON variable above
selection.appendChild(txt); // This line appends each value as it is read.
selection.setAttribute("value",test[i]); // This line sets each value read in as a value for the drop down
selectvalue.insertBefore(selection,selectvalue.lastChild); //This reads eah value into the dropdown based on the order in the "TEST" above.
 }
</script>
</body>
</html>

Step 4 – What the output looks like!

From step 2, these are values we asked to be used in Javascript to populate a dropdown:

name = [‘Joe’,’John’,’Jim’,’Paul’,’Niall’,’Tom’]

Python variable passed to Javascript

How to pass multiple lists to a function and compare

Estimated reading time: 3 minutes

Have you ever been faced with a situation where you have multiple lists you want to compare, and need a function that will read them in and show you what is different between them all?

In this scenario, this can be handy where you have large data sets and need a quick way to find those differences and fix them where appropriate.

Comparing two lists

Let’s look and see what is going on below.

First of all, we have defined two lists. The only difference between the two is that one has a value of six, the other does not.

Next, we have the function “comparelists”. What this is doing is taking in the two lists as parameters (a,b), and then processing them.

The lists are passed as arguments to the parameters in this line ===> comparelists(list1,list2)

The parameters a are assigned to list1, and b is assigned to list2.

The main bit of the function is the list comprehension, and it is doing the following:

  1. x for x is basically creating a variable x, and starting a loop that goes through all the values of b.
  2. Each iteration of x is stored and compared with a.
  3. “If x not in a” completes the comparison and if true returns the value, otherwise moves to the next value.

As a result of this logic, it can be seen that six is the only value returned, and this is what we are expecting based on a visual inspection.

#Reading in two lists
list1 = [1,2,3,4,5] # parameter a below
list2 = [1,2,3,4,5,6] # parameter b below

def comparelists(a,b):
    z = [x for x in b if x not in a] #list comprehension
    print(z)

comparelists(list1,list2)

Output:

[6]

Comparing more than two lists

In the logic above, we saw that two lists can be compared. But what if you want to compare one list against two other lists?

That is easily done, with some tweaking to the code.

As before the three lists are defined, created as arguments in comparelists(list1,list2,list3), and then passed to the function parameters a,b,c.

The only difference in this logic is that the list comprehension is written a little different as follows:

  1. x for x is basically creating a variable x, and starting a loop that goes through all the values of b. ====> This is the same as above
  2. Each iteration of x is stored and compared with a and c ===> This line is different as comparing to two lists now.
  3. “if x not in a and x not in c” ====> Here we have two comparisons
    • Checking for the value x from b in a
    • Checking for the value x from b in c
  4. The value of six is correct as it is not in either a or c.
  5. Note the if statement is specific as to what to check for with the “and” statement. This can be changed to suit your needs.
list1 = [1,2,3,4,5] # parameter a below
list2 = [1,2,3,4,5,6] # parameter b below
list3 = [1,2,3,7,8,9] # parameter c below

def comparelists(a,b,c):
    z = [x for x in b
         if x not in a and x not in c] #list comprehension
    print(z)

comparelists(list1,list2,list3)

Output:
[6]

How to run Python directly from Javascript

Estimated reading time: 3 minutes

Are you developing a website and looking to execute some Python script directly from the website?

Here we take you through how to achieve this using Python Flask, the scenarios we demonstrate are as follows:

  1. Javascript on load event – execute Python script
  2. Javascript on click event – execute a Python script

In both scenarios above you will be presented with pop-up boxes like below, but the python code can be changed to whatever you like.

Page load message box
On button press, calls the python script to load a message box

The code you will need to run this is split into two. (A) The Index.HTML logic and (B) the app.py logic

The code for the INDEX.HTML page

This is the page that the user is presented with. As you can see the logic has two pieces of javascript that when run, go over to the python code and run some logic that it wishes the webpage to run.

In this instance, the app.py python script holds the commands that need to be executed.

Note the page load event goes to the route in app.py, namely ” @app.route(‘/’)”

Whereas the button click event is pointed directly at “@app.route(‘/test’ )” – essentially this will not run until the javascript asks it to, on the button click event.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Sample Home Page</title>
</head>
<body>
<! -- Calls a function on the page load. -- >
<element onload="myfunction_onload">
<button type="submit" onclick='myfunction_clickevent()'>Run my Python!</button>
<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
<! -- This script runs the python script on the page load -->
<script>
    function myfunction_onload(){
        $.ajax({
            url: "app.py",
             context: document.body
            })
        }
    </script>
<! -- This script runs the python script when the button is clicked -->
<script>
    function myfunction_clickevent(){
        $.ajax({
            url:"/test",
            context: document.body});}
</script>


</body>
</html>
How to run Python directly from Javascript

The code for the APP.PY Python script

Here we are linking to the index.html page and taking commands from it and executing those requests.

In this scenario, the message boxes are rendered by using the win32api package.

The app is using the flask package to run the website.

The powerful thing here is you can start customising this logic to do what you like, examples include:

(A) Return graphs to the webpage.

(B) Process data received and return statistics on the data.

(C) Validate data received in Python and return a response. An example here could be a user logging into a database.

import win32api
from flask import Flask, render_template

app = Flask(__name__)


#Using the below, the popup message appears on the page load of index.html
#0x00001000 - This makes the popup appear over the browser window
@app.route('/')
def index():
    win32api.MessageBox(0, 'You have just run a python script on the page load!', 'Running a Python Script via Javascript', 0x00001000)
    return render_template('index.html')

#Using the below, the popup message appears when the button is clicked on the webpage.
#0x00001000 - This makes the popup appear over the browser window
@app.route('/test')
def test():
    win32api.MessageBox(0, 'You have just run a python script on the button press!', 'Running a Python Script via Javascript', 0x00001000)
    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)