Sunday, March 7, 2021
Latest:

Data Analytics Ireland

Data Analytics and Video Tutorials

Python

This programming language helps to analyse data and show the output in a way that is easily used.

Python type error 

TypeError: cannot unpack non-iterable int object

admin 113 Views 0 Comments , , , , ,

Often when working on a data analytics project it requires you to split data out into its constituent parts.

There are a number of reasons for this, it can be confusing when you get errors as with the title of this post.

Before we explain this error and what it means, lets us first explain unpacking

Unpacking basically means splitting something up into a number of its parts that make it up.

To demonstrate if you take a,b,c = 123, and look to unpack it, it throws out the error, but why?

Well pure and simple, we have three values on the left “a,b,c”, looking for three values on the right.

a,b,c = 123
print(a)

Output:
 a,b,c = 123
TypeError: cannot unpack non-iterable int object

If you would like to fix this problem change the right hand side to have three values.

a,b,c = 1,2,3
print(a)
print(b)
print(c)
print(a,b,c)

Output:
1
2
3
1 2 3

Process finished with exit code 0

In essence, what is going on is that an evaluation checking that both sides have the same amount of values.

It is important to remember, the code above we used to show the error is an integer, which cannot be unpacked.

So if you take 123 for an example, which we used here it cannot be split into say 100 and 10 and 13.

In this case, even though when they are added up to 123, integers cannot be unpacked.

For this reason in the code for our solution, the difference is that the values used are tuples as follows:

a,b,c = 1,2,3
print(a)
print(b)
print(c)
print(a,b,c)
print(type((a,b,c)))

Or 
a,b,c = (1,2,3)
print(a)
print(b)
print(c)
print(a,b,c)
print(type((a,b,c)))

yield the same result:

1
2
3
1 2 3
<class 'tuple'>

Process finished with exit code 0

So in summary:

When unpacking there are a number of things to remember:

  • Integers on their own cannot be unpacked.
  • You need to make sure that if you have a number of variables, that you have the same number of integers if they the values.
    • This will make it a tuple and unpacking can then happen.

Python type error 

TypeError object of type ‘int’ has no len()

admin 138 Views 0 Comments , , , , , , , , , ,

I have seen this data type error come up numerous times while working on my data analytics projects, and recently decided to investigate further. On initial inspection, it can seem a bit of a funny one, but in actual fact, it is quite straight forward.

Lets break it down and see what is going on

So in the below code, there are a number of things:

On line 1 we have a variable that is an integer. If we think about this logically, something that is a single numeric number cannot have a length.

An integer by design is purely to count up a number of apples or no of people, it cannot be viewed as having a length as it is descriptive of the number of occurrences of an object.

data = 100
print(type(data))
print(len(data))

Output Error:
<class 'int'>
Traceback (most recent call last):
  File "object of type int.py", line 3, in <module>
    print(len(data))
TypeError: object of type 'int' has no len()

So for it to in anyway allow a length to be calculated, the object needs to be one of the following data types:

  • List
  • String
  • Tuple
  • Dictionary

Opposite to an integer, these are datatypes that have values that would be more appropriate to having values that a length can be calculated on.

data = "100"
print(type(data))
print("Length of string is: ", len(data))

data = [100,200,300]
print(type(data))
print("Length of list is: ", len(data))

data = (100,200,300)
print(type(data))
print("Length of tuple is: ", len(data))

data = {"Age": 1, "Name": 2}
print(type(data))
print("Length of dictionary is: ", len(data))

And the output is:
<class 'str'>
Length of string is:  3
<class 'list'>
Length of list is:  3
<class 'tuple'>
Length of tuple is:  3
<class 'dict'>
Length of dictionary is:  2

In summary, to understand this error and fix it:

An integer describes the number of things that exist for an object, they are actually not the actual object in existence.

Anything that can have a length method applied to it actually exists and can be counted. In the above four examples, they are actually values that you could describe as existing as you can count each one of them.

The explanation here hopefully clears up the matter, if you have any questions leave a comment and I will answer for you!

Python Python working with excel 

how to copy/paste special a range of cells with xlwings

admin 163 Views 4 Comments , , , ,

Are you using Microsoft Excel in conjunction with Python for your data analytics projects, but have a need to automate certain tasks?

In this blog post we will take you through how to remove formulas in a cell , and replace them with their returned values.

This is achieved through using xlwings, a very powerful library that can be used with Python.

So what we want to do is remove the formulas in an excel sheet, normally this is achieved through “copy and paste special values” in excel.

Below is a screenshot of the before:

In order to remove the formulas we use the following code:

This code basically loads the file( input) and looks for the range F2:F5.

Then using the the xlwings functionality, it makes the old file range values equal to the new range values.

The difference is that it looks at what the vlookup returned value to the cell and not the formula.

from openpyxl import load_workbook
import xlwings as xlfile

filepath_input = r'your file path here'
filepath_output = r'your file path here'

input_workbook = load_workbook(filepath_input)
output_workbook = load_workbook(filepath_output)

ws = input_workbook['Sheet1']

### Removing formulas in the spreadsheet

oldlist = xlfile.Book(filepath_input)
newlist = xlfile.Book(filepath_output)

my_values = oldlist.sheets['Sheet1'].range('F2:F5').options(ndim=2).value

my_values1 = newlist.sheets['Sheet1'].range('F2:F5').options(ndim=2).value

newlist.sheets['Sheet1'].range('F2:F5').value = my_values1

The output is a new file , with the formulas removed!

And there you go, there are other options though.

Theoretically you don’t have to create a new sheet like I did above, that was done to show the before and after, otherwise the input file is overwritten, and if that is what you need then your problem is solved!

In rolling out this solution, there are other options out there as well, I found this the simplest to implement.

Openpyxl can be used and it was the most common suggestion , but I found its implementation not as straight forward.

Python python dictionaries 

how do I merge two dictionaries in Python?

admin 125 Views 0 Comments , , , , ,

Python dictionaries which are used in data analytics frequently and by their nature are enclosed in {} and have key:value pairs, so the data in them can be retrieved easily.

There maybe a scenario where you need to merge two dictionaries, but how would you acheive this?

The good thing is that Python dictionaries are unordered and mutable, meaning that what makes them up can be changed.

Lets start off by creating two dictionaries

dict1 = {"A":"1", "B":"2", "C":"3"}
dict2 = {"D":"4", "E":"5", "F":"6"}
print("dictionary 1 is:", dict1)
print("dictionary 2 is:", dict2)
print(type(dict1))
print(type(dict2))

Its output is as follows:
dictionary 1 is: {'A': '1', 'B': '2', 'C': '3'}
dictionary 2 is: {'D': '4', 'E': '5', 'F': '6'}
<class 'dict'>
<class 'dict'>

So the objective is to get these two dictionaries into one, how is this achieved?

Approach 1 – Use PEP 448

This approach uses PEP 448 which allows * iterable unpacking operator and ** dictionary unpacking operators to be implemented.

As can be seen below , it is a quick and efficient way to quickly merge, without impacting the two dictionaries structure.

dict3 = {**dict1, **dict2}
print(dict3)
print(type(3))

With output:
{'A': '1', 'B': '2', 'C': '3', 'D': '4', 'E': '5', 'F': '6'}
<class 'int'>

Approach 2 – Update when some values not required.

You maybe faced with a situation where you only want certain values from the second dictionary.

In the below there are common keys to both dictionares, namely “A” and “B”.

What the update is doing is it keeps all the values of dictionary 1, and adds in any key value pair that is not A or B.

This scenario could be encountered where dict1 is the master dictionary and always correct, and just needs new values added that do not exist already.

dict1 = {"A":"1", "B":"2", "C":"3"}
dict2 = {"A":"2", "E":"5", "B":"6"}
dict2.update((dict1))
print(dict2)
print(type(dict2))

Resulting in:
{'A': '1', 'E': '5', 'B': '2', 'C': '3'}
<class 'dict'>

Approach 3 – Iterating over the dictionaries

In this scenario, there are a few things going on, that should be explained.

dict1.copy ===> This is done so that you have the original, as it maybe updated if there where duplicate keys.

The loop then just goes through dict2 key value pairs, and adds the key value pairs to dict3, which was originally dict1.

dict1 = {"A":"1", "B":"2", "C":"3"}
dict2 = {"D":"4", "E":"5", "F":"6"}

dict3 = dict1.copy()

for key,value in dict2.items():
    dict3[key] = value
print(dict3)

Which gives you:
{'A': '1', 'B': '2', 'C': '3', 'D': '4', 'E': '5', 'F': '6'}
<class 'dict'>

Summing it all up

In conclusion, depending on what way you would like to approach, we have outlined options.

Probably the most important thing that came out of this, is that dictionaries can be changed, as a result when applying some of the

techniques above, before proceeding be sure to check if you want to keep the original values.

class CRUD Python SQLite 

create read update delete using Tkinter

admin 272 Views 0 Comments , , ,

Estimated reading time: 3 minutes

CRUD ( create, read, update, delete) is a programming language acronym notably for how to manage updates to a database.

It is commonly used when talking about storing data on a database and follows the following rules:

  • The ability to create or add new records.
  • Be able to read and or retrieve the records.
  • If an update is needed, then allow those updates to be posted to the database successfully.
  • To ensure records are maintained correctly and deleted where a delete request is requested.

Generally speaking, with crud operations, it is related to persistent storage principles, more information can be found here basic functions of persistent storage (datacadamia.com)

Given these points when using graphical user interfaces, and trying to introduce crud functionality to your database applications, together with the design of the application, should yield:

  • Good database design.
  • Reducing complexity.
  • Eliminate duplication.
  • Have consistency.

We have spoken about the use of CRUD, but what are its benefits?

Security roles can be enabled

In addition, CRUD brings they bring structure to what actually can happen on a server, in essence, the ability to apply updates is managed tightly.

Helps put structure around what an application can do

Equally important, in building an application for use by users, knowing what they will do can prompt the designers to ask questions about what the user will actually do.

For example probably when Twitter was been designed, the things that probably where thought of :

Create a tweet – Functionality for the user to create and post a tweet, that gets saved to the database.

Read a tweet – Load all tweets from the database to the users interface.

Update – Allow a user to update their account profile or tweet timeline.

Delete – A user can delete their own tweets, their profile and or account.

Data flows between servers can be managed easier

Similarly, with the modern use of technology and processes moving online, data flows around between lots of people and organizations.

With this in mind, the need to send and receive data has to be managed more efficiently and securely.

As a result, data coming into the server can be controlled as to where it is received, and updated to. This can be accomplished by security roles discussed above.

Using classes with CRUD

The use of Python classes is commonly seen in many applications, for this reason we have used them again below.

Uniquely classes will only help to enhance your computer programme and organization of code, as a result of removing duplication and simplifying the code.

Classes are a very useful way to manage the structure of your code, thus keeping everything centralized.

Sooner or later if this was not implemented the project would become too difficult to manage, and maintenance and updates would become difficult to manage.

In the below video we take you through the steps involved in applying this methodology, using an SQLite database.

We use Python Classes to manage the different requests by the user, the details can be found here

  • Main Tkinter window creation
  • Exporting data to Excel
  • Updating the records
  • Filter the records based on user choice
  • Deletion of a record.

When applying these updates, we have written the code that will apply the following SQL updates:

  • Select
  • Update
  • Insert
  • Delete

As can be seen these four SQL commands are the commonly used across any application to perform these requests in a CRUD application.

It is also important that your data types are synced between your Tkinter application and your SQL logic.

On our YouTube channel you can subscribe to find out more information about SQLite, SQL, Tkinter and many more tips and tricks!

Python Python Functions tkinter 

Tkinter GUI tutorial python – how to clean excel data

admin 367 Views 0 Comments , , , , , ,

Estimated reading time: 2 minutes

Tkinter is an application within Python that allows users to create GUI or graphical user interfaces to manage data in a more user-friendly way.

We are building our data analytics capability here, and looking to provide the user with the functionality they use in their work or college projects.

We have tested this code over 100,000 records sitting on the Microsoft OneDrive network so in a way, for this reason, its speeds were quite good.

As a result over five tests, they all were under 100s from start to finish.

data cleansing data cleansing fixed

In this Tkinter GUI tutorial python, you will be shown how to find the data errors, clean them and then export the final result to excel.

We will take you through the following:

  • Creation of the Tkinter interface.
  • Methods/ functions to find errors.
  • Methods/functions to clean the data.
  • Exporting the clean data to an excel file.

 

To sum up:

The video walks through the creation of a Tkinter window using a canvas and a frame to store the data frame.

Then it looks at importing the data through pd.read_excel, to load the data into a pandas data frame.

Next, there is a function and or method that will extract the errors through str.extract , which is loaded into separate columns

Finally, I have exported the clean dataset using rawdata.to_excel , and saved the file as a separate new spreadsheet.

machine learning Python 

planning your machine learning model

admin 402 Views 0 Comments , ,

Estimated reading time: 1 minute

Planning your machine learning model is one of the most important steps you will take in order to achieve the best results you are looking for.

In looking at how to plan a machine learning project, this video takes you through 3 steps:

a. Researching

b. Building your model

c. Testing your model

❤Subscribe for more free YouTube tips: Subscribe to Data analytics Ireland

❤Share this video with a YouTuber friend: Planning your machine learning model

 

Python 

python constructor self and __init__explained

admin 376 Views 0 Comments , , , ,

Here in python constructor self and __init__ explained, we will take you through an explanation of self and __init__, how you can use, and the concepts behind them.

We show how to create an object, initiate a class and pass the parameters to the __init__ constructor.

Also we take you through the concept of self and how it can be used to process an object’s parameters and return values where necessary.

❤Subscribe for more free YouTube tips: Subscribe to Data analytics Ireland

❤Share this video with a YouTuber friend: Python constructors explained

 

Python tkinter 

how to create charts in Tkinter

admin 397 Views 0 Comments , ,

Estimated reading time: 1 minute

In how to create charts in Tkinter, it is one of many libraries that can be used to create different chart types and graphical user interfaces

With python programming, as a result of its ease of use and ability to quickly roll out GUI applications, it makes it one of the most popular in use today.

Because the application allows the creation of interfaces that users will be familiar with, its roll out across an organization will make adoption a lot easier.

In fact, the applications created can also have the ability to create an executable file that can be placed on users’ desktops.

In the below video, after importing Tkinter we use many of its built-in functions to create:

  • Pie chart, a bar chart, and a line chart.
  • We also use functions and classes to help manage the creation of the charts.
  • It is recommended you follow the video to the end as it will give you a great insight into how these can be used.

❤Subscribe for more free YouTube tips: Subscribe to Data analytics Ireland

❤Share this video with a YouTuber friend: How to create Tkinter charts

numpy Python 

how to add sine and cosine in python code

admin 334 Views 0 Comments , , , , , ,

Explaining what this post is about
I was recently online and providing help to a fellow python coder, and a query came up about how you would rewrite some code that had included the sine function  and the cosine function

We were asked to see if we could translate this code:

L_1=1;
L_2=1.5;
q_0=0.5;
q_1=pi()/4;
q_2=pi()/6;
%Position of the end effetor
x= q_0+L_1*cos(q_1)-L_2*cos(pi()-q_1-q_2)
y=L_1*sin(q_1)+L_2*sin(pi()-q_1-q_2)

into its python equivalent.

Some background about the output

In order to get the desired result, there is a need to import a package to provide some of the mathematical analysis required, and was achieved through using Numpy statistical analysis

This package allows the following functions to be used in the logic:

  • Pi
  • Cosine
  • Sine

And the result is

As a result, the below shows the output of the above question converted to its Python equivalent:

import numpy as np

a= np.pi
print("PI value is ", a)

L_1=1
L_2=1.5
q_0=0.5
q_1=a/4
q_2=a/6

print("L_1 value is",L_1)
print("L_2 value is",L_2)
print("q_0 value is",q_0)
print("q_1 value is",q_1)
print("q_2 value is",q_2)


x= (q_0+(L_1*(np.cos(q_1)))-(L_2*(np.cos(a-q_1-q_2))))
y= (L_1*(np.sin(q_1))+(L_2*(np.sin(a-q_1-q_2))))

print("x value is " , x)
print("y value is " , y)

with its output showing:

PI value is  3.141592653589793
L_1 value is 1
L_2 value is 1.5
q_0 value is 0.5
q_1 value is 0.7853981633974483
q_2 value is 0.5235987755982988
0.7071067811865476
-0.25881904510252085
x value is  1.5953353488403288
y value is  2.15599552062015