Benford’s Law is a mathematical law that describes the behavior of naturally-occurring numbers in some kinds of numerical data sets. I recommend that you watch this video before proceeding to get an explanation:
Benford’s law is useful for distinguishing naturally occurring data from randomized or made-up data.
It has been used in the real world to detect election fraud in the 2009 Iranian election and (maybe) in the 2020 US election.
It has also been used as evidence in criminal cases in the US.
In this PA, you’ll be writing a program that reads in a data set, and prints out the plot of first-digits.
Then, you can look at the plot to determine if it conforms to the law or not!
Name your file benfords_law.py.
You should organize the code into several functions: main, one for loading the file, one for counting the occurrences, and one for printing the plot.
Your program should as the user for the name of an input file, which your program should expect to be formatted as CSV.
If you don’t know what a CSV file is, or forgot, go watch the video quiz that covered it!
Shown below is an example of the program prompting the user for a file name, the types file name (places.csv), and then printing out the plot.
Data file name:
places.csv
1 | ###############################
2 | ##############
3 | ##########
4 | ########
5 | #######
6 | #####
7 | ####
8 | ####
9 | ####
Follows Benford's Law
After opening up the input file, the program should search through the CSV data for numerical values. The way you should do so is as follows:
0, then convert the string to a float and append to the list of numbersFor example, say you had this data file named places.csv:
region,population
pima,1234
georgia,145
steele,10
tampa,1700
greece,1729
rome,1711
milan,219
tucson,231
tuscany,20001
florence,301
nigeria,3879
newyork,404
phoenix,40123
belgium,505
madrid,502
nogales,601
brussels,712
tempe,81231
anthem,91231
After reading it in, your numbers list should be:
numbers = [1234.0, 145.0, 10.0, 1700.0, 1729.0, 1711.0, 219.0, 231.0, 20001.0, 301.0,
3879.0, 404.0, 40123.0, 505.0, 502.0, 601.0, 712.0, 81231.0, 9123.0]
In the next step, you should use this list of numbers to build the plot
In order to create the plot, you will first have to loop through the numbers list and count how many times a number starts with the digit 1, the digit 2, the digit 3, and so on up to 9.
I recommend that you use a dictionary for this counting.
If you have a floating-point number x, you can get the first digit as an int by doing int(str(x)[0]).
Based on the places.csv data shown earlier, the counts dictionary should be as follows after counting:
counts = {1: 6, 2: 3, 3: 2, 4: 2, 5: 2, 6: 1, 7: 1, 8: 1, 9: 1}
If you forgot how to use a dictionary to count things, go watch the video quiz where I showed how to do so!
After counting, loop through the numbers 1 through 9 and figure out the percentage that each occurs.
You will use these percentages both to print out the bar chart, and to check if the data follows the law.
The way that you would calculate the percentage for a particular digit, as an integer, is:
The number of # for a digit in the plot should be the same as the percentage of the data that digit appears first.
For example, in the places.csv data, there were 3 numbers that started with the digit 2 and there were a total of 19 numbers from the data set, then you should print out int((3 / 19) * 100) = 15%.
Thus, 15 hashtags for 2.
For each row of the plot, print out the digit, a vertical bar, and then the hashtags.
The plot that should print based on the places.csv example is:
1 | ###############################
2 | ###############
3 | ##########
4 | ##########
5 | ##########
6 | #####
7 | #####
8 | #####
9 | #####
The other thing you should determine is if the data follows Benford’s Law. For the purposes of this PA, a data set will follow Benford’s law if the percentage of occurrences of each digits follows the following percentages, plus 10% or minus 5%.
| digit | percent |
|---|---|
| 1 | 30% |
| 2 | 17% |
| 3 | 12% |
| 4 | 9% |
| 5 | 7% |
| 6 | 6% |
| 7 | 5% |
| 8 | 5% |
| 9 | 4% |
If every digit follows, then print out Follows Benford's Law.
Otherwise, print out Does not follow Benford's Law.
The populations.csv file contains population information from many countries across the map. If you download this file and run it with your code, you should get:
Data file name:
populations.csv
1 | ##################################
2 | ###############
3 | ###########
4 | ########
5 | ########
6 | ######
7 | ####
8 | #####
9 | ####
Follows Benford's Law
The stocks.csv file contains open, max, min, and closing prices for stocks traded on the NYSE from 10/7/2019. If you run the code with this data, you should get:
Data file name:
stocks.csv
1 | ##############################
2 | #########################
3 | ##########
4 | #######
5 | #######
6 | #####
7 | ####
8 | ###
9 | ####
Follows Benford's Law
The random_numbers.csv file contains a bunch of randomly generated numbers. Due to this, we should NOT expect it to follow Benford’s law. When the code is run with these numbers, you should get:
Data file name:
random_numbers.csv
1 | ##########
2 | ###########
3 | ############
4 | ############
5 | ##########
6 | ############
7 | #########
8 | ##########
9 | ###########
Does not follow Benford's Law
This is due on 11/24/2020 on Gradescope. Make sure that you follow the style guide rules.