What it shows
The program reads a text file, cleans the words and counts them in a dictionary. It shows how text analysis works and which words dominate ordinary writing.
The science
A dictionary can hold each word as a key and its count as the value. The get method returns zero for words not yet seen, so one line updates any count. Sorting by value shows the most common words. In most English texts the same small words such as the, of and and make up a large share of all words.
Materials
Cost: free. Many items come from home or from waste; spare parts such as LEDs, motors and wire are in the store.
- A computer, laptop or phone with a browser (free)
- Python 3 with IDLE, or the free Thonny editor, or an online Python runner (all free)
- A text file story.txt containing a chapter or an essay you wrote or a free public-domain story, saved in Notepad
- A notebook and a pencil
Safety
This project uses safe, everyday materials. Still, read the rules below with a parent before you start.
Never use mains electricity: power models with cells, batteries or a low-voltage adapter. Sharp and hot tools, flames and lab chemicals need an adult or your teacher beside you. Keep magnets, button cells and small parts away from young children, and wash your hands after making.
All the safety rulesStep by step
8 steps. Do them in order, and test each part before you go on to the next. Write what you notice in a notebook as you go.
Save a few paragraphs in story.txt in the same folder as your program.
Create words.py and read the file with with open("story.txt", encoding="utf-8") as f: and text = f.read().lower().
Remove punctuation by looping over ".,;:!?\"()" and calling text = text.replace(p, " ") for each p.
Split into words with words = text.split() and print(len(words), "words in total").
Create counts = {} and loop for w in words: with counts[w] = counts.get(w, 0) + 1.
Sort with top = sorted(counts.items(), key=lambda pair: pair[1], reverse=True).
Print the first ten with for w, c in top[:10]: print(w, c).
Try two different texts, for example an English story and a news report, and compare their top words.
What you should see
The program prints the total word count and the ten most common words with their counts. Small words such as the and and appear at the top for most English texts.
Viva questions
A teacher or a judge will ask how it works. Try to answer each question before you open it, then compare.
1What does counts.get(w, 0) do?
It returns the existing count, or 0 if the word is not yet in the dictionary.
2Why convert to lower case?
So The and the count as the same word.
3What does split do?
It breaks the text into a list of words at the spaces.
4Why sort by value?
To see the commonest words first.
Take it further
Skip a list of stop words such as the, a and of to reveal the topic words of the text.
Change one thing, measure what happens, and add a table or chart of your readings: that is what turns a model into your own investigation.
Free ebooks for this project
Computer & office
Computer Fundamentals Study Book
The complete study notes, practice questions and glossary for Computer Fundamentals