What it shows
A first machine learning project should show how a computer decides from examples. The k-nearest neighbours method classifies a new flower or wine by looking at its closest known examples. A student tests how the choice of k and scaling of the features change accuracy.
The science
KNN has no training step: it stores all examples. To classify a new point, it finds the k closest stored points by Euclidean distance and takes a majority vote. A small k follows noise and a large k blurs the classes. Features with large numeric ranges dominate the distance, so scaling them (for example standardisation) is important. A confusion matrix shows which classes are mixed up.
Materials
Cost: free. Many items come from home or from waste; spare parts such as LEDs, motors and wire are in the store.
- A laptop with Python 3, numpy, pandas, matplotlib and scikit-learn
- The iris and wine data sets that come with scikit-learn (open data)
- Jupyter or Thonny
- A notebook for the report
Safety
For this project: No special safety needs. Save your work often.
Never use mains electricity: power models with cells, batteries or a low-voltage adapter. Sharp and hot tools, flames and lab chemicals need an adult or your teacher beside you. Keep magnets, button cells and small parts away from young children, and wash your hands after making.
All the safety rulesStep by step
10 steps. Do them in order, and test each part before you go on to the next. Write what you notice in a notebook as you go.
Load the iris data and print the number of samples, features and classes.
Draw scatter plots of pairs of features coloured by class to see which are easy to separate.
Split the data 70 to 30 with stratification and a fixed random seed.
Train KNeighborsClassifier with k = 5 and measure accuracy on the test set.
Try k from 1 to 25, plot accuracy against k and choose a good value.
Print the confusion matrix and discuss which two flower types are confused.
Repeat the whole process for the wine data without scaling and note the accuracy.
Apply StandardScaler to the features and repeat; compare the accuracies.
Use five-fold cross validation to get a more reliable accuracy estimate and its spread.
Write the report with plots, the table of k against accuracy and a paragraph on why scaling matters.
What you should see
Iris accuracy is about 95 to 100 percent. For the wine data, accuracy is around 70 percent without scaling and rises above 95 percent after scaling, which shows clearly why preprocessing matters.
Viva questions
A teacher or a judge will ask how it works. Try to answer each question before you open it, then compare.
1What does k mean in KNN?
The number of nearest neighbours that vote on the class of a new point.
2Why is scaling needed?
Distance would otherwise be controlled by features with large values, such as proline in the wine set.
3What is overfitting in KNN?
Using k = 1 follows every noisy point, so training accuracy is perfect but test accuracy may drop.
4What does a confusion matrix show?
How many cases of each true class were predicted as each class.
Take it further
Collect measurements of 30 leaves from your neighbourhood and train a classifier to recognise two local plant species.
Change one thing, measure what happens, and add a table or chart of your readings: that is what turns a model into your own investigation.
Free ebooks for this project
Programming
Flowcharts, Algorithms and Logic Building
Think like a programmer before you write any code