"Good enough" control can be obtained by wiring a potentiometer in series with the motor, i.e. as a rheostat (they are the same thing wired differently). More resistance, less speed. The voltage control is not linear because it forms a voltage divider. It's also continuous - it doesn't "snap" to 0.1V or 0.2V. Maybe you can include a voltmeter in your display, so it shows the current voltage, and then you turn the knob until you get the voltage you want. You can also write a non-linear scale behind the knob. This is the simplest it can possibly be.
If you want a linear control you can wire it as a potentiometer. But connecting the motor will cause the voltage to sag, severely, from what you set it to. So you would need a potentiometer to adjust the voltage, and then a circuit to replicate the same voltage to the motor without affecting it. That is something that can be built, with a handful of parts including an IC (maybe someone else knows a design without an IC). Since you seem to be a complete beginner I wouldn't call it simple for you, but it would be simple for someone who knew how to build circuits with transistors and ICs.
You can find an already-built voltage converter module online. Buying these is pretty common for electronics hobbyists because it's easier than building your own. It will be a circuit board with some places to screw the input (battery) and output (motor) wires and some way to adjust the voltage, either a small knob or up/down buttons. Often it takes a screwdriver You may be able to find one that has a knob and also has a little screen that shows the voltage. A note: a "buck converter" can decrease voltage (motor voltage lower than battery voltage); a "boost converter" can increase it (motor voltage higher than battery); a "buck/boost converter" can do both.
For the science fair context, this may be a bit of a dangerous option since these are usually just circuit boards and people could touch the circuit board with metal things and break it - and you'd have to let them touch it so they could adjust the voltage. But maybe you're handy enough to make a little cardboard or wooden or plastic case.
The voltage from your batteries will probably decrease over the course of the science fair so if you label different voltages on the knob they will be wrong by the end. If you choose to buy a voltage converter circuit, it will automatically adjust to keep the output voltage the same. You might want to consider plugging your circuit into the wall - not dangerous - you can find transformers that plug into the wall and output 12 or 9 or 5 volts that you can use for your circuit. This only works if your science fair will have a place to plug it in.
I haven't gone into too much detail on any of these options because I think the main point of this question was to figure out what your options are. If you need more help implementing one you can ask a separate question. Note that "where can I buy this part?" questions are not allowed because the answers always go out of date.
My recommendation: If the goal is just to measure how the motor reacts to different voltages (you do not care about explaining the rest of the circuit), and you can find a way to protect the circuit from damage from kids, I would try to find a voltage converter module that has the right user interface for your project. Otherwise, I would use the first option (rheostat in series and a voltmeter) as it is very simple.