The breaker in your panel is sized for the complete circuit, say 20amps, the overload on the starter/contactor is to protect the motor. The overload would be set at say 15 amps to trip. If there is no overload the breaker in the panel would eventually trip depending on what it is but it may be too late thus burning the motor up.
See if there is a CED, Rexel, or Graybar electrical distributor in your area. They should have a Manual motor starter. Ask them for a 2 pole manual starter, then you will need to find out the full load amp on the motor unless you know the horsepower. (the dist. should be able to size the overloads based on HP and voltage)
As pook said, you are probably fine, but at the least change that switch out to a Double pole single throw so that you are turning off both hot leads to the motor. This will get you by until you have more time to go back and install a starter.
A starter is nothing more than a contactor with an overload built in or bolted onto the load side of the contactor. Some are manual and require no control circuit others are magnetic and require a control circuit to energize a coil that magnetically pulls the contacts together.
The easiest for your app would be a manual starter.
If it is a small motor, pulling no more than 15 amps. see if you can find a GE or equal. (i sell ge, so i have the part numbers)
GE Part# CR101H1
this is a 2 pole manual starter
You will need a heater/overload as well
Depending on the load, Say 14.8amp the part # would be
CR123H171B this is just an example since i don't know exactly what motor you have.
Total cost should be no more than $60 to $70
This is as cheap as you can go but still protect the motor.
Tony