I recently had to throw out some of my peppers because they were infested with beetles. The beetles would dig holes in the leaves & stem.
Its a possibility that some type of insects are leaving dropings on your plants.
Did you see any of the 'spots' mooving?
I picked a whole bunch of apples today. Were gunna wait until they get a bit older to make some home-aid apple-sauce and other delicious baked apple treats.
I just posted this pic in case any of you didn't know what Stevia is :-}
I used to have stevia in my herb garden for about 3 months but then dried up and died.
I would only pick the larger leaves for now and wait for it to re-grow.
And unless your seelling the herbs, you can pick some leaves everytime you need them for cooking.