^ I'm sorry if you were offended as it was not my intention, but the type of tomato matters quite a bit when referencing peppers relative to tomatoes.
Realize that everyone has different growing conditions so a pot size that may work for one, may not work for another. As I started to suggest there are too many variables to give you a complete answer without complete information which is probably more than either of us want to cover.
We don't know your grow zone, nor whether you start with nursery peppers and how large, or seed and when if you do. All these things matter, peppers do not stop growing except from lack of sun or warmth. There is no limit to how large a pot can be, to beneficial in the right climate up to a point. 20 gallons is none too big in my region and I don't have a long grow season, though I start to have diminishing returns past 10 gallons.
The increase in pot size needed for a plant with 6 good months of growth instead of 5, is a larger increase than needed for 5 good months instead of 4, and for 4 good months instead of 3 and so on. Every additional month the growth accelerates more if you have a large enough pot and nutrients to handle it, and a long enough growing season.
If you want to use 5 gallons that will work too. The plants may not get as large but you can increase # of plants to make up the difference, or vice versa you could use 3 gallon pots but that's about the smallest I would bother with for a whole season, unless it has to live indoors on a window sill.