What matters is the quality of your show. In my opinion, numbers will fluctuate but you need to be honest with what you are offering to your audience. There are shows that have super high numbers but then you can't even sit through those.