This is an interesting topic, and there is a lot to cover, so this will probably extend past this one post. The Torah gives a time to start counting until Shavuot, and the definition of that time is a famous dispute. It is after the Shabbat. But is that "Shabbat" a Saturday? Or is it the day after the first day of Yom Tov of Pesach? Tzedukim, and Karaites IIUC, say Saturday, while Chazal say that the Shabbat is Pesach.
After reviewing a bit of the data, I heavily lean towards it being Pesach, as a matter of peshat. And context and theme are my primary motivators, things which do not always strike me as motivating certain classic meforshim. But there is what to argue and discuss. In this post, however, what I see that makes me lean towards Pesach, and this is what I present as an opening salvo.
As others point out, Shabbat can refer to other Jewish holidays. Thus, in Vayikra 16, about Yom Kippur:
and in the very perek under discussion in parshat Emor, perek 23, again about Yom Kippur:
where Shabbatchem means "your Shabbat." A bit earlier in that same perek, about Rosh HaShanah:
and later in that same perek, about Succot:
In terms of shemitta, in perek 25, the Torah uses Shabbat Shabbaton to discuss a period of imposed rest:
But that is all mere background. I believe that in the beginning of perek 23, the Torah is trying to first give a definition of the term Shabbat before using it, in such important contexts as from when to start counting. Thus, in perek 23:
Why say this? Do we not know there are festivals? Have we not heard about Shabbat?! Rather, this is done to introduce the festivals, and to connect it to the famous Shabbat Bereishit, of which everyone knows.
More than that, we are told that the seventh day of the week is a "Shabbat Shabbaton," and then we are carefully told what this entails. It is "a holy convocation; ye shall do no manner of work." Why, because it is a "sabbath" unto Hashem. This is definitional. We are being told what a "Shabbat" is.
Then, continuing in the perek:
In pasuk 4, we are introduced to the מוֹעֲדֵי ה מִקְרָאֵי קֹדֶשׁ. These have already been defined above by the term Shabbat. Then, after giving the particular days, we hear in pasuk 7 and pasuk 8 that these days of Yom Tov are mikra kodesh, such that כָּל-מְלֶאכֶת עֲבֹדָה לֹא תַעֲשׂוּ. That was precisely what had been defined above as a Shabbat, and it had been defined above as that for a purpose -- to introduce terminology. A Shabbat, day of rest, is one on which one does no work, as opposed to the intervening chol haMoed.
Next, we have:
This is after having introduced Pesach, so the context of "day of rest" is the Pesach. Perhaps the first day of Pesach, perhaps the last day. (More on my theory of the last day in a separate post.) But Shabbat has been established as a technical term.
It gets a bit more complicated a little lower, when Shabbat is used in the context of seven Shabbats:
Here we need to alternate between the meaning "day of rest" and "weeks." Presumably the Karaites would not need to do that, for if we start after Shabbat, we can count Saturdays, rather than weeks. But language can and does alternate like this, and I get the impression that not only is Shabbaton equivalent to Shabbat for these purposes, part of the point of the introduction in the beginning of the perek was to define Shabbat as a day in which no rest is done, as is to be learnt from Saturday.
Whether or not Chazal are correct, something certainly influenced them to adopt this understanding and reject the seemingly simpler explanation of it as Saturday. Perhaps this is simply the lunar calendar in play rather than solar calendar. Perhaps it was this introductory material defining Shabbat, as speaks to me. Or perhaps it is this pasuk in sefer Yehoshua, which some meforshim discuss and some reject. In Yehoshua 5:
No mention is made here of bringing the omer sacrifice. And see whether this is the 15th day or the 16th day. Is this the daylight following the night of the Pesach offering, or is it after the first day of Pesach? See Ibn Ezra. But still, there is an idea of eating produce of the land of Israel מִמָּחֳרַת הַפֶּסַח. And this language is highly reminiscent of the Pesach associated omer-offering making things Yashan, with the words מִמָּחֳרַת הַשַּׁבָּת. I do not feel like this is an accident, but rather was a word choice by someone who knew the wording used in the Torah. Which makes me strongly consider it to mean the day after Pesach, from a text-internal perspective